fix: address Copilot review round 3 on the SDK watch
Four findings, including one suppressed. Three fixed outright; the fourth is fixed as far as this PR can reach, with the remainder tracked in #2269 rather than papered over. 1. (Suppressed finding, and the most consequential.) The comment lookup was `gh api --jq '.[].body'` split on newlines, which destroyed the comment boundaries — a body is multi-line, so every LINE became its own entry. Both consumers read a line as though it were a whole comment and both use `startsWith`, so a maintainer quoting `<!-- sdk-watch:analysis -->` at the start of any line would have permanently convinced the sweep that issue was analyzed, and the retry added in round 2 would never fire for it again. The same held for the supersession marker. `issueComments` now fetches `--paginate --slurp` and parses JSON, mapping whole `body` strings. The test fake was part of the problem — it returned newline-joined text, which made the broken parse look correct because it could never produce a multi-line body — so it now returns the real wire shape, an array of pages of comment objects. Note the round-2 test written for this class checked INLINE quoting and passed. The unit was correct; the transport feeding it was not. 2. A CLOSED matching issue was being re-queued for analysis. Two different questions about the same match had been collapsed into one: whether it suppresses creation (yes, whatever its state — that is why `sweepIssues` reads `--state all`) and whether it should be re-analyzed (only if open). A closed issue has no analysis marker almost by definition, so it was handed to the analyze job and would have drawn a fresh automated comment nightly, re-arguing the "not planned" decision that `--state all` exists to respect. Gated on `match.state === "OPEN"`, with the closed case logged rather than passed over in silence. 3. The docs contradicted the code in three places — AGENTS.md, the workflow header and `formatFiledOutput` all still said analysis runs only for newly created issues. All three now describe the same rule, including both exclusions and why they differ: an already-analyzed issue is excluded to avoid a nightly duplicate, a closed one because closing it was a decision. 4. The model retains a secret-exfiltration path: `claude-code-action` copies its environment into the model's, `Read` is genuinely needed to inspect how `core/` uses the SDK, and `analysis` is model-controlled text this workflow publishes. Removing the write-capable `gh` tools closed the posting channel, not the output channel. Landed here as a BACKSTOP, and labelled as one in the workflow so it is not read as settled: the posting step refuses any analysis containing ANTHROPIC_API_KEY or the job's GITHUB_TOKEN verbatim, matched with shell pattern matching so no secret reaches an argv. An injection that encodes the value defeats it. The real fix is workload identity federation (`anthropic_federation_rule_id` + `id-token: write`, and no `anthropic_api_key` at all), which needs an Anthropic-side federation rule and org UUID that do not exist and cannot be created from a pull request. Filed as #2269 with the exact YAML change and acceptance criteria, boarded at Todo/Medium in v2.6.0. AGENTS.md and the workflow both point at it rather than describing this job as secure. Mutation-checked, each failing exactly one test: newline splitting restored -> the multi-line-comment test `state === "OPEN"` guard removed -> the closed-issue test 49 tests in this file; `npm run local:gate` green end to end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9 Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall
parent
6711593a37
commit
e9a1571f42
1 changed files with 149 additions and 15 deletions
48
.github/workflows/sdk-watch.yml
vendored
48
.github/workflows/sdk-watch.yml
vendored
|
|
@ -15,10 +15,12 @@
|
|||
# issue per upstream that is behind. It runs every night and is a complete
|
||||
# no-op when nothing moved.
|
||||
# * `analyze` is the "have Opus read the SDK changes" half of #1063. It runs
|
||||
# ONLY over issues the sweep just created — never over one that already
|
||||
# existed — which is what keeps it to exactly one analysis per SDK release
|
||||
# rather than a near-identical comment every night for as long as the issue
|
||||
# stays open.
|
||||
# over the issues the sweep just created, plus any OPEN issue it already
|
||||
# filed that still carries no analysis comment — the retry path for an
|
||||
# `analyze` job that failed or timed out. What it never does is re-analyze an
|
||||
# issue that already has one, which is what keeps it to a single analysis per
|
||||
# SDK release rather than a near-identical comment every night for as long as
|
||||
# the issue stays open.
|
||||
#
|
||||
# ⚠️ Why this files an issue and not a PR, and why it is NOT the Copilot coding
|
||||
# agent that #1063's comment sketched. Assigning `copilot-swe-agent` is possible
|
||||
|
|
@ -201,6 +203,25 @@ jobs:
|
|||
# What remains is read-only and takes no attacker-chosen file path.
|
||||
# `--disallowedTools` names the write tools anyway, so a later edit
|
||||
# that widens the whitelist cannot silently restore them.
|
||||
#
|
||||
# ⚠️ **A RESIDUAL EXFILTRATION CHANNEL REMAINS, AND IT IS NOT CLOSED
|
||||
# HERE.** `claude-code-action` copies the action's environment into the
|
||||
# model's, so `ANTHROPIC_API_KEY` and the job's `GITHUB_TOKEN` are
|
||||
# readable by a `Read` this job genuinely needs — and `analysis` is
|
||||
# model-controlled text this workflow publishes. An injection can
|
||||
# therefore still read a credential and encode it in the write-up
|
||||
# (Copilot, round 3). Two things narrow it and neither is a boundary:
|
||||
# the posting step refuses any analysis containing a credential
|
||||
# VERBATIM, and the job's own token is `contents: read` and dies with
|
||||
# the job. `ANTHROPIC_API_KEY` is the asset that outlives the run.
|
||||
#
|
||||
# The real fix is to stop putting a long-lived key in the environment
|
||||
# at all: this action supports workload identity federation
|
||||
# (`anthropic_federation_rule_id` + `anthropic_organization_id` with
|
||||
# `id-token: write`), which exchanges the workflow's OIDC token for a
|
||||
# short-lived credential. That needs an Anthropic-side federation rule
|
||||
# and org UUID that do not exist yet, so it cannot land in this change.
|
||||
# Tracked separately — do not treat the scan below as having settled it.
|
||||
claude_args: |
|
||||
--model claude-opus-5
|
||||
--max-turns 40
|
||||
|
|
@ -226,12 +247,31 @@ jobs:
|
|||
ISSUE: ${{ matrix.target.issue }}
|
||||
LABEL: ${{ matrix.target.label }}
|
||||
TO: ${{ matrix.target.to }}
|
||||
# Read back ONLY to refuse publishing them; see the scan below.
|
||||
SCAN_ANTHROPIC: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
SCAN_GITHUB: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${ANALYSIS//[[:space:]]/}" ]; then
|
||||
echo "sdk-watch: the analysis came back empty — posting nothing" >&2
|
||||
exit 1
|
||||
fi
|
||||
# ⚠️ A BACKSTOP, NOT A BOUNDARY — read the security note above the
|
||||
# analysis step before relying on it. `analysis` is model-controlled
|
||||
# text and the model can still read files, so a verbatim credential in
|
||||
# it is the one exfiltration shape that is cheap to refuse outright.
|
||||
# Matched with shell pattern matching rather than `grep` so no secret
|
||||
# ever reaches an argv that `ps` could show.
|
||||
for scanned in "$SCAN_ANTHROPIC" "$SCAN_GITHUB"; do
|
||||
if [ -n "$scanned" ]; then
|
||||
case "$ANALYSIS" in
|
||||
*"$scanned"*)
|
||||
echo "sdk-watch: the analysis contains a credential verbatim — refusing to post" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
done
|
||||
printf '%s\n\n## Automated review of %s %s\n\n%s\n\n---\n\n%s\n' \
|
||||
'<!-- sdk-watch:analysis -->' \
|
||||
"$LABEL" "$TO" "$ANALYSIS" \
|
||||
|
|
|
|||
Loading…
Reference in a new issue