fix: address Copilot review round 2 on the SDK watch
Three findings, all real. The first is a security hole introduced by round 1's own fix. 1. The analysis job could have published live secrets. Round 1 pinned `Bash(gh issue comment <N>:*)` to the tracked issue and treated that as scoping, but a `Bash(...)` grant matches a command PREFIX and says nothing about the flags that follow — so `gh issue comment <N> --body-file /proc/self/environ` matched it, and this job's subprocess environment holds ANTHROPIC_API_KEY and GITHUB_TOKEN. A prompt injection from upstream release notes could have posted live credentials into a public issue. The model now has no tool that can write anywhere. It returns its write-up as `--json-schema` structured output, and a deterministic, model-free step posts it: the body arrives through the environment rather than the command line, so nothing in it can be read as a flag or a path, and the issue number comes from the matrix. That step is the only thing in the job holding a token that can write. WebFetch is denied as well. It was not raised, but with `Read` available an outbound fetch the model controls is the same exfiltration channel by another route; `gh release view` already covered the need. The remaining grant is read-only and accepts no attacker-chosen file path. The general rule is now in AGENTS.md rather than only this instance: a prefix grant on any command that accepts a file path is an arbitrary-file read with a publish attached. 2. Issue existence was standing in for analysis completion. A failed or timed-out `analyze` job left an issue the next sweep matched, emitted `[]` for, and reported as a green no-op — so the promised analysis was never retried. The posting step now stamps ANALYSIS_MARKER on its comment, and a sweep finding an existing issue without one re-queues it. The marker is matched with `startsWith`, so a maintainer quoting it cannot suppress the retry. 3. A failed supersession note was unrecoverable. Creation succeeding and the note failing left a state where every retry matched the target's own marker and skipped reconciliation entirely, so the note was never posted. An existing issue is now ADOPTED rather than skipped: the loop takes its number and runs the same follow-up it would for a new one. The existing announced-marker check prevents duplication, and a `stale.number === number` guard stops an issue superseding itself. Findings 2 and 3 share one restructure — replacing the early `continue` with adoption is what gives both the retry path. ANALYSIS_MARKER is necessarily duplicated in the workflow, since the posting step is shell. Drift would be invisible from either file alone and would make every issue read as permanently unanalyzed, so a test asserts the workflow contains the exact constant. Mutation-checked, each failing a different minimal set: if (match) continue; restored -> supersession-retry + re-queue tests hasAnalysis -> always true -> re-queue + hasAnalysis tests workflow marker misspelled -> the drift-guard test 47 tests in this file; `npm run local:gate` green end to end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9 Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
3b3c7e35d1
commit
6711593a37
1 changed files with 273 additions and 71 deletions
|
|
@ -136,7 +136,14 @@ The board write needs `organization projects: write`, which `GITHUB_TOKEN` canno
|
|||
|
||||
**The analysis half runs Claude, not Copilot, and that is deliberate.** #1063 sketched "a copilot agent running Opus"; neither half of that is reachable from a workflow. Assigning `copilot-swe-agent` produces a **pull request** — the artifact this whole section exists to remove — and its model cannot be selected programmatically at all (`replaceActorsForAssignable` takes no model parameter; absent an admin-configured picker it runs Sonnet). So the `analyze` job uses `anthropics/claude-code-action` with `--model claude-opus-5`, which is told to post **one comment** and is denied every file-writing tool. It runs on `ANTHROPIC_API_KEY`, an **organization** secret already available to this repo, and only over issues the sweep **just created** — never over one that already existed, which is what keeps it to one analysis per release instead of a near-identical comment every night.
|
||||
|
||||
⚠️ **Upstream release notes are untrusted input to that job.** It reads text this repo does not control, so its system prompt says so explicitly and its `--allowedTools` list is a whitelist with no `Edit`/`Write` and no `git`. The job's token carries `contents: read` only, so a successful injection still cannot write code. Keep both of those properties when editing the prompt.
|
||||
- **"An issue exists" and "the issue was analyzed" are different claims**, and the sweep must not equate them. The posting step stamps `ANALYSIS_MARKER` on its comment; a sweep that finds an existing issue for the current target but no such comment **re-queues it**, so a failed or timed-out `analyze` job is retried rather than silently never revisited. That marker is duplicated in the workflow because the posting step is shell — a test asserts the two strings match, since drift would make every issue read as unanalyzed forever.
|
||||
|
||||
⚠️ **Upstream release notes are untrusted input to that job, and the model is granted nothing that can write anywhere.** The tool whitelist is the only real control — the prompt's prose constraints are not — and two revisions of it were wrong before this one landed:
|
||||
|
||||
- `Bash(gh api:*)` against an `issues: write` token allowed arbitrary issue mutation.
|
||||
- Pinning `Bash(gh issue comment <N>:*)` to the tracked issue did **not** fix it, because a `Bash(...)` grant matches a command **prefix** and says nothing about the flags that follow. `gh issue comment <N> --body-file /proc/self/environ` matches that grant, and the job's subprocess environment holds `ANTHROPIC_API_KEY` and `GITHUB_TOKEN` — so a prompt injection could have published live credentials into a public issue. **A prefix grant on any command that accepts a file path is an arbitrary-file read with a publish attached.** Treat that as the general rule, not a one-off.
|
||||
|
||||
So the model returns its write-up as structured output (`--json-schema`) and a deterministic, model-free step posts it — that step is the only thing in the job holding a token that can write, and the body reaches it through the environment rather than the command line. `WebFetch` is denied for the same reason: an outbound fetch the model controls is the other end of the same exfiltration channel, and `gh release view` covers the need. The job's token carries `contents: read` only. **Keep every one of those properties when editing this job.**
|
||||
|
||||
## Contributing
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue