modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

fix: address Copilot review round 6, and close out the federation follow-up

Browse source 
Two unrelated things, both touching the same security notes.

## Round 6 findings

1. The credential scan ran too late to be a backstop. The analysis was
   uploaded as an artifact and only scanned in the posting job — but this
   repo is public, so an artifact holding a verbatim credential is
   downloadable for its whole retention window. Refusing to POST text
   that has already left the job refuses nothing. The scan now gates the
   WRITE, in the same step that writes `analysis.md` (not two steps: a
   failure between a passing scan and a separate write would leave the
   file for the upload to collect), and the upload is gated on
   `hashFiles('analysis.md')`. The posting job scans again, since the
   artifact is the boundary and each side should check what it handles.

2. The permissions test failed open on the input it existed to catch.
   GitHub accepts the shorthand `permissions: write-all`, and
   `Object.entries("write-all")` iterates the string's CHARACTERS — so
   the scan for a `write` value found nothing and the test passed green
   while the model held every scope. It now rejects a non-mapping
   outright and asserts the exact value.

3. (Suppressed.) The security prose in AGENTS.md still described the
   same-job, environment-variable handoff that two rounds had already
   replaced. Rewritten around the two boundaries the write-up actually
   crosses now. Worth noting the pattern rather than just the instance:
   this was the third round running in which I changed the mechanism and
   left the prose describing the previous one, and stale security prose
   is worse than none — the next person preserves properties the code no
   longer has.

## Closing #2269

Filed during round 3 to replace the long-lived ANTHROPIC_API_KEY with
workload identity federation. Closed as not planned, because the threat
model behind it does not hold.

Every round escalated on "the model reads UNTRUSTED upstream release
notes" and I accepted that framing without checking it. Both upstreams
this sweep watches — modelcontextprotocol/typescript-sdk and
modelcontextprotocol/ext-apps — are in this repository's own org, with
release notes written by the same maintainer group. That is first-party
content, and anyone able to plant an injection in it already holds
release rights here.

Federation was reachable, so this is a judgement rather than a
capitulation: issuers and rules are creatable via
`client.beta.organization.federation.*`, and for GitHub Actions the
issuer is the standard token.actions.githubusercontent.com with JWKS
discovery. But those endpoints reject plain API keys and need an
`org:admin` OAuth token, making it org-admin work on the Anthropic
organization — disproportionate against our own changelogs.

The assessment is recorded rather than deleted. AGENTS.md and the
workflow comment now state the residual channel AND what makes it
acceptable, so pointing this at an upstream outside the org, or at
third-party content, is a documented trigger to re-open the question.

Mutation-checked, each failing a different set:

  permissions: write-all on analyze -> no model runs in a job that can write
  upload re-gated on structured_output -> the scan-ordering test

62 tests in this file; `npm run local:gate` green end to end.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-06 11:19:17 -04:00
commit 7deb8f25d2
3 changed files with 122 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

80
.github/workflows/sdk-watch.yml vendored
View file

@ -272,24 +272,28 @@ jobs:
# needing a prompt no headless run can answer, and `--disallowedTools`
# denies the rest by name as a third layer.
#
# ⚠️ **A RESIDUAL EXFILTRATION CHANNEL REMAINS, AND IT IS NOT CLOSED
# HERE.** `claude-code-action` copies the action's environment into the
# model's, so `ANTHROPIC_API_KEY` and the job's `GITHUB_TOKEN` are
# readable by a `Read` this job genuinely needs — and `analysis` is
# model-controlled text this workflow publishes. An injection can
# therefore still read a credential and encode it in the write-up
# (Copilot, round 3). Two things narrow it and neither is a boundary:
# the posting step refuses any analysis containing a credential
# VERBATIM, and the job's own token is `contents: read` and dies with
# the job. `ANTHROPIC_API_KEY` is the asset that outlives the run.
# ⚠️ **A RESIDUAL CHANNEL REMAINS, AND IT IS ACCEPTED ON PURPOSE.**
# `claude-code-action` copies the action's environment into the model's,
# so `ANTHROPIC_API_KEY` is readable by a `Read` this job genuinely
# needs, and `analysis` is model-controlled text this workflow
# publishes. The verbatim-credential scan catches the naive shape only;
# an encoded value passes it.
#
# The real fix is to stop putting a long-lived key in the environment
# at all: this action supports workload identity federation
# (`anthropic_federation_rule_id` + `anthropic_organization_id` with
# `id-token: write`), which exchanges the workflow's OIDC token for a
# short-lived credential. That needs an Anthropic-side federation rule
# and org UUID that do not exist yet, so it cannot land in this change.
# Tracked separately — do not treat the scan below as having settled it.
# What makes that acceptable is WHAT THIS JOB READS, not the controls
# around it. Both upstreams — `modelcontextprotocol/typescript-sdk` and
# `modelcontextprotocol/ext-apps` — are in this repository's own org, so
# their release notes are first-party content, and anyone able to plant
# an injection in them already holds release rights here. Closing the
# channel properly means workload identity federation instead of a
# long-lived key (`anthropic_federation_rule_id` + `id-token: write`);
# that is org-admin work on the Anthropic organization and is
# disproportionate against our own changelogs. See #2269, closed as not
# planned, for the full reasoning.
#
# ⚠️ **Re-open that judgement if the inputs change.** Point this at an
# upstream outside the org, or at third-party content, and federation —
# or at least a dedicated CI-scoped key with a spend cap — becomes the
# next step rather than another grant to narrow.
claude_args: |
--model claude-opus-5
--max-turns 40
@ -302,14 +306,52 @@ jobs:
# Still inside the READ-ONLY job. This writes the model's text to a file so
# the separately-permissioned `post` job can pick it up; no token capable of
# writing anything exists in this job at all.
# ⚠️ **The credential scan runs HERE, BEFORE anything is written or
# uploaded.** It used to run only in the posting job, which was too late to
# be the backstop it claimed to be: this repository is public, so an
# analysis containing a credential verbatim would have been uploaded as a
# downloadable artifact and sat there for a day, even though the later step
# correctly refused to post it (Copilot, round 6). Refusing at the comment
# is not refusing at all if the text has already left the job.
#
# The same scan is repeated in `post` as defense in depth — the artifact is
# the boundary between the two jobs, so each side checks what it handles.
# A workflow test pins both the presence and the ordering.
- name: Stage the analysis for the posting job
if: ${{ steps.analysis.outputs.structured_output != '' }}
env:
ANALYSIS: ${{ fromJSON(steps.analysis.outputs.structured_output).analysis }}
run: printf '%s' "$ANALYSIS" > analysis.md
# Read back ONLY to refuse writing them out; never logged or posted.
SCAN_ANTHROPIC: ${{ secrets.ANTHROPIC_API_KEY }}
SCAN_GITHUB: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
if [ -z "${ANALYSIS//[[:space:]]/}" ]; then
echo "sdk-watch: the analysis came back empty — staging nothing" >&2
exit 1
fi
# Shell pattern matching rather than `grep`, so no secret ever reaches
# an argv that `ps` could show. A BACKSTOP, NOT A BOUNDARY: it catches a
# verbatim credential and nothing cleverer — see the note above the
# analysis step.
for scanned in "$SCAN_ANTHROPIC" "$SCAN_GITHUB"; do
if [ -n "$scanned" ]; then
case "$ANALYSIS" in
*"$scanned"*)
echo "sdk-watch: the analysis contains a credential verbatim — refusing to stage or upload it" >&2
exit 1
;;
esac
fi
done
printf '%s' "$ANALYSIS" > analysis.md
# Gated on the FILE, not on the model's output being non-empty: the staging
# step above deliberately refuses to write it when the scan trips, and this
# condition is what makes that refusal mean "nothing leaves the job" rather
# than relying on step-failure ordering alone.
- name: Upload the analysis
if: ${{ steps.analysis.outputs.structured_output != '' }}
if: ${{ hashFiles('analysis.md') != '' }}
uses: actions/upload-artifact@v7
with:
name: sdk-watch-analysis-${{ matrix.target.issue }}

11
AGENTS.md
View file

@ -154,9 +154,16 @@ The board write needs `organization projects: write`, which `GITHUB_TOKEN` canno
⚠️ **A marker is not evidence — this repo is public.** Anyone can open an issue or write a comment whose body starts with any string, and every marker here drives automation. Untrusted, an outsider could file (and close) an issue carrying the current target's marker to suppress the real upgrade issue indefinitely, post `ANALYSIS_MARKER` to suppress analysis retries forever, or forge a supersession note so the genuine one is never posted. So the sweep trusts a marker **only** on something the automation wrote: an issue must be authored by `github-actions` *and* carry the `chore` + `dependencies` labels an outsider cannot set, and a comment must come from `github-actions[bot]`. Marker versions are validated with `semver.valid` too, since a malformed target would otherwise throw in `semver.lt` and fail the sweep every run.
So the model returns its write-up as structured output (`--json-schema`) and a deterministic, model-free step posts it — that step is the only thing in the job holding a token that can write, and the body reaches it through the environment rather than the command line. `WebFetch` is denied for the same reason: an outbound fetch the model controls is the other end of the same exfiltration channel, and `gh release view` covers the need. The job's token carries `contents: read` only. **Keep every one of those properties when editing this job.**
So the model returns its write-up as structured output (`--json-schema`) and never posts anything itself. The write-up crosses **two** boundaries before it becomes a comment, and each one is a property to preserve:
⚠️ **One channel is still open, and the workflow says so rather than implying otherwise.** `claude-code-action` copies the action's environment into the model's, so `ANTHROPIC_API_KEY` is readable by a `Read` the analysis genuinely needs, and `analysis` is model-controlled text this workflow publishes. Two things narrow it and **neither is a boundary**: the posting step refuses any analysis containing a credential *verbatim*, and the job's own token expires with the job. The real fix is to stop putting a long-lived key in the environment — the action supports **workload identity federation** (`anthropic_federation_rule_id` + `id-token: write`), which needs an Anthropic-side federation rule this org does not have yet. Tracked in #2269. Do not read the verbatim scan as having settled it.
1. **Out of the model's job.** A step in `analyze` scans the text and, only if it passes, writes `analysis.md` and uploads it as an artifact. ⚠️ **The scan gates the write, in the same step, and the upload is gated on that file existing** — scanning later would be no protection at all, since this repo is public and an artifact holding a verbatim credential is downloadable for as long as it is retained. Refusing to *post* text that has already left the job refuses nothing.
2. **Into the comment.** `post` downloads the artifact, scans it again (defense in depth — the artifact is the boundary, so each side checks what it handles), and streams it to `gh issue comment --body-file -` over **stdin**, so nothing in the body can be read as a flag or a path.
`WebFetch` is denied and the model has no shell, so it controls no outbound channel; release notes are prefetched with `gh api` by a deterministic step. **Keep every one of those properties when editing this job**, and note that a test pins the scan-before-upload ordering specifically, because ordering is the whole control.
⚠️ **One channel is still open, and it is accepted deliberately rather than closed.** `claude-code-action` copies the action's environment into the model's, so `ANTHROPIC_API_KEY` is readable by a `Read` the analysis genuinely needs, and `analysis` is model-controlled text this workflow publishes. The verbatim-credential scan catches only the naive shape; an encoded value passes it. **What makes that acceptable is the threat model, not the controls: both upstreams this sweep reads (`modelcontextprotocol/typescript-sdk`, `modelcontextprotocol/ext-apps`) are in this repository's own org**, so the release notes are first-party content, and anyone able to plant an injection in them already holds release rights here. Closing it properly means workload identity federation instead of a long-lived key (`anthropic_federation_rule_id` + `id-token: write`) — possible, but org-admin work on the Anthropic organization, and disproportionate against our own changelogs (#2269, closed as not planned, records the full reasoning).
**That assessment is what to re-open if the inputs change.** Point the analysis at an upstream outside this org, or feed it third-party content, and the trade changes — at which point federation, or a dedicated CI-scoped key with a spend cap, is the next step rather than another grant to narrow.
## Contributing

57
scripts/sdk-watch.test.mjs
View file

@ -215,13 +215,20 @@ test("no model runs in a job that can write", () => {
// A job with no `permissions:` block inherits the top-level one, which here
// includes `issues: write` — so an omitted block is a failure, not a default.
const permissions = job.permissions ?? inherited;
const writes = Object.entries(permissions).filter(
([, level]) => level === "write",
// ⚠️ GitHub also accepts the SHORTHAND forms `permissions: write-all` and
// `read-all`. Scanning entries for a `write` value silently passes on those,
// because `Object.entries("write-all")` iterates the string's CHARACTERS and
// finds nothing equal to `write` — so the test would go green while the model
// held every scope there is (Copilot). Reject a non-mapping outright, then
// assert the exact value.
assert.ok(
permissions !== null && typeof permissions === "object",
`job "${name}" runs a model and uses the shorthand permissions "${permissions}" — write it as an explicit mapping`,
);
assert.deepEqual(
writes,
[],
`job "${name}" runs a model and holds write scope: ${JSON.stringify(permissions)}`,
permissions,
{ contents: "read" },
`job "${name}" runs a model, so its permissions must be exactly {contents: read}, not ${JSON.stringify(permissions)}`,
);
assert.equal(
(job.steps ?? []).some((s) => /gh issue comment/.test(s.run ?? "")),
@ -275,6 +282,46 @@ test("the model's tool availability is restricted to reading, with no shell", ()
}
});
test("the credential scan runs before the analysis can leave the job", () => {
// ⚠️ Ordering IS the control here. This repo is public, so an artifact holding
// a verbatim credential is downloadable for as long as it is retained —
// refusing to post it afterwards is not refusing at all, because the text has
// already left the job (Copilot). The scan therefore gates the WRITE, and the
// upload is gated on the file the scan refused to produce.
const steps = analyzeJob().steps;
const scanAt = steps.findIndex((s) =>
/contains a credential/.test(s.run ?? ""),
);
const uploadAt = steps.findIndex((s) =>
String(s.uses ?? "").startsWith("actions/upload-artifact"),
);
assert.ok(scanAt !== -1, "expected a credential scan in the analyze job");
assert.ok(uploadAt !== -1, "expected an upload step in the analyze job");
assert.ok(
scanAt < uploadAt,
"the credential scan must run before the artifact is uploaded",
);
// The scan and the write must be the SAME step, or a failure between them
// would leave the file on disk for the upload to pick up anyway.
assert.match(
steps[scanAt].run,
/> analysis\.md/,
"the scan must gate the write of analysis.md, in the same step",
);
assert.match(
String(steps[uploadAt].if ?? ""),
/hashFiles\('analysis\.md'\)/,
"the upload must be gated on the file the scan refuses to write",
);
// Kept on the far side of the artifact boundary too, as defense in depth.
const post = workflowDoc().jobs.post;
assert.ok(
(post.steps ?? []).some((s) => /contains a credential/.test(s.run ?? "")),
"the posting job must scan what it received as well",
);
});
test("the release notes are fetched by a step the model does not run in", () => {
// The model has no shell, so the notes must arrive some other way — if this
// step is ever dropped, the analysis silently degrades to guesswork.