inspector/scripts
Repository files (latest commit first)
Filename Latest commit message Latest commit date
cliffhall 7deb8f25d2 fix: address Copilot review round 6, and close out the federation follow-up
Two unrelated things, both touching the same security notes.

## Round 6 findings

1. The credential scan ran too late to be a backstop. The analysis was
   uploaded as an artifact and only scanned in the posting job — but this
   repo is public, so an artifact holding a verbatim credential is
   downloadable for its whole retention window. Refusing to POST text
   that has already left the job refuses nothing. The scan now gates the
   WRITE, in the same step that writes `analysis.md` (not two steps: a
   failure between a passing scan and a separate write would leave the
   file for the upload to collect), and the upload is gated on
   `hashFiles('analysis.md')`. The posting job scans again, since the
   artifact is the boundary and each side should check what it handles.

2. The permissions test failed open on the input it existed to catch.
   GitHub accepts the shorthand `permissions: write-all`, and
   `Object.entries("write-all")` iterates the string's CHARACTERS — so
   the scan for a `write` value found nothing and the test passed green
   while the model held every scope. It now rejects a non-mapping
   outright and asserts the exact value.

3. (Suppressed.) The security prose in AGENTS.md still described the
   same-job, environment-variable handoff that two rounds had already
   replaced. Rewritten around the two boundaries the write-up actually
   crosses now. Worth noting the pattern rather than just the instance:
   this was the third round running in which I changed the mechanism and
   left the prose describing the previous one, and stale security prose
   is worse than none — the next person preserves properties the code no
   longer has.

## Closing #2269

Filed during round 3 to replace the long-lived ANTHROPIC_API_KEY with
workload identity federation. Closed as not planned, because the threat
model behind it does not hold.

Every round escalated on "the model reads UNTRUSTED upstream release
notes" and I accepted that framing without checking it. Both upstreams
this sweep watches — modelcontextprotocol/typescript-sdk and
modelcontextprotocol/ext-apps — are in this repository's own org, with
release notes written by the same maintainer group. That is first-party
content, and anyone able to plant an injection in it already holds
release rights here.

Federation was reachable, so this is a judgement rather than a
capitulation: issuers and rules are creatable via
`client.beta.organization.federation.*`, and for GitHub Actions the
issuer is the standard token.actions.githubusercontent.com with JWKS
discovery. But those endpoints reject plain API keys and need an
`org:admin` OAuth token, making it org-admin work on the Anthropic
organization — disproportionate against our own changelogs.

The assessment is recorded rather than deleted. AGENTS.md and the
workflow comment now state the residual channel AND what makes it
acceptable, so pointing this at an upstream outside the org, or at
third-party content, is a documented trigger to re-open the question.

Mutation-checked, each failing a different set:

  permissions: write-all on analyze -> no model runs in a job that can write
  upload re-gated on structured_output -> the scan-ordering test

62 tests in this file; `npm run local:gate` green end to end.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-06 11:19:17 -04:00
..
lib feat(skills): measure a skill reached from another skill (#2204) 2026-09-04 19:07:26 -04:00
dependabot-alerts.mjs chore(deps): address Copilot review round 15 on #2243 2026-09-04 09:26:20 -04:00
dependabot-alerts.test.mjs chore(deps): address Copilot review round 15 on #2243 2026-09-04 09:26:20 -04:00
dependency-refresh.mjs chore(deps): address Copilot review round 1 on #2243 2026-09-04 09:06:32 -04:00
dependency-refresh.test.mjs chore(deps): paginate the release list so the highest version is actual 2026-09-04 09:06:31 -04:00
install-clients.mjs feat: port CLI, TUI, and launcher from v1.5 into v2 (#1452) 2026-06-14 20:27:49 -04:00
install-smoke-browser.mjs fix(smoke): reject an empty SMOKE_BROWSER; test the installer's argument logic 2026-08-25 22:52:02 -04:00
install-smoke-browser.test.mjs fix(smoke): reject an empty SMOKE_BROWSER; test the installer's argument logic 2026-08-25 22:52:02 -04:00
pack-and-verify.mjs ci: drop the Firefox smoke job; keep the engines as an on-demand tool 2026-08-25 23:55:19 -04:00
run-engine-smokes.mjs fix(smoke): route the Chromium tier through ENGINE_SMOKES too 2026-08-26 08:55:51 -04:00
run-engine-smokes.test.mjs Merge remote-tracking branch 'origin/v2/main' into v2/chore/2146-rename-local-gate 2026-08-27 00:16:14 -04:00
sdk-watch.mjs fix: address Copilot review round 5 on the SDK watch 2026-09-06 11:00:01 -04:00
sdk-watch.test.mjs fix: address Copilot review round 6, and close out the federation follow-up 2026-09-06 11:19:17 -04:00
skill-eval.mjs Review round 8: using a fixture, not connecting, is the trigger 2026-09-05 18:03:37 -04:00
skill-eval.test.mjs feat(skills): address Copilot review round 3 on #2204 2026-09-04 20:28:37 -04:00
smoke-cli.mjs chore(scripts): build test-servers unconditionally from one shared helper 2026-08-25 00:57:48 -04:00
smoke-launcher.mjs refactor: self-validating clients, fast validate + separate coverage/smoke (#1504) (#1505) 2026-06-21 11:41:33 -04:00
smoke-tui.mjs fix(scripts): make the test stubs drain, and the raw-mode guard testable (#2147) 2026-08-26 22:13:35 -04:00
smoke-web-app.mjs chore(scripts): rename the local pre-push gate off ci, and guard it out of CI (#2146) 2026-08-26 23:56:34 -04:00
smoke-web-browser.mjs chore(scripts): rename the local pre-push gate off ci, and guard it out of CI (#2146) 2026-08-26 23:56:34 -04:00
smoke-web-elicitation.mjs Merge remote-tracking branch 'origin/v2/main' into v2/chore/2146-rename-local-gate 2026-08-27 00:16:14 -04:00
smoke-web-tabs.mjs Merge remote-tracking branch 'origin/v2/main' into v2/chore/2146-rename-local-gate 2026-08-27 00:16:14 -04:00
smoke-web.mjs fix(smoke): await the server's exit before removing its catalog, and test it 2026-08-11 14:32:20 -04:00
verify-build-gate.mjs chore(scripts): rename the local pre-push gate off ci, and guard it out of CI (#2146) 2026-08-26 23:56:34 -04:00
verify-bundle-externals.mjs test(proxy): isolate proxy env per file; word the guard failure by candidate source 2026-08-23 23:00:28 -04:00
verify-bundle-externals.test.mjs test(proxy): isolate proxy env per file; word the guard failure by candidate source 2026-08-23 23:00:28 -04:00
verify-dep-lockstep.main.test.mjs chore: address Copilot review round 1 (#2226) 2026-09-05 16:09:44 -04:00
verify-dep-lockstep.mjs chore: address Copilot review round 3 (#2226) 2026-09-05 17:14:44 -04:00
verify-dep-lockstep.test.mjs chore: address Copilot review round 1 (#2226) 2026-09-05 16:09:44 -04:00
verify-format-coverage.main.test.mjs chore: address Copilot review round 20 2026-08-29 05:32:12 -04:00
verify-format-coverage.mjs chore: address Copilot review round 9 2026-08-29 03:04:03 -04:00
verify-skills-cli.mjs chore: address Copilot review round 17 2026-08-29 04:54:04 -04:00
verify-skills-cli.test.mjs chore: address Copilot review round 16 2026-08-29 04:40:48 -04:00
verify-skills.main.test.mjs feat(skills): measure a skill reached from another skill (#2204) 2026-09-04 19:07:26 -04:00
verify-skills.mjs feat(skills): measure a skill reached from another skill (#2204) 2026-09-04 19:07:26 -04:00
verify-typecheck-coverage.main.test.mjs Merge origin/v2/main: move the #1939 npx fix into the shared tsc-program lib 2026-08-16 17:07:50 -07:00
verify-typecheck-coverage.mjs Merge origin/v2/main: move the #1939 npx fix into the shared tsc-program lib 2026-08-16 17:07:50 -07:00
verify-typecheck-coverage.test.mjs chore(scripts): derive dep-lockstep candidates from what enters each tsc program 2026-08-16 12:17:15 -04:00