inspector/scripts
Repository files (latest commit first)
Filename Latest commit message Latest commit date
cliffhall e9a1571f42 fix: address Copilot review round 3 on the SDK watch
Four findings, including one suppressed. Three fixed outright; the
fourth is fixed as far as this PR can reach, with the remainder tracked
in #2269 rather than papered over.

1. (Suppressed finding, and the most consequential.) The comment lookup
   was `gh api --jq '.[].body'` split on newlines, which destroyed the
   comment boundaries — a body is multi-line, so every LINE became its
   own entry. Both consumers read a line as though it were a whole
   comment and both use `startsWith`, so a maintainer quoting
   `<!-- sdk-watch:analysis -->` at the start of any line would have
   permanently convinced the sweep that issue was analyzed, and the retry
   added in round 2 would never fire for it again. The same held for the
   supersession marker.

   `issueComments` now fetches `--paginate --slurp` and parses JSON,
   mapping whole `body` strings. The test fake was part of the problem —
   it returned newline-joined text, which made the broken parse look
   correct because it could never produce a multi-line body — so it now
   returns the real wire shape, an array of pages of comment objects.

   Note the round-2 test written for this class checked INLINE quoting
   and passed. The unit was correct; the transport feeding it was not.

2. A CLOSED matching issue was being re-queued for analysis. Two
   different questions about the same match had been collapsed into one:
   whether it suppresses creation (yes, whatever its state — that is why
   `sweepIssues` reads `--state all`) and whether it should be
   re-analyzed (only if open). A closed issue has no analysis marker
   almost by definition, so it was handed to the analyze job and would
   have drawn a fresh automated comment nightly, re-arguing the "not
   planned" decision that `--state all` exists to respect. Gated on
   `match.state === "OPEN"`, with the closed case logged rather than
   passed over in silence.

3. The docs contradicted the code in three places — AGENTS.md, the
   workflow header and `formatFiledOutput` all still said analysis runs
   only for newly created issues. All three now describe the same rule,
   including both exclusions and why they differ: an already-analyzed
   issue is excluded to avoid a nightly duplicate, a closed one because
   closing it was a decision.

4. The model retains a secret-exfiltration path: `claude-code-action`
   copies its environment into the model's, `Read` is genuinely needed to
   inspect how `core/` uses the SDK, and `analysis` is model-controlled
   text this workflow publishes. Removing the write-capable `gh` tools
   closed the posting channel, not the output channel.

   Landed here as a BACKSTOP, and labelled as one in the workflow so it
   is not read as settled: the posting step refuses any analysis
   containing ANTHROPIC_API_KEY or the job's GITHUB_TOKEN verbatim,
   matched with shell pattern matching so no secret reaches an argv. An
   injection that encodes the value defeats it.

   The real fix is workload identity federation
   (`anthropic_federation_rule_id` + `id-token: write`, and no
   `anthropic_api_key` at all), which needs an Anthropic-side federation
   rule and org UUID that do not exist and cannot be created from a pull
   request. Filed as #2269 with the exact YAML change and acceptance
   criteria, boarded at Todo/Medium in v2.6.0. AGENTS.md and the workflow
   both point at it rather than describing this job as secure.

Mutation-checked, each failing exactly one test:

  newline splitting restored     -> the multi-line-comment test
  `state === "OPEN"` guard removed -> the closed-issue test

49 tests in this file; `npm run local:gate` green end to end.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-06 10:25:36 -04:00
..
lib feat(skills): measure a skill reached from another skill (#2204) 2026-09-04 19:07:26 -04:00
dependabot-alerts.mjs chore(deps): address Copilot review round 15 on #2243 2026-09-04 09:26:20 -04:00
dependabot-alerts.test.mjs chore(deps): address Copilot review round 15 on #2243 2026-09-04 09:26:20 -04:00
dependency-refresh.mjs chore(deps): address Copilot review round 1 on #2243 2026-09-04 09:06:32 -04:00
dependency-refresh.test.mjs chore(deps): paginate the release list so the highest version is actual 2026-09-04 09:06:31 -04:00
install-clients.mjs feat: port CLI, TUI, and launcher from v1.5 into v2 (#1452) 2026-06-14 20:27:49 -04:00
install-smoke-browser.mjs fix(smoke): reject an empty SMOKE_BROWSER; test the installer's argument logic 2026-08-25 22:52:02 -04:00
install-smoke-browser.test.mjs fix(smoke): reject an empty SMOKE_BROWSER; test the installer's argument logic 2026-08-25 22:52:02 -04:00
pack-and-verify.mjs ci: drop the Firefox smoke job; keep the engines as an on-demand tool 2026-08-25 23:55:19 -04:00
run-engine-smokes.mjs fix(smoke): route the Chromium tier through ENGINE_SMOKES too 2026-08-26 08:55:51 -04:00
run-engine-smokes.test.mjs Merge remote-tracking branch 'origin/v2/main' into v2/chore/2146-rename-local-gate 2026-08-27 00:16:14 -04:00
sdk-watch.mjs fix: address Copilot review round 3 on the SDK watch 2026-09-06 10:25:36 -04:00
sdk-watch.test.mjs fix: address Copilot review round 3 on the SDK watch 2026-09-06 10:25:36 -04:00
skill-eval.mjs Review round 8: using a fixture, not connecting, is the trigger 2026-09-05 18:03:37 -04:00
skill-eval.test.mjs feat(skills): address Copilot review round 3 on #2204 2026-09-04 20:28:37 -04:00
smoke-cli.mjs chore(scripts): build test-servers unconditionally from one shared helper 2026-08-25 00:57:48 -04:00
smoke-launcher.mjs refactor: self-validating clients, fast validate + separate coverage/smoke (#1504) (#1505) 2026-06-21 11:41:33 -04:00
smoke-tui.mjs fix(scripts): make the test stubs drain, and the raw-mode guard testable (#2147) 2026-08-26 22:13:35 -04:00
smoke-web-app.mjs chore(scripts): rename the local pre-push gate off ci, and guard it out of CI (#2146) 2026-08-26 23:56:34 -04:00
smoke-web-browser.mjs chore(scripts): rename the local pre-push gate off ci, and guard it out of CI (#2146) 2026-08-26 23:56:34 -04:00
smoke-web-elicitation.mjs Merge remote-tracking branch 'origin/v2/main' into v2/chore/2146-rename-local-gate 2026-08-27 00:16:14 -04:00
smoke-web-tabs.mjs Merge remote-tracking branch 'origin/v2/main' into v2/chore/2146-rename-local-gate 2026-08-27 00:16:14 -04:00
smoke-web.mjs fix(smoke): await the server's exit before removing its catalog, and test it 2026-08-11 14:32:20 -04:00
verify-build-gate.mjs chore(scripts): rename the local pre-push gate off ci, and guard it out of CI (#2146) 2026-08-26 23:56:34 -04:00
verify-bundle-externals.mjs test(proxy): isolate proxy env per file; word the guard failure by candidate source 2026-08-23 23:00:28 -04:00
verify-bundle-externals.test.mjs test(proxy): isolate proxy env per file; word the guard failure by candidate source 2026-08-23 23:00:28 -04:00
verify-dep-lockstep.main.test.mjs chore: address Copilot review round 1 (#2226) 2026-09-05 16:09:44 -04:00
verify-dep-lockstep.mjs chore: address Copilot review round 3 (#2226) 2026-09-05 17:14:44 -04:00
verify-dep-lockstep.test.mjs chore: address Copilot review round 1 (#2226) 2026-09-05 16:09:44 -04:00
verify-format-coverage.main.test.mjs chore: address Copilot review round 20 2026-08-29 05:32:12 -04:00
verify-format-coverage.mjs chore: address Copilot review round 9 2026-08-29 03:04:03 -04:00
verify-skills-cli.mjs chore: address Copilot review round 17 2026-08-29 04:54:04 -04:00
verify-skills-cli.test.mjs chore: address Copilot review round 16 2026-08-29 04:40:48 -04:00
verify-skills.main.test.mjs feat(skills): measure a skill reached from another skill (#2204) 2026-09-04 19:07:26 -04:00
verify-skills.mjs feat(skills): measure a skill reached from another skill (#2204) 2026-09-04 19:07:26 -04:00
verify-typecheck-coverage.main.test.mjs Merge origin/v2/main: move the #1939 npx fix into the shared tsc-program lib 2026-08-16 17:07:50 -07:00
verify-typecheck-coverage.mjs Merge origin/v2/main: move the #1939 npx fix into the shared tsc-program lib 2026-08-16 17:07:50 -07:00
verify-typecheck-coverage.test.mjs chore(scripts): derive dep-lockstep candidates from what enters each tsc program 2026-08-16 12:17:15 -04:00