`ServerImportJsonModal` > "guards against a live edit made before the
debounce re-validates" opened its window on the real clock: it pasted
invalid JSON and clicked Add Server, relying on less than
VALIDATE_DEBOUNCE_MS (300ms) of wall time elapsing in between. When more
did, the debounce landed first, `canAdd` went false, and the click hit a
disabled button — a no-op that sets no submit error, so the
`findByText(/Fix the validation errors/)` timed out. The first assertion
(`onAddServer` not called) still passed, which is why the failure read as
mysterious rather than as a disabled button.
Run the test on fake timers end to end instead. The first validation is
landed explicitly with `advanceTimersByTimeAsync(VALIDATE_DEBOUNCE_MS)`
— imported from the hook, so the test cannot drift from the value it
depends on — and after the second paste the timers are simply not
advanced. The pending re-validation therefore cannot land at all, the
window stays open by construction, and the final assertion becomes a
synchronous `getByText`.
Verified with a throwaway probe holding both shapes side by side with a
deterministic 400ms stall injected between the paste and the click: the
old shape fails exactly as reported, the new one passes.
No timeout was widened and no production code changed.
Closes#2250
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQgwZ1kGzhkdkMJ81JVg42
Signed-off-by: cliffhall <cliff@futurescale.com>
Re-measured the two committed chain cases under the stricter turn-boundary
scoring from 313a5809: 100% and 33% at RUNS=3, against 33% and 33% before.
That rise is NOT an effect of the change and the doc now says so. The turn
rule only ever rejects matches a flatter reading accepted, so it cannot
raise a score — the two runs simply straddle a 67-point swing on the same
prompt at RUNS=3, where one sample is worth 33 points. Reading it as an
improvement would be exactly the mistake the eval exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
One finding, and a sharp one: position in the stream is not causation.
`collectSkillInvocations` flattened every `tool_use` block across every
assistant event into one ordered array, and `chainHit` read position
alone. But the model can emit several `Skill` blocks in a SINGLE assistant
message, and it has not seen the first skill's body when it does — those
are parallel guesses, not a hand-off. A run that emitted `testing` and
`test-servers` together in one message scored as `A` led to `B`, and would
have gone on scoring after the pointer in `testing`'s body was deleted:
exactly the false pass this feature exists to rule out, arriving by a
second route (Copilot).
Each recorded invocation now carries the assistant event it came from —
`{payload, turn}` — and `chainHit` requires every link after the first to
land in a STRICTLY later turn. The turn count is threaded through
`collectSkillInvocations`'s `turnOffset`/`nextTurn` so a stream read in
chunks stays monotonic rather than restarting per read; without that, a
chunk boundary would reset the numbering and reintroduce the same bug on
long runs.
The scan stays greedy, which is still correct under the constraint: taking
the earliest occurrence of a link can only leave more room for the rest,
so no later starting point succeeds where the greedy one fails.
`sampleHit` is deliberately unaffected — a first-move case asks only
whether a skill fired at all, so it reads names and ignores turns.
Four new assertions in one case: a same-turn pair scores false; the same
two skills a turn apart score true; a same-turn pair does not poison a
later genuine hand-off; and only the chain's first link is unconstrained.
Plus a `collectSkillInvocations` case pinning the turn tagging and the
cross-chunk offset.
`docs/skill-authoring.md` now lists this as the second way a chained case
can false-pass, alongside a prompt carrying the target's own trigger.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
Closes#2244Closes#2225
All three outstanding advisories were stale lockfile resolutions, not
upward-blocked pins: every fixed version already sits inside the range its
declaring parent asks for, so refreshing the lock entry is the whole fix.
fast-uri 3.1.5 -> 3.1.7 ajv@8.18.0 asks for ^3.0.1 (root)
qs 6.15.3 -> 6.16.0 express@5.2.1 asks for ^6.14.0 (root)
browserslist 4.28.2 -> 4.28.9 via @babel/core (clients/tui)
No `overrides` entry is added, and no manifest changes. AGENTS.md's rule is
that a transitive is pinned with `overrides` rather than with `npm audit fix`
-- it does not call for a pin where none is needed. A permanent pin here would
buy nothing and would later hold a package back: `fast-uri: ^3.1.6` forbids
fast-uri 4.x for as long as it stands, including after ajv moves to it. The
monthly refresh sweep and the daily alert sweep are what catch a regression.
`npm audit` is clean in all five installs (root, web, cli, tui, launcher).
Reachability, assessed rather than assumed:
- fast-uri is the one that ships. ajv is a root runtime dependency, and Vite
pre-bundles it into the published `clients/web/dist`, so the vulnerable code
was inlined into the SPA rather than merely resolved at install time. Its
input is attacker-influenced: `schemaUtils.ts` compiles the `outputSchema`
a server under test supplies, and ajv resolves that schema's `$id`/`$ref`
through fast-uri (`ajv/dist/runtime/uri.js`). Impact is bounded well below
the advisory headlines, though: both SSRF advisories need a consumer that
fetches the parsed URI, and ajv never performs a network request. The
realistic worst case is the host-confusion pair mis-normalizing a crafted
`$id`, giving a wrong or failed validation of one tool's output.
- qs is installed in production, but nothing shipped runs it. express is a root
devDependency, but that is not its only path: `npm ls express --omit=dev`
shows it reaching a production install through
@modelcontextprotocol/server-legacy@2.0.0 (a root runtime dependency, also
via express-rate-limit) and through @modelcontextprotocol/ext-apps ->
@modelcontextprotocol/sdk@1.30.0. So qs is present in every user install.
What holds is that no shipped module ever instantiates it: nothing in
`core/`, `clients/*/src` or `clients/web/server` calls `express()`, the web
backend is Hono, and server-legacy is imported only from `test-servers/src`,
which the root `files` list does not publish. Both advisories need express to
parse an attacker-supplied query string, which requires a live express app.
- browserslist is dev tooling. It arrives in the clients/tui install through
eslint-plugin-react-hooks -> @babel/core, is reached only by lint, and is in
no published bundle. The root install already resolved a patched 4.28.8 from
the same plugin at the same version, which is what identified tui's copy as a
stale lock rather than a constrained one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YahVxMTGpigLbZBh1JGPDr
Signed-off-by: cliffhall <cliff@futurescale.com>
Two findings, both correct, and the first one means round 1's containment
fix was only half a fix.
**`--allowedTools` is not the availability filter.** It grants permission;
it does not restrict which tools exist. So a tool a user's or a plugin's
settings already permit stayed reachable across all 14 turns, and the
bound still rested on the deny list being complete — the exact property
round 1 set out to stop depending on. `--tools` is the real restriction
(it selects from the built-in set), and both flags now carry the same
four names: `--tools` to bound, `--allowedTools` so those four never need
a permission prompt no headless run can answer. Verified against the real
CLI — the full flag set runs to a clean `end_turn`. The test asserts both
flags, and both probe snippets in `docs/skill-authoring.md` carry
`--tools` with a ⚠️ saying which one does the bounding.
**The threshold guard let `NaN` and negatives through.**
`CHAIN_THRESHOLD=abc` is `NaN`, which fails every comparison and prints an
`above NaN%` summary; a negative bar passes every chain unconditionally.
Both turn an advertised env knob into a measurement that quietly means
nothing. `CHAIN_THRESHOLD` must now be in `[0, 1)` — half-open because the
chain bar is strict — and `THRESHOLD` in `[0, 1]`, inclusive because a
first-move floor of 1.0 is meetable. The message names the offending
value. A new case spawns the script for each bad value rather than
asserting on the predicate, since these guards live in `main`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
Five findings, all real.
**The chain prompts carried the target skill's own trigger.** `test-servers`
claims "a change needs a real server to exercise it", and both cases said
"against a real/live server" — so the model could load `testing`, then pick
`test-servers` from the ORIGINAL prompt, in that order, and score a hit that
would have survived deleting the pointer from `testing` entirely. Rewritten to
carry no server cue ("end to end"), and the measured rate fell from 100% / 67%
to 33% / 33%. That is the size of the artefact, and 33% is what the pointer
alone is worth. Both cases are now red against the 0.5 bar and stay that way:
`skills:eval` is not a gate, the number IS the finding, and lowering the bar to
turn the column green would discard the only signal this feature adds. Filed as
#2247 with the trap written down so a fix cannot re-measure the artefact.
**A deny list cannot bound a 14-turn run.** It only names tools known when it
was written, and this checkout configures an HTTP `mcp-docs` server in
`.mcp.json` while a contributor's own MCP servers and plugins add more. Now
`--allowedTools Read,Glob,Grep,Skill` (what the harness actually needs) plus
`--strict-mcp-config` with no `--mcp-config`, which drops every configured
server. The deny list stays alongside, since a deny is unconditional while an
allow list governs pre-approval. What remains outside all three is a
contributor's plugin tools; `--bare` would remove those and skills with them,
which would measure nothing — so that residual is stated rather than papered
over.
**The chain threshold compared inclusively.** "More often than not" is `> 0.5`,
and `>=` passes exactly half whenever RUNS is even — 2/4 would report a result
the criterion does not license. `passesThreshold` is now strict for chains and a
floor for first moves, and a strict bound of 1.0 is rejected up front rather
than failing every case while looking like a trigger problem.
**The doc's probe command still denied only four tools**, contradicting the
section's own promise that a probe and a scored case see the same policy.
Updated, with a note that it must move with the harness, plus a hand-off probe
form.
**The reporting had no automated coverage** — the one thing that could silently
merge the two measurements lived inside `main`. Extracted as `formatReport` and
tested: both headings, per-group thresholds, the two separate summaries, the
combined exit, and both single-kind selections.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
#2222 reported that `core/auth/revocation.ts` breaks a client secret
containing `+`, on the basis that `encodeURIComponent` leaves `+` bare.
It does not — it escapes it as `%2B`. The characters it leaves bare are
exactly `!'()*-._~` and alphanumerics, every one of which a form-
urldecoder passes through unchanged, so its output decodes identically
under both algorithms; verified over every code point up to U+2FFF.
Switching to `URLSearchParams` would be the literal algorithm §2.3.1
names and a small regression: it encodes a space as `+`, which a lenient
server decoding with `decodeURIComponent` alone reads as a literal `+`,
while `%20` is understood by both. The encoder is therefore unchanged
and the reasoning is recorded on it.
The report's second half stands and is the real defect: the fixture
decoded with `decodeURIComponent`, the encoder's own inverse, so the
round trip succeeded for every input by construction and no test could
have failed on an encoding mistake. `/oauth/revoke` now runs a genuine
RFC 6749 Appendix B decode, with unit cases for `+`, a space, `:` and
`%` (asserted on the wire and through an independent decoder), an
end-to-end client whose secret is base64 with a `+` in it, and a guard
that an unencoded credential is refused — the case that separates the
two decoders.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BhVaufrEZAfFAXcg6456hK
Signed-off-by: cliffhall <cliff@futurescale.com>
`skills:eval` ran every case with `--max-turns 1`, so it measured one
thing: whether a skill is the model's FIRST tool call. That is right for a
skill a user reaches directly, and it leaves a whole class unmeasured — a
skill reached from inside another skill's body. `testing` tells the model
that picking a fixture is `/test-servers` and that it has to load it, and
`test-servers` scores 5/5 while every one of those five cases asks for it
by name. A skill only ever reachable through a hand-off would score a
clean 100% with the hand-off silently never firing.
An eval case is now one of two shapes, and exactly one:
{ "prompt": "…", "expect": "test-servers" } first move, 1 turn
{ "prompt": "…", "chain": ["testing", "test-servers"] } hand-off, 14 turns
A chain must END with the skill whose file it lives in. The case exists to
measure whether THIS skill is reachable, so the file that goes red is the
one belonging to the skill that stopped being reached; anchoring on the
first link would file a `test-servers` measurement under `testing`, where
a `test-servers` description edit would never be seen. Links are checked
against the repo's model-invoked set, which is why both `verify:skills`
and `collectCases` now validate in a second pass — `test-servers` sorts
before `testing`, so a per-directory check would reject a live chain as
unknown purely because of where the alphabet put it.
Scoring is an ordered SUBSEQUENCE, not a prefix and not a contiguous run:
the model may load something before the chain starts and something
unrelated in between, and neither changes the claim that A led to B.
`collectSkillInvocations` therefore returns an ordered array rather than a
Set — a B, A, B run has to stay distinguishable from one that never
reached B from A.
The two rates are reported in separate columns and never summed. A
hand-off is a second-hop load over many turns; a first-move rate is the
model's opening move. `CHAIN_MAX_TURNS` (14) and `CHAIN_THRESHOLD` are
their own knobs for the same reason, and `CHAIN_THRESHOLD` defaults to
**0.5** rather than inheriting 0.8: see the measurement below.
Read-only containment no longer leans on `--max-turns 1`, which was doing
much of it by itself. The deny list gains the agentic and network tools —
`Task` in particular, whose subagent the flag does not reach.
Measured, `skills:eval -- test-servers`:
First move (1 turn) 7/7 at 100%, unchanged
Hand-off (14 turns) 67% and 33% at RUNS=3
So the hand-off is real and unreliable — which is the fact nothing could
observe before this change, and the reason 0.5 rather than 0.8 is the
default bar: at 0.8 both committed cases are red no matter how strongly
the first skill points at the second, and the column stops carrying
signal. A `["pr-flow", "test-servers"]` probe measured 0% and was dropped
rather than kept: `pr-flow` says nothing about fixtures, so the case had
no lever short of broadening a description onto another skill's ground.
Both findings are written up in `docs/skill-authoring.md`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
Record each advisory's disposition during the loop instead of deriving
"can this be filed?" from the groups beforehand.
Membership in a group only means an alert exists. Whether an issue
tracks it is decided by the ecosystem check, the manifest read and the
range probe that follow — so if GitHub revised first_patched_version
while that manifest happened to be unparseable, the replacement group
was skipped and the old-key issue was still cleared as "superseded",
claiming an issue that was never filed.
Each advisory is now noted as tracked, not-exposed or indeterminate as
the loop reaches it, and reconciliation refuses to clear while anything
still open is indeterminate or absent. That also splits the clear reason
honestly: a bump whose copies simply left range now says so, rather than
claiming a replacement issue exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
- Reconcile on (package, manifest, GHSA) tuples, not GHSAs alone.
Dependabot alerts are per manifest and this repo has five lockfiles,
so the same advisory legitimately covers the root install and a
client. Keyed on the GHSA alone, another manifest's still-filable
alert could vouch for this one and clear an issue whose own alert is
open with no bump available.
- Rename the cleared-date test: it asserts the NEW date is taken when
the reason changes, which is the opposite of what its name said.
- AGENTS.md no longer implies both sweeps select a dated milestone.
Only the security sweep filters undated buckets; the monthly one's
selection is raised on #2239.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Three findings, all of them the same invariant reached by new routes: a
still-open advisory must never stand itself down, and a cleared issue
must be written once.
- Never clear an advisory that is open but unpatched. groupAlerts drops
an alert with no first_patched_version, so it has no replacement
group and no replacement issue — calling it "superseded" was false
and clearing it removed the only thing tracking a live exposure. Such
an issue is now left exactly as it is, and said so in the log.
- Stop re-editing cleared issues daily. A cleared issue stays open, so
the next run regenerated its body with a new date and edited it
again, forever. The date is now read back off the existing body, so
only a real change writes — and a real change takes today's date.
- Distinguish an absent manifest from an unparseable one. Both returned
null, so a malformed lockfile was treated as "the manifest is gone"
and cleared the issue. A read error is evidence of nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Build the open-GHSA set from the raw alert feed, not from the grouped
alerts. groupAlerts deliberately drops an alert with no
first_patched_version, so an advisory that stays OPEN but loses its
patched version vanished from both the keys and that set — and
reconciliation would have called it "fixed or dismissed".
Same wrong direction as the superseded case fixed last round, reached a
different way: a still-open advisory must never stand itself down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
- Filter to the npm ecosystem. Dependabot alerts are not npm-only and
this repo has a Dockerfile, so an alert against it would have been
parsed as a lockfile, thrown, and aborted the whole daily sweep
before any npm group ran. A non-npm alert is now reported loudly with
its GHSAs so a human can file it, and readManifest no longer lets one
unparseable manifest take the run down.
- Distinguish a superseded bump from a closed one. GitHub can revise
first_patched_version, moving an advisory to a different key while it
stays open; reporting that as "fixed or dismissed" would stand down a
live exposure. The reason now reads from the open GHSA set.
- Only claim security-update PRs are off when the run actually read the
setting. The guard degrades to UNVERIFIED, and an issue asserting
what the run could not confirm is worse than one that stays quiet.
- README.md and AGENTS.md: scripts/ now holds repo automation run from
CI, not only build/verify tooling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Three reviews I missed: my poll called the reviews endpoint without
--paginate, so it only ever read page one — the same pagination bug
this PR was reviewed for.
- Reconcile marked issues whose bump has left the open alert feed
entirely. openAlerts asks for state=open, so a fixed or dismissed
alert simply vanishes: its group is never built, the loop never
visits it, and its issue kept asserting a vulnerability with a live
Todo/High card. Both cases are covered — the zero-alert run, which
used to return before loading issues at all, and a vanished group
while other groups remain.
- Ask for a PARENT-SCOPED overrides entry when the package is also
declared directly. npm rejects an override contradicting a direct
dependency with EOVERRIDE, so the guidance for the mixed case would
not have applied. The issue now prints the exact nested JSON.
- Correct the body's NOTE: severity, range, GHSA and CVE are the
advisory's own. Only the installed versions, their paths and range
applicability are verified against v2/main.
- Stop the new-advisory comment claiming the body marker "now covers"
the advisory; it is posted before the edit, so that is false at
posting time. It speaks for its own marker instead.
- Cover the successful two-field board placement, not only its failure
modes.
- AGENTS.md: the monthly sweep NEVER boards; only the security sweep
does, and only with a PAT. The old wording promised a card the
monthly job cannot create.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Both findings are consequences of round 6's staleness fix, in the two
places it did not reach.
- Resolve the matching open issue BEFORE the skip paths, and rewrite it
to a cleared state when the exposure is gone. Yesterday's issue is
still open today, so a manifest that has since been removed or copies
that have moved out of range left its body asserting a vulnerability
that no longer exists and its Todo/High card live indefinitely. The
marker is retained so the issue is reused if the advisory returns; the
issue is not auto-closed, because whether the card belongs in Done or
should be deleted depends on why the exposure went away.
- The body's prose counted the marker's monotonic history rather than
the advisories that apply, so it could claim two open alerts while the
title and table correctly showed one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Both were suppressed "previously missed" findings, and both were real.
- isPermissionDenied matched on status alone, so a rate limit (also a
403) and a bad token (401) were waved through as "missing scope" —
contradicting the comment right above it, which said those must stop
the sweep. It now excludes rate-limit wording and drops 401.
- The no-op path asked "were advisories ADDED?" when the question is
"did the issue CHANGE?". An issue filed for A+B whose branch moved so
only B applies has nothing added, yet its table, severity, affected
copies and remediation are all stale. The rendered title and body are
now compared against the issue, and a comment stays reserved for
genuinely new advisories.
- The title counts the advisories that APPLY, matching the body, so it
tracks shrinking exposure as well as growth. The marker's GHSA list
stays monotonic — its job is to remember what has been announced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Both findings were "previously missed" suppressed ones; the two against
scripts/dependency-refresh.mjs and its workflow are relayed to #2239.
- Serialize the sweep with a fixed concurrency group and
cancel-in-progress: false. The marker check is a read-before-write and
a workflow_dispatch can land on top of the scheduled run, so two runs
could both see no open issue and both file one — the duplicate the
whole idempotency design exists to prevent. The queued run must wait
and re-read, never be cancelled.
- Pick the milestone in JS, not in jq. jq sorts null before every
string, so sort_by(.due_on) | .[0] returns an UNDATED open milestone
in preference to every dated one. An undated bucket has no due date
and so cannot be the nearest; pickMilestone drops it, and files the
issue unmilestoned if nothing dated is open.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
- Narrow each group to the advisories the installed copies are actually
in range of. Two advisories can share a package, manifest and patched
version while having different vulnerable ranges, so a group-level
"does any match?" left the marker, title, severity and table all
claiming an advisory that does not apply on this branch.
- Escape every free-form Markdown cell, the vulnerable range included.
A semver range may contain `||`, which is also the column separator.
- Paginate the open-issue lookup instead of capping it at 100. The
marker lookup is what makes the sweep idempotent, so a truncated list
would file a duplicate for every issue it could not see — at the same
scale the alert fetch is built to handle. Pull requests, which the
issues endpoint also returns, are dropped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Derive the remediation from WHICH copies are vulnerable, not from
whether the package is declared. A manifest can declare a safe `pkg@4`
while a dependency drags a vulnerable `pkg@3` into a nested folder; the
old boolean called that "direct" and asked for a range bump that would
have changed nothing, omitting the override the nested copy needs.
- `lockfileEntries` keeps each copy's tree path and whether it is the
hoisted one; `lockfileVersions` is now derived from it.
- `remediation(affected, declared)` returns both flags, so the issue can
ask for a range bump, an overrides pin, or explicitly both. The body
lists the vulnerable copies and their paths.
- An undeclared hoisted copy counts as transitive: it got there the same
way any other transitive copy did, and no declared range reaches it.
(Found by a test written for this change.)
The `vulnerability-alerts: read` finding is declined — verified against
a real runner, which reports `VulnerabilityAlerts: read` and reads the
alerts successfully. See the PR comment for the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
- Remove the literal NUL bytes that were separating the grouping key's
fields; they classified the whole source file as binary, so repo
searches skipped it. The key is a JSON array now, with no separator
left to justify.
- De-duplicate advisory comments per GHSA rather than per whole set. A
run that comments and then fails before the marker edit left the next
run computing a different set, which matched nothing and announced the
same advisory twice.
- Refresh the issue title on edit; it carries the advisory count, so it
went stale as soon as an issue grew past what it was filed with.
- Correct the test file's header, which still claimed main() was left
to workflow_dispatch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
- `--slurp` the paginated alert feed; a bare `--paginate` emits one JSON
array per page and `JSON.parse` rejects it past 100 open alerts.
- Put `fixedIn` in the marker and in the existing-issue lookup, so a
second bump of one package cannot merge into the first one's issue.
- Only an authorization-shaped failure of the `automated-security-fixes`
read becomes UNVERIFIED; a rate limit or 5xx now throws.
- Distinguish "card never added" (benign, triage picks it up) from
"card added, field not set" — the latter finishes every group, then
fails the run.
- Don't board an unmilestoned issue at Todo; `Incoming` <=> no milestone.
- Comment before rewriting the marker, and give the comment its own
marker, so a failed comment cannot be skipped forever.
- Ask a direct dependency's range to be raised, not widened to `>=`.
- Test `main()` through an injected spawn, as the sibling sweep does.
- Correct the docs that named the removed release-time `npm audit fix`,
the ones promising an unconditional guard, and the sibling sweep's
issue body claiming security updates remain enabled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Closes#2233 — the alert-consuming half of #2229. #2235 removed
`.github/dependabot.yml`, ending Dependabot's version-update PRs; this
ends its security-update PRs and replaces them with a daily sweep that
turns the alerts into ordinary board items.
- `scripts/dependabot-alerts.mjs` groups alerts by
`(package, manifest_path, first_patched_version)` — one issue per
BUMP, not per advisory — re-checks each vulnerable range against
`v2/main`'s own lockfile before filing, and keys idempotency on a
marker comment naming every GHSA the issue covers.
- `.github/workflows/dependabot-alerts.yml` runs it daily and on
`workflow_dispatch`, with `vulnerability-alerts: read`.
- `semver` is declared at the repo root, per Dependency placement:
`scripts/` is root-owned code with no manifest of its own.
- The sibling `dependency-refresh` comments no longer claim security
updates are unaffected, and AGENTS.md gains the flow both halves
now follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Addresses Copilot review round 7 on #2239.
The lookup read one page of 100 and the doc comment justified it with an
assumption rather than a guarantee: "the greatest version is within it
for any real action". The list is ordered by release DATE, not version,
so an action that keeps cutting maintenance releases on lower majors
pushes the genuine maximum off page 1. Ranking what remained would
report a lower major as highest — the exact false-current result that
dropping `releases/latest` in round 3 was meant to prevent, arrived at
by a different route.
Now passes `--paginate`. Cost is unchanged for every action this repo
uses (57 and 64 releases are the largest, so still one request each);
it only costs more where correctness actually required it.
The command-shape test now requires `--paginate` as well as the list
endpoint. It has to: pagination is invisible in the result — one page
and every page return an identical-looking tag list until the day they
do not — so nothing about the returned value can detect its absence.
Verified by mutation: removing `--paginate` fails exactly one test.
Adding the flag also moved the URL out of args[1], which the test's own
filter had hardcoded — it caught that itself. Both the filter and
fakeSpawn's matcher are now argument-position agnostic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGtzPg3UxMLszysQXXfqax
Signed-off-by: cliffhall <cliff@futurescale.com>
Addresses Copilot review round 5 on #2239.
Round 4 made every non-zero release-list response fatal and deleted
`isMissingRelease`, but `latestReleaseTag`'s JSDoc still promised
`@throws when the lookup fails for any reason other than a 404` — the
exact behavior that change removed. The PR description carried the same
stale claim.
A contract that documents the opposite of the code is worse than none:
the next reader reasonably trusts it, and here it would tell them a
missing action repo is silently tolerated when it now fails the sweep.
The contract now reserves `null` for a successful response carrying no
usable release — an action that has never cut one, or whose tags are all
unparseable — and states that a failed lookup is never `null`.
Documentation only; no behavior change, and the 29 tests are untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGtzPg3UxMLszysQXXfqax
Signed-off-by: cliffhall <cliff@futurescale.com>
Addresses Copilot review round 4 on #2239. Both findings are regressions
I introduced in round 3 and both are correct.
Round 3 moved the lookup from `releases/latest` to the release LIST, to
rank by version rather than by GitHub's designated latest — but kept the
404 suppression that only made sense for the old endpoint. The list
endpoint answers "this action cuts no releases" with a successful empty
array, so a 404 there does not mean that at all: it means the repository
is missing or inaccessible, i.e. a `uses:` ref the sweep cannot check.
Converting it to null silently dropped a broken or renamed action from
the flow that replaced Dependabot — the same silent-success shape this
PR has been closing everywhere else, reintroduced one layer down.
Every non-zero status is now fatal and `isMissingRelease` is gone
entirely; the benign no-releases case is the successful empty array,
covered by its own test.
`buildClearedBody` also overstated its all-clear. Refs pinned to a commit
SHA or a branch return null from `parseVersionRef` and are never ranked,
so "no workflow `uses:` ref is behind" asserted a check that had not
happened for them. It now says version-pinned refs and names the
exclusion explicitly.
Both guards are mutation-checked: restoring the 404 suppression, and
dropping the SHA/branch qualification, each fail exactly one test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGtzPg3UxMLszysQXXfqax
Signed-off-by: cliffhall <cliff@futurescale.com>
Closes#2235.
Switches the old dependency flow off now that #2232 landed the
replacement. Removes the five npm ecosystem entries (root plus each
client under clients/*) and, resolving the question #2229 left open, the
github-actions entry as well — so .github/dependabot.yml goes away
outright rather than being emptied, which its schema does not allow.
The github-actions entry had exactly the property #2229 exists to
remove: it opened a grouped monthly PR carrying no `Closes #N` and no
board card, the one standing exception to "every PR references an
issue". Deleting it unreplaced would have left 9 actions unwatched, and
`npm outdated` says nothing about actions, so the monthly sweep now also
checks every `uses:` ref under .github/workflows and renders the stale
ones as one more section of the same tracking issue.
Ranking comes from the release LIST, not `releases/latest`. That
endpoint returns the release GitHub designates most recent, not the
greatest version, so an action publishing a maintenance release for an
older major (a v6.9.1 cut after v8.0.0) would make a workflow pinned to
v7 compare against v6 and read as current — silently missing a whole
major upgrade, the one thing this check exists to catch.
Staleness is compared only to the precision the ref specifies. `v7` is a
moving major tag that GitHub repoints at every v7.x release, so `v7`
against a highest of `v7.0.1` is current and only `v8` makes it stale;
an exactly-pinned `v7.0.0` is behind `v7.0.1`; a SHA pin is deliberately
immovable and is never reported.
A release lookup suppresses only a 404 — the legitimate "this action has
never cut a release" answer — and throws on anything else. Treating a
rate limit or an expired token as "no release" is indistinguishable from
"not stale", and since every action here already sits on its latest
major, the resulting empty section is byte-identical to a healthy run.
`buildClearedBody` now speaks for both halves: once actions are in scope,
its npm-only wording would assert a clean bill of health the sweep never
checked.
Dependabot security updates are unaffected — they are configured in repo
settings, not in this file, and kept working while it was missing
entirely (#1833, #1840). That note moves into the workflow header rather
than dying with the file; #2233 is where they are turned off
deliberately. The script header, the workflow header and the generated
issue body all say version-update PRs rather than claiming Dependabot is
replaced wholesale.
Also renames the workflow's npm-outdated job to dependency-sweep now
that the sweep is no longer npm-only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGtzPg3UxMLszysQXXfqax
Signed-off-by: cliffhall <cliff@futurescale.com>
`main refuses to run without a repo` called `main(undefined, …)` so the default
parameter would fire, then asserted it throws. The default reads
`process.env.GITHUB_REPOSITORY` — which GitHub Actions sets on every run — so
the assertion held only where the variable happened to be unset. It passed
locally and failed CI, the one environment where the default is always
populated.
The test now clears the variable and restores it, and a second test covers the
other half of the default: with it set, `main(undefined, …)` uses it instead of
throwing. Between them the default's behavior is pinned in both environments
rather than inherited from whichever one is running.
Verified both ways: 14 pass with the variable unset and with it set, and the
full `test:scripts` suite is 411/411 under the CI environment shape.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWMJENjs1mExXE5XPJTEei
Signed-off-by: cliffhall <cliff@futurescale.com>
Five findings, all valid.
`runOutdated` accepted every exit status and returned `result.stdout ?? ""`.
A failed `npm outdated` (exit 2 on a registry or config error) also prints
nothing, so the empty string parsed to an empty package list — and across five
installs a total outage produced output byte-identical to a clean sweep. It now
throws on any status other than the documented 0/1.
`main()` was untested, which is how that reached review: the helper-only suite
passed while the silent-success path was live. It now takes its spawn function
as a parameter and is driven by a fake, covering npm failure, create vs. edit,
milestone handling, both no-op paths and the missing-repo guard.
The no-op path returned before looking for the marker issue, so once every
install caught up, a still-open tracking issue kept its obsolete package table
indefinitely — contradicting the body's own promise to update in place. It now
rewrites through `buildClearedBody()`. It deliberately does not close the
issue: the sweep takes no board actions, and closing one whose card a
maintainer may have moved would make the board assert work shipped that this
script cannot verify shipped.
The release step mandated `npm audit fix`, which AGENTS.md forbids. The reason
is not `--force`: plain `audit fix` resolves an advisory with no upward escape
in range by silently downgrading, as it did to esbuild across three installs in
#2058, and `local:gate` has no version-regression check to catch it. The step
is now `npm audit` report-only, with fixes applied deliberately via a direct
bump or an `overrides` entry.
An unmilestoned issue is swept into Incoming, not Todo — Todo asserts a
maintainer signed off. Message corrected and pinned by a test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWMJENjs1mExXE5XPJTEei
Signed-off-by: cliffhall <cliff@futurescale.com>
The npm-entry removal in `.github/dependabot.yml` is split out of this PR so
that every change to that one file lands in a single reviewable piece —
alongside the `github-actions` entry, which #2229 had declared out of scope but
which opens the same issue-less, board-cardless PR the decision exists to
remove.
Reverts `.github/dependabot.yml` to its `v2/main` state, leaving this PR with
the release-time `npm audit fix` and the monthly `npm outdated` sweep: the
replacement flow, not the switch-off.
Both new files' header comments claimed the switch-off had already happened,
which is false until #2235 lands. They now say #2235 does it, and that until
then Dependabot's npm PRs and this sweep overlap — duplicate signal rather than
conflicting action.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWMJENjs1mExXE5XPJTEei
Signed-off-by: cliffhall <cliff@futurescale.com>
Adds npm audit fix (root + every client) as the first sub-step of the
release skill's bump step, before npm version, so a release is never
gated on remembering to check separately. Never --force; anything
audit fix can't resolve is left to the dependabot-alert pipeline
(#2229) or a follow-up issue.
Disables Dependabot npm version updates in dependabot.yml (a
version-update PR carries no issue and no board card) and replaces
them with a monthly scheduled sweep that runs npm outdated across the
root install and every client, filing or updating one idempotent
tracking issue instead of an auto-generated PR.
Closes#2231
Part of #2229
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017N9ha27Qg6LHpBQP7G7zPY
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot: the catalog entry is mutable and the live client is not. A card
can be edited while connected — `onConfigSubmit` writes the catalog and
does not rebuild the client — so after A connects to X and is edited to
Y, the entry reads Y while the session is still authorized against X.
Both consumers read the entry, so both got it wrong in the same way:
- the hook compared a cleared entry against A's *entry* URL, so clearing
another entry still at X missed the match and deleted and revoked the
live client's X-keyed credentials without disconnecting it — the exact
failure this PR exists to prevent;
- `runClear` locked on the entry's key, so a clear of edited-A (`url:Y`)
and a clear of entry B at X (`url:X`) took different locks while both
performed the same X-keyed operation through the same live client.
`InspectorClient.getTransportConfig()` returns the config it was
constructed with, which is what its credentials are keyed under, so both
now resolve from that and fall back to the catalog entry only when there
is no client.
The rule moves into one exported function, `resolveOAuthClearIdentity`.
Two copies of it that disagree is precisely the class of bug #2217 is,
and the two consumers each re-deriving it is how this round's finding
came to exist at all.
Both fakes gain a `getTransportConfig`, and three tests cover the split:
the resolver's own two (the client's URL wins; an active clear locks on
the client's key) and a hook case asserting an edited active entry still
disconnects when the entry sharing the client's real URL is cleared.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGbw6TezNWnXZSCSp7sqb4
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, and the repo's own rule against an unjustified `as unknown as`.
The `useServers` cast was the more useful catch: removing it surfaced
that the mock omitted `reorderServers` and `importSource`, so App code
calling either would have read `undefined` with this test still passing
type-check. The full result shape is supplied instead, matching the
`mockServersWith` helper above.
The client cast becomes an intersection — the instances are already typed
`EventTarget`, so naming the test-only spy on top of that expresses what
is needed without erasing the rest.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGbw6TezNWnXZSCSp7sqb4
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 1.
Refreshes are concurrent — an `oauthComplete` can start a second read while
the first is still in flight — and nothing makes them settle in order. The
new catch made that visible: a slow earlier read could reject *after* a
newer one succeeded and clear the fresh result. Guard both handlers with a
monotonically increasing sequence so only the newest read writes.
The two tests were also too weak to detect their own guards, as the review
pointed out. Both started from an undefined panel, so neither could tell a
correct handler from one that merely swallowed the error:
- The rejection test now seeds a successful read first, then rejects an
`oauthComplete` refresh and asserts the already-loaded details clear.
- The post-cleanup test now reconnects with a replacement client that
populates details, then rejects the stale read, and asserts the current
details survive.
- A third test covers the ordering guard directly.
Each of the three guards has exactly one test that fails without it,
verified by mutation: dropping the sequence check, dropping the clear, and
dropping the `cancelled` check each break one test and no others.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBkPzbuxgyqiz39ytmQzMB
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot: `oauthClearKey`'s unit tests prove the derivation, but a
regression that dropped the `has` check, added or deleted the wrong key,
or invoked the clear twice would still have passed.
Adds an App.test.tsx case that connects entry A, holds its clear open on
a deferred `clearOAuthTokens`, then drives a clear for entry B — a
different catalog id against the same URL — and asserts the underlying
clear ran exactly once, and once more after the first settles, so the
key is proven released rather than leaking a permanently dead control.
Verified to fail against the id-keyed guard.
Two supporting changes it needs:
- an `open-settings-b` control on the mocked InspectorView, the only way
to reach a settings modal for a server other than the active one;
- `@inspector/core/mcp/remote/index.js` becomes a partial mock.
`getWebProxiedFetch` reaches for `createRemoteFetch` there, and the
bare stub threw a missing-export error before the clear under test ever
ran — the failure surfaced as the generic "Could not clear the stored
OAuth state" toast.
The case carries an explicit 20s timeout and `delay: null`: it drives
three full modal interaction sequences against the whole App tree, which
runs past the 5s default when the suite is under load.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGbw6TezNWnXZSCSp7sqb4
Signed-off-by: cliffhall <cliff@futurescale.com>
Clearing the retry from the mirror effect raced: an async command continuation
can open a prompt and install its retry before a previous commit's passive
effect flushes, and that effect's stale `pendingStepUp === null` snapshot would
then delete the new prompt's retry.
Install it in `trySetPendingStepUp` instead, in the same synchronous step that
opens the prompt — `null` for an ambient prompt, the operation for a
command-scoped one. Every prompt comes through there and `handleStepUpAuthorize`
is the only reader, so a slot cleared as a side effect can leave a stale closure
behind harmlessly: the next prompt overwrites it. A refused prompt still
installs nothing (#2165).
Copilot round 3 on #2237.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S4WZSxx6XMKzRdgPce9CAd
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot: `runClear`'s double-click guard deduped by `server.id`, which is
the same identity mismatch this PR fixes, one layer up. Two catalog
entries against one URL share one credential blob, one grant and one
revocation, so an id-keyed guard let exactly the race it exists to
prevent through between them — concurrent store writes, concurrent RFC
7009 requests, and two contradictory teardown toasts.
The guard now keys on `oauthClearKey`, a pure helper that resolves the
OAuth storage key and falls back to the entry id for a config that has
none (stdio), where there is no shared state to collide over. The id
fallback keeps such entries distinct from each other rather than
collapsing them onto one key, and both forms are prefixed so a
user-supplied id can never impersonate a URL.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGbw6TezNWnXZSCSp7sqb4
Signed-off-by: cliffhall <cliff@futurescale.com>
