Both findings from the v2.5.0 milestone-merge review (#2215).
1. `progressToastId` collided across distinct progress streams.
`ProgressToken` is `string | number`, so `String(token)` mapped the
numeric token 7 and the string token "7" onto one id — and because
notifications keyed by the same id are *replaced* rather than
stacked, two concurrent streams overwrote each other's toast. The
absent case was worse: it hardcoded the sentinel "default", which a
server is free to send as a genuine string token. The id now carries
the token's type (`progress-n:7` / `progress-s:7`) and gives the
no-token case a prefix of its own (`progress-none`) that no token
value can produce.
2. `newAttemptId`'s fallback now prefers `crypto.getRandomValues`.
`randomUUID` needs a secure context; `getRandomValues` does not and
exists in every browser that has `crypto` at all — so the exact
situation the fallback exists for (a `file://` page, a plain-HTTP
non-loopback host) still has a CSPRNG on hand. `Math.random` stays
as the last resort for a `crypto`-less global, and the "never a
security token" comment stays too. Retires CodeQL alert 72
(`js/insecure-randomness`) honestly rather than by dismissing it.
Tests: collision cases for 7 vs "7" and the absent token vs "default" /
"none", a distinctness sweep over the whole id space; and the attempt-id
fallback test split into a getRandomValues arm (asserting the CSPRNG is
used and `Math.random` is not) and a no-crypto-at-all arm.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uw1s4LRBUzrFwLzPT4mAJn
Signed-off-by: cliffhall <cliff@futurescale.com>
Round 16 reported no inline comments and three suppressed findings, all
valid:
- `getSkillsExtension` treated a non-object extension value (`false`,
`"skills"`) as a declaration. SEP-2133 declares an extension as an
object of sub-options, so a primitive is not one — now rejected,
matching `appElicitation.ts`.
- The `onGetSkill` prop doc claimed the two responses "must agree" and
that a difference means a broken server, contradicting the
implementation, which reports a difference as a finding rather than a
fault. Reworded to what the code does.
- No real-transport test exercised the fixture's `skills/*` handlers.
Added `src/test/integration/mcp/inspectorClient-skills.test.ts`,
parameterized over both protocol eras: capability advertisement, the
paged `skills/list` walk (direct and through `ManagedSkillsState`),
`skills/get`, the `-32602` for an unknown URI, `resources/read` of a
skill file, and delegation of an ordinary resource through the
fixture's `resources/read` wrapper.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Round 15 generated no new inline comments; all five suppressed findings
were real and unaddressed.
- SkillsScreen: claim the attempt BEFORE issuing the request, for both the
SKILL.md preview and `skills/get`. Recording it only on settle left a
window where an older request that happened to resolve first was still
considered current and published while a newer one was in flight —
`verifyRow` already claimed its row up front, these two did not. Tests
answer the OLDER request first and assert nothing is published.
- SkillsScreen: "No skills listed", not "No skills". SEP-2640 lets a server
return an empty or partial catalog and says an empty result is not proof
it has none — an unlisted skill is still fetchable by URI — so the old
copy had the tool asserting something the protocol explicitly does not.
- SkillsScreen: each row's Verify button gets an `aria-label` carrying its
URI. Every row's visible text is "Verify" and the URI cell is not
programmatically associated with the button, so a screen-reader user had
no way to tell which file each control checked.
- inspectorClient: attribute a rejected `skills/get` envelope with
`markResponseRejected`, so the Protocol tab stops rendering it as a clean
success while the screen shows an error. In the client rather than a
store because `skills/get` has none. Gated on `isClientDecodeRejection`,
with a test that a transport failure is NOT attributed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
The review finally quoted the normative text, which is what I asked for
across five rounds before declining: SEP-2640's `skills/list` section says
"In protocol versions 2026-07-28 and later, the result also carries …
`ttlMs` and `cacheScope`." So the era-aware validation is in.
- skillsSchemas: `ModernListSkillsResultSchema` = the page plus the base
list envelope, with field shapes mirroring `ModernResultEnvelopeSchema`
in `listSalvage.ts` so this repo's two statements of a modern envelope
cannot drift. `InspectorClient.listSkills` picks it from the negotiated
era; the legacy shape stays permissive, because those are 2026-era
attributes a legacy server has no business sending.
Nothing else catches this: `skills/*` is consumer-owned, so it is absent
from the SDK's cacheable-method registry and a modern server answering
`{ skills: [] }` was reaching the conformance UI as a clean list.
- skills.ts: count `frontmatter.description` (and `name`) length in Unicode
CODE POINTS. `String.length` counts UTF-16 code units, so 600 non-BMP
characters measured as 1200 and a perfectly valid description was
reported `malformed-description` — a conforming server failed by an
off-by-encoding, which is the direction this module works hardest to
avoid. The Agent Skills reference validator uses Python `len()`.
- #2248 narrowed: the `skills/list` half is settled here; what remains is
whether `skills/get` carries the same attributes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot round 3 on #2256: both detectors were one-sided. Each measured
the sub-floor remainder against the real clock, so a preempted worker
overshoots the deadline, the remainder goes negative, and the unfixed
`remainingMs <= 0` bound takes the same branch and prints the same
message — the test passes on an implementation with no floor. It could
never fail wrongly, but it could silently stop testing anything, which on
a flake fix is the wrong half of that guarantee to keep.
Both now stub `performance.now()`: the web case advances it inside the
fetch so the second grant's remainder is exactly
MIN_REVOCATION_REQUEST_BUDGET_MS - 1, and the CLI case freezes it so the
remainder at the check is the budget itself. Positive and under the floor
on every machine, which is the only state that separates the two bounds.
Verified as detectors with the clock stubbed: reverting the core floor
gives 1 failed / 57 passed, reverting the CLI floor 1 failed / 14 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017wEXtbs8UEHUMxDEAxbs99
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot round 1 on #2256: the `budgetMs: 0` case passes under the old
`remainingMs <= 0` bound too, so nothing in the CLI suite detected the
floor added to `sendPlans`.
The new case gives it a budget that is positive and below the floor, and
asserts on *this loop's* exhaustion message — the one naming the server
URL. That distinction is the test: with the floor removed the plan is
handed a 4ms budget and core's identical floor declines it one level
down, so a bare "budget was exhausted" match passes either way. Verified
as a detector — 1 failed / 14 passed with the CLI floor reverted, 15
passed with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017wEXtbs8UEHUMxDEAxbs99
Signed-off-by: cliffhall <cliff@futurescale.com>
- SkillsScreen: put the SESSION in the invalidation key. This screen stays
mounted across a disconnect, so content alone cannot tell server A's entry
from an identical-looking one on server B — A's in-flight verification
could land afterwards and report `verified` for a file never read from B,
and a retained batch entry could leave B's Verify all disabled.
`useInspectorStores` now exposes a `sessionNonce`, bumped on both create
and destroy so it never repeats across a reconnect, and App keys on
`${activeServerId}:${sessionNonce}` — the server id alone would repeat,
which is one of the crossings this exists to prevent.
- SkillsScreen: match the selection by `skillUriIdentity`, in both the
lookup and the NavLink active check. A refresh that canonicalizes
`skill://demo/%53KILL.md` names the same skill, and the detail pane was
emptying out because the server changed its spelling — the last raw-string
URI comparison left after round 12.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
The shared teardown deadline was decided on `remainingMs > 0` against
`Date.now()`. Both halves of that are noise-sensitive: the deadline is
enforced by a `setTimeout`, and a wall clock at millisecond resolution
can still read a hair short of it when the loop comes round, so the next
grant inherits a fractional budget and spends it on a request that cannot
possibly complete. The same run then issues one request or two depending
on scheduling — which is what makes the "shares one deadline across
grants" test intermittent on CI (#2252).
Measure the deadline with `performance.now()` (monotonic, so an NTP step
or a suspend cannot expire or extend the budget, and sub-millisecond, so
none is lost to rounding), and skip any grant reaching the loop with less
than MIN_REVOCATION_REQUEST_BUDGET_MS left. The skipped grant is already
reported as `failed`, so nothing is silently dropped — the needless call
to the authorization server is.
The CLI's outer per-plan budget in `sendPlans` shares both the shape and
the defect, so it gets the same treatment.
One consequence worth naming: a monotonic clock hands `revokeToken` a
fractional budget, and Node's `AbortSignal.timeout` throws
`ERR_OUT_OF_RANGE` on a non-integer delay — before the fetch, so the
request would never be sent and the revocation would report that as its
failure. `revokeToken` now rounds its own budget to whole milliseconds,
which also covers any other caller passing a fractional one.
Tests: the epsilon skip is pinned by a grant that lands inside the floor
but above zero (removing the floor fails it, verified); the fractional
budget by a `revokeToken` call at 19.996ms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017wEXtbs8UEHUMxDEAxbs99
Signed-off-by: cliffhall <cliff@futurescale.com>
`ServerImportJsonModal` > "guards against a live edit made before the
debounce re-validates" opened its window on the real clock: it pasted
invalid JSON and clicked Add Server, relying on less than
VALIDATE_DEBOUNCE_MS (300ms) of wall time elapsing in between. When more
did, the debounce landed first, `canAdd` went false, and the click hit a
disabled button — a no-op that sets no submit error, so the
`findByText(/Fix the validation errors/)` timed out. The first assertion
(`onAddServer` not called) still passed, which is why the failure read as
mysterious rather than as a disabled button.
Run the test on fake timers end to end instead. The first validation is
landed explicitly with `advanceTimersByTimeAsync(VALIDATE_DEBOUNCE_MS)`
— imported from the hook, so the test cannot drift from the value it
depends on — and after the second paste the timers are simply not
advanced. The pending re-validation therefore cannot land at all, the
window stays open by construction, and the final assertion becomes a
synchronous `getByText`.
Verified with a throwaway probe holding both shapes side by side with a
deterministic 400ms stall injected between the paste and the click: the
old shape fails exactly as reported, the new one passes.
No timeout was widened and no production code changed.
Closes#2250
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQgwZ1kGzhkdkMJ81JVg42
Signed-off-by: cliffhall <cliff@futurescale.com>
All three findings were the same underlying inconsistency: URI comparisons
on raw strings, in a flow that elsewhere deliberately treats normalized
equivalents as the same resource. One `skillUriIdentity` helper now backs
every comparison, so they cannot disagree.
- skills.ts: `manifest-missing-self` compares identities, so a manifest
listing the RFC-equivalent `skill://demo/%53KILL.md` is recognized as the
entry's own file — it is fetchable as that file, so reporting it missing
was the tool disagreeing with itself.
- skills.ts: `skillEntriesMatch` normalizes the entry URI and every manifest
URI before comparing, so a server that canonicalizes an escape between the
listing and the fetch is not reported as a changed snapshot.
- SkillsScreen: the `wrongUri` check uses the same identity, so a
canonicalizing server is not accused of answering for a different skill —
the read path already accepts exactly that equivalence.
- skills.ts: apply the Agent Skills name grammar to the RAW value. Trimming
first let `" demo "` pass, and whitespace is not in the grammar — so an
entry could report "Conforms" with a name that can never equal its URI
path segment. The trimmed copy now only distinguishes absent from invalid.
- skills.ts: add `malformed-description` for a description above the
1024-character Agent Skills limit, with boundary coverage.
- inspectorClient-skills.test: justify the `as unknown as` per AGENTS.md —
both fields are `private` with no public setter, the asserted shape is
exactly what the class declares, and the alternative is a live connection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- skills.ts: apply the two RFC 3986 §6.2.2 percent-encoding normalizations
the URL parser does not — decode an escape standing for an unreserved
character, and upper-case the hex of every escape that remains. `URL.href`
leaves `%72eference.md` encoded, so a server echoing an RFC-equivalent
form of the URI we asked for was rejected by `onReadSkillFile` as a
different resource, and an encoded name segment produced a false
`name-path-mismatch`. Both are the tool calling a conforming server wrong.
- oauthResume.test: assert the Skills restore path. It was wired into the
setters but never checked, and the comprehensive present-value and
undefined-fallback cases omitted Skills entirely. Adds it to both, plus a
case restoring a REAL saved selection rather than the EMPTY value — a bug
that always wrote the default would have passed the others.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- skills.ts: validate `frontmatter.name` against the Agent Skills format
(1–64 lowercase alphanumerics and hyphens, no leading, trailing or
consecutive hyphen). Checking only for non-emptiness let `Bad Name` read
as "Conforms" — and the name is not decorative: it must equal the URI
path segment, so a name that cannot appear in a URI is a contradiction
the entry cannot satisfy. Suppressed when the name is absent entirely,
which `missing-name` already reports.
- skills.ts: detect duplicates on the NORMALIZED identity. Containment and
the read that fetches the bytes both treat normalized-equivalents as one
resource, so `skill://demo/SKILL.md` and `skill://demo/x/../SKILL.md`
were passing as two distinct files while naming one. The finding still
reports the raw URI, and two different unparseable URIs stay distinct.
- SkillsScreen: run `checkSkillConformance` on the fetched `skills/get`
entry and treat an error — or an entry answering for a different URI —
as invalid (red), not as "a different snapshot" (yellow). A fresh
point-in-time read excuses a CHANGE; it does not excuse a violation, and
answering with another skill is never a refresh of the one requested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- SkillsScreen: track "Verify all" batches in a MAP keyed by manifest, not
one slot. A slot remembers only the most recent batch, so A running →
start B → return to A left A's button enabled and a second worker pool
could be started on top of A's first, doubling the concurrency cap. The
round-7 test switched to B but never STARTED B's batch, which is exactly
why it missed this; it now does, and fails against the old single slot.
- test-servers/skills.ts: pass `result` schemas to `setRequestHandler` for
both custom methods, and type the three builders from them, so a shape
change in `toEntry` / `listSkillsPage` fails `tsc` rather than shipping a
fixture that claims to conform. Declared locally rather than imported
from `core/mcp/skillsSchemas.ts`: a fixture validated against the
client's own schema could never catch the client being wrong.
One correction to the review's premise on that second point: the SDK does
NOT require `{ params, result }`, and supplying `result` does not validate
the response. Its own doc on `RequestHandlerSchemas` says `result` is
optional and "no runtime validation is performed on the result" — it types
the handler's return value. The change is worth making for that
compile-time check; it does not do what was claimed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- useServerCommands: require the returned URI to match the requested one
in `onReadSkillFile`. The "a single-block response must be the block we
asked for" fallback is right for a viewer and wrong here: these bytes
are hashed against the requested URI's advertised digest, so accepting
a block the server labelled `b.md` would verify one file's content
against another file's digest — and could report it `verified`. A
normalized equivalent is still accepted (a server may echo a resolved
`..`), but two unparseable URIs do not compare equal, and the error
names what came back instead.
- test-servers/skills.ts: build each `SKILL.md` FROM its frontmatter
object. Three fixtures listed one description and served another, which
is an undocumented extra violation — SEP-2640 requires the listed and
served frontmatter to be identical — and would have made phase 3's
frontmatter check report a finding these fixtures were not built to
demonstrate. Deriving one from the other makes that drift impossible
rather than merely fixed.
The PR description's severity table also still described the round-1
contract (limits as errors, digest/size/description as warnings) while
the code says the reverse. Rewritten to match, with the MUST/SHOULD split
stated rather than implied.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- skills.ts: stop sorting `resources` arrays recursively. Only the entry's
OWN manifest is a set; `frontmatter` is verbatim arbitrary JSON, so a
custom `frontmatter.metadata.resources` array was being sorted too and
two genuinely different frontmatters compared equal. The manifest sort is
now done once, at the top level, and every other array keeps its order.
- SkillsScreen: invalidate on the whole selected entry, not just its
manifest. The `skills/get` comparison covers `frontmatter`, so a refresh
that changed only a description left a stale "matches skills/list"
verdict on screen — computed against an entry that no longer existed.
Re-verifying after a metadata-only refresh is the cheap direction to be
wrong in; showing a match against a different entry is not.
- SkillsScreen: give each "Verify all" invocation a token, so a finalizer
can only ever clear its own batch rather than any batch with a matching
manifest key.
- SkillsScreen.stories: supply `onGetSkill`. Every story rendered the
"Fetch with skills/get" button, and clicking it threw.
On the batch token: the A → B → A sequence the review describes is not
actually reachable, because the button is disabled whenever the in-flight
batch's key matches the selection — returning to A finds it disabled, not
clickable. The token is defence in depth for the same property, and the
test asserts what is really guaranteed rather than the unreachable path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Most of this round re-reported code the previous commit already changed
(it generated 0 new comments). Two findings were genuinely open.
- SkillsScreen: give "View SKILL.md" and "Fetch with skills/get" the same
attempt ordering `verifyRow` got in round 3. Both were still
last-completion-wins within one manifest: a double click whose older
request settled last replaced the newer result. One shared `isStale`
helper now covers all three actions — a result is dropped when its
manifest was invalidated OR a later click already wrote.
- test-servers/skills.ts + docs: stop calling all three edge cases
"non-conforming". `resources: "dynamic"` is a LEGAL wire form for
generated content — it is unverifiable, not invalid, which is exactly
why the checker reports it as a warning. The prose contradicted the
code it was describing. Now: one legal-but-unverifiable case and two
actual violations.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Two of these reverse round 4, which asked for the opposite. Where the
readings conflict I took the permissive one: a wrong tightening rejects
conforming servers, a wrong loosening only fails to report something.
- skills.ts: stop requiring the `skill:` scheme. SEP-2640 says a server
SHOULD use `skill://` and explicitly allows a domain-native scheme, so
round 4's requirement handed a conforming `github://…` skill a false
`malformed-uri` and skipped its name and root checks. Containment is
scheme-independent — it compares against THIS entry's own root — so the
traversal fix that requirement came packaged with is unaffected.
- skills.ts: the 512-entry and 16 MiB limits are warnings. Both are
SHOULD NOTs for a server and MAYs for a host, so calling them errors
contradicted the "error = a MUST was broken" contract round 4
established and told authors a permitted skill was invalid.
- skills.ts: `skillEntriesMatch` compares canonically — keys sorted,
manifest sorted by URI. `JSON.stringify` treated key order and manifest
order as differences, though neither carries meaning.
- SkillsScreen: a differing `skills/get` is "a different snapshot"
(yellow), not a disagreement (red). The SEP defines it as a fresh
point-in-time read, so a skill that changed since the listing
legitimately differs.
- useServerCommands: route `onGetSkill` through the command-scoped OAuth
recovery like every other server command. An expired authorization
rendered an error and stopped there — no reauthorization, no retry.
- test-servers/skills.ts: stamp the modern base result envelope
(`resultType` / `ttlMs` / `cacheScope`) on every skills result. The SDK
stamps it only for methods in its own codec, and `skills/*` are
consumer-owned, so a 2026-era connection was receiving results without
it. Unconditional rather than per era: the modern leg builds a server
per request, so there is no era to branch on at registration time, and
on legacy they are three members no codec inspects.
- composable-test-server: remove the `skills.directoryRead` option. It was
publicly settable and advertised a method nothing serves, producing the
exact false capability the fixture exists to help catch. `skills` is now
a plain boolean; the flag returns in phase 3 (#2248) with its handler.
Still not changed: whether a modern `skills/list` RESULT SCHEMA must
require that envelope. Stamping it server-side is free; requiring it
client-side would reject servers on a reading nobody has checked against
the normative text. #2248 owns settling it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Three things the Skills work got wrong in that modal, all found by review
of the PR screenshots.
- The "Skills Extension" section repeated `io.modelcontextprotocol/skills`
as its value — the identifier the "Server Extensions" section two rows
above already lists, so the section added nothing. What a flat key list
*cannot* show is the extension's sub-options, which is the fact a server
author opens this modal to check. Renamed "Skills Extension Options" and
rendered as a ✓/✗ row for `directoryRead`, in the same vocabulary as the
capability columns, so it reads as the same kind of claim.
- The extension sections' contents were bold (`ValueText`, the value half
of a label/value pair) while sitting directly beneath the capability
checklists, which are plain. They are lists of items, not values, so
they now use the same `Text` the checklist rows do.
- Those lists were comma-joined into one line, which wraps mid-identifier
in a half-width column. One row per identifier.
The `skills-directory-read` tests now assert `data-supported` rather than
the copy: "Not supported" contains "Supported", so a text assertion passed
for either answer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- SkillsScreen: actually call `skills/get`. The client method and its
tests existed but no production caller invoked it, so #2234's acceptance
criterion ("skills/get retrieves a single entry") was unmet and a server
author's required handler could not be exercised. The detail pane now
fetches the selected URI on demand and reports whether the fetched entry
AGREES with the one skills/list advertised — both describe the same
skill, so a disagreement is a server bug only a side-by-side fetch shows.
- core/mcp/sha256.ts: a dependency-free SHA-256, used when `crypto.subtle`
is absent. `SubtleCrypto` needs a secure context, and this app is
documented as servable over plain HTTP on a LAN IP — where every skill
verification threw and the UI reported a read failure for files it had
fetched fine. Checked against the FIPS 180-4 vectors and differentially
against WebCrypto.
- skills.ts: require the `skill:` scheme in `normalizeSkillUri`. Checking
only that a URI was hierarchical let `https://demo/SKILL.md` pass the
name and root checks — a manifest pointing anywhere on the web, reported
as conforming.
- skills.ts: `missing-digest` and `missing-size` are errors. Both are
required fields, and an omitted `size` is what lets a server slip past
the 16 MiB pre-fetch limit while the UI reports zero errors. `warning`
is now reserved for what is legal yet unverifiable — `"dynamic"`.
- SkillsScreen: key the "Verify all" batch guard to the manifest. A global
flag left a newly selected skill's button disabled until the previous
skill's reads settled — forever, if one hung.
Not changed, deliberately: whether a modern-era `skills/list` result MUST
carry the SEP-2549 caching attributes. #2234's analysis records it as open
and the review asserts the opposite; neither reading was checked against
the normative text. Leaving the schema permissive accepts a server that
omits them, while tightening on a wrong reading would reject conforming
responses — the more expensive direction. `skillsSchemas.ts` states this
and #2248 settles it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- skills.ts: decide root containment on NORMALIZED URIs. A raw prefix
check passed `skill://acme/billing/refunds/../other.md`, which starts
with the advertised root but resolves outside it — a traversal the
conformance report was reporting as clean.
- skills.ts: `normalizeSkillUri` also rejects a relative string, so
`demo/SKILL.md` is `malformed-uri` rather than a skill path, and an
opaque-path URI (`skill:demo/..`), which the parser leaves un-normalized
and on which containment cannot be decided.
- skills.ts: `missing-description` is an error. SEP-2640 requires it, so
an absent one must not read as "0 errors" in the conformance summary.
- managedSkillsState: call `markResponseRejected` for a decode rejection,
as every managed list does. An invalid `skills/list` result was showing
in the Protocol tab as a clean success.
- SkillsScreen: per-row attempt token. The manifest key cannot tell two
verifications of the SAME row apart, so a double click (or a row button
pressed during "Verify all") let an older read finish last and overwrite
the newer verdict.
- SkillsScreen: disable "Verify all" while a batch runs. The concurrency
cap is per invocation, so repeated clicks stacked pools — 4, then 8,
then 12.
- SkillsScreen: include the finding index in each issue alert's key. Three
identical URIs produce two `duplicate-resource` findings with the same
code and URI, and React was free to drop the extras — hiding findings in
exactly the malformed input this view exists to inspect.
- test-servers/skills.ts: register `skills/list` and `skills/get` through
the PUBLIC `setRequestHandler` with explicit param schemas. The private
`_requestHandlers` map is now reached only to wrap `resources/read`,
which has to chain onto the SDK's handler rather than replace it — the
one thing the public API cannot express, and the comment now says so
instead of citing the tasks fixture.
- test-servers/skills.ts: raise `-32602` for an unknown `skills/get` URI.
A plain Error mapped to a generic server failure, making the fixture
non-conforming outside its three documented bad cases.
- clients/web/README.md: the paragraph named four contracts while still
saying the smoke "drives all three".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- SkillsScreen: key verdicts and React elements by manifest ROW INDEX,
not URI. The checker deliberately tolerates a duplicated URI so it can
report `duplicate-resource`; a URI key collided those rows into one
verdict, so verifying either updated both and "Verify all" raced two
different digest/size declarations into the same slot.
- SkillsScreen: hold the invalidation generation in React state keyed by
the manifest, not a ref bumped during render. `useValueChange` runs its
callback during render and requires setState-only purity — an abandoned
render left the ref incremented and silently discarded a live
verification.
- SkillsScreen: title a size disagreement "Size mismatch" and render its
reason. `verifySkillResource` catches it before hashing, so the alert
was showing "actual undefined" under "Digest mismatch".
- SkillsScreen: pass `contents` to ContentViewer so a base64 SKILL.md
renders. The text-block form substituted "" and painted a blank preview
for a file verification had just read correctly.
- skills.ts: add `malformed-size` for a size that is not a non-negative
safe integer, and exclude such values from the 16 MiB total. A negative
one could pull the sum back under the limit and hide a violation.
- managedSkillsState: make the overlap guard per-session instead of a
boolean. A disconnect during an in-flight walk left it set, so the
reconnect's own load no-oped and was never retried — permanently, if
the stale request never settled. A stale `finally` can no longer clear
the live session's guard either.
- skillsSchemas: require the `{ skill }` envelope for `skills/get`. The
accepted SEP settles it, and normalizing an inline entry would let a
non-conforming response past the one place that could report it.
- #2248 and the PR description: corrected — they said
`ReadResourceDirectoryResultSchema` was already declared, which round 1
removed. #2248 now owns defining it, and records the frontmatter
cross-check gap too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
- skills.ts: report a manifest that omits the skill's own SKILL.md
(an empty list included), duplicate URIs, entries outside the skill
root, and a missing size. `Conforms` was reachable for manifests that
break invariants SEP-2640 states.
- skills.ts: cross-check the declared byte length before hashing. A size
that disagrees fails verification on its own — the digest is taken over
the bytes the server served, so agreeing with it says nothing about
whether the manifest describes them.
- skills.ts: copy the view instead of slicing its backing store in
`sha256Digest`. `SharedArrayBuffer.prototype.slice()` returns another
SharedArrayBuffer, which `crypto.subtle.digest` rejects — the cast
claimed to handle the exact input that would have thrown. No cast now.
- skills.ts: state the one obligation NOT checked here — that an entry's
frontmatter matches the fetched SKILL.md's. The digest cannot cover it,
and closing it needs a YAML parser, so it is tracked on #2248.
- managedSkillsState: cap the walk at LIST_MAX_PAGES. The repeated-cursor
guard only catches a server stuck on one cursor; endlessly unique ones
walked forever. Raises rather than truncating, like the salvage walk.
- managedSkillsState: gate every write on a session generation, so a walk
that resolves after a disconnect or destroy cannot repopulate a cleared
store or deliver the previous session's skills into the next.
- inspectorClient: send a cursor when it is `!== undefined`, not when it
is truthy. An opaque cursor may be `""`, and dropping it re-requested
page one — which the store then reported as a repeated-cursor failure.
- SkillsScreen: key verdict invalidation on the manifest (URI + digests +
sizes), not the URI alone. A Refresh that changed the manifest left a
green badge attached to a digest nothing had checked.
- SkillsScreen: epoch-guard every read continuation, so a fetch that
resolves after the selection moved on cannot write into the new one.
- SkillsScreen: bound "Verify all" to 4 concurrent reads. A conforming
manifest may hold 512 files.
- skillsSchemas: drop the guessed `resources/directory/read` result
schema. Nothing calls it, so an unverified shape could sit wrong
indefinitely; phase 3 adds it against the normative text.
- skills-http.json: declare the extension bare. It advertised
`directoryRead: true` with no handler, so Connection Info reported
"Supported" for a method that answers -32601.
- types.ts: restore the Tasks doc comment the Skills interface displaced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Adds detection, enumeration, and conformance checking for the Skills
extension (`io.modelcontextprotocol/skills`) — phases 1 and 2 of #2234,
which is everything its Acceptance list names.
Skills is a *server*-declared extension, read off the connecting server's
`capabilities.extensions`, so it is deliberately absent from
`ADVERTISABLE_EXTENSIONS` — that registry is what the Inspector advertises
and the user toggles, and an entry there would be a meaningless toggle.
`skills/list` and `skills/get` are consumer-owned extension methods that
neither era codec defines, so the SDK's era gate skips them and they go out
as ordinary `client.request` calls with explicit result schemas. The
raw-wire channel modern `tasks/*` needs is not used, and the Skills tab is
not era-gated: a legacy-era server that declares the extension is serving it.
The checks are the point, not the list view. `core/mcp/skills.ts` reports
each obligation SEP-2640 states — the name/path invariant, the digest
format, the 512-entry and 16 MiB limits, and `resources: "dynamic"`, which
means integrity cannot be verified at all. Digest verification hashes the
fetched bytes with WebCrypto and returns a mismatch with both digests
attached rather than throwing, because showing a mismatch loudly is the
whole value proposition.
Files are fetched on demand: SEP-2640 is explicit that a `resources/read`
of a `SKILL.md` is not a load and confers no standing, so none of the SEP's
host machinery is implemented.
The `skills-http` fixture serves four skills over two pages, three of them
deliberately non-conforming — without those the verification code is
untestable.
Phase 3 (CLI, TUI, `resources/directory/read`, paged mode) is #2248.
Closes#2234
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
Re-measured the two committed chain cases under the stricter turn-boundary
scoring from 313a5809: 100% and 33% at RUNS=3, against 33% and 33% before.
That rise is NOT an effect of the change and the doc now says so. The turn
rule only ever rejects matches a flatter reading accepted, so it cannot
raise a score — the two runs simply straddle a 67-point swing on the same
prompt at RUNS=3, where one sample is worth 33 points. Reading it as an
improvement would be exactly the mistake the eval exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
One finding, and a sharp one: position in the stream is not causation.
`collectSkillInvocations` flattened every `tool_use` block across every
assistant event into one ordered array, and `chainHit` read position
alone. But the model can emit several `Skill` blocks in a SINGLE assistant
message, and it has not seen the first skill's body when it does — those
are parallel guesses, not a hand-off. A run that emitted `testing` and
`test-servers` together in one message scored as `A` led to `B`, and would
have gone on scoring after the pointer in `testing`'s body was deleted:
exactly the false pass this feature exists to rule out, arriving by a
second route (Copilot).
Each recorded invocation now carries the assistant event it came from —
`{payload, turn}` — and `chainHit` requires every link after the first to
land in a STRICTLY later turn. The turn count is threaded through
`collectSkillInvocations`'s `turnOffset`/`nextTurn` so a stream read in
chunks stays monotonic rather than restarting per read; without that, a
chunk boundary would reset the numbering and reintroduce the same bug on
long runs.
The scan stays greedy, which is still correct under the constraint: taking
the earliest occurrence of a link can only leave more room for the rest,
so no later starting point succeeds where the greedy one fails.
`sampleHit` is deliberately unaffected — a first-move case asks only
whether a skill fired at all, so it reads names and ignores turns.
Four new assertions in one case: a same-turn pair scores false; the same
two skills a turn apart score true; a same-turn pair does not poison a
later genuine hand-off; and only the chain's first link is unconstrained.
Plus a `collectSkillInvocations` case pinning the turn tagging and the
cross-chunk offset.
`docs/skill-authoring.md` now lists this as the second way a chained case
can false-pass, alongside a prompt carrying the target's own trigger.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
Closes#2244Closes#2225
All three outstanding advisories were stale lockfile resolutions, not
upward-blocked pins: every fixed version already sits inside the range its
declaring parent asks for, so refreshing the lock entry is the whole fix.
fast-uri 3.1.5 -> 3.1.7 ajv@8.18.0 asks for ^3.0.1 (root)
qs 6.15.3 -> 6.16.0 express@5.2.1 asks for ^6.14.0 (root)
browserslist 4.28.2 -> 4.28.9 via @babel/core (clients/tui)
No `overrides` entry is added, and no manifest changes. AGENTS.md's rule is
that a transitive is pinned with `overrides` rather than with `npm audit fix`
-- it does not call for a pin where none is needed. A permanent pin here would
buy nothing and would later hold a package back: `fast-uri: ^3.1.6` forbids
fast-uri 4.x for as long as it stands, including after ajv moves to it. The
monthly refresh sweep and the daily alert sweep are what catch a regression.
`npm audit` is clean in all five installs (root, web, cli, tui, launcher).
Reachability, assessed rather than assumed:
- fast-uri is the one that ships. ajv is a root runtime dependency, and Vite
pre-bundles it into the published `clients/web/dist`, so the vulnerable code
was inlined into the SPA rather than merely resolved at install time. Its
input is attacker-influenced: `schemaUtils.ts` compiles the `outputSchema`
a server under test supplies, and ajv resolves that schema's `$id`/`$ref`
through fast-uri (`ajv/dist/runtime/uri.js`). Impact is bounded well below
the advisory headlines, though: both SSRF advisories need a consumer that
fetches the parsed URI, and ajv never performs a network request. The
realistic worst case is the host-confusion pair mis-normalizing a crafted
`$id`, giving a wrong or failed validation of one tool's output.
- qs is installed in production, but nothing shipped runs it. express is a root
devDependency, but that is not its only path: `npm ls express --omit=dev`
shows it reaching a production install through
@modelcontextprotocol/server-legacy@2.0.0 (a root runtime dependency, also
via express-rate-limit) and through @modelcontextprotocol/ext-apps ->
@modelcontextprotocol/sdk@1.30.0. So qs is present in every user install.
What holds is that no shipped module ever instantiates it: nothing in
`core/`, `clients/*/src` or `clients/web/server` calls `express()`, the web
backend is Hono, and server-legacy is imported only from `test-servers/src`,
which the root `files` list does not publish. Both advisories need express to
parse an attacker-supplied query string, which requires a live express app.
- browserslist is dev tooling. It arrives in the clients/tui install through
eslint-plugin-react-hooks -> @babel/core, is reached only by lint, and is in
no published bundle. The root install already resolved a patched 4.28.8 from
the same plugin at the same version, which is what identified tui's copy as a
stale lock rather than a constrained one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YahVxMTGpigLbZBh1JGPDr
Signed-off-by: cliffhall <cliff@futurescale.com>
Two findings, both correct, and the first one means round 1's containment
fix was only half a fix.
**`--allowedTools` is not the availability filter.** It grants permission;
it does not restrict which tools exist. So a tool a user's or a plugin's
settings already permit stayed reachable across all 14 turns, and the
bound still rested on the deny list being complete — the exact property
round 1 set out to stop depending on. `--tools` is the real restriction
(it selects from the built-in set), and both flags now carry the same
four names: `--tools` to bound, `--allowedTools` so those four never need
a permission prompt no headless run can answer. Verified against the real
CLI — the full flag set runs to a clean `end_turn`. The test asserts both
flags, and both probe snippets in `docs/skill-authoring.md` carry
`--tools` with a ⚠️ saying which one does the bounding.
**The threshold guard let `NaN` and negatives through.**
`CHAIN_THRESHOLD=abc` is `NaN`, which fails every comparison and prints an
`above NaN%` summary; a negative bar passes every chain unconditionally.
Both turn an advertised env knob into a measurement that quietly means
nothing. `CHAIN_THRESHOLD` must now be in `[0, 1)` — half-open because the
chain bar is strict — and `THRESHOLD` in `[0, 1]`, inclusive because a
first-move floor of 1.0 is meetable. The message names the offending
value. A new case spawns the script for each bad value rather than
asserting on the predicate, since these guards live in `main`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
Five findings, all real.
**The chain prompts carried the target skill's own trigger.** `test-servers`
claims "a change needs a real server to exercise it", and both cases said
"against a real/live server" — so the model could load `testing`, then pick
`test-servers` from the ORIGINAL prompt, in that order, and score a hit that
would have survived deleting the pointer from `testing` entirely. Rewritten to
carry no server cue ("end to end"), and the measured rate fell from 100% / 67%
to 33% / 33%. That is the size of the artefact, and 33% is what the pointer
alone is worth. Both cases are now red against the 0.5 bar and stay that way:
`skills:eval` is not a gate, the number IS the finding, and lowering the bar to
turn the column green would discard the only signal this feature adds. Filed as
#2247 with the trap written down so a fix cannot re-measure the artefact.
**A deny list cannot bound a 14-turn run.** It only names tools known when it
was written, and this checkout configures an HTTP `mcp-docs` server in
`.mcp.json` while a contributor's own MCP servers and plugins add more. Now
`--allowedTools Read,Glob,Grep,Skill` (what the harness actually needs) plus
`--strict-mcp-config` with no `--mcp-config`, which drops every configured
server. The deny list stays alongside, since a deny is unconditional while an
allow list governs pre-approval. What remains outside all three is a
contributor's plugin tools; `--bare` would remove those and skills with them,
which would measure nothing — so that residual is stated rather than papered
over.
**The chain threshold compared inclusively.** "More often than not" is `> 0.5`,
and `>=` passes exactly half whenever RUNS is even — 2/4 would report a result
the criterion does not license. `passesThreshold` is now strict for chains and a
floor for first moves, and a strict bound of 1.0 is rejected up front rather
than failing every case while looking like a trigger problem.
**The doc's probe command still denied only four tools**, contradicting the
section's own promise that a probe and a scored case see the same policy.
Updated, with a note that it must move with the harness, plus a hand-off probe
form.
**The reporting had no automated coverage** — the one thing that could silently
merge the two measurements lived inside `main`. Extracted as `formatReport` and
tested: both headings, per-group thresholds, the two separate summaries, the
combined exit, and both single-kind selections.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
#2222 reported that `core/auth/revocation.ts` breaks a client secret
containing `+`, on the basis that `encodeURIComponent` leaves `+` bare.
It does not — it escapes it as `%2B`. The characters it leaves bare are
exactly `!'()*-._~` and alphanumerics, every one of which a form-
urldecoder passes through unchanged, so its output decodes identically
under both algorithms; verified over every code point up to U+2FFF.
Switching to `URLSearchParams` would be the literal algorithm §2.3.1
names and a small regression: it encodes a space as `+`, which a lenient
server decoding with `decodeURIComponent` alone reads as a literal `+`,
while `%20` is understood by both. The encoder is therefore unchanged
and the reasoning is recorded on it.
The report's second half stands and is the real defect: the fixture
decoded with `decodeURIComponent`, the encoder's own inverse, so the
round trip succeeded for every input by construction and no test could
have failed on an encoding mistake. `/oauth/revoke` now runs a genuine
RFC 6749 Appendix B decode, with unit cases for `+`, a space, `:` and
`%` (asserted on the wire and through an independent decoder), an
end-to-end client whose secret is base64 with a `+` in it, and a guard
that an unencoded credential is refused — the case that separates the
two decoders.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BhVaufrEZAfFAXcg6456hK
Signed-off-by: cliffhall <cliff@futurescale.com>
`skills:eval` ran every case with `--max-turns 1`, so it measured one
thing: whether a skill is the model's FIRST tool call. That is right for a
skill a user reaches directly, and it leaves a whole class unmeasured — a
skill reached from inside another skill's body. `testing` tells the model
that picking a fixture is `/test-servers` and that it has to load it, and
`test-servers` scores 5/5 while every one of those five cases asks for it
by name. A skill only ever reachable through a hand-off would score a
clean 100% with the hand-off silently never firing.
An eval case is now one of two shapes, and exactly one:
{ "prompt": "…", "expect": "test-servers" } first move, 1 turn
{ "prompt": "…", "chain": ["testing", "test-servers"] } hand-off, 14 turns
A chain must END with the skill whose file it lives in. The case exists to
measure whether THIS skill is reachable, so the file that goes red is the
one belonging to the skill that stopped being reached; anchoring on the
first link would file a `test-servers` measurement under `testing`, where
a `test-servers` description edit would never be seen. Links are checked
against the repo's model-invoked set, which is why both `verify:skills`
and `collectCases` now validate in a second pass — `test-servers` sorts
before `testing`, so a per-directory check would reject a live chain as
unknown purely because of where the alphabet put it.
Scoring is an ordered SUBSEQUENCE, not a prefix and not a contiguous run:
the model may load something before the chain starts and something
unrelated in between, and neither changes the claim that A led to B.
`collectSkillInvocations` therefore returns an ordered array rather than a
Set — a B, A, B run has to stay distinguishable from one that never
reached B from A.
The two rates are reported in separate columns and never summed. A
hand-off is a second-hop load over many turns; a first-move rate is the
model's opening move. `CHAIN_MAX_TURNS` (14) and `CHAIN_THRESHOLD` are
their own knobs for the same reason, and `CHAIN_THRESHOLD` defaults to
**0.5** rather than inheriting 0.8: see the measurement below.
Read-only containment no longer leans on `--max-turns 1`, which was doing
much of it by itself. The deny list gains the agentic and network tools —
`Task` in particular, whose subagent the flag does not reach.
Measured, `skills:eval -- test-servers`:
First move (1 turn) 7/7 at 100%, unchanged
Hand-off (14 turns) 67% and 33% at RUNS=3
So the hand-off is real and unreliable — which is the fact nothing could
observe before this change, and the reason 0.5 rather than 0.8 is the
default bar: at 0.8 both committed cases are red no matter how strongly
the first skill points at the second, and the column stops carrying
signal. A `["pr-flow", "test-servers"]` probe measured 0% and was dropped
rather than kept: `pr-flow` says nothing about fixtures, so the case had
no lever short of broadening a description onto another skill's ground.
Both findings are written up in `docs/skill-authoring.md`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABdTnqyBodw6mHVVkGehqe
Signed-off-by: cliffhall <cliff@futurescale.com>
Record each advisory's disposition during the loop instead of deriving
"can this be filed?" from the groups beforehand.
Membership in a group only means an alert exists. Whether an issue
tracks it is decided by the ecosystem check, the manifest read and the
range probe that follow — so if GitHub revised first_patched_version
while that manifest happened to be unparseable, the replacement group
was skipped and the old-key issue was still cleared as "superseded",
claiming an issue that was never filed.
Each advisory is now noted as tracked, not-exposed or indeterminate as
the loop reaches it, and reconciliation refuses to clear while anything
still open is indeterminate or absent. That also splits the clear reason
honestly: a bump whose copies simply left range now says so, rather than
claiming a replacement issue exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
- Reconcile on (package, manifest, GHSA) tuples, not GHSAs alone.
Dependabot alerts are per manifest and this repo has five lockfiles,
so the same advisory legitimately covers the root install and a
client. Keyed on the GHSA alone, another manifest's still-filable
alert could vouch for this one and clear an issue whose own alert is
open with no bump available.
- Rename the cleared-date test: it asserts the NEW date is taken when
the reason changes, which is the opposite of what its name said.
- AGENTS.md no longer implies both sweeps select a dated milestone.
Only the security sweep filters undated buckets; the monthly one's
selection is raised on #2239.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Three findings, all of them the same invariant reached by new routes: a
still-open advisory must never stand itself down, and a cleared issue
must be written once.
- Never clear an advisory that is open but unpatched. groupAlerts drops
an alert with no first_patched_version, so it has no replacement
group and no replacement issue — calling it "superseded" was false
and clearing it removed the only thing tracking a live exposure. Such
an issue is now left exactly as it is, and said so in the log.
- Stop re-editing cleared issues daily. A cleared issue stays open, so
the next run regenerated its body with a new date and edited it
again, forever. The date is now read back off the existing body, so
only a real change writes — and a real change takes today's date.
- Distinguish an absent manifest from an unparseable one. Both returned
null, so a malformed lockfile was treated as "the manifest is gone"
and cleared the issue. A read error is evidence of nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
Build the open-GHSA set from the raw alert feed, not from the grouped
alerts. groupAlerts deliberately drops an alert with no
first_patched_version, so an advisory that stays OPEN but loses its
patched version vanished from both the keys and that set — and
reconciliation would have called it "fixed or dismissed".
Same wrong direction as the superseded case fixed last round, reached a
different way: a still-open advisory must never stand itself down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
- Filter to the npm ecosystem. Dependabot alerts are not npm-only and
this repo has a Dockerfile, so an alert against it would have been
parsed as a lockfile, thrown, and aborted the whole daily sweep
before any npm group ran. A non-npm alert is now reported loudly with
its GHSAs so a human can file it, and readManifest no longer lets one
unparseable manifest take the run down.
- Distinguish a superseded bump from a closed one. GitHub can revise
first_patched_version, moving an advisory to a different key while it
stays open; reporting that as "fixed or dismissed" would stand down a
live exposure. The reason now reads from the open GHSA set.
- Only claim security-update PRs are off when the run actually read the
setting. The guard degrades to UNVERIFIED, and an issue asserting
what the run could not confirm is worse than one that stays quiet.
- README.md and AGENTS.md: scripts/ now holds repo automation run from
CI, not only build/verify tooling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
