Four findings, all real.
- No catalog-level bound. The per-skill caps bound what ONE entry can
cost; nothing bounded how many entries there are, and SEP-2640 puts no
ceiling on a catalog — every entry costs at least one resources/read,
so a large listing made `--verify` run indefinitely and transfer
unboundedly. Adds SKILL_MAX_CATALOG_SKILLS / SKILL_MAX_CATALOG_BYTES.
Entries past the budget are still reported, with their static findings
and an `incomplete` reason, rather than dropped or failed: they were
not checked, which is neither a pass nor a verdict against the server.
- `responseBytes` charged `text.length`, which counts UTF-16 code units.
That undercharges non-ASCII by up to 3x, so a decoy block of emoji kept
the counter under 16 MiB while the wire carried far more. Adds
`utf8Length`, an allocation-free UTF-8 byte count — deliberately not
TextEncoder, which would copy a payload the server chose the size of.
- `checkSkillNameCollisions` was O(N^2) in both work and output for a
group of N: every entry filtered all N URIs and embedded the other
N-1. Duplicate names are legal and a server controls N. Now names a
bounded sample and counts the rest.
- `--verify` help documented exit 7 but not exit 8.
The three-tier bounding (per skill, per skill on the wire, per run) is
now a table in clients/cli/README.md, since the run bound is this tool's
limit rather than the spec's and should not read as a conformance rule.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
Three findings, all in paths hardened by earlier rounds.
- The 16 MiB read budget was charged only after a content block had
matched by URI *and* decoded. A server could answer every manifest row
with one enormous block labelled a different URI (so `contentsFor`
found nothing) or one enormous invalid-base64 blob (so `skillFileBytes`
threw), bank zero against the cap, and have the walk issue up to 512
more. `responseBytes` now charges the raw response before selection or
decoding; the exact decoded length is substituted when there is one, so
the common path is still measured precisely.
- The Directory section checked that a child was inside the skill root
but not that it was a child of the directory actually being read. A
grandchild, or the directory echoing itself back, passed containment
and was rendered as navigable. Non-direct entries are now shown and
labelled rather than linked — `resources/directory/read` answers with
direct children, so an entry that is not one is itself the finding.
- Containment was decided on the normalized URI while navigation sent and
stored the raw one. For `skill://r/a/../templates` the first Up produced
`skill://r/a/..` and a second walked into `skill://r/a`, a directory
nothing had validated. Directory descent passes the normalized URI, and
the Up arithmetic moved into `parentOfSkillUri`, documented as valid
only on a normalized URI.
The suppressed comment (two stderr lines on a failing `--verify`) is
declined again: `--strict` produces the same shape in `emit-result.ts`,
and clients/cli/README.md already documents it as such.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
Four of the five findings share one root cause: the tri-state `outcome`
added in round 15 was not propagated to consumers, which kept branching
on `ok` — true for an `incomplete` report, since nothing that was checked
was wrong.
- CLI `summarizeSkillVerification` counts off `outcome`, not `ok`, so a
truncated walk no longer prints "no conformance errors" one line before
exiting SKILL_INCOMPLETE. A mixed failed/incomplete catalog reports
both counts rather than letting the louder verdict hide the quieter.
- `verifySkills` sets the truncation reason only when entries were
actually left unread. A file crossing the byte budget as the FINAL
entry stopped nothing, and "Stopped after 3 of 3" both read as a
contradiction and demoted a fully-read skill out of `verified`.
- The TUI status line switches on `outcome` through a `Record` over the
union, so the INCOMPLETE arm is reachable and a fourth outcome would
be a type error rather than a silently missing label.
- The `incomplete` doc paragraph said it makes `ok` false. It does not,
deliberately — corrected, and it now names `outcome` as the thing a
consumer must branch on.
- The web sidebar composes in the per-URI `duplicate-name` warning, so
two colliding skills are badged in the catalog instead of looking
clean until one is selected.
The suppressed `selfAttempted` finding is stale — fixed in round 16.
The existing TUI INCOMPLETE test was passing for the wrong reason: its
fixture understated every size, which is itself a mismatch, so the report
was `failed` and only the old `ok`-first branch printed INCOMPLETE. It is
rebuilt with honest digests and sizes, so truncation comes from the
declared-size prefilter and the report is genuinely incomplete.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
**A hole my own round-13 change opened.** The byte budget added a `break`,
and `manifestListsSelf` described the bounded slice rather than the rows the
walk reached — so breaking before a later self-entry left the flag true,
suppressed the fallback, and skipped the mandatory frontmatter check
entirely. It is `selfAttempted` now, set inside the loop, and marked before
the read: a row the walk reached but could not read has still been attempted,
and its failure belongs to the loop rather than to a second fetch.
**A preview could overwrite a verification's own bytes.** The digest verdict
on screen was computed from the verification's fetch; replacing only the text
let the frontmatter findings describe different bytes, recreating the
mixed-fetch verdict that state exists to prevent. `entryTextVerified` marks
which read produced it, and a preview no longer wins over a verification.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
**Verification is a tri-state now, because two states could not be right.**
Round 12 was correct that `ok: true` for a manifest whose unread 513th file
may be tampered with is a false pass. Round 15 is correct that forcing `ok:
false` calls a server nonconformant for exceeding limits SEP-2640 states as
SHOULD NOT, with hosts free to support more — contradicting this module's own
rule that a warning never fails a report. Both hold, so:
verified -> exit 0 everything checked, everything passed
failed -> exit 7 a MUST was broken
incomplete -> exit 8 nothing checked was wrong; the walk was cut short
`ok` keeps its narrow meaning, `allSkillsVerified` is stricter than
`every(r => r.ok)` since it selects the exit code, and `anySkillFailed`
separates 7 from 8. A job that tolerates oversized catalogs can allow 8 and
still fail on 7.
**`manifestKey` still used `JSON.stringify` on the entry** — the same crash I
fixed in the TUI in round 12, one file over.
**Directory children were navigable outside the skill root.** A server could
return `skill://other-skill/...` and clicking it left the tree, with "Up"
only comparing equality against the root. Containment is checked on the
normalized URI now, and an outside child renders as text rather than a link.
Worth recording from the tests: `..` cannot escape the authority, so
`skill://a/nested/../x.md` resolves back inside and must stay navigable.
**The TUI matched report rows by raw URI**, which missed a row recorded under
an equivalent spelling while `extraReportFiles` suppressed it as covered.
Normalized, like the membership test beside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
**The TUI ignored `incomplete`.** Round 12 added that field precisely so a
consumer could tell "not fully checked" from a real failure, and then the one
client that reads the report did not consume it — which made the field
decorative. The pane shows an Incomplete: block with the report's own reason,
and the status line reads INCOMPLETE rather than FAILED.
It sits ABOVE the manifest, not beside the status line: it explains the list
that follows — only the first N rows were fetched, the rest stay marked `·`
because nobody looked at them — and below a 512-row manifest it would be
off-screen, which is the same as absent. Found by writing the test, whose
first version asserted against a 600-entry fixture and could see neither
string. The test triggers truncation through the BYTE budget instead, so the
manifest stays three rows and the assertions measure the pane rather than the
test's viewport.
**The capped self-entry was reported under the wrong URI.** A manifest may
write its self-entry in a normalized-equivalent form; the fallback recorded
`entry.uri`, so manifest rows keyed on the declared spelling matched nothing
while the normalized extra-files filter suppressed it as already covered. The
verdict existed in the report and appeared nowhere on screen. Recorded under
`declaredSelf.uri` now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
**The byte budget trusted server-declared sizes.** A manifest advertising
`size: 1` — or, since the wire schema deliberately accepts it for reporting,
no size at all — sailed through the declared budget and then served
arbitrarily large bodies. The 16 MiB safeguard protected against an honest
server only, which is no safeguard.
The walk now tracks bytes ACTUALLY received and stops issuing reads once the
real total crosses the cap, marking the report `incomplete` and so `ok:
false`. The declared prefilter is kept rather than replaced: it refuses to
schedule an obviously oversized set, and the received-byte counter stops one
that lied. The count happens after verifying the crossing file, so that file
is still reported rather than fetched and discarded.
⚠️ This bounds the total across responses, not any single one: a first
response larger than the cap is already in memory before it can be measured,
which would need a streaming read the client API does not expose. Stated in
the code rather than left to look closed.
**A modern `skills/get` did not require `resultType`.** The "left open"
question SEP-2640 states covers `ttlMs` / `cacheScope` and only those;
`resultType` is base-protocol per SEP-2322 and appears in the SEP's own
`skills/get` example. Requiring it of `resources/directory/read` but not here
was an inconsistency in the module rather than a distinction the spec draws.
Era-selected now, with the caching attributes still optional.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
**The read bounds could produce a false `ok: true`.** Entries past the cap are
never fetched, `fileFailed` only examines fetched rows, and
`resource-limit-exceeded` is a WARNING — so a manifest whose 513th file was
tampered with reported the skill verified. That trades a denial of service
for a wrong answer, which is the failure this whole PR argues against. A
truncated read is now recorded as `incomplete` and forces `ok: false`,
reported separately so a consumer can tell "this skill is wrong" from "this
skill was not fully checked". And a self entry excluded by the cap is now
verified against its declared digest, not merely read for frontmatter —
otherwise the skill's own SKILL.md was the one file nobody checked.
**`entryKey` could crash the TUI.** It called `JSON.stringify` on the whole
server-controlled entry, during render. `skillEntryKey` is decided by the
same `jsonGraphError` guard that bounds the frontmatter comparison, with a
coarse fallback that still separates two unrepresentable entries by URI —
collapsing them would show one skill's verdict under another's name.
**The tab bar wraps and `tabsHeight` was hard-coded to 1.** An OAuth-capable
stdio server serving Skills needs ~107 columns, so it wraps at 132 as well as
at 80, and every pane below was sized a row too tall. `tabBarRows` derives it
from the same list `Tabs` renders, via a shared `visibleTabs(flags)` — the
duplicated filter being how the two would drift apart again.
Nine tests in a new `tabsConfig.test.ts`, including the 80-column regression.
Two assert properties rather than instances: rows never decrease as the
terminal narrows, and a tab wider than the row gets its own rather than
looping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
**Reads were bounded by entry count but not by total bytes.** A manifest can
sit at exactly 512 entries and declare a gigabyte each, so round 9's cap
still let a server dictate unbounded bandwidth and time after
`size-limit-exceeded` had already been reported. `boundedManifest` caps on
both interoperability limits now.
Sizes are summed exactly as `totalSkillBytes` sums them, so the bound and the
finding that reports the overage cannot disagree about the total. A read is
skipped only when the running total would CROSS the limit, so a conforming
skill — at most 16 MiB by definition — is never truncated; there is a test
for that, because a bound that protected against a hostile server by giving a
wrong answer about a good one would be a poor trade. The self-file fallback
still applies, so the mandatory frontmatter check survives either cap.
**The `vitest.shared.mts` pin rationale had gone stale.** It said `yaml` is
reached only through `test-servers/src`, which this PR made untrue by
importing it from `core/mcp/skillFile.ts`. That comment is what a future
dependency-placement change would read before removing or reclassifying the
pin, so it now names the `core/` path and the bundler `external` lists that
follow from it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
**Verification reads were unbounded.** The 512-entry limit is checked by
`checkSkillConformance` — as a warning, since the SEP makes it an
interoperability bound rather than a MUST — but checking it constrains
nothing, so a hostile server advertising a million entries had the tool
perform a million sequential reads after the report already knew the manifest
was over. Capped at `SKILL_MAX_RESOURCE_ENTRIES`, with the finding still
reported so bounding the work does not silence the reason it was
unnecessary. `manifestListsSelf` is computed over the READ slice, so a
self-entry pushed past the cap still reaches the fallback: the frontmatter
comparison is mandatory and must not be lost to a limit that bounds other
files.
**A malformed skill URI still produced a directory root.** `skillUriIdentity`
falls back to the raw string, so `not a uri/SKILL.md` yielded the root `not a
uri` and enabled the Directory section — letting the UI build a request from
a URI the conformance checks had already rejected. The comment above it
claimed otherwise. `normalizeSkillUri` now, with the rule written down:
identity is for COMPARING two spellings, not for deciding a URI is
well-formed enough to build a request from.
The existing regression test passed against a weaker input that failed for
lacking the suffix and never reached the fallback; it is a table of three
shapes now.
**The test-server guide overstated its own coverage,** claiming four of the
five conformance scenarios map onto fixtures. Three do: there is no
size-mismatch fixture, because the fixture can override an advertised digest
and nothing else, so that path is covered by unit tests. Corrected, with what
adding one would take.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
Eight findings. Two were visible in a terminal capture I had already taken
and read past.
**A digest mismatch rendered with no digests.** `verifySkillResource` sets
`reason` for a SIZE mismatch — which short-circuits before hashing — while a
digest mismatch carries `expectedDigest` / `actualDigest` and no `reason`, so
the TUI printed a bare `✗ notes.md` with nothing to act on. `failureDetail`
now falls back to the expected/actual pair, truncated to fit the pane.
**A dynamic skill's read failure rendered nowhere.** Its synthetic
`read-error` row lives in the report, not the manifest, and the manifest is
empty for such a skill by definition — so the pane said only "Verification
FAILED". There is a Read failures block for any report file the manifest does
not cover, matched on normalized identity.
**The non-finite sentinel could be aliased.** A listing whose value genuinely
was `{"#non-finite":"NaN"}` canonicalized identically to a served `.nan`, so
the round-3 fix moved the bug rather than closing it. Any encoding into the
value space can be aliased by a document containing the encoding, so the
comparison is structural now (`jsonLikeEqual`) and there is no sentinel at
all; `Object.is` gives NaN === NaN while keeping ±Infinity distinct.
Serialization is used only for the message.
**`parseSkillFrontmatter("null")` returned `{ fields: {} }`,** contradicting
its own non-mapping contract: an explicit null scalar parses to exactly what
an empty block does. They are told apart by the source, since the parsed
value cannot.
Also: the `files` doc comment still claimed it is empty for a dynamic skill,
which the round-2 fix made untrue; and three inline Mantine elements with two
static styling props each are extracted to `.withProps()` constants, per the
convention the rest of the file follows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
Four findings, two of them silent holes in the verification itself.
**A blob-served SKILL.md skipped the frontmatter check entirely.** The entry's
own file was captured as `contents.text`, so a server returning the markdown
as a base64 `blob` — a legal `resources/read` shape this module already
decodes for the digest — never reached the comparison, and the report still
said `ok`. A MUST that quietly did not run. The file is now held as bytes and
the text derived from them, which also guarantees the digest and the
frontmatter describe one snapshot.
**A dynamic skill whose SKILL.md could not be read reported `ok: true`.** It
has no manifest rows, so `files` stayed empty, and its only static finding is
a warning — a verification that could not be performed was reported as one
that passed. The fallback read now records a `read-error` for all three ways
it can fail. It is gated on `manifestListsSelf` (by normalized identity) so a
self-entry the manifest listed and failed to read is not read, or reported,
twice.
**A served `.nan` compared equal to a listed `null`.** YAML expresses
non-finite numbers, JSON does not, and `JSON.stringify` turns all of them
into `null` — so the canonical comparison reported a real mismatch as
agreement. Verified against the parser before fixing. They now canonicalize
to a form no JSON scalar can equal, and the value is named in the finding.
Also: the self-entry match is by normalized identity rather than raw string,
which stops a second read and stops this function disagreeing with
`checkSkillConformance`; and `SkillsTab.tsx` gains the file header AGENTS.md
requires.
The four `err instanceof Error` ternaries are one `reasonOf` helper now —
extracted because the coverage gate found every one of their non-Error arms
uncovered, and one honestly-tested branch beats four ignores.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
SEP-2640: hosts MUST NOT assume name uniqueness, and when two entries in one
listing collide on `name` a host MUST disambiguate them rather than silently
discarding or preferring one.
`checkSkillConformance` structurally cannot see this — it takes one entry and
a collision is a property of the pair — so `checkSkillNameCollisions` walks
the listing and returns a finding per colliding entry, each naming the
others. All three clients merge it into the entry's own findings, so it
carries through the header badge, the CLI report and the TUI row marks with
no new surface.
**It is a `warning`, not an `error`,** and that is the severity split doing
its job. The obligation is on the *consumer*: a server may legitimately
publish two skills with the same name under different paths, and the SEP's
own `acme/billing/refunds` example is exactly that shape. Calling it an error
would tell a conforming author their catalog is invalid. `--verify` therefore
still exits 0 for a collision.
In the web screen it renders as a banner directly under the Conformance
header and is filtered out of the findings list, the same treatment
`dynamic-resources` gets: it changes how everything below it should be read,
and stating one fact twice reads as two findings.
Fixed while adding it: an entry whose only finding was a banner one rendered
an EMPTY findings container instead of "no structural issues", because the
list's presence was decided on the unfiltered set while its contents were
filtered. `listedIssues` is now derived once and used for both.
The fixture gains `acme/reports` + `globex/reports`, both fully conforming
and sharing the name `reports` — also the only fixture with a multi-segment
skill path, which nothing else exercised. Eight skills over four pages now,
and the integration test walks the cursor to exhaustion rather than asserting
a fixed page count, so a future fixture does not require editing it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
Five findings, all real.
**Critical — `verifySkills` selected `contents[0]`.** These bytes are hashed
against the file's advertised digest, so accepting a block the server
labelled something else verifies one file's content against another file's
digest, and can report that as `verified`. A false pass is worse than a
missing check. `contentsFor(result, uri)` now selects by NORMALIZED identity,
which keeps the canonicalized-echo case that motivated the original code
while rejecting an unrelated block, and reports a read error when nothing
answers for the URI. This is what `onReadSkillFile` already did; the two
paths hash bytes against digests and must not disagree about which bytes.
**`skills/get` was dispatched without the extension gate** that `skills/list`
had. Hoisted into `assertSkillsSupported` so the two cannot drift — declaring
the extension commits a server to both. Without it, an undeclared server's
-32601 is indistinguishable to a script from the -32602 a declared server
returns for a URI it does not serve.
**The TUI could strand a user on a hidden Skills tab.** The tab left the bar
when the gate went false but `activeTab` did not, so the pane kept rendering
for a server that never declared the extension. Reset to `info`, following
the Auth precedent — additionally gated on `connected`, because this gate
reads a server declaration and would otherwise fire during a reconnect.
**The TUI keyed a verdict by URI**, so a refresh replacing the manifest under
the same URI left hashes computed for the previous snapshot describing the
new one. Keyed on the entry now, as the web screen already was.
**A failed "Load more" discarded the pages on screen and the retry cursor.**
Both are preserved now, and the success and failure paths share one
staleness-guarded `commit` helper so they cannot drift on which results they
may write — which is how they came to disagree.
Two nits: the six-skill counts. The docs sentence was more wrong than
reported (five awkward skills of six, three outright violations) and is
rewritten rather than renumbered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
Closes#2248.
Completes SEP-2640 support across all three clients, and closes the one
obligation #2234 deliberately left open.
- `checkSkillFrontmatterMatch` compares a served SKILL.md's own YAML
frontmatter against the entry the listing advertised, field by field. No
digest can cover this: a digest is taken over the bytes the server served,
so it proves the file was not altered in transit and says nothing about
whether the listing described it honestly. Needs a real YAML parser; `yaml`
was already a root dependency, so this adds no package — but it does newly
put it on core/'s import graph, hence the three bundler `external` lists.
- `resources/directory/read`: result schemas defined against the normative
text, `InspectorClient.readResourceDirectory`, a Directory section on the
Skills screen, and `--method resources/directory/read` in the CLI. The
client refuses the call locally when the server did not declare
`directoryRead`, which is the SEP's MUST NOT. A child the directory lists
but the entry does not is marked `not listed` rather than merged into the
manifest — the SEP calls a directory result a live observation and forbids
treating it as extending the manifest.
- CLI: `skills/list`, `skills/get` and `resources/directory/read`, plus
`--verify` — one NDJSON report per skill, a summary on stderr, exit 7 on a
violation. Reads are sequential; a conforming manifest may declare 512
entries.
- TUI: a Skills pane, shown only when the server declares the extension. Each
row carries its conformance verdict as a glyph as well as a colour, since
the pane is read over ssh and through script(1). Enter verifies.
- `verifySkills` re-throws `AuthRecoveryRequiredError` rather than recording
it per file: it says the session's authorization expired, so absorbing it
would report N identical read failures and swallow the error the TUI and
the web commands key off to reauthorize.
Two open questions settled, both in code comments where they will be found:
`skills/get` carries no caching attributes because SEP-2640 leaves the
question open in as many words, so requiring them would fail a conforming
server; and there is no `PagedSkillsState` because every consumer of this
list is a whole-catalog verdict — computed over page one of three, "this
server's skills conform" is not merely partial but wrong.
The fixture grows two skills, both for checks that were otherwise
undemonstrable: `lying-listing` (listing and file disagree, digest still
verifies) and `stale-manifest` (serves a file its manifest does not declare).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
cliffhall