Commit graph inspector/clients/cli/__tests__/run-method-skills.test.ts
Author SHA1 Message Date
cliffhall
d6b1e36e7e fix: address Copilot review round 7 on #2248
Four findings.

**The CLI dropped what it promised to print.** `getSkill` transforms the
`skills/get` envelope to a `SkillEntry`, so every other member the loose
schema accepted was gone before the CLI saw it — including the `ttlMs` /
`cacheScope` SEP-2640 explicitly leaves open. The code contradicted a comment
two lines above it saying a CLI whose contract is "print the result" must not
reshape one. `getSkillResult` returns the envelope whole; `getSkill` is a
one-line unwrap over it, which matches the callers: the UIs want the entry,
the CLI wants the result.

**The frontmatter verdict and the digest came from different reads.** The
preview fetch and the Verify fetch are separate `resources/read` calls, so a
resource changing between them could pair a verified digest with a
frontmatter verdict for other bytes. `VerificationState.entryText` now holds
the SKILL.md as the verification read it, written in the SAME state update as
the verdict — a separate write was clobbered, because `write` returns a fresh
object and drops anything not named in it. The preview stays the fallback so
a reader who has not clicked Verify still gets the check.

**The Conformance badge excluded the frontmatter findings** it renders, so it
said `0 error(s)` above a red mismatch. Digest and size mismatches stay in
their own badge: "the listing is wrong" and "the bytes are wrong" are
different answers.

**A repeated skill URI collided two React keys** in the TUI list, in a pane
whose job is to show both entries. Index-keyed like the manifest rows.

The web fixture also gains its collision pair in the served-file lookup —
without it they were handed another skill's SKILL.md, which the badge fix
correctly began reporting as a frontmatter error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-07 20:44:51 -04:00
cliffhall
775107776d fix: address Copilot review round 1 on #2248
Five findings, all real.

**Critical — `verifySkills` selected `contents[0]`.** These bytes are hashed
against the file's advertised digest, so accepting a block the server
labelled something else verifies one file's content against another file's
digest, and can report that as `verified`. A false pass is worse than a
missing check. `contentsFor(result, uri)` now selects by NORMALIZED identity,
which keeps the canonicalized-echo case that motivated the original code
while rejecting an unrelated block, and reports a read error when nothing
answers for the URI. This is what `onReadSkillFile` already did; the two
paths hash bytes against digests and must not disagree about which bytes.

**`skills/get` was dispatched without the extension gate** that `skills/list`
had. Hoisted into `assertSkillsSupported` so the two cannot drift — declaring
the extension commits a server to both. Without it, an undeclared server's
-32601 is indistinguishable to a script from the -32602 a declared server
returns for a URI it does not serve.

**The TUI could strand a user on a hidden Skills tab.** The tab left the bar
when the gate went false but `activeTab` did not, so the pane kept rendering
for a server that never declared the extension. Reset to `info`, following
the Auth precedent — additionally gated on `connected`, because this gate
reads a server declaration and would otherwise fire during a reconnect.

**The TUI keyed a verdict by URI**, so a refresh replacing the manifest under
the same URI left hashes computed for the previous snapshot describing the
new one. Keyed on the entry now, as the web screen already was.

**A failed "Load more" discarded the pages on screen and the retry cursor.**
Both are preserved now, and the success and failure paths share one
staleness-guarded `commit` helper so they cannot drift on which results they
may write — which is how they came to disagree.

Two nits: the six-skill counts. The docs sentence was more wrong than
reported (five awkward skills of six, three outright violations) and is
rewritten rather than renumbered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-07 18:43:58 -04:00
cliffhall
9a8488681e feat: Skills extension phase 3 — CLI, TUI, directory reads, frontmatter check
Closes #2248.

Completes SEP-2640 support across all three clients, and closes the one
obligation #2234 deliberately left open.

- `checkSkillFrontmatterMatch` compares a served SKILL.md's own YAML
  frontmatter against the entry the listing advertised, field by field. No
  digest can cover this: a digest is taken over the bytes the server served,
  so it proves the file was not altered in transit and says nothing about
  whether the listing described it honestly. Needs a real YAML parser; `yaml`
  was already a root dependency, so this adds no package — but it does newly
  put it on core/'s import graph, hence the three bundler `external` lists.

- `resources/directory/read`: result schemas defined against the normative
  text, `InspectorClient.readResourceDirectory`, a Directory section on the
  Skills screen, and `--method resources/directory/read` in the CLI. The
  client refuses the call locally when the server did not declare
  `directoryRead`, which is the SEP's MUST NOT. A child the directory lists
  but the entry does not is marked `not listed` rather than merged into the
  manifest — the SEP calls a directory result a live observation and forbids
  treating it as extending the manifest.

- CLI: `skills/list`, `skills/get` and `resources/directory/read`, plus
  `--verify` — one NDJSON report per skill, a summary on stderr, exit 7 on a
  violation. Reads are sequential; a conforming manifest may declare 512
  entries.

- TUI: a Skills pane, shown only when the server declares the extension. Each
  row carries its conformance verdict as a glyph as well as a colour, since
  the pane is read over ssh and through script(1). Enter verifies.

- `verifySkills` re-throws `AuthRecoveryRequiredError` rather than recording
  it per file: it says the session's authorization expired, so absorbing it
  would report N identical read failures and swallow the error the TUI and
  the web commands key off to reauthorize.

Two open questions settled, both in code comments where they will be found:
`skills/get` carries no caching attributes because SEP-2640 leaves the
question open in as many words, so requiring them would fail a conforming
server; and there is no `PagedSkillsState` because every consumer of this
list is a whole-catalog verdict — computed over page one of three, "this
server's skills conform" is not merely partial but wrong.

The fixture grows two skills, both for checks that were otherwise
undemonstrable: `lying-listing` (listing and file disagree, digest still
verifies) and `stale-manifest` (serves a file its manifest does not declare).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-07 18:10:37 -04:00