feat: Skills extension phase 3 — CLI, TUI, directory reads, frontmatter check

Closes #2248.

Completes SEP-2640 support across all three clients, and closes the one
obligation #2234 deliberately left open.

- `checkSkillFrontmatterMatch` compares a served SKILL.md's own YAML
  frontmatter against the entry the listing advertised, field by field. No
  digest can cover this: a digest is taken over the bytes the server served,
  so it proves the file was not altered in transit and says nothing about
  whether the listing described it honestly. Needs a real YAML parser; `yaml`
  was already a root dependency, so this adds no package — but it does newly
  put it on core/'s import graph, hence the three bundler `external` lists.

- `resources/directory/read`: result schemas defined against the normative
  text, `InspectorClient.readResourceDirectory`, a Directory section on the
  Skills screen, and `--method resources/directory/read` in the CLI. The
  client refuses the call locally when the server did not declare
  `directoryRead`, which is the SEP's MUST NOT. A child the directory lists
  but the entry does not is marked `not listed` rather than merged into the
  manifest — the SEP calls a directory result a live observation and forbids
  treating it as extending the manifest.

- CLI: `skills/list`, `skills/get` and `resources/directory/read`, plus
  `--verify` — one NDJSON report per skill, a summary on stderr, exit 7 on a
  violation. Reads are sequential; a conforming manifest may declare 512
  entries.

- TUI: a Skills pane, shown only when the server declares the extension. Each
  row carries its conformance verdict as a glyph as well as a colour, since
  the pane is read over ssh and through script(1). Enter verifies.

- `verifySkills` re-throws `AuthRecoveryRequiredError` rather than recording
  it per file: it says the session's authorization expired, so absorbing it
  would report N identical read failures and swallow the error the TUI and
  the web commands key off to reauthorize.

Two open questions settled, both in code comments where they will be found:
`skills/get` carries no caching attributes because SEP-2640 leaves the
question open in as many words, so requiring them would fail a conforming
server; and there is no `PagedSkillsState` because every consumer of this
list is a whole-catalog verdict — computed over page one of three, "this
server's skills conform" is not merely partial but wrong.

The fixture grows two skills, both for checks that were otherwise
undemonstrable: `lying-listing` (listing and file disagree, digest still
verifies) and `stale-manifest` (serves a file its manifest does not declare).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-07 18:10:37 -04:00
commit 9a8488681e

View file

@ -3,6 +3,10 @@ import { InspectorClient } from "@inspector/core/mcp/inspectorClient.js";
import { createTransportNode } from "@inspector/core/mcp/node/transport.js";
import { getSkillsExtension } from "@inspector/core/mcp/skills.js";
import { ManagedSkillsState } from "@inspector/core/mcp/state/managedSkillsState.js";
import {
allSkillsVerified,
verifySkills,
} from "@inspector/core/mcp/skillsVerification.js";
import {
createTestServerHttp,
type TestServerHttp,
@ -96,9 +100,11 @@ describe("Skills extension over a real transport (#2234)", () => {
it("advertises the extension in its capabilities", async () => {
const started = await startSkillsServer(modern);
const connected = await connect(started.url, modern);
// Bare, per the fixture: no `directoryRead` until phase 3 serves it.
// `directoryRead` is declared because the fixture now serves the
// method (#2248) — the declaration and the handler are one switch, so
// this can never report a sub-option the server does not answer.
expect(getSkillsExtension(connected.getCapabilities())).toEqual({
directoryRead: false,
directoryRead: true,
});
});
@ -115,13 +121,18 @@ describe("Skills extension over a real transport (#2234)", () => {
// below — so a modern page missing the envelope surfaces here as a
// rejection rather than as a missing property.
//
// The fixture pages at two, so a client that stops here sees half.
// The fixture pages at two over five skills, so a client that stops
// here sees less than half.
expect(first.skills).toHaveLength(2);
expect(first.nextCursor).toBeDefined();
const second = await connected.listSkills(first.nextCursor);
expect(second.skills).toHaveLength(2);
expect(second.nextCursor).toBeUndefined();
expect(second.nextCursor).toBeDefined();
const third = await connected.listSkills(second.nextCursor);
expect(third.skills).toHaveLength(2);
expect(third.nextCursor).toBeUndefined();
});
it("walks every page through the managed store", async () => {
@ -134,9 +145,11 @@ describe("Skills extension over a real transport (#2234)", () => {
"data-analysis",
"tampered-notes",
"dynamic-report",
"stale-manifest",
"lying-listing",
"right-name",
]);
expect(store.getPagination()).toEqual({ pageCount: 2 });
expect(store.getPagination()).toEqual({ pageCount: 3 });
} finally {
store.destroy();
}
@ -171,6 +184,127 @@ describe("Skills extension over a real transport (#2234)", () => {
expect("text" in block && block.text).toContain("Column rules");
});
it("reads a directory and pages through its children", async () => {
// The whole `resources/directory/read` round trip against a real
// server: the client's `directoryRead` gate, the era-selected result
// schema, and the fixture's cursor.
const started = await startSkillsServer(modern);
const connected = await connect(started.url, modern);
const first = await connected.readResourceDirectory(
"skill://data-analysis",
);
// Pages at one child, so a client ignoring `nextCursor` is visibly
// wrong here rather than merely lucky.
expect(first.resources).toHaveLength(1);
expect(first.nextCursor).toBeDefined();
const second = await connected.readResourceDirectory(
"skill://data-analysis",
first.nextCursor,
);
expect(second.nextCursor).toBeUndefined();
expect(
[...first.resources, ...second.resources].map((r) => r.uri).sort(),
).toEqual([
"skill://data-analysis/SKILL.md",
"skill://data-analysis/reference.md",
]);
});
it("lists a dynamic skill's files, which is what the method is for", async () => {
// `dynamic-report` advertises no manifest, so a directory read is the
// only way its files are discoverable at all — the case SEP-2640 says
// directory reading earns its place for.
const started = await startSkillsServer(modern);
const connected = await connect(started.url, modern);
const page = await connected.readResourceDirectory(
"skill://dynamic-report",
);
expect(page.resources[0].uri).toBe("skill://dynamic-report/SKILL.md");
});
it("lists a file the entry's manifest does not declare", async () => {
// The stale-snapshot case SEP-2640 governs: a directory read is "a
// live observation" that may run ahead of the held entry, and hosts
// MUST NOT treat it as extending the manifest. The entry itself is
// fully conforming — only the two views disagree — so nothing but this
// comparison can surface it.
const started = await startSkillsServer(modern);
const connected = await connect(started.url, modern);
const entry = await connected.getSkill(
"skill://stale-manifest/SKILL.md",
);
const declared = new Set(
(entry.resources === "dynamic" ? [] : entry.resources).map(
(r) => r.uri,
),
);
expect(declared).toEqual(new Set(["skill://stale-manifest/SKILL.md"]));
const first = await connected.readResourceDirectory(
"skill://stale-manifest",
);
const second = await connected.readResourceDirectory(
"skill://stale-manifest",
first.nextCursor,
);
const children = [...first.resources, ...second.resources].map(
(r) => r.uri,
);
expect(children).toContain("skill://stale-manifest/added-later.md");
expect(declared.has("skill://stale-manifest/added-later.md")).toBe(
false,
);
// And the entry still verifies clean — the disagreement is the whole
// defect, and no digest check can see it.
const [report] = await verifySkills(connected, [entry]);
expect(report.ok).toBe(true);
});
it("answers -32602 for a URI that is not a directory resource", async () => {
const started = await startSkillsServer(modern);
const connected = await connect(started.url, modern);
await expect(
connected.readResourceDirectory("skill://data-analysis/SKILL.md"),
).rejects.toThrow(/Not a directory resource/);
});
it("verifies the whole catalog, failing exactly the three bad skills", async () => {
// End to end against the fixture: conformance, digests and the
// frontmatter cross-check, over a real transport. The three failures
// are one per violation class, and `dynamic-report` passing is the
// assertion that a warning does not fail a report.
const started = await startSkillsServer(modern);
const connected = await connect(started.url, modern);
const store = new ManagedSkillsState(connected);
try {
const skills = await store.refresh();
const reports = await verifySkills(connected, skills);
expect(reports.filter((r) => !r.ok).map((r) => r.name)).toEqual([
"tampered-notes",
"lying-listing",
"right-name",
]);
expect(allSkillsVerified(reports)).toBe(false);
const tampered = reports.find((r) => r.name === "tampered-notes")!;
expect(tampered.files.some((f) => f.status === "mismatch")).toBe(
true,
);
// The one violation only the frontmatter check can catch: its digest
// verifies, because the digest is over the bytes the server served.
const lying = reports.find((r) => r.name === "lying-listing")!;
expect(lying.files.every((f) => f.status === "verified")).toBe(true);
expect(lying.frontmatter[0].code).toBe("frontmatter-mismatch");
const dynamic = reports.find((r) => r.name === "dynamic-report")!;
expect(dynamic.ok).toBe(true);
} finally {
store.destroy();
}
});
it("still serves an ordinary resource — the wrapper delegates", async () => {
// The one thing the `resources/read` wrap must not break.
const started = await startSkillsServer(modern);