modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

feat: Skills extension phase 3 — CLI, TUI, directory reads, frontmatter check

Browse source 
Closes #2248.

Completes SEP-2640 support across all three clients, and closes the one
obligation #2234 deliberately left open.

- `checkSkillFrontmatterMatch` compares a served SKILL.md's own YAML
  frontmatter against the entry the listing advertised, field by field. No
  digest can cover this: a digest is taken over the bytes the server served,
  so it proves the file was not altered in transit and says nothing about
  whether the listing described it honestly. Needs a real YAML parser; `yaml`
  was already a root dependency, so this adds no package — but it does newly
  put it on core/'s import graph, hence the three bundler `external` lists.

- `resources/directory/read`: result schemas defined against the normative
  text, `InspectorClient.readResourceDirectory`, a Directory section on the
  Skills screen, and `--method resources/directory/read` in the CLI. The
  client refuses the call locally when the server did not declare
  `directoryRead`, which is the SEP's MUST NOT. A child the directory lists
  but the entry does not is marked `not listed` rather than merged into the
  manifest — the SEP calls a directory result a live observation and forbids
  treating it as extending the manifest.

- CLI: `skills/list`, `skills/get` and `resources/directory/read`, plus
  `--verify` — one NDJSON report per skill, a summary on stderr, exit 7 on a
  violation. Reads are sequential; a conforming manifest may declare 512
  entries.

- TUI: a Skills pane, shown only when the server declares the extension. Each
  row carries its conformance verdict as a glyph as well as a colour, since
  the pane is read over ssh and through script(1). Enter verifies.

- `verifySkills` re-throws `AuthRecoveryRequiredError` rather than recording
  it per file: it says the session's authorization expired, so absorbing it
  would report N identical read failures and swallow the error the TUI and
  the web commands key off to reauthorize.

Two open questions settled, both in code comments where they will be found:
`skills/get` carries no caching attributes because SEP-2640 leaves the
question open in as many words, so requiring them would fail a conforming
server; and there is no `PagedSkillsState` because every consumer of this
list is a whole-catalog verdict — computed over page one of three, "this
server's skills conform" is not merely partial but wrong.

The fixture grows two skills, both for checks that were otherwise
undemonstrable: `lying-listing` (listing and file disagree, digest still
verifies) and `stale-manifest` (serves a file its manifest does not declare).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-07 18:10:37 -04:00
commit 9a8488681e
10 changed files with 4750 additions and 191 deletions
Download patch file Download diff file Expand all files Collapse all files

66
clients/cli/README.md
View file

@ -108,11 +108,12 @@ Options that specify the MCP server (catalog/config file, ad-hoc command/URL, en
| Option | Description |
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--method <method>` | MCP method to invoke. Supports `initialize` (connect-only probe → `{serverInfo, protocolVersion, capabilities, instructions}`), `tools/list`, `tools/call`, `resources/list`, `resources/read`, `resources/templates/list`, `prompts/list`, `prompts/get`, `logging/setLevel`, plus catalog-only `servers/list` / `servers/show` (no MCP connect). Stream / session-only methods (e.g. `logging/tail`) are rejected. |
| `--method <method>` | MCP method to invoke. Supports `initialize` (connect-only probe → `{serverInfo, protocolVersion, capabilities, instructions}`), `tools/list`, `tools/call`, `resources/list`, `resources/read`, `resources/templates/list`, `prompts/list`, `prompts/get`, `logging/setLevel`, `skills/list`, `skills/get`, `resources/directory/read`, plus catalog-only `servers/list` / `servers/show` (no MCP connect). Stream / session-only methods (e.g. `logging/tail`) are rejected. |
| `--tool-name <name>` | Tool name (for `tools/call`). |
| `--tool-arg <key=value>` | Tool argument; repeat for multiple. Use `key='{"json":true}'` for JSON. Values are coerced (JSON-parsed, so `count=1` becomes a number). |
| `--tool-args-json <json>` | Tool arguments as a single JSON object (e.g. `'{"zip":"10001"}'`). Passed verbatim — no `key=value` coercion, so `"012"` stays a string. Mutually exclusive with `--tool-arg`. |
| `--uri <uri>` | Resource URI (for `resources/read`). |
| `--uri <uri>` | Resource URI (`resources/read`), directory URI (`resources/directory/read`), or skill URI (`skills/get`). |
| `--cursor <cursor>` | Opaque pagination cursor for `resources/directory/read` — pass back the `nextCursor` from the previous page. The listing is not recursive and pages are not aggregated: SEP-2640 gives the cursor to the client, and descending is the caller's job. |
| `--prompt-name <name>` | Prompt name (for `prompts/get`). |
| `--prompt-args <key=value>` | Prompt arguments; repeat for multiple. |
| `--log-level <level>` | Logging level for `logging/setLevel` (e.g. `debug`, `info`). |
@ -121,6 +122,7 @@ Options that specify the MCP server (catalog/config file, ad-hoc command/URL, en
| `--connect-timeout <ms>` | Connection timeout in ms. Defaults to `15000` for ad-hoc `--server-url`/target runs (so a black-holed host fails fast) and to the file-level timeout for `--catalog`/`--config` runs. `0` disables the timeout. |
| `--app-info` | Probe a tool's MCP App UI metadata without invoking it. With `--method tools/call --tool-name <name>`: prints one JSON line (`hasApp`, `resourceUri`, `csp`, `permissions`, `domain`, …) and exits `0` if the tool has an app or `2` (`no_app`) if not. With `--method tools/list`: emits NDJSON — one app-info line per tool over a single connection. |
| `--strict` | With `--method tools/list`: report tool-schema portability problems in full (path, issue, suggested fix) on stderr, and exit `6` if any is error-severity. Without it, a one-line count is printed instead. See [Schema portability](#schema-portability---strict). |
| `--verify` | With `--method skills/list` or `--method skills/get`: run the SEP-2640 conformance, digest and frontmatter checks over the skills returned, emit one JSON report per skill on stdout, and exit `7` if any fails. See [Skill verification](#skill-verification---verify). |
| `--format <text\|json>` | Output format. `text` (default) pretty-prints the result. `json` emits a single JSON object on stdout (`{ "result": … }`, plus `{ "appInfo": … }` as a sibling key for App tools) with no banners, so the whole output pipes cleanly into `jq`. |
| `--relogin` | Delete stored OAuth for this server URL from the shared store before connect; interactive login still only runs if the server requires auth. Requires an HTTP/SSE URL (rejected for stdio). Conflicts with `--stored-auth-only` / `--use-stored-auth` / `--wait-for-auth` / catalog short-circuits. |
| `--no-revoke` | With `--relogin`, skip the [RFC 7009](https://datatracker.ietf.org/doc/html/rfc7009) revocation request that would otherwise end the grant at the authorization server when the local state is deleted. The per-server `oauth.revokeOnClear` setting is the persistent form of the same opt-out; either one is enough to skip it. See [Revoking on `--relogin`](#revoking-on---relogin). |
@ -333,6 +335,65 @@ mcp-inspector --cli --transport http --server-url https://api.example/mcp \
--wait-for-auth 120 --method tools/list
```
#### Skill verification (`--verify`)
SEP-2640 puts real obligations on whoever consumes a skill: verify each fetched
file against the digest its manifest advertised, check that the served
`SKILL.md`'s frontmatter matches the one the listing advertised, and honour the
per-skill limits. `--verify` runs all of them over a whole catalog and turns the
answer into an exit code, so a server author can gate CI on it:
```sh
mcp-inspector --cli <server> --method skills/list --verify
```
Stdout is **NDJSON, one report per skill**, in listing order:
```json
{
"uri": "skill://tampered-notes/SKILL.md",
"name": "tampered-notes",
"conformance": [],
"frontmatter": [],
"files": [
{ "uri": "skill://tampered-notes/SKILL.md", "status": "verified", "…": "…" },
{ "uri": "skill://tampered-notes/notes.md", "status": "mismatch", "…": "…" }
],
"ok": false
}
```
Stderr gets a one-line summary, so a reader who piped stdout into `jq` still
sees the verdict. `--method skills/get --uri <skill>` verifies exactly one
skill, in the same shape.
**What fails the run.** `ok` is false — and the exit code is `7` — for anything
SEP-2640 makes a MUST: an error-severity conformance finding, a digest or size
mismatch, or a manifest file that could not be read. A **warning** does not fail
it. That distinction matters most for `resources: "dynamic"`, which is a
*conforming* wire form for generated content: it means integrity cannot be
verified, which is worth reporting, but failing CI for it would tell server
authors their valid skill is broken.
**Three checks, three different jobs**, and the second is the one nothing else
covers:
- **Conformance** — structural checks against the entry as listed (name grammar,
the name/URI invariant, digest and size formats, manifest completeness, the
interoperability limits).
- **Frontmatter** — the served `SKILL.md`'s own YAML frontmatter, compared field
by field against the frontmatter the listing advertised. A digest cannot cover
this: it is taken over the bytes the server served, so it proves the file was
not altered in transit and says nothing about whether the *listing* described
it honestly. A server can advertise one description, serve another, and pass
every digest check.
- **Files** — each manifest entry fetched and hashed. Reads are sequential: a
conforming manifest may declare 512 entries, and a parallel walk would open
512 `resources/read` calls against the server under test.
A read failure is recorded against the file it happened on and the walk
continues, so one unreadable file never hides the findings after it.
## Exit codes & error envelopes
Every non-zero exit maps to a stable failure class, so a programmatic caller
@ -348,6 +409,7 @@ prose from stderr:
| `4` | Server unreachable (DNS, connection refused, timeout, `fetch failed`). |
| `5` | Tool error (`tools/call` returned `isError:true`, or the tool was not found). |
| `6` | `--strict` found an error-severity tool-schema portability problem (`schema_unportable` — the schema is valid JSON Schema, just not portable). |
| `7` | `--verify` found a SEP-2640 violation (`skills_nonconformant` — a conformance error, a digest or size mismatch, or an unreadable manifest file). |
On any non-zero exit the CLI also writes a single JSON line to **stderr** — the
`ErrorEnvelope`:

222
clients/cli/__tests__/run-method-skills.test.ts Normal file
View file

@ -0,0 +1,222 @@
import { describe, it, expect, vi } from "vitest";
import { runMethod } from "../src/handlers/run-method.js";
import { summarizeSkillVerification } from "../src/handlers/skills-verify.js";
import { EXIT_CODES } from "../src/error-handler.js";
import type { InspectorClient } from "@inspector/core/mcp/index.js";
import type { SkillEntry } from "@inspector/core/mcp/skillsSchemas.js";
import { sha256Digest } from "@inspector/core/mcp/skills.js";
import type { SkillVerifyReport } from "@inspector/core/mcp/skillsVerification.js";
/**
* The three SEP-2640 methods the CLI gained in #2248, plus `--verify`.
*
* The store's cursor walk and the verification checks are covered where they
* live (`managedSkillsState.test.ts`, `skillsVerification.test.ts`); what these
* pin is the dispatcher's own decisions — which method reaches which client
* call, what shape leaves as a result, and when the report sets a non-zero exit
* code.
*/
const SKILL_MD = "---\nname: demo\ndescription: A demo\n---\n\n# Demo\n";
async function cleanEntry(): Promise<SkillEntry> {
const bytes = new TextEncoder().encode(SKILL_MD);
return {
uri: "skill://demo/SKILL.md",
frontmatter: { name: "demo", description: "A demo" },
resources: [
{
uri: "skill://demo/SKILL.md",
digest: await sha256Digest(bytes),
size: bytes.byteLength,
},
],
};
}
function mockClient(overrides: Record<string, unknown> = {}): InspectorClient {
return {
addEventListener: vi.fn(),
removeEventListener: vi.fn(),
getStatus: vi.fn().mockReturnValue("connected"),
getSkillsExtension: vi.fn().mockReturnValue({ directoryRead: true }),
listSkills: vi.fn().mockResolvedValue({ skills: [] }),
getSkill: vi.fn(),
readResourceDirectory: vi.fn(),
readResource: vi.fn().mockResolvedValue({
result: { contents: [{ uri: "skill://demo/SKILL.md", text: SKILL_MD }] },
}),
...overrides,
} as unknown as InspectorClient;
}
describe("runMethod skills dispatch (#2248)", () => {
it("returns the walked list for skills/list", async () => {
const entry = await cleanEntry();
const client = mockClient({
listSkills: vi.fn().mockResolvedValue({ skills: [entry] }),
});
const outcome = await runMethod(client, { method: "skills/list" });
expect(outcome).toEqual({
kind: "result",
result: { skills: [entry] },
appInfo: undefined,
});
});
it("rejects skills/list with a usage exit code when the server declares no extension", async () => {
// The store answers "no extension" with an empty list, which is right for
// a UI that must render something and wrong for a CLI: "no skills" and
// "does not serve skills" are answers a script has to tell apart.
const client = mockClient({
getSkillsExtension: vi.fn().mockReturnValue(undefined),
});
await expect(runMethod(client, { method: "skills/list" })).rejects.toThrow(
/does not declare/i,
);
await expect(
runMethod(client, { method: "skills/list" }),
).rejects.toMatchObject({ exitCode: EXIT_CODES.USAGE });
});
it("keeps the { skill } envelope on skills/get", async () => {
// The client unwraps it for callers that want the entry; a CLI whose
// contract is "print the result" must not quietly reshape the wire form.
const entry = await cleanEntry();
const client = mockClient({ getSkill: vi.fn().mockResolvedValue(entry) });
const outcome = await runMethod(client, {
method: "skills/get",
uri: entry.uri,
});
expect(outcome).toMatchObject({ result: { skill: entry } });
});
it("requires --uri for skills/get", async () => {
await expect(
runMethod(mockClient(), { method: "skills/get" }),
).rejects.toThrow(/URI is required/);
});
it("requires --uri for resources/directory/read", async () => {
await expect(
runMethod(mockClient(), { method: "resources/directory/read" }),
).rejects.toThrow(/URI is required/);
});
it("returns one page of resources/directory/read and forwards the cursor", async () => {
// One page, not a walk: the SEP says the listing is not recursive and the
// client descends, so aggregating here would present a subtree as a
// directory.
const page = { resources: [], nextCursor: "2" };
const readResourceDirectory = vi.fn().mockResolvedValue(page);
const client = mockClient({ readResourceDirectory });
const outcome = await runMethod(client, {
method: "resources/directory/read",
uri: "skill://demo",
cursor: "1",
});
expect(readResourceDirectory).toHaveBeenCalledWith(
"skill://demo",
"1",
undefined,
);
expect(outcome).toMatchObject({ result: page });
});
it("--verify emits one NDJSON report per skill with no exit code when clean", async () => {
const entry = await cleanEntry();
const client = mockClient({
listSkills: vi.fn().mockResolvedValue({ skills: [entry] }),
});
const outcome = await runMethod(client, {
method: "skills/list",
verify: true,
});
expect(outcome.kind).toBe("ndjson");
if (outcome.kind !== "ndjson") throw new Error("unreachable");
expect(outcome.lines).toHaveLength(1);
expect((outcome.lines[0] as SkillVerifyReport).ok).toBe(true);
expect(outcome.summary).toMatch(/no conformance errors/);
expect(outcome.exitCode).toBeUndefined();
});
it("--verify sets the skills exit code when a skill fails", async () => {
const entry = await cleanEntry();
const client = mockClient({
listSkills: vi.fn().mockResolvedValue({ skills: [entry] }),
readResource: vi.fn().mockResolvedValue({
result: {
contents: [{ uri: entry.uri, text: "totally different bytes" }],
},
}),
});
const outcome = await runMethod(client, {
method: "skills/list",
verify: true,
});
if (outcome.kind !== "ndjson") throw new Error("unreachable");
expect(outcome.exitCode).toBe(EXIT_CODES.SKILL_NONCONFORMANT);
// Its own code, not SCHEMA_UNPORTABLE — an unportable tool schema and a
// tampered skill digest are different CI failures.
expect(EXIT_CODES.SKILL_NONCONFORMANT).not.toBe(
EXIT_CODES.SCHEMA_UNPORTABLE,
);
});
it("--verify works on a single skills/get", async () => {
const entry = await cleanEntry();
const client = mockClient({ getSkill: vi.fn().mockResolvedValue(entry) });
const outcome = await runMethod(client, {
method: "skills/get",
uri: entry.uri,
verify: true,
});
expect(outcome.kind).toBe("ndjson");
if (outcome.kind !== "ndjson") throw new Error("unreachable");
expect(outcome.lines).toHaveLength(1);
});
});
describe("summarizeSkillVerification (#2248)", () => {
const report = (
over: Partial<SkillVerifyReport> = {},
): SkillVerifyReport => ({
uri: "skill://demo/SKILL.md",
name: "demo",
conformance: [],
frontmatter: [],
files: [{ uri: "skill://demo/SKILL.md", status: "verified" }],
ok: true,
...over,
});
it("reports a clean run with singular wording for one skill", () => {
expect(summarizeSkillVerification([report()])).toBe(
"Verified 1 skill and 1 file: no conformance errors.",
);
});
it("pluralizes for more than one", () => {
expect(summarizeSkillVerification([report(), report()])).toBe(
"Verified 2 skills and 2 files: no conformance errors.",
);
});
it("counts failures and digest mismatches separately", () => {
// A skill can fail on a conformance error with no mismatched file at all,
// so collapsing the two counts would misreport the cause.
const failed = report({
ok: false,
files: [{ uri: "skill://demo/SKILL.md", status: "mismatch" }],
});
expect(summarizeSkillVerification([report(), failed])).toBe(
"1 of 2 skills failed verification (1 digest/size mismatch across 2 files).",
);
});
it("reports a failure with no mismatched file", () => {
const failed = report({ ok: false, files: [] });
expect(summarizeSkillVerification([failed])).toBe(
"1 of 1 skill failed verification (0 digest/size mismatch across 0 files).",
);
});
});

150
clients/cli/__tests__/skills-verify-cli.test.ts Normal file
View file

@ -0,0 +1,150 @@
import { describe, it, expect } from "vitest";
import { runCli } from "../src/cli.js";
import { consumeMethodOutcome } from "../src/handlers/consume-outcome.js";
import { EXIT_CODES } from "../src/error-handler.js";
/**
* `--verify`'s argument validation and its NDJSON consumption path (#2248).
*
* The validation sits with `--strict`'s, ahead of every short-circuit return in
* `parseArgs`, for the same reason: the returns below it never reach
* `runMethod`, so a check placed further down would let the flag be accepted
* and then silently ignored.
*/
describe("--verify argument validation", () => {
it("is rejected with a method other than skills/list or skills/get", async () => {
await expect(
runCli([
"node",
"cli",
"--cli",
"--method",
"tools/list",
"--verify",
"--server-url",
"http://127.0.0.1:1/mcp",
]),
).rejects.toThrow(
"--verify requires --method skills/list or --method skills/get.",
);
});
it.each([
["servers/list", ["--method", "servers/list"]],
["--list-stored-auth", ["--method", "servers/list", "--list-stored-auth"]],
])(
"is rejected on the %s short-circuit path, which never reaches the report",
async (_label, extra) => {
await expect(
runCli(["node", "cli", "--cli", "--verify", ...extra]),
).rejects.toThrow(
"--verify requires --method skills/list or --method skills/get.",
);
},
);
it("is accepted with skills/get", async () => {
// Reaches the connect and fails there — which is the point: the flag
// itself was not what was rejected.
await expect(
runCli([
"node",
"cli",
"--cli",
"--method",
"skills/get",
"--uri",
"skill://demo/SKILL.md",
"--verify",
"--server-url",
"http://127.0.0.1:1/mcp",
]),
).rejects.not.toThrow(/--verify requires/);
});
});
describe("consumeMethodOutcome NDJSON summary and exit code (#2248)", () => {
function captureStreams() {
let stdout = "";
let stderr = "";
const write = (sink: (s: string) => void) =>
((chunk: unknown, ...rest: unknown[]) => {
sink(typeof chunk === "string" ? chunk : String(chunk));
const cb = rest.find((r) => typeof r === "function") as
| (() => void)
| undefined;
cb?.();
return true;
}) as typeof process.stdout.write;
const originalOut = process.stdout.write;
const originalErr = process.stderr.write;
process.stdout.write = write((s) => (stdout += s));
process.stderr.write = write((s) => (stderr += s));
return {
get stdout() {
return stdout;
},
get stderr() {
return stderr;
},
restore() {
process.stdout.write = originalOut;
process.stderr.write = originalErr;
},
};
}
it("writes the summary to stderr so it cannot contaminate the NDJSON", async () => {
const streams = captureStreams();
try {
await consumeMethodOutcome(
{ kind: "ndjson", lines: [{ ok: true }], summary: "all good" },
{},
);
} finally {
streams.restore();
}
expect(JSON.parse(streams.stdout.trim())).toEqual({ ok: true });
expect(streams.stderr).toBe("all good\n");
});
it("throws the exit code AFTER writing the report", async () => {
// The report is the output a CI job reads; failing before writing it would
// give the reader an exit code and nothing to act on.
const streams = captureStreams();
let thrown: unknown;
try {
await consumeMethodOutcome(
{
kind: "ndjson",
lines: [{ ok: false }],
summary: "one failed",
exitCode: EXIT_CODES.SKILL_NONCONFORMANT,
},
{},
);
} catch (err) {
thrown = err;
} finally {
streams.restore();
}
expect(streams.stdout.trim()).toBe('{"ok":false}');
expect(thrown).toMatchObject({
exitCode: EXIT_CODES.SKILL_NONCONFORMANT,
envelope: { code: "skills_nonconformant" },
});
});
it("leaves an --app-info NDJSON outcome unchanged", async () => {
// No summary, no exit code — the field is additive and the older caller
// must behave exactly as before.
const streams = captureStreams();
try {
await consumeMethodOutcome({ kind: "ndjson", lines: [{ a: 1 }] }, {});
} finally {
streams.restore();
}
expect(streams.stderr).toBe("");
expect(streams.stdout.trim()).toBe('{"a":1}');
});
});

29
clients/cli/src/cli.ts
View file

@ -679,7 +679,14 @@ async function parseArgs(argv?: string[]): Promise<ParseResult> {
parseKeyValuePair,
{},
)
.option("--uri <uri>", "URI of the resource (for resources/read method)")
.option(
"--uri <uri>",
"URI of the resource (resources/read, resources/directory/read) or of the skill (skills/get)",
)
.option(
"--cursor <cursor>",
"Opaque pagination cursor (for resources/directory/read; pass back the nextCursor from the previous page).",
)
.option(
"--prompt-name <promptName>",
"Name of the prompt (for prompts/get method)",
@ -743,6 +750,10 @@ async function parseArgs(argv?: string[]): Promise<ParseResult> {
"--strict",
"Report tool-schema portability problems in full (path, issue, suggested fix) on stderr, and exit 6 if any is error-severity. Use with --method tools/list. Without it, a one-line count is printed instead.",
)
.option(
"--verify",
"Run the SEP-2640 conformance and digest checks over the skills returned, emit one JSON report per skill on stdout, and exit 7 if any fails. Use with --method skills/list or --method skills/get.",
)
.option(
"--connect-timeout <ms>",
`Connection timeout in ms (default ${DEFAULT_CONNECT_TIMEOUT_MS} for ad-hoc --server-url / target invocations; 0 = no timeout).`,
@ -848,6 +859,8 @@ async function parseArgs(argv?: string[]): Promise<ParseResult> {
header?: Record<string, string>;
appInfo?: boolean;
strict?: boolean;
verify?: boolean;
cursor?: string;
connectTimeout?: number;
format?: OutputFormat;
toolArgsJson?: string;
@ -919,6 +932,18 @@ async function parseArgs(argv?: string[]): Promise<ParseResult> {
}
}
// `--verify` is checked here for exactly the reason `--strict` is: the
// short-circuit returns below never reach `runMethod`, so validating further
// down would let `--verify --method servers/list` succeed while silently
// ignoring a flag documented as skills-only.
if (options.verify) {
if (options.method !== "skills/list" && options.method !== "skills/get") {
throw new Error(
"--verify requires --method skills/list or --method skills/get.",
);
}
}
// State-path precedence (getStateFilePath): MCP_INSPECTOR_OAUTH_STATE_PATH →
// <MCP_STORAGE_DIR>/oauth.json → ~/.mcp-inspector/storage/oauth.json — the
// same file the web backend writes, so tokens are shared across surfaces.
@ -1147,6 +1172,8 @@ async function parseArgs(argv?: string[]): Promise<ParseResult> {
toolMeta: options.toolMetadata,
appInfo: options.appInfo === true,
strict: options.strict === true,
verify: options.verify === true,
cursor: options.cursor,
format: options.format,
};

10
clients/cli/src/error-handler.ts
View file

@ -25,6 +25,16 @@ export const EXIT_CODES = {
UNREACHABLE: 4,
TOOL_ERROR: 5,
SCHEMA_UNPORTABLE: 6,
/**
* `--verify` found a SEP-2640 violation: a conformance error, a digest or
* size mismatch, or a manifest file that could not be read (#2248).
*
* Its own code rather than reusing `SCHEMA_UNPORTABLE`, for the reason that
* one exists at all: a CI job that fails on an unportable tool schema and a
* CI job that fails on a tampered skill digest are different jobs, and
* collapsing them would make `if [ $? -eq 6 ]` ambiguous.
*/
SKILL_NONCONFORMANT: 7,
} as const;
/** Machine-readable error envelope written as one JSON line on stderr. */

14
clients/cli/src/handlers/consume-outcome.ts
View file

@ -1,4 +1,5 @@
import { awaitableLog } from "../utils/awaitable-log.js";
import { awaitableError, awaitableLog } from "../utils/awaitable-log.js";
import { CliExitCodeError } from "../error-handler.js";
import { emitResult } from "./emit-result.js";
import type { MethodArgs, MethodOutcome } from "./method-types.js";
@ -21,6 +22,17 @@ export async function consumeMethodOutcome(
for (const line of outcome.lines) {
await awaitableLog(JSON.stringify(line) + "\n");
}
// Summary on **stderr**, after the report, so it cannot contaminate the
// NDJSON a consumer is parsing on stdout.
if (outcome.summary) await awaitableError(`${outcome.summary}\n`);
// Thrown rather than returned so it routes through the CLI's single exit
// path — the report has already been written, which is why this is the
// last thing that happens.
if (outcome.exitCode) {
throw new CliExitCodeError(outcome.exitCode, outcome.summary ?? "", {
code: "skills_nonconformant",
});
}
return;
}

34
clients/cli/src/handlers/method-types.ts
View file

@ -34,6 +34,18 @@ export type MethodArgs = {
taskId?: string;
/** When true, tools/call uses callToolStream (task-augmented). */
task?: boolean;
/**
* `--verify`: run the SEP-2640 conformance and digest checks over the skills
* a `skills/list` / `skills/get` returned, emit one NDJSON report per skill,
* and exit non-zero when any fails (#2248).
*/
verify?: boolean;
/**
* Opaque pagination cursor. Used by `resources/directory/read`, whose result
* pages exactly as `resources/list` does — and where the caller descends the
* tree itself, so there is no store to walk it.
*/
cursor?: string;
/** roots/set payload (JSON array of {uri, name?}). */
rootsJson?: string;
/** prompts/complete: argument name / value / ref. */
@ -53,7 +65,18 @@ export type McpResponse = Record<string, unknown>;
export type MethodOutcome =
| { kind: "result"; result: McpResponse; appInfo?: CliAppInfo }
/** One JSON object per line (e.g. tools/list --app-info). Caller writes stdout. */
| { kind: "ndjson"; lines: unknown[] }
| {
kind: "ndjson";
lines: unknown[];
/**
* A line for **stderr**, written after the NDJSON. `--verify` uses it for
* its one-line summary, so a reader who piped stdout into `jq` still sees
* the verdict; `--app-info` sets nothing and behaves as before.
*/
summary?: string;
/** Non-zero when the emitted report is itself a failure (`--verify`). */
exitCode?: number;
}
| {
kind: "stream";
/** Human label for errors. */
@ -76,6 +99,7 @@ export const SESSION_RPC_METHODS = [
"resources/list",
"resources/read",
"resources/templates/list",
"resources/directory/read",
"resources/subscribe",
"resources/unsubscribe",
"prompts/list",
@ -89,6 +113,8 @@ export const SESSION_RPC_METHODS = [
"tasks/result",
"roots/list",
"roots/set",
"skills/list",
"skills/get",
] as const;
export type SessionRpcMethod = (typeof SESSION_RPC_METHODS)[number];
@ -109,6 +135,12 @@ export const ONE_SHOT_METHODS = [
"prompts/list",
"prompts/get",
"logging/setLevel",
// SEP-2640. All three are ordinary one-shot request/response calls — no
// stream, no long-lived subscription — so they belong here alongside the
// other list verbs rather than being reachable only from the session CLI.
"skills/list",
"skills/get",
"resources/directory/read",
] as const;
export type OneShotMethod = (typeof ONE_SHOT_METHODS)[number];

84
clients/cli/src/handlers/run-method.ts
View file

@ -6,10 +6,17 @@ import {
ManagedResourceTemplatesState,
ManagedPromptsState,
ManagedRequestorTasksState,
ManagedSkillsState,
MessageLogState,
} from "@inspector/core/mcp/state/index.js";
import { SKILLS_EXTENSION_KEY } from "@inspector/core/mcp/skillsSchemas.js";
import { CliExitCodeError, EXIT_CODES } from "../error-handler.js";
import { collectAppInfo } from "./collect-app-info.js";
import { summarizeSkillVerification } from "./skills-verify.js";
import {
allSkillsVerified,
verifySkills,
} from "@inspector/core/mcp/skillsVerification.js";
import type {
CliAppInfo,
McpResponse,
@ -35,6 +42,7 @@ export async function runMethod(
null;
let managedPromptsState: ManagedPromptsState | null = null;
let managedTasksState: ManagedRequestorTasksState | null = null;
let managedSkillsState: ManagedSkillsState | null = null;
try {
let result: McpResponse;
@ -283,6 +291,81 @@ export async function runMethod(
result = (await inspectorClient.getRequestorTaskResult(
args.taskId,
)) as McpResponse;
} else if (args.method === "skills/list") {
// The store's cursor walk is reused rather than re-implemented — it
// carries the repeated-cursor and page-cap guards, and a second copy of
// a pagination walk is how the two come to disagree. What the CLI adds
// is the check below: the store answers "no extension" with an empty
// list, which is right for a UI that must render *something*, and wrong
// for a CLI where "this server has no skills" and "this server does not
// serve skills at all" are different answers a script has to tell apart.
if (!inspectorClient.getSkillsExtension()) {
throw new CliExitCodeError(
EXIT_CODES.USAGE,
`Server does not declare the ${SKILLS_EXTENSION_KEY} extension, so ${args.method} is not available.`,
{ code: "skills_unsupported" },
);
}
managedSkillsState = new ManagedSkillsState(inspectorClient);
const skills = await managedSkillsState.refresh(args.metadata);
if (args.verify) {
const reports = await verifySkills(
inspectorClient,
skills,
args.metadata,
);
return {
kind: "ndjson",
lines: reports,
summary: summarizeSkillVerification(reports),
...(allSkillsVerified(reports)
? {}
: { exitCode: EXIT_CODES.SKILL_NONCONFORMANT }),
};
}
result = { skills };
} else if (args.method === "skills/get") {
if (!args.uri) {
throw new Error(
"URI is required for skills/get method. Use --uri to specify the skill URI.",
);
}
const skill = await inspectorClient.getSkill(args.uri, args.metadata);
if (args.verify) {
const reports = await verifySkills(
inspectorClient,
[skill],
args.metadata,
);
return {
kind: "ndjson",
lines: reports,
summary: summarizeSkillVerification(reports),
...(allSkillsVerified(reports)
? {}
: { exitCode: EXIT_CODES.SKILL_NONCONFORMANT }),
};
}
// The `{ skill }` envelope is restored here because it is what the wire
// carries: `GetSkillResultSchema` unwraps it for callers that want the
// entry, and a CLI whose contract is "print the result" must not quietly
// reshape one.
result = { skill };
} else if (args.method === "resources/directory/read") {
if (!args.uri) {
throw new Error(
"URI is required for resources/directory/read. Use --uri to specify the directory URI.",
);
}
// One page, not a walk. SEP-2640 says the listing is not recursive and
// clients descend by calling again on a child, so aggregating pages here
// would present a subtree as a directory — and the cursor is exposed as
// `--cursor` precisely so a script can do the descending.
result = await inspectorClient.readResourceDirectory(
args.uri,
args.cursor,
args.metadata,
);
} else if (args.method === "roots/list") {
result = { roots: inspectorClient.getRoots() };
} else if (args.method === "roots/set") {
@ -318,6 +401,7 @@ export async function runMethod(
managedResourcesState?.destroy();
managedResourceTemplatesState?.destroy();
managedPromptsState?.destroy();
managedSkillsState?.destroy();
managedTasksState?.destroy();
}
}

36
clients/cli/src/handlers/skills-verify.ts Normal file
View file

@ -0,0 +1,36 @@
/**
* `--verify`: the scriptable SEP-2640 conformance report (#2248).
*
* The Skills screen in the web client can verify a skill, but only by hand, one
* file at a time, in a browser. A server author wants the same verdict in CI,
* over the whole catalog, with an exit code — which is exactly the argument
* `--strict` makes for the tool-schema lint, so this follows that handler's
* shape rather than inventing a second one.
*
* The walk itself is `core/mcp/skillsVerification.ts`, shared with the TUI's
* Skills pane. What is left here is presentation: the one-line stderr summary,
* which is a CLI concern and nothing else's.
*/
import type { SkillVerifyReport } from "@inspector/core/mcp/skillsVerification.js";
/**
* A one-line human summary for stderr, so a reader who piped stdout to `jq`
* still learns the verdict.
*/
export function summarizeSkillVerification(
reports: readonly SkillVerifyReport[],
): string {
const failed = reports.filter((report) => !report.ok).length;
const files = reports.reduce((sum, report) => sum + report.files.length, 0);
const mismatched = reports.reduce(
(sum, report) =>
sum + report.files.filter((file) => file.status === "mismatch").length,
0,
);
const skillWord = reports.length === 1 ? "skill" : "skills";
const fileWord = files === 1 ? "file" : "files";
return failed === 0
? `Verified ${reports.length} ${skillWord} and ${files} ${fileWord}: no conformance errors.`
: `${failed} of ${reports.length} ${skillWord} failed verification (${mismatched} digest/size mismatch across ${files} ${fileWord}).`;
}

7
clients/cli/tsup.config.ts
View file

@ -55,6 +55,13 @@ export default defineConfig({
"atomically",
"open",
"zod",
// Newly on `core/`'s runtime import graph as of #2248:
// `core/mcp/skillFile.ts` parses a served SKILL.md's YAML frontmatter to
// check it against the entry the listing advertised (SEP-2640). Already a
// root `dependency` — it was reached from `test-servers/src` — so this
// adds no package, but a root-declared dependency `core/` imports must be
// named in all three `external` lists or tsup inlines it here.
"yaml",
// Reached through `core/` but not through this client's own code today.
// AGENTS.md requires every root-declared package `core/` imports at runtime
// in ALL three lists regardless, because which client reaches one is a