feat: require the modern list envelope on skills/list (review round 14)

The review finally quoted the normative text, which is what I asked for
across five rounds before declining: SEP-2640's `skills/list` section says
"In protocol versions 2026-07-28 and later, the result also carries …
`ttlMs` and `cacheScope`." So the era-aware validation is in.

- skillsSchemas: `ModernListSkillsResultSchema` = the page plus the base
  list envelope, with field shapes mirroring `ModernResultEnvelopeSchema`
  in `listSalvage.ts` so this repo's two statements of a modern envelope
  cannot drift. `InspectorClient.listSkills` picks it from the negotiated
  era; the legacy shape stays permissive, because those are 2026-era
  attributes a legacy server has no business sending.
  Nothing else catches this: `skills/*` is consumer-owned, so it is absent
  from the SDK's cacheable-method registry and a modern server answering
  `{ skills: [] }` was reaching the conformance UI as a clean list.
- skills.ts: count `frontmatter.description` (and `name`) length in Unicode
  CODE POINTS. `String.length` counts UTF-16 code units, so 600 non-BMP
  characters measured as 1200 and a perfectly valid description was
  reported `malformed-description` — a conforming server failed by an
  off-by-encoding, which is the direction this module works hardest to
  avoid. The Agent Skills reference validator uses Python `len()`.
- #2248 narrowed: the `skills/list` half is settled here; what remains is
  whether `skills/get` carries the same attributes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-05 02:01:18 -04:00
commit 11ac134975
6 changed files with 178 additions and 27 deletions

View file

@ -27,6 +27,7 @@ describe("InspectorClient skills methods (#2234)", () => {
}; };
interface SkillsInternals { interface SkillsInternals {
protocolEra: string | undefined;
client: { client: {
request: ( request: (
req: { method: string; params: Record<string, unknown> }, req: { method: string; params: Record<string, unknown> },
@ -146,6 +147,40 @@ describe("InspectorClient skills methods (#2234)", () => {
).rejects.toBeDefined(); ).rejects.toBeDefined();
}); });
it("requires the modern list envelope on a modern connection", async () => {
// SEP-2640: "In protocol versions 2026-07-28 and later, the result also
// carries … `ttlMs` and `cacheScope`." Nothing else validates it —
// `skills/*` is consumer-owned, so the SDK codec never sees it.
const client = makeClient();
internals(client).protocolEra = "modern";
stubRequest(client, { skills: [] });
await expect(client.listSkills()).rejects.toBeDefined();
});
it("accepts a modern result that carries the envelope", async () => {
const client = makeClient();
internals(client).protocolEra = "modern";
stubRequest(client, {
resultType: "complete",
ttlMs: 0,
cacheScope: "public",
skills: [ENTRY],
});
await expect(client.listSkills()).resolves.toMatchObject({
skills: [ENTRY],
});
});
it("does NOT require the envelope on a legacy connection", async () => {
// Those are 2026-era attributes; failing a legacy server for their absence
// would reject a conforming server.
const client = makeClient();
stubRequest(client, { skills: [ENTRY] });
await expect(client.listSkills()).resolves.toMatchObject({
skills: [ENTRY],
});
});
it("rejects a skills/list result that is not a skills page", async () => { it("rejects a skills/list result that is not a skills page", async () => {
// The explicit result schema is the whole client-side mechanism for a // The explicit result schema is the whole client-side mechanism for a
// consumer-owned extension method, so a nonconforming result must fail // consumer-owned extension method, so a nonconforming result must fail

View file

@ -378,6 +378,24 @@ describe("checkSkillConformance", () => {
expect(issues[0].severity).toBe("error"); expect(issues[0].severity).toBe("error");
}); });
it("counts description length in code points, not UTF-16 code units", () => {
// 600 non-BMP characters are 1200 code units. Measuring those would report
// a perfectly valid description as over the 1024-character limit — a
// conforming server failed by an off-by-encoding.
const issues = checkSkillConformance(
entry({ frontmatter: { name: "demo", description: "𝄞".repeat(600) } }),
);
expect(issues).toEqual([]);
});
it("still reports a description over the limit in code points", () => {
const issues = checkSkillConformance(
entry({ frontmatter: { name: "demo", description: "𝄞".repeat(1025) } }),
);
expect(issues.map((i) => i.code)).toEqual(["malformed-description"]);
expect(issues[0].message).toContain("1025");
});
it("accepts a description exactly at the limit", () => { it("accepts a description exactly at the limit", () => {
const issues = checkSkillConformance( const issues = checkSkillConformance(
entry({ frontmatter: { name: "demo", description: "d".repeat(1024) } }), entry({ frontmatter: { name: "demo", description: "d".repeat(1024) } }),

View file

@ -3,6 +3,7 @@ import {
DYNAMIC_RESOURCES, DYNAMIC_RESOURCES,
GetSkillResultSchema, GetSkillResultSchema,
ListSkillsResultSchema, ListSkillsResultSchema,
ModernListSkillsResultSchema,
SKILLS_EXTENSION_KEY, SKILLS_EXTENSION_KEY,
SKILLS_GET_METHOD, SKILLS_GET_METHOD,
SKILLS_LIST_METHOD, SKILLS_LIST_METHOD,
@ -87,6 +88,56 @@ describe("ListSkillsResultSchema", () => {
}); });
}); });
describe("ModernListSkillsResultSchema", () => {
const envelope = { resultType: "complete", ttlMs: 0, cacheScope: "public" };
it("accepts a modern page carrying the base list envelope", () => {
const parsed = ModernListSkillsResultSchema.parse({
...envelope,
skills: [ENTRY],
});
expect(parsed.skills).toHaveLength(1);
});
it("rejects a modern page that omits the caching attributes", () => {
// The whole reason for the era split: `skills/*` is consumer-owned, so the
// SDK codec validates none of it, and `{ skills: [] }` would otherwise
// reach the conformance UI as a clean list.
expect(() => ModernListSkillsResultSchema.parse({ skills: [] })).toThrow();
expect(() =>
ModernListSkillsResultSchema.parse({
...envelope,
ttlMs: undefined,
skills: [],
}),
).toThrow();
});
it("rejects a malformed ttlMs rather than accepting the envelope loosely", () => {
for (const ttlMs of [-1, 0.5]) {
expect(() =>
ModernListSkillsResultSchema.parse({ ...envelope, ttlMs, skills: [] }),
).toThrow();
}
});
it("rejects an unknown cacheScope", () => {
expect(() =>
ModernListSkillsResultSchema.parse({
...envelope,
cacheScope: "shared",
skills: [],
}),
).toThrow();
});
it("the LEGACY schema still accepts a page without the envelope", () => {
// Those are 2026-era attributes; a legacy server has no business sending
// them and must not be failed for their absence.
expect(ListSkillsResultSchema.parse({ skills: [] }).skills).toEqual([]);
});
});
describe("GetSkillResultSchema", () => { describe("GetSkillResultSchema", () => {
it("unwraps the envelope to the entry", () => { it("unwraps the envelope to the entry", () => {
expect(GetSkillResultSchema.parse({ skill: ENTRY })).toEqual(ENTRY); expect(GetSkillResultSchema.parse({ skill: ENTRY })).toEqual(ENTRY);

View file

@ -147,6 +147,7 @@ import { buildClientExtensions } from "./extensions.js";
import { import {
GetSkillResultSchema, GetSkillResultSchema,
ListSkillsResultSchema, ListSkillsResultSchema,
ModernListSkillsResultSchema,
SKILLS_GET_METHOD, SKILLS_GET_METHOD,
SKILLS_LIST_METHOD, SKILLS_LIST_METHOD,
type SkillEntry, type SkillEntry,
@ -5561,11 +5562,20 @@ export class InspectorClient extends InspectorClientEventTarget {
// conforming server made to look broken. // conforming server made to look broken.
...(cursor !== undefined ? { cursor } : {}), ...(cursor !== undefined ? { cursor } : {}),
}; };
// Era-aware: a modern (2026-07-28+) `skills/list` result also carries the
// base list envelope (`resultType` / `ttlMs` / `cacheScope`). `skills/*` is
// consumer-owned, so the SDK codec validates none of it — without picking
// the schema here a modern server could answer `{ skills: [] }` and the
// conformance UI would show a clean list. Legacy stays permissive: those
// are 2026-era attributes.
const resultSchema = this.isModernEra()
? ModernListSkillsResultSchema
: ListSkillsResultSchema;
const response = await this.invokeMcpClient( const response = await this.invokeMcpClient(
() => () =>
this.client!.request( this.client!.request(
{ method: SKILLS_LIST_METHOD, params }, { method: SKILLS_LIST_METHOD, params },
ListSkillsResultSchema, resultSchema,
this.getRequestOptions(this.progressTokenOf(metadata)), this.getRequestOptions(this.progressTokenOf(metadata)),
), ),
{ method: SKILLS_LIST_METHOD }, { method: SKILLS_LIST_METHOD },

View file

@ -71,6 +71,21 @@ const SKILL_NAME_MAX_LENGTH = 64;
/** The Agent Skills limit on `frontmatter.description`. */ /** The Agent Skills limit on `frontmatter.description`. */
const SKILL_DESCRIPTION_MAX_LENGTH = 1024; const SKILL_DESCRIPTION_MAX_LENGTH = 1024;
/**
* Length in Unicode **code points**, not UTF-16 code units.
*
* `String.prototype.length` counts code units, so every non-BMP character
* (emoji, many CJK extension characters) counts twice — a perfectly valid
* 600-character description would be measured as 1200 and reported as
* `malformed-description`. The Agent Skills limit is in characters, and its
* reference validator uses Python's `len()`, which counts code points. Getting
* this wrong fails a conforming server, which is the direction this module
* works hardest to avoid.
*/
function codePointLength(value: string): number {
return [...value].length;
}
/** /**
* What the server declared under `io.modelcontextprotocol/skills`. The only * What the server declared under `io.modelcontextprotocol/skills`. The only
* sub-option SEP-2640 defines is `directoryRead`, which gates * sub-option SEP-2640 defines is `directoryRead`, which gates
@ -269,7 +284,7 @@ export function checkSkillConformance(entry: SkillEntry): SkillIssue[] {
message: "frontmatter.name is required but missing or empty.", message: "frontmatter.name is required but missing or empty.",
}); });
} else if ( } else if (
declaredName.length > SKILL_NAME_MAX_LENGTH || codePointLength(declaredName) > SKILL_NAME_MAX_LENGTH ||
!SKILL_NAME_PATTERN.test(declaredName) !SKILL_NAME_PATTERN.test(declaredName)
) { ) {
// Reaches here for `" demo "` too: the grammar sees the untrimmed value. // Reaches here for `" demo "` too: the grammar sees the untrimmed value.
@ -288,11 +303,11 @@ export function checkSkillConformance(entry: SkillEntry): SkillIssue[] {
severity: "error", severity: "error",
message: "frontmatter.description is required but missing or empty.", message: "frontmatter.description is required but missing or empty.",
}); });
} else if (rawDescription.length > SKILL_DESCRIPTION_MAX_LENGTH) { } else if (codePointLength(rawDescription) > SKILL_DESCRIPTION_MAX_LENGTH) {
issues.push({ issues.push({
code: "malformed-description", code: "malformed-description",
severity: "error", severity: "error",
message: `frontmatter.description is ${rawDescription.length} characters, above the ${SKILL_DESCRIPTION_MAX_LENGTH}-character limit.`, message: `frontmatter.description is ${codePointLength(rawDescription)} characters, above the ${SKILL_DESCRIPTION_MAX_LENGTH}-character limit.`,
}); });
} }
if (uriName === undefined) { if (uriName === undefined) {

View file

@ -11,17 +11,21 @@
* hatch that modern `tasks/*` needs is deliberately NOT used here — `tasks/*` * hatch that modern `tasks/*` needs is deliberately NOT used here — `tasks/*`
* are spec names the 2026 codec deleted, which is a different problem. * are spec names the 2026 codec deleted, which is a different problem.
* *
* **This module is the whole wire surface.** SEP-2640 is Accepted, so the method * **This module is the wire surface for the two methods the Inspector calls.**
* names and the entry shape are settled, but the skill *format* is delegated to * SEP-2640 is Accepted, so the method names and the entry shape are settled;
* the independently-versioned Agent Skills specification and the SEP leaves the * the skill *format* is delegated to the independently-versioned Agent Skills
* `skills/get` caching attributes (SEP-2549 `ttlMs` / `cacheScope`) open. Keeping * specification. Keeping every wire type here makes a spec revision a
* every wire type here makes a spec revision a single-file edit (#2234). * single-file edit (#2234).
* *
* Schemas are deliberately permissive (`looseObject`, and a `digest` typed as a * **Permissive where a defect should be reported; strict where a shape is
* plain string rather than a hex-constrained one) so a non-conforming server is * settled.** `looseObject`, and a `digest` typed as a plain string rather than
* *surfaced* rather than rejected — the Inspector is a conformance tool, and a * a hex-constrained one, so a non-conforming server is *surfaced* by
* malformed digest is a finding to report, not a parse error to swallow. The * `checkSkillConformance` rather than rejected at the parse — a malformed
* structural checks live in `skills.ts`. * digest is a finding, not a parse error to swallow. But a settled shape is
* enforced, because silently normalizing one past the checks would defeat the
* point: `skills/get` requires its `{ skill }` envelope, and a modern
* `skills/list` result requires the base list envelope (see
* `ModernListSkillsResultSchema`).
*/ */
import { z } from "zod/v4"; import { z } from "zod/v4";
@ -97,19 +101,8 @@ export const SkillEntrySchema = z.looseObject({
export type SkillEntry = z.infer<typeof SkillEntrySchema>; export type SkillEntry = z.infer<typeof SkillEntrySchema>;
/** /**
* `skills/list` result: a page of entries plus the opaque cursor. * `skills/list` result on a **legacy** connection: a page of entries plus the
* * opaque cursor, and nothing required beyond that.
* ⚠️ **Whether a modern-era (2026-07-28) result must also carry the SEP-2549
* caching attributes `ttlMs` / `cacheScope` is unsettled here and deliberately
* not guessed.** #2234's analysis records it as an open point; a review of that
* PR asserted the opposite. Neither reading was checked against the normative
* text, and the two mistakes are not symmetric: leaving the schema permissive
* means a server that omits them is accepted (they pass through untouched when
* sent), while tightening on a wrong reading would *reject* conforming
* responses. `resources/directory/read` was removed from this module for the
* same reason. #2248 settles it against the spec. Note the SDK is no help
* either way — `skills/list` is consumer-owned, so it is absent from the
* cacheable-method registry and nothing stamps or validates these fields.
*/ */
export const ListSkillsResultSchema = z.looseObject({ export const ListSkillsResultSchema = z.looseObject({
skills: z.array(SkillEntrySchema), skills: z.array(SkillEntrySchema),
@ -118,6 +111,35 @@ export const ListSkillsResultSchema = z.looseObject({
export type ListSkillsResult = z.infer<typeof ListSkillsResultSchema>; export type ListSkillsResult = z.infer<typeof ListSkillsResultSchema>;
/**
* `skills/list` result on a **modern** (2026-07-28+) connection: the page plus
* the base list envelope.
*
* SEP-2640's `skills/list` section states: *"In protocol versions 2026-07-28
* and later, the result also carries … `ttlMs` and `cacheScope`."* The field
* shapes mirror `ModernResultEnvelopeSchema` in `listSalvage.ts`, which is this
* repo's existing statement of a modern result envelope and is already applied
* to modern list results on the salvage path — so the two cannot drift.
*
* The era split is load-bearing rather than defensive. `skills/*` is a
* consumer-owned method, so it is absent from the SDK's cacheable-method
* registry and **nothing else validates this**: without an era-aware schema a
* modern server could answer `{ skills: [] }` and the conformance UI would
* present it as a clean list. The legacy shape stays permissive because these
* are 2026-era attributes a legacy server has no business sending.
*
* ⚠️ Picked by `InspectorClient.listSkills` from the negotiated era — a schema
* cannot know it on its own.
*/
export const ModernListSkillsResultSchema = ListSkillsResultSchema.extend({
resultType: z.literal("complete"),
// A non-negative INTEGER, matching the codec: `ttlMs: -1` or `0.5` is an
// envelope violation, and accepting it here would hide exactly the kind of
// defect this schema exists to surface.
ttlMs: z.int().min(0),
cacheScope: z.enum(["public", "private"]),
});
/** /**
* The `skills/get` result envelope: the entry wrapped under `skill`. * The `skills/get` result envelope: the entry wrapped under `skill`.
* *