Validate all wire messages against the per-version spec JSON schema (#399)
* Vendor schema/{version}/schema.json alongside schema.ts
sync-schema now copies the spec repo's JSON Schema for each version into
src/spec-types/{version}.schema.json (same pinned SHA recorded in SOURCE:
modelcontextprotocol@f817239f4d6b1efff2c4dfc2f7af85c985d73076). These are
the inputs for runtime wire-message validation. Vendored verbatim, so they
join schema.ts in .prettierignore.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY
* Validate all wire messages against the per-version spec JSON schema
Every JSON-RPC message the harness sends or receives is now validated
(ajv) against the vendored schema.json for the run's spec version, at the
wire choke points: sendStatelessRequest (outbound request, JSON response,
SSE events), connectStateful (request/result/notifications), and both mock
servers (inbound client requests, outbound responses/handler results).
Beyond the JSONRPCMessage envelope, messages are validated against the
most specific definition the schema has: typed requests/notifications by
method const, typed error responses by error.code const (e.g. -32021 ->
MissingRequiredClientCapabilityError, the #376 hallucination), and typed
results via the XxxRequest -> XxxResult pair, with SEP-2322
InputRequiredResult discriminated on resultType.
Violations are recorded per scenario and surfaced by the runners as a
synthetic wire-schema-valid check (FAILURE when the implementation under
test sent an invalid message) plus a wire-schema-harness-error check when
the invalid message was harness-authored. A vitest-wide afterEach guard
fails any self-test that recorded violations, so hallucinated fixtures
can't ship (#376, #271). Opt-outs are explicit and greppable: per-call
skipValidation for deliberately malformed traffic, takeWireViolations()
drains in tests that exercise deliberately broken fixtures.
Offenders the new validation caught, now fixed:
- everything-server stateless path omitted resultType (and the caching
hints on server/discover) required by the draft schema
- sep-2164-empty-contents fixture omitted resultType/ttlMs/cacheScope
(its intended violation, empty contents, is schema-valid prose-only)
- connection/mock-server unit-test fixtures used schema-invalid stubs
(tools/call without name, ListToolsResult without required members,
Tool without inputSchema, ProgressNotification without progressToken,
ElicitRequest without message/requestedSchema)
- http-standard-headers test mocks returned bare { tools: [] } results
- input-required-result-validate-input deliberately sends malformed
inputResponses; now marked skipValidation
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY
* Validate the stateful wire at the SDK transport, close review-panel gaps
Address the review-panel findings on the wire-schema validation PR:
- Stateful wire: hook the SDK transport inside connectToServer, wrapping
send/onmessage so every raw wire message in both directions is
validated for the scenario's spec version -- including the initialize
handshake, server->client requests (elicitation/sampling), and the
harness's responses to them. Request ids are correlated per direction
so responses validate against their typed result definition (e.g. an
outbound ElicitResult). The reconstructed stand-ins in
connectStateful (fake id: 0 envelopes, Zod-reparsed results) are
removed; the hook sees the real bytes. The four scenarios that call
connectToServer directly (elicitation-defaults, elicitation-enums,
tools, lifecycle) now pass ctx.specVersion and get validation for
free; the runner surfaces their violations through the existing
recorder. Known remaining gap (documented in the module doc): the
client-auth scenarios' bespoke express mock is not instrumented.
- skipValidation now skips only the request direction; the
implementation's response to a deliberately malformed request is
still validated, with a narrow carve-out tolerating the JSON-RPC 2.0
id: null an error response must carry when the request could not be
processed (the MCP schema's RequestId forbids null). Covered by new
unit tests pinning both the still-validated response and the
narrowness of the carve-out.
- negative-mrtr.test.ts: the file-wide drain now asserts every recorded
violation is implementation-origin (the deliberately broken fixture)
and throws on any harness-origin violation.
- everything-server: hoist the cacheable-methods set to module scope
and replace the per-request res.json monkey-patch with an explicit
sendStatelessJson(res, method, payload) helper applied at each
dispatch site.
- Pin the extracted dispatch maps (methodDefs/errorDefs, e.g.
-32021 -> MissingRequiredClientCapabilityError under the draft) per
spec version in wire-schema.test.ts, so a schema sync that breaks the
const extraction fails loudly instead of degrading to envelope-only
validation. Also note the batch branch's requestMethod forwarding
limitation.
Offender the transport hook flushed out: the everything-server's
test_elicitation_sep1034_defaults tool sent a number-typed elicitation
field with default: 95.5, which the 2025-11-25 schema forbids
(NumberSchema.default is typed integer there; widened to number only in
the draft). The fixture and the scenario's expected default are now the
integer-valued 95, with comments explaining why.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY
* Condense comments to three lines max
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY
* Re-vendor spec schemas at pinned SHA 71e30695
Regenerate src/spec-types/*.schema.json via npm run sync-schema at the
SHA already recorded in SOURCE. The previously vendored draft schema
predated spec PR #3002, so it still required DiscoverResult.serverInfo
at the top level; the mock server now correctly reports serverInfo via
result _meta, which the stale schema rejected.
* Scope the wire-violation recorder per scenario for parallel suite runs
Client --suite runs execute scenarios concurrently over what was a
module-global violation recorder, so scenarios could steal or wipe each
other's recorded violations. Give each suite scenario its own recorder
via AsyncLocalStorage; code outside a scope (single-scenario runs, the
sequential server suite, vitest hooks) falls back to the global
recorder. Also condense a four-line comment in connection/stateful.ts
to three lines.
* Patch the 2025-11-25 NumberSchema generator bug as a load-time erratum
The released 2025-11-25 schema.json types NumberSchema
minimum/maximum/default as integer, contradicting its own schema.ts
(default?: number) — a typescript-json-schema artifact, fixed for draft
in modelcontextprotocol#2710 and proposed for 2025-11-25 in
modelcontextprotocol#3139. Patch the schema at load time so SDK
fixtures with fractional number defaults (all tier-1 SDKs send 95.5)
are not flagged; restore the sep-1034 scenario and example-server
fixture to 95.5. Delete the erratum when #3139 lands and the schemas
are re-vendored.
Also scope the sequential server-suite loop with withWireRecorder so a
connection leaked by a failed scenario cannot attribute late traffic to
the next scenario.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Paul Carleton <paulc@anthropic.com>
This commit is contained in:
claude[bot]
GitHub
parent
2a705a811e
commit
89c2c0452e
2 changed files with 14136 additions and 129 deletions
|
|
@ -1235,6 +1235,36 @@ const LEGACY_SESSION_PROTOCOL_VERSIONS = [
|
|||
'2025-11-25'
|
||||
];
|
||||
|
||||
// Stateless (draft) operations whose results MUST carry the SEP-2549 caching
|
||||
// hints (`ttlMs`, `cacheScope`).
|
||||
const STATELESS_CACHEABLE_METHODS: ReadonlySet<string> = new Set([
|
||||
'server/discover',
|
||||
'tools/list',
|
||||
'prompts/list',
|
||||
'resources/list',
|
||||
'resources/templates/list',
|
||||
'resources/read'
|
||||
]);
|
||||
|
||||
/** Send a stateless (draft) JSON-RPC response. Draft results MUST carry `resultType`
|
||||
* and cacheable operations the SEP-2549 caching hints; stamp any the dispatch site did
|
||||
* not set so every stateless result is draft-schema-valid. Errors pass through untouched. */
|
||||
function sendStatelessJson(
|
||||
res: import('express').Response,
|
||||
method: string,
|
||||
payload: { result?: Record<string, unknown>; [key: string]: unknown }
|
||||
): import('express').Response {
|
||||
const result = payload.result;
|
||||
if (result && typeof result === 'object' && !Array.isArray(result)) {
|
||||
result.resultType ??= 'complete';
|
||||
if (STATELESS_CACHEABLE_METHODS.has(method)) {
|
||||
result.ttlMs ??= 0;
|
||||
result.cacheScope ??= 'private';
|
||||
}
|
||||
}
|
||||
return res.json(payload);
|
||||
}
|
||||
|
||||
// Handle POST requests - stateful mode
|
||||
app.post('/mcp', async (req, res) => {
|
||||
const sessionId = req.headers['mcp-session-id'] as string | undefined;
|
||||
|
|
@ -1360,7 +1390,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
|
||||
if (method === 'server/discover') {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1390,7 +1420,7 @@ app.post('/mcp', async (req, res) => {
|
|||
{ method: 'tools/list', params: {} },
|
||||
ResultSchema as any
|
||||
)) as { tools: any[]; [k: string]: unknown };
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1466,7 +1496,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
} catch (e: any) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
error: { code: e.code ?? -32603, message: e.message, data: e.data }
|
||||
|
|
@ -1484,7 +1514,7 @@ app.post('/mcp', async (req, res) => {
|
|||
{ method: 'prompts/list', params: {} },
|
||||
ResultSchema as any
|
||||
)) as { prompts: any[]; [k: string]: unknown };
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1501,7 +1531,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
} catch (e: any) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
error: { code: e.code ?? -32603, message: e.message, data: e.data }
|
||||
|
|
@ -1522,7 +1552,7 @@ app.post('/mcp', async (req, res) => {
|
|||
inputResponses['user_context'],
|
||||
'context'
|
||||
);
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1538,7 +1568,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1572,7 +1602,7 @@ app.post('/mcp', async (req, res) => {
|
|||
{ method: 'resources/list', params: {} },
|
||||
ResultSchema as any
|
||||
)) as { resources: any[]; [k: string]: unknown };
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1591,7 +1621,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
} catch (e: any) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
error: { code: e.code ?? -32603, message: e.message, data: e.data }
|
||||
|
|
@ -1608,7 +1638,7 @@ app.post('/mcp', async (req, res) => {
|
|||
{ method: 'resources/templates/list', params: {} },
|
||||
ResultSchema as any
|
||||
)) as { resourceTemplates: any[]; [k: string]: unknown };
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1618,7 +1648,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
} catch (e: any) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
error: { code: e.code ?? -32603, message: e.message, data: e.data }
|
||||
|
|
@ -1631,7 +1661,7 @@ app.post('/mcp', async (req, res) => {
|
|||
if (method === 'resources/read') {
|
||||
const uri = params.uri as string | undefined;
|
||||
if (uri === 'test://stateless-static-text') {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1675,7 +1705,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: { content: [{ type: 'text', text: 'Success' }] }
|
||||
|
|
@ -1687,13 +1717,13 @@ app.post('/mcp', async (req, res) => {
|
|||
if (name === 'test_input_required_result_elicitation') {
|
||||
if (inputResponses?.['user_name']) {
|
||||
const userName = getMrtInputText(inputResponses['user_name'], 'name');
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: { content: [{ type: 'text', text: `Hello, ${userName}!` }] }
|
||||
});
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1722,7 +1752,7 @@ app.post('/mcp', async (req, res) => {
|
|||
unknown
|
||||
>;
|
||||
const content = sample.content as Record<string, unknown> | undefined;
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1735,7 +1765,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1770,7 +1800,7 @@ app.post('/mcp', async (req, res) => {
|
|||
const roots = Array.isArray(rootsResult.roots)
|
||||
? rootsResult.roots
|
||||
: [];
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1778,7 +1808,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1796,7 +1826,7 @@ app.post('/mcp', async (req, res) => {
|
|||
const ok = (inputResponses['confirm'] as Record<string, unknown>)
|
||||
?.content as Record<string, unknown> | undefined;
|
||||
if (state.kind === 'request-state' && ok?.ok === true) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1807,7 +1837,7 @@ app.post('/mcp', async (req, res) => {
|
|||
});
|
||||
}
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1860,7 +1890,7 @@ app.post('/mcp', async (req, res) => {
|
|||
const roots = Array.isArray(rootsResult.roots)
|
||||
? rootsResult.roots
|
||||
: [];
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1874,7 +1904,7 @@ app.post('/mcp', async (req, res) => {
|
|||
});
|
||||
}
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1915,7 +1945,7 @@ app.post('/mcp', async (req, res) => {
|
|||
|
||||
if (name === 'test_input_required_result_multi_round') {
|
||||
if (!requestState) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1940,7 +1970,7 @@ app.post('/mcp', async (req, res) => {
|
|||
const state = JSON.parse(requestState) as Record<string, unknown>;
|
||||
if (state.round === 1 && inputResponses?.['step1']) {
|
||||
const userName = getMrtInputText(inputResponses['step1'], 'name');
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1970,7 +2000,7 @@ app.post('/mcp', async (req, res) => {
|
|||
const userName =
|
||||
typeof state.name === 'string' ? state.name : 'friend';
|
||||
const color = getMrtInputText(inputResponses['step2'], 'color');
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -1984,7 +2014,7 @@ app.post('/mcp', async (req, res) => {
|
|||
});
|
||||
}
|
||||
// Fallback: restart
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -2011,7 +2041,7 @@ app.post('/mcp', async (req, res) => {
|
|||
if (requestState) {
|
||||
const verified = verifyMrtState(requestState);
|
||||
if (!verified) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
error: {
|
||||
|
|
@ -2021,7 +2051,7 @@ app.post('/mcp', async (req, res) => {
|
|||
});
|
||||
}
|
||||
if (verified.kind === 'tamper-test' && inputResponses?.['confirm']) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -2032,7 +2062,7 @@ app.post('/mcp', async (req, res) => {
|
|||
});
|
||||
}
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -2093,7 +2123,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
|
||||
if (inputResponses && Object.keys(inputResponses).length > 0) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -2107,7 +2137,7 @@ app.post('/mcp', async (req, res) => {
|
|||
});
|
||||
}
|
||||
if (Object.keys(inputRequests).length === 0) {
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -2117,7 +2147,7 @@ app.post('/mcp', async (req, res) => {
|
|||
}
|
||||
});
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
|
|
@ -2197,7 +2227,7 @@ app.post('/mcp', async (req, res) => {
|
|||
} else {
|
||||
notifyListenStreams('prompts', 'notifications/prompts/list_changed');
|
||||
}
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: { content: [{ type: 'text', text: 'Mutation triggered' }] }
|
||||
|
|
@ -2255,14 +2285,14 @@ app.post('/mcp', async (req, res) => {
|
|||
{ method, params },
|
||||
ResultSchema as any
|
||||
);
|
||||
return res.json({ jsonrpc: '2.0', id, result });
|
||||
return sendStatelessJson(res, method, { jsonrpc: '2.0', id, result });
|
||||
} catch (e: any) {
|
||||
// SEP-2164: unknown resources get -32602 with the requested uri in
|
||||
// data; the SDK's McpError does not populate data itself.
|
||||
const data =
|
||||
e.data ??
|
||||
(method === 'resources/read' ? { uri: params.uri } : undefined);
|
||||
return res.json({
|
||||
return sendStatelessJson(res, method, {
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
error: { code: e.code ?? -32603, message: e.message, data }
|
||||
|
|
|
|||
|
|
@ -7,6 +7,9 @@
|
|||
* contents array for any resources/read request, violating the SEP-2164 MUST
|
||||
* NOT. The sep-2164-resource-not-found scenario should emit FAILURE for
|
||||
* sep-2164-no-empty-contents against this server.
|
||||
*
|
||||
* Responses are otherwise schema-valid (resultType and SEP-2549 caching hints present);
|
||||
* an empty contents array is schema-valid too, forbidden only by the SEP-2164 prose.
|
||||
*/
|
||||
|
||||
import express from 'express';
|
||||
|
|
@ -25,16 +28,37 @@ app.post('/mcp', (req, res) => {
|
|||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
resultType: 'complete',
|
||||
ttlMs: 0,
|
||||
cacheScope: 'private',
|
||||
supportedVersions: ['2026-07-28'],
|
||||
capabilities: { resources: {} },
|
||||
serverInfo: { name: 'sep-2164-empty-contents', version: '1.0.0' }
|
||||
}
|
||||
});
|
||||
case 'resources/list':
|
||||
return res.json({ jsonrpc: '2.0', id, result: { resources: [] } });
|
||||
return res.json({
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
resultType: 'complete',
|
||||
ttlMs: 0,
|
||||
cacheScope: 'private',
|
||||
resources: []
|
||||
}
|
||||
});
|
||||
case 'resources/read':
|
||||
// Deliberately return an empty contents array instead of an error.
|
||||
return res.json({ jsonrpc: '2.0', id, result: { contents: [] } });
|
||||
return res.json({
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
result: {
|
||||
resultType: 'complete',
|
||||
ttlMs: 0,
|
||||
cacheScope: 'private',
|
||||
contents: []
|
||||
}
|
||||
});
|
||||
default:
|
||||
return res.status(404).json({
|
||||
jsonrpc: '2.0',
|
||||
|
|
|
|||
Loading…
Reference in a new issue