Make the draft-spec scenarios internally consistent (fixes #311, #312, #315) (#319)

* feat(everything-server): serve resources and caching hints on the stateless draft path

The stateless (SEP-2575) request path now answers resources/list,
resources/templates/list and resources/read — including the SEP-2164
-32602 + data.uri error for unknown URIs — and includes the SEP-2549
ttlMs/cacheScope hints on its cacheable list results, matching what the
draft-spec scenarios exercise.

* feat: make the harness's own draft-spec traffic conformant

Fixes the class of issues where a strictly-conformant SDK rejects the
harness's requests for reasons unrelated to the behaviour under test
(#311, #312, #315).

- add a shared stateless draft request helper (draft-client.ts) that puts
  the cross-cutting SEP-2243 headers (Mcp-Method, Mcp-Name,
  MCP-Protocol-Version, Accept) and SEP-2575 _meta (protocolVersion,
  clientInfo, clientCapabilities) on every request, with overrides for
  negative cases and the SHOULD-level retry on
  UnsupportedProtocolVersionError
- migrate the draft scenarios onto it: the SEP-2322 helpers and
  server-stateless now send Mcp-Method/Mcp-Name (#312); the SEP-2243
  server scenarios drop the legacy initialize handshake and carry _meta on
  every raw request (#311); caching and sep-2164-resource-not-found are
  stateless DRAFT-2026-v1 requests instead of an SDK 2025-11-25 session
  (#315)
- add a draft self-conformance gate (draft-self-conformance.test.ts) that
  points the harness's own draft drivers at the existing client-testing
  judge scenarios (request-metadata, http-standard-headers) and requires
  zero failures or warnings
- align the request-metadata mock's unsupported-version rejection with the
  draft schema (-32004) and teach the everything-client to recognize it
- rewrite the SEP-2549/SEP-2164 negative example servers as stateless
  draft servers so they keep exercising the violation, not the transport

* refactor: rename the stateless request helpers and harden them per review

- rename draft-client.ts to stateless-client.ts and name the helpers by
  feature (sendStatelessRequest, buildStandardHeaders, buildRequestMeta,
  withRequestMeta, CONFORMANCE_CLIENT_INFO); 'draft' now only appears in
  DRAFT_PROTOCOL_VERSION, the one name that tracks the moving target
- retry only genuine UnsupportedProtocolVersionError rejections (-32004,
  tolerating -32001/-32602) and only with recognized versions; record the
  retry on the response
- parse text/event-stream responses incrementally and resolve on the
  matching event instead of blocking until the timeout
- keep the MCP-Protocol-Version header and _meta protocolVersion in sync
  when the version is overridden via meta overrides
- make server-stateless's sendRpc SSE-tolerant
- emit explicit setup failures in the SEP-2243 server scenarios when the
  tools/list discovery fails instead of misleading per-case results
- latch the worst status per check in the request-metadata judge so an
  early violation cannot be masked by a later conformant request
- rename the gate to harness-traffic-conformance.test.ts and derive its
  driver list from the draft suite with documented per-judge exclusions
  (35 pairings)
- everything-server: advertise resources in stateless server/discover and
  use -32004 for unsupported protocol version
- add unit tests for the helper (retry guard, SSE handling, header/_meta
  version sync, default headers)

* refactor: slim the stateless client helper for the initial merge
This commit is contained in:
Paul Carleton 2026-05-26 21:10:23 +01:00 • committed by GitHub
commit 076b417b23
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 894 additions and 596 deletions

View file

@ -155,7 +155,12 @@ async function runRequestMetadataClient(serverUrl: string): Promise<void> {
const clone = response.clone(); const clone = response.clone();
try { try {
const errorResult = await clone.json(); const errorResult = await clone.json();
if (errorResult.error?.code === -32001) { // -32004 is UnsupportedProtocolVersionError in the draft schema;
// -32001 is tolerated for servers that predate the dedicated code.
if (
errorResult.error?.code === -32004 ||
errorResult.error?.code === -32001
) {
logger.debug( logger.debug(
'Received UnsupportedProtocolVersionError, starting negotiation...' 'Received UnsupportedProtocolVersionError, starting negotiation...'
); );

View file

@ -1227,13 +1227,13 @@ app.post('/mcp', async (req, res) => {
}); });
} }
// Protocol Version Negotiation Matrix (-32602, HTTP 400) // Protocol Version Negotiation Matrix (-32004, HTTP 400)
if (metaVersion !== 'DRAFT-2026-v1') { if (metaVersion !== 'DRAFT-2026-v1') {
return res.status(400).json({ return res.status(400).json({
jsonrpc: '2.0', jsonrpc: '2.0',
id, id,
error: { error: {
code: -32602, code: -32004,
message: 'UnsupportedProtocolVersionError', message: 'UnsupportedProtocolVersionError',
data: { supported: ['DRAFT-2026-v1'] } data: { supported: ['DRAFT-2026-v1'] }
} }
@ -1296,7 +1296,10 @@ app.post('/mcp', async (req, res) => {
supportedVersions: ['DRAFT-2026-v1'], supportedVersions: ['DRAFT-2026-v1'],
capabilities: { capabilities: {
tools: { listChanged: true }, // Explicitly announce dynamic capabilities matching Section 7 expectations tools: { listChanged: true }, // Explicitly announce dynamic capabilities matching Section 7 expectations
prompts: { listChanged: true } prompts: { listChanged: true },
// resources/list, resources/templates/list and resources/read are
// served on this path, so the capability must be declared too.
resources: {}
}, },
serverInfo: { name: 'everything-stateless-server', version: '1.0.0' } serverInfo: { name: 'everything-stateless-server', version: '1.0.0' }
} }
@ -1371,7 +1374,10 @@ app.post('/mcp', async (req, res) => {
description: 'Diagnostic logging validator tool', description: 'Diagnostic logging validator tool',
inputSchema: { type: 'object', properties: {} } inputSchema: { type: 'object', properties: {} }
} }
] ],
// SEP-2549 caching hints are required on cacheable list results.
ttlMs: 300000,
cacheScope: 'public'
} }
}); });
} }
@ -1387,7 +1393,9 @@ app.post('/mcp', async (req, res) => {
name: 'test_input_required_result_prompt', name: 'test_input_required_result_prompt',
description: 'MRTR: prompt that requires elicitation input' description: 'MRTR: prompt that requires elicitation input'
} }
] ],
ttlMs: 300000,
cacheScope: 'public'
} }
}); });
} }
@ -1442,6 +1450,69 @@ app.post('/mcp', async (req, res) => {
} }
} }
// Resources on the stateless path (SEP-2575): SEP-2549 hints + SEP-2164 errors.
if (method === 'resources/list') {
return res.json({
jsonrpc: '2.0',
id,
result: {
resources: [
{
uri: 'test://stateless-static-text',
name: 'Stateless Static Text',
description: 'A static text resource served on the draft path',
mimeType: 'text/plain'
}
],
ttlMs: 300000,
cacheScope: 'public'
}
});
}
if (method === 'resources/templates/list') {
return res.json({
jsonrpc: '2.0',
id,
result: {
resourceTemplates: [],
ttlMs: 300000,
cacheScope: 'public'
}
});
}
if (method === 'resources/read') {
const uri = params.uri as string | undefined;
if (uri === 'test://stateless-static-text') {
return res.json({
jsonrpc: '2.0',
id,
result: {
contents: [
{
uri,
mimeType: 'text/plain',
text: 'Static text content from the stateless draft path.'
}
],
ttlMs: 300000,
cacheScope: 'private'
}
});
}
// SEP-2164: unknown resources get -32602 with the requested uri in data.
return res.status(200).json({
jsonrpc: '2.0',
id,
error: {
code: -32602,
message: 'Resource not found',
data: { uri }
}
});
}
if (method === 'tools/call') { if (method === 'tools/call') {
const name = params.name; const name = params.name;
const inputResponses = params.inputResponses as const inputResponses = params.inputResponses as

View file

@ -3,58 +3,44 @@
/** /**
* SEP-2164 Negative Test Server * SEP-2164 Negative Test Server
* *
* Returns an empty contents array for any resources/read request, violating * Speaks the stateless wire protocol (SEP-2575) but returns an empty
* the SEP-2164 MUST NOT. The sep-2164-resource-not-found scenario should * contents array for any resources/read request, violating the SEP-2164 MUST
* emit FAILURE for sep-2164-no-empty-contents against this server. * NOT. The sep-2164-resource-not-found scenario should emit FAILURE for
* sep-2164-no-empty-contents against this server.
*/ */
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import {
ListResourcesRequestSchema,
ReadResourceRequestSchema
} from '@modelcontextprotocol/sdk/types.js';
import express from 'express'; import express from 'express';
function createServer() {
const server = new Server(
{ name: 'sep-2164-empty-contents', version: '1.0.0' },
{ capabilities: { resources: {} } }
);
server.setRequestHandler(ListResourcesRequestSchema, async () => ({
resources: []
}));
server.setRequestHandler(ReadResourceRequestSchema, async () => ({
contents: []
}));
return server;
}
const app = express(); const app = express();
app.use(express.json()); app.use(express.json());
app.post('/mcp', async (req, res) => { app.post('/mcp', (req, res) => {
try { const body = req.body || {};
const server = createServer(); const id = body.id ?? null;
const transport = new StreamableHTTPServerTransport({ const method = body.method;
sessionIdGenerator: undefined
}); switch (method) {
await server.connect(transport); case 'server/discover':
await transport.handleRequest(req, res, req.body); return res.json({
} catch (error) {
if (!res.headersSent) {
res.status(500).json({
jsonrpc: '2.0', jsonrpc: '2.0',
error: { id,
code: -32603, result: {
message: `Internal error: ${error instanceof Error ? error.message : String(error)}` supportedVersions: ['DRAFT-2026-v1'],
}, capabilities: { resources: {} },
id: null serverInfo: { name: 'sep-2164-empty-contents', version: '1.0.0' }
}
});
case 'resources/list':
return res.json({ jsonrpc: '2.0', id, result: { resources: [] } });
case 'resources/read':
// Deliberately return an empty contents array instead of an error.
return res.json({ jsonrpc: '2.0', id, result: { contents: [] } });
default:
return res.status(404).json({
jsonrpc: '2.0',
id,
error: { code: -32601, message: 'Method not found' }
}); });
}
} }
}); });

View file

@ -3,151 +3,96 @@
/** /**
* SEP-2549 Negative Test Server * SEP-2549 Negative Test Server
* *
* Returns list and read results WITHOUT ttlMs and cacheScope fields, * Speaks the stateless wire protocol (SEP-2575) but returns list and
* violating the SEP-2549 MUST. The caching scenario should emit FAILURE * read results WITHOUT ttlMs and cacheScope fields, violating the SEP-2549
* for presence checks against this server. * MUST. The caching scenario should emit FAILURE for presence checks against
* this server.
*/ */
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import {
ListToolsRequestSchema,
ListPromptsRequestSchema,
ListResourcesRequestSchema,
ListResourceTemplatesRequestSchema,
ReadResourceRequestSchema
} from '@modelcontextprotocol/sdk/types.js';
import express from 'express'; import express from 'express';
import { randomUUID } from 'crypto';
const transports: Record<string, StreamableHTTPServerTransport> = {};
function isInitializeRequest(body: any): boolean {
return body?.method === 'initialize';
}
function createServer() {
const server = new Server(
{ name: 'sep-2549-no-caching-hints', version: '1.0.0' },
{
capabilities: {
tools: {},
resources: {},
prompts: {}
}
}
);
// Deliberately omit ttlMs and cacheScope from all responses
server.setRequestHandler(ListToolsRequestSchema, async () => ({
tools: [
{
name: 'test_tool',
description: 'A test tool',
inputSchema: { type: 'object' as const }
}
]
}));
server.setRequestHandler(ListPromptsRequestSchema, async () => ({
prompts: [
{
name: 'test_prompt',
description: 'A test prompt'
}
]
}));
server.setRequestHandler(ListResourcesRequestSchema, async () => ({
resources: [
{
uri: 'test://static-text',
name: 'Static Text',
description: 'A static text resource'
}
]
}));
server.setRequestHandler(ListResourceTemplatesRequestSchema, async () => ({
resourceTemplates: []
}));
server.setRequestHandler(ReadResourceRequestSchema, async () => ({
contents: [
{
uri: 'test://static-text',
mimeType: 'text/plain',
text: 'Static text content.'
}
]
}));
return server;
}
const app = express(); const app = express();
app.use(express.json()); app.use(express.json());
app.post('/mcp', async (req, res) => { app.post('/mcp', (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined; const body = req.body || {};
const id = body.id ?? null;
const method = body.method;
try { // Deliberately omit ttlMs and cacheScope from every result below.
if (sessionId && transports[sessionId]) { switch (method) {
await transports[sessionId].handleRequest(req, res, req.body); case 'server/discover':
return; return res.json({
} jsonrpc: '2.0',
id,
if (!sessionId && isInitializeRequest(req.body)) { result: {
const server = createServer(); supportedVersions: ['DRAFT-2026-v1'],
const transport = new StreamableHTTPServerTransport({ capabilities: { tools: {}, resources: {}, prompts: {} },
sessionIdGenerator: () => randomUUID(), serverInfo: { name: 'sep-2549-no-caching-hints', version: '1.0.0' }
onsessioninitialized: (newSessionId) => {
transports[newSessionId] = transport;
} }
}); });
transport.onclose = () => { case 'tools/list':
const sid = transport.sessionId; return res.json({
if (sid) delete transports[sid];
};
await server.connect(transport);
await transport.handleRequest(req, res, req.body);
return;
}
res.status(400).json({
jsonrpc: '2.0',
error: { code: -32000, message: 'Invalid or missing session ID' },
id: null
});
} catch (error) {
if (!res.headersSent) {
res.status(500).json({
jsonrpc: '2.0', jsonrpc: '2.0',
error: { id,
code: -32603, result: {
message: `Internal error: ${error instanceof Error ? error.message : String(error)}` tools: [
}, {
id: null name: 'test_tool',
description: 'A test tool',
inputSchema: { type: 'object' }
}
]
}
});
case 'prompts/list':
return res.json({
jsonrpc: '2.0',
id,
result: {
prompts: [{ name: 'test_prompt', description: 'A test prompt' }]
}
});
case 'resources/list':
return res.json({
jsonrpc: '2.0',
id,
result: {
resources: [
{
uri: 'test://static-text',
name: 'Static Text',
description: 'A static text resource'
}
]
}
});
case 'resources/templates/list':
return res.json({
jsonrpc: '2.0',
id,
result: { resourceTemplates: [] }
});
case 'resources/read':
return res.json({
jsonrpc: '2.0',
id,
result: {
contents: [
{
uri: 'test://static-text',
mimeType: 'text/plain',
text: 'Static text content.'
}
]
}
});
default:
return res.status(404).json({
jsonrpc: '2.0',
id,
error: { code: -32601, message: 'Method not found' }
}); });
}
}
});
app.get('/mcp', async (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined;
if (sessionId && transports[sessionId]) {
await transports[sessionId].handleRequest(req, res);
} else {
res.status(400).json({ error: 'Invalid or missing session ID' });
}
});
app.delete('/mcp', async (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined;
if (sessionId && transports[sessionId]) {
await transports[sessionId].handleRequest(req, res);
} else {
res.status(400).json({ error: 'Invalid or missing session ID' });
} }
}); });

View file

@ -104,9 +104,10 @@ async function incompatibleVersionClient(serverUrl: string) {
if (response.status === 400) { if (response.status === 400) {
const body = await response.json(); const body = await response.json();
if (body.error?.code === -32001) { if (body.error?.code === -32004 || body.error?.code === -32001) {
return body; // Abort cleanly return body; // Abort cleanly
} }
return body;
} }
return response.json(); return response.json();
} }

View file

@ -3,9 +3,23 @@ import {
Scenario, Scenario,
ScenarioUrls, ScenarioUrls,
ConformanceCheck, ConformanceCheck,
CheckStatus,
DRAFT_PROTOCOL_VERSION DRAFT_PROTOCOL_VERSION
} from '../../types'; } from '../../types';
/**
* Severity ranking used to latch per-id check results: a single
* non-conformant request is a violation even if later requests are
* conformant, so a later better status must never overwrite a worse one.
*/
const STATUS_SEVERITY: Record<CheckStatus, number> = {
FAILURE: 3,
WARNING: 2,
SUCCESS: 1,
INFO: 1,
SKIPPED: 0
};
/** /**
* Every check ID this scenario can emit. Declared-but-unemitted checks are * Every check ID this scenario can emit. Declared-but-unemitted checks are
* backfilled as FAILURE by getChecks(), so the emitted ID set is the same for * backfilled as FAILURE by getChecks(), so the emitted ID set is the same for
@ -92,10 +106,17 @@ export class RequestMetadataScenario implements Scenario {
private addOrUpdateCheck(check: ConformanceCheck): void { private addOrUpdateCheck(check: ConformanceCheck): void {
const index = this.checks.findIndex((c) => c.id === check.id); const index = this.checks.findIndex((c) => c.id === check.id);
if (index !== -1) { if (index === -1) {
this.checks[index] = check;
} else {
this.checks.push(check); this.checks.push(check);
return;
}
// Keep the worst status observed for this id (FAILURE > WARNING > SUCCESS
// > SKIPPED): an equal-or-worse result replaces the stored check (so its
// details stay fresh), but a better result must not erase a violation
// recorded from an earlier request.
const existing = this.checks[index];
if (STATUS_SEVERITY[check.status] >= STATUS_SEVERITY[existing.status]) {
this.checks[index] = check;
} }
} }
@ -256,7 +277,8 @@ export class RequestMetadataScenario implements Scenario {
jsonrpc: '2.0', jsonrpc: '2.0',
id: request.id ?? null, id: request.id ?? null,
error: { error: {
code: -32001, // UnsupportedProtocolVersionError per the draft schema.
code: -32004,
message: 'Unsupported protocol version', message: 'Unsupported protocol version',
data: { data: {
supported: [DRAFT_PROTOCOL_VERSION] supported: [DRAFT_PROTOCOL_VERSION]

View file

@ -10,14 +10,7 @@ import {
ConformanceCheck, ConformanceCheck,
DRAFT_PROTOCOL_VERSION DRAFT_PROTOCOL_VERSION
} from '../../types'; } from '../../types';
import { connectToServer } from './client-helper'; import { sendStatelessRequest } from './stateless-client';
import {
ListToolsResultSchema,
ListPromptsResultSchema,
ListResourcesResultSchema,
ListResourceTemplatesResultSchema,
ReadResourceResultSchema
} from '@modelcontextprotocol/sdk/types.js';
const SPEC_REFS = [ const SPEC_REFS = [
{ {
@ -98,259 +91,181 @@ Servers MUST include \`ttlMs\` (integer >= 0) and \`cacheScope\` ("public" or "p
const checks: ConformanceCheck[] = []; const checks: ConformanceCheck[] = [];
const allFields: Array<{ endpoint: string; fields: CachingFields }> = []; const allFields: Array<{ endpoint: string; fields: CachingFields }> = [];
try { // SEP-2549 only exists in the draft spec, so each cacheable endpoint is
const connection = await connectToServer(serverUrl); // queried over the stateless path (SEP-2575): protocolVersion DRAFT-2026-v1
// plus the cross-cutting _meta and standard headers (issue #315).
// 1. tools/list const queryEndpoint = async (
checkId: string,
checkName: string,
endpoint: string,
params?: Record<string, unknown>
): Promise<Record<string, unknown> | undefined> => {
const description = `${endpoint} response includes ttlMs and cacheScope caching hints`;
try { try {
const toolsResult = await connection.client.request( const response = await sendStatelessRequest(
{ method: 'tools/list', params: {} }, serverUrl,
ListToolsResultSchema endpoint,
params
); );
const fields = extractCachingFields( const result = response.body?.result;
toolsResult as Record<string, unknown> if (!result) {
); const error = response.body?.error;
allFields.push({ endpoint: 'tools/list', fields });
checks.push(
buildPresenceCheck(
'sep-2549-tools-list-caching-hints',
'ToolsListCachingHints',
'tools/list',
fields
)
);
} catch (error) {
checks.push({
id: 'sep-2549-tools-list-caching-hints',
name: 'ToolsListCachingHints',
description:
'tools/list response includes ttlMs and cacheScope caching hints',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `tools/list request failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS
});
}
// 2. prompts/list
try {
const promptsResult = await connection.client.request(
{ method: 'prompts/list', params: {} },
ListPromptsResultSchema
);
const fields = extractCachingFields(
promptsResult as Record<string, unknown>
);
allFields.push({ endpoint: 'prompts/list', fields });
checks.push(
buildPresenceCheck(
'sep-2549-prompts-list-caching-hints',
'PromptsListCachingHints',
'prompts/list',
fields
)
);
} catch (error) {
checks.push({
id: 'sep-2549-prompts-list-caching-hints',
name: 'PromptsListCachingHints',
description:
'prompts/list response includes ttlMs and cacheScope caching hints',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `prompts/list request failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS
});
}
// 3. resources/list
let firstResourceUri: string | undefined;
try {
const resourcesResult = await connection.client.request(
{ method: 'resources/list', params: {} },
ListResourcesResultSchema
);
const fields = extractCachingFields(
resourcesResult as Record<string, unknown>
);
allFields.push({ endpoint: 'resources/list', fields });
checks.push(
buildPresenceCheck(
'sep-2549-resources-list-caching-hints',
'ResourcesListCachingHints',
'resources/list',
fields
)
);
// Capture the first resource URI for the resources/read check
if (resourcesResult.resources && resourcesResult.resources.length > 0) {
firstResourceUri = resourcesResult.resources[0].uri;
}
} catch (error) {
checks.push({
id: 'sep-2549-resources-list-caching-hints',
name: 'ResourcesListCachingHints',
description:
'resources/list response includes ttlMs and cacheScope caching hints',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `resources/list request failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS
});
}
// 4. resources/templates/list
try {
const templatesResult = await connection.client.request(
{ method: 'resources/templates/list', params: {} },
ListResourceTemplatesResultSchema
);
const fields = extractCachingFields(
templatesResult as Record<string, unknown>
);
allFields.push({ endpoint: 'resources/templates/list', fields });
checks.push(
buildPresenceCheck(
'sep-2549-resources-templates-list-caching-hints',
'ResourcesTemplatesListCachingHints',
'resources/templates/list',
fields
)
);
} catch (error) {
checks.push({
id: 'sep-2549-resources-templates-list-caching-hints',
name: 'ResourcesTemplatesListCachingHints',
description:
'resources/templates/list response includes ttlMs and cacheScope caching hints',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `resources/templates/list request failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS
});
}
// 5. resources/read — use first resource from resources/list
if (firstResourceUri) {
try {
const readResult = await connection.client.request(
{
method: 'resources/read',
params: { uri: firstResourceUri }
},
ReadResourceResultSchema
);
const fields = extractCachingFields(
readResult as Record<string, unknown>
);
allFields.push({ endpoint: 'resources/read', fields });
checks.push(
buildPresenceCheck(
'sep-2549-resources-read-caching-hints',
'ResourcesReadCachingHints',
'resources/read',
fields
)
);
} catch (error) {
checks.push({ checks.push({
id: 'sep-2549-resources-read-caching-hints', id: checkId,
name: 'ResourcesReadCachingHints', name: checkName,
description: description,
'resources/read response includes ttlMs and cacheScope caching hints',
status: 'FAILURE', status: 'FAILURE',
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
errorMessage: `resources/read request failed: ${error instanceof Error ? error.message : String(error)}`, errorMessage: error
specReferences: SPEC_REFS ? `${endpoint} returned JSON-RPC error ${error.code}: ${error.message}`
: `${endpoint} returned HTTP ${response.status} with no result`,
specReferences: SPEC_REFS,
details: { httpStatus: response.status, error }
}); });
return undefined;
} }
const fields = extractCachingFields(result);
allFields.push({ endpoint, fields });
checks.push(buildPresenceCheck(checkId, checkName, endpoint, fields));
return result;
} catch (error) {
checks.push({
id: checkId,
name: checkName,
description,
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `${endpoint} request failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS
});
return undefined;
} }
};
// 6. Aggregate: ttlMs must be a non-negative integer // 1. tools/list
const ttlErrors: string[] = []; await queryEndpoint(
const endpointsWithTtl = allFields.filter((f) => f.fields.hasTtlMs); 'sep-2549-tools-list-caching-hints',
if (endpointsWithTtl.length === 0) { 'ToolsListCachingHints',
ttlErrors.push('no endpoints returned ttlMs'); 'tools/list'
} else { );
for (const { endpoint, fields } of endpointsWithTtl) {
const val = fields.ttlMs;
if (typeof val !== 'number') {
ttlErrors.push(
`${endpoint}: ttlMs is ${typeof val}, expected number`
);
} else if (!Number.isInteger(val)) {
ttlErrors.push(`${endpoint}: ttlMs is ${val}, expected integer`);
} else if (val < 0) {
ttlErrors.push(`${endpoint}: ttlMs is ${val}, must be >= 0`);
}
}
}
checks.push({ // 2. prompts/list
id: 'sep-2549-ttl-non-negative', await queryEndpoint(
name: 'TtlNonNegative', 'sep-2549-prompts-list-caching-hints',
description: 'All ttlMs values are non-negative integers', 'PromptsListCachingHints',
status: ttlErrors.length === 0 ? 'SUCCESS' : 'FAILURE', 'prompts/list'
timestamp: new Date().toISOString(), );
errorMessage: ttlErrors.length > 0 ? ttlErrors.join('; ') : undefined,
specReferences: SPEC_REFS,
details: {
endpoints: allFields.map((f) => ({
endpoint: f.endpoint,
ttlMs: f.fields.ttlMs
}))
}
});
// 7. Aggregate: cacheScope must be "public" or "private" // 3. resources/list
const scopeErrors: string[] = []; const resourcesResult = await queryEndpoint(
const endpointsWithScope = allFields.filter( 'sep-2549-resources-list-caching-hints',
(f) => f.fields.hasCacheScope 'ResourcesListCachingHints',
'resources/list'
);
// 4. resources/templates/list
await queryEndpoint(
'sep-2549-resources-templates-list-caching-hints',
'ResourcesTemplatesListCachingHints',
'resources/templates/list'
);
// 5. resources/read — use first resource from resources/list
const resources = resourcesResult?.resources as
| Array<{ uri?: string }>
| undefined;
const firstResourceUri = resources?.[0]?.uri;
if (firstResourceUri) {
await queryEndpoint(
'sep-2549-resources-read-caching-hints',
'ResourcesReadCachingHints',
'resources/read',
{ uri: firstResourceUri }
); );
if (endpointsWithScope.length === 0) { } else {
scopeErrors.push('no endpoints returned cacheScope'); // Keep the emitted check-ID set stable even when there is nothing to
} else { // read (resources/list failed or the server exposes no resources).
for (const { endpoint, fields } of endpointsWithScope) {
const val = fields.cacheScope;
if (val !== 'public' && val !== 'private') {
scopeErrors.push(
`${endpoint}: cacheScope is ${JSON.stringify(val)}, expected "public" or "private"`
);
}
}
}
checks.push({ checks.push({
id: 'sep-2549-cache-scope-valid', id: 'sep-2549-resources-read-caching-hints',
name: 'CacheScopeValid', name: 'ResourcesReadCachingHints',
description: 'All cacheScope values are "public" or "private"', description:
status: scopeErrors.length === 0 ? 'SUCCESS' : 'FAILURE', 'resources/read response includes ttlMs and cacheScope caching hints',
status: 'SKIPPED',
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
errorMessage: errorMessage:
scopeErrors.length > 0 ? scopeErrors.join('; ') : undefined, 'resources/read was not exercised: resources/list failed or returned no resources.',
specReferences: SPEC_REFS,
details: {
endpoints: allFields.map((f) => ({
endpoint: f.endpoint,
cacheScope: f.fields.cacheScope
}))
}
});
await connection.close();
} catch (error) {
// Connection-level failure — push a single failure check
checks.push({
id: 'sep-2549-caching-connection',
name: 'CachingConnection',
description: 'Caching hints scenario failed to connect',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Connection failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS specReferences: SPEC_REFS
}); });
} }
// 6. Aggregate: ttlMs must be a non-negative integer
const ttlErrors: string[] = [];
const endpointsWithTtl = allFields.filter((f) => f.fields.hasTtlMs);
if (endpointsWithTtl.length === 0) {
ttlErrors.push('no endpoints returned ttlMs');
} else {
for (const { endpoint, fields } of endpointsWithTtl) {
const val = fields.ttlMs;
if (typeof val !== 'number') {
ttlErrors.push(
`${endpoint}: ttlMs is ${typeof val}, expected number`
);
} else if (!Number.isInteger(val)) {
ttlErrors.push(`${endpoint}: ttlMs is ${val}, expected integer`);
} else if (val < 0) {
ttlErrors.push(`${endpoint}: ttlMs is ${val}, must be >= 0`);
}
}
}
checks.push({
id: 'sep-2549-ttl-non-negative',
name: 'TtlNonNegative',
description: 'All ttlMs values are non-negative integers',
status: ttlErrors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: ttlErrors.length > 0 ? ttlErrors.join('; ') : undefined,
specReferences: SPEC_REFS,
details: {
endpoints: allFields.map((f) => ({
endpoint: f.endpoint,
ttlMs: f.fields.ttlMs
}))
}
});
// 7. Aggregate: cacheScope must be "public" or "private"
const scopeErrors: string[] = [];
const endpointsWithScope = allFields.filter((f) => f.fields.hasCacheScope);
if (endpointsWithScope.length === 0) {
scopeErrors.push('no endpoints returned cacheScope');
} else {
for (const { endpoint, fields } of endpointsWithScope) {
const val = fields.cacheScope;
if (val !== 'public' && val !== 'private') {
scopeErrors.push(
`${endpoint}: cacheScope is ${JSON.stringify(val)}, expected "public" or "private"`
);
}
}
}
checks.push({
id: 'sep-2549-cache-scope-valid',
name: 'CacheScopeValid',
description: 'All cacheScope values are "public" or "private"',
status: scopeErrors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: scopeErrors.length > 0 ? scopeErrors.join('; ') : undefined,
specReferences: SPEC_REFS,
details: {
endpoints: allFields.map((f) => ({
endpoint: f.endpoint,
cacheScope: f.fields.cacheScope
}))
}
});
return checks; return checks;
} }
} }

View file

@ -20,7 +20,7 @@ import {
ConformanceCheck, ConformanceCheck,
DRAFT_PROTOCOL_VERSION DRAFT_PROTOCOL_VERSION
} from '../../types'; } from '../../types';
import { connectToServer } from './client-helper'; import { withRequestMeta, sendStatelessRequest } from './stateless-client';
const SPEC_REFERENCE = { const SPEC_REFERENCE = {
id: 'SEP-2243-Server-Validation', id: 'SEP-2243-Server-Validation',
@ -264,51 +264,39 @@ export class HttpHeaderValidationScenario implements ClientScenario {
async run(serverUrl: string): Promise<ConformanceCheck[]> { async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = []; const checks: ConformanceCheck[] = [];
let sessionId: string | null = null;
try { try {
// Establish a session via normal SDK initialization // Discover the server's tools with a fully-conformant stateless request
const connection = await connectToServer(serverUrl); // (SEP-2575) — that wire protocol has no initialize handshake or sessions.
const toolsResult = await connection.client.listTools(); const toolsResponse = await sendStatelessRequest(serverUrl, 'tools/list');
await connection.close(); if (!toolsResponse.body?.result) {
// The server under test could not even answer a conformant tools/list:
// Get a fresh session for raw requests // report a single explicit setup failure instead of misleading
const initResponse = await sendRawRequest( // per-case results against a broken server.
serverUrl, const rpcError = toolsResponse.body?.error;
{ checks.push({
jsonrpc: '2.0', id: 'sep-2243-server-standard-setup',
id: 1, name: 'HttpHeaderValidationSetup',
method: 'initialize', description: 'Setup for header validation tests',
params: { status: 'FAILURE',
protocolVersion: DRAFT_PROTOCOL_VERSION, timestamp: new Date().toISOString(),
capabilities: {}, errorMessage:
clientInfo: { `tools/list discovery failed: HTTP ${toolsResponse.status}` +
name: 'conformance-test-raw-client', (rpcError
version: '1.0.0' ? `, JSON-RPC error ${rpcError.code}: ${rpcError.message}`
} : ', no result in response body'),
} specReferences: [SPEC_REFERENCE],
}, details: { httpStatus: toolsResponse.status, error: rpcError }
{ 'Mcp-Method': 'initialize' } });
); return checks;
if (initResponse.status === 200) {
const rawSid = initResponse.headers['mcp-session-id'];
sessionId = (Array.isArray(rawSid) ? rawSid[0] : rawSid) || null;
const notifHeaders: Record<string, string> = {
'Mcp-Method': 'notifications/initialized'
};
if (sessionId) notifHeaders['mcp-session-id'] = sessionId;
await sendRawRequest(
serverUrl,
{ jsonrpc: '2.0', method: 'notifications/initialized' },
notifHeaders
);
} }
const toolsResult = toolsResponse.body.result as {
tools?: Array<{ name: string; inputSchema?: unknown }>;
};
const baseHeaders: Record<string, string> = { const baseHeaders: Record<string, string> = {
'MCP-Protocol-Version': DRAFT_PROTOCOL_VERSION 'MCP-Protocol-Version': DRAFT_PROTOCOL_VERSION
}; };
if (sessionId) baseHeaders['mcp-session-id'] = sessionId;
let idCounter = 100; let idCounter = 100;
const nextId = () => idCounter++; const nextId = () => idCounter++;
@ -501,7 +489,13 @@ export class HttpHeaderValidationScenario implements ClientScenario {
details: Record<string, unknown> details: Record<string, unknown>
): Promise<void> { ): Promise<void> {
try { try {
const requestBody = { ...body, id: body.id === 0 ? nextId() : body.id }; // Issue #311: every raw request carries the SEP-2575 _meta fields — the
// header-validation cases only mangle headers, never the body metadata.
const requestBody = {
...body,
id: body.id === 0 ? nextId() : body.id,
params: withRequestMeta(body.params)
};
const response = await sendRawRequest(serverUrl, requestBody, { const response = await sendRawRequest(serverUrl, requestBody, {
...baseHeaders, ...baseHeaders,
...extraHeaders ...extraHeaders
@ -565,12 +559,37 @@ export class HttpCustomHeaderServerValidationScenario implements ClientScenario
async run(serverUrl: string): Promise<ConformanceCheck[]> { async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = []; const checks: ConformanceCheck[] = [];
let sessionId: string | null = null;
try { try {
const connection = await connectToServer(serverUrl); // Discover the server's tools with a fully-conformant stateless request
const toolsResult = await connection.client.listTools(); // (SEP-2575) — that wire protocol has no initialize handshake or sessions.
await connection.close(); const toolsResponse = await sendStatelessRequest(serverUrl, 'tools/list');
if (!toolsResponse.body?.result) {
// The server under test could not even answer a conformant tools/list:
// report a single explicit setup failure (and backfill the declared
// checks, mirroring the catch path) instead of pretending the
// requirements are not applicable to a broken server.
const rpcError = toolsResponse.body?.error;
checks.push({
id: 'sep-2243-server-custom-setup',
name: 'HttpCustomHeaderServerValidationSetup',
description: 'Setup for custom header server validation tests',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage:
`tools/list discovery failed: HTTP ${toolsResponse.status}` +
(rpcError
? `, JSON-RPC error ${rpcError.code}: ${rpcError.message}`
: ', no result in response body'),
specReferences: [SPEC_REFERENCE_CUSTOM],
details: { httpStatus: toolsResponse.status, error: rpcError }
});
this.failDeclaredChecks(checks);
return checks;
}
const toolsResult = toolsResponse.body.result as {
tools?: Array<{ name: string; inputSchema?: unknown }>;
};
// Find a tool with x-mcp-header annotations // Find a tool with x-mcp-header annotations
const xMcpTool = toolsResult.tools?.find((tool) => { const xMcpTool = toolsResult.tools?.find((tool) => {
@ -602,43 +621,9 @@ export class HttpCustomHeaderServerValidationScenario implements ClientScenario
return checks; return checks;
} }
// Get a fresh session for raw requests
const initResponse = await sendRawRequest(
serverUrl,
{
jsonrpc: '2.0',
id: 1,
method: 'initialize',
params: {
protocolVersion: DRAFT_PROTOCOL_VERSION,
capabilities: {},
clientInfo: {
name: 'conformance-test-base64-client',
version: '1.0.0'
}
}
},
{ 'Mcp-Method': 'initialize' }
);
if (initResponse.status === 200) {
const rawSid2 = initResponse.headers['mcp-session-id'];
sessionId = (Array.isArray(rawSid2) ? rawSid2[0] : rawSid2) || null;
const notifHeaders: Record<string, string> = {
'Mcp-Method': 'notifications/initialized'
};
if (sessionId) notifHeaders['mcp-session-id'] = sessionId;
await sendRawRequest(
serverUrl,
{ jsonrpc: '2.0', method: 'notifications/initialized' },
notifHeaders
);
}
const baseHeaders: Record<string, string> = { const baseHeaders: Record<string, string> = {
'MCP-Protocol-Version': DRAFT_PROTOCOL_VERSION 'MCP-Protocol-Version': DRAFT_PROTOCOL_VERSION
}; };
if (sessionId) baseHeaders['mcp-session-id'] = sessionId;
// Find the first x-mcp-header annotated STRING property // Find the first x-mcp-header annotated STRING property
// that is callable with minimal arguments to avoid schema validation failures // that is callable with minimal arguments to avoid schema validation failures
@ -828,6 +813,16 @@ export class HttpCustomHeaderServerValidationScenario implements ClientScenario
// Declared-but-unemitted -> FAILURE. Reached only when setup threw partway // Declared-but-unemitted -> FAILURE. Reached only when setup threw partway
// through (the gate-out paths emit SKIPPED rows and the happy path emits // through (the gate-out paths emit SKIPPED rows and the happy path emits
// every declared ID). // every declared ID).
this.failDeclaredChecks(checks);
return checks;
}
/**
* Backfill every declared-but-unemitted check ID as FAILURE when setup
* failed before the cases could run, keeping the emitted ID set stable.
*/
private failDeclaredChecks(checks: ConformanceCheck[]): void {
for (const id of CUSTOM_HEADER_SERVER_DECLARED_CHECK_IDS) { for (const id of CUSTOM_HEADER_SERVER_DECLARED_CHECK_IDS) {
if (checks.some((c) => c.id === id)) continue; if (checks.some((c) => c.id === id)) continue;
checks.push({ checks.push({
@ -841,8 +836,6 @@ export class HttpCustomHeaderServerValidationScenario implements ClientScenario
specReferences: [SPEC_REFERENCE_CUSTOM] specReferences: [SPEC_REFERENCE_CUSTOM]
}); });
} }
return checks;
} }
/** /**
@ -890,10 +883,12 @@ export class HttpCustomHeaderServerValidationScenario implements ClientScenario
jsonrpc: '2.0', jsonrpc: '2.0',
id: nextId(), id: nextId(),
method: 'tools/call', method: 'tools/call',
params: { // Issue #311: the body always carries the SEP-2575 _meta fields —
// these cases only vary the Mcp-Param header value.
params: withRequestMeta({
name: toolName, name: toolName,
arguments: { ...defaultArgs, [paramName]: bodyValue } arguments: { ...defaultArgs, [paramName]: bodyValue }
} })
}, },
{ {
...baseHeaders, ...baseHeaders,
@ -974,10 +969,12 @@ export class HttpCustomHeaderServerValidationScenario implements ClientScenario
jsonrpc: '2.0', jsonrpc: '2.0',
id: nextId(), id: nextId(),
method: 'tools/call', method: 'tools/call',
params: { // Issue #311: the body always carries the SEP-2575 _meta fields —
// this case only omits the Mcp-Param header.
params: withRequestMeta({
name: toolName, name: toolName,
arguments: { ...defaultArgs, [paramName]: 'test-value' } arguments: { ...defaultArgs, [paramName]: 'test-value' }
} })
}, },
{ {
...baseHeaders, ...baseHeaders,

View file

@ -5,60 +5,32 @@
* and a stateless JSON-RPC transport helper. * and a stateless JSON-RPC transport helper.
*/ */
import { DRAFT_PROTOCOL_VERSION } from '../../types';
// ─── JSON-RPC Types ────────────────────────────────────────────────────────── // ─── JSON-RPC Types ──────────────────────────────────────────────────────────
export interface JsonRpcResponse { export type { JsonRpcResponse } from './stateless-client';
jsonrpc: '2.0';
id: number;
result?: Record<string, unknown>;
error?: { code: number; message: string; data?: unknown };
}
// ─── Stateless RPC Helper ──────────────────────────────────────────────────── // ─── Stateless RPC Helper ────────────────────────────────────────────────────
let nextId = 1; import { sendStatelessRequest, JsonRpcResponse } from './stateless-client';
/** /**
* Send a stateless JSON-RPC request (SEP-2575 pattern). * Send a stateless JSON-RPC request (SEP-2575 pattern).
* Automatically injects _meta with protocolVersion, clientInfo, clientCapabilities. * The shared stateless helper injects the cross-cutting requirements: _meta
* (protocolVersion, clientInfo, clientCapabilities) and the standard
* MCP-Protocol-Version / Mcp-Method / Mcp-Name headers (SEP-2243).
*/ */
export async function sendRpc( export async function sendRpc(
serverUrl: string, serverUrl: string,
method: string, method: string,
params?: Record<string, unknown> params?: Record<string, unknown>
): Promise<JsonRpcResponse> { ): Promise<JsonRpcResponse> {
const id = nextId++; const response = await sendStatelessRequest(serverUrl, method, params);
if (!response.body) {
const enrichedParams = { throw new Error(
...params, `Expected a JSON-RPC response for ${method}, got HTTP ${response.status} (${response.contentType ?? 'no content-type'})`
_meta: { );
'io.modelcontextprotocol/protocolVersion': DRAFT_PROTOCOL_VERSION, }
'io.modelcontextprotocol/clientInfo': { return response.body as JsonRpcResponse;
name: 'conformance-test-client',
version: '1.0.0'
},
'io.modelcontextprotocol/clientCapabilities': {
sampling: {},
elicitation: {},
roots: { listChanged: true }
},
...(params?._meta as Record<string, unknown> | undefined)
}
};
const response = await fetch(serverUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Accept: 'application/json',
'MCP-Protocol-Version': DRAFT_PROTOCOL_VERSION
},
body: JSON.stringify({ jsonrpc: '2.0', id, method, params: enrichedParams })
});
return (await response.json()) as JsonRpcResponse;
} }
// ─── InputRequiredResult Types ─────────────────────────────────────────────── // ─── InputRequiredResult Types ───────────────────────────────────────────────

View file

@ -8,10 +8,10 @@ import {
DRAFT_PROTOCOL_VERSION DRAFT_PROTOCOL_VERSION
} from '../../types'; } from '../../types';
import { connectToServer } from './client-helper'; import { connectToServer } from './client-helper';
import { sendStatelessRequest } from './stateless-client';
import { import {
TextResourceContents, TextResourceContents,
BlobResourceContents, BlobResourceContents
McpError
} from '@modelcontextprotocol/sdk/types.js'; } from '@modelcontextprotocol/sdk/types.js';
export class ResourcesListScenario implements ClientScenario { export class ResourcesListScenario implements ClientScenario {
@ -484,9 +484,13 @@ This scenario does not require the server to register any specific resource —
} }
]; ];
let connection; // SEP-2164 is a draft-spec requirement, so the request is sent statelessly
// with the draft protocol version and the cross-cutting _meta/headers.
let response;
try { try {
connection = await connectToServer(serverUrl); response = await sendStatelessRequest(serverUrl, 'resources/read', {
uri: nonexistentUri
});
} catch (error) { } catch (error) {
checks.push({ checks.push({
id: 'sep-2164-error-code', id: 'sep-2164-error-code',
@ -495,19 +499,16 @@ This scenario does not require the server to register any specific resource —
'Server returns -32602 (Invalid Params) for non-existent resource', 'Server returns -32602 (Invalid Params) for non-existent resource',
status: 'FAILURE', status: 'FAILURE',
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
errorMessage: `Failed to connect: ${error instanceof Error ? error.message : String(error)}`, errorMessage: `resources/read request failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences specReferences
}); });
return checks; return checks;
} }
let caughtError: unknown; const result = response.body?.result as
let result: { contents: unknown[] } | undefined; | { contents?: unknown[] }
try { | undefined;
result = await connection.client.readResource({ uri: nonexistentUri }); const rpcError = response.body?.error;
} catch (error) {
caughtError = error;
}
// Check 1: MUST NOT return an empty contents array // Check 1: MUST NOT return an empty contents array
const returnedEmptyContents = const returnedEmptyContents =
@ -531,13 +532,12 @@ This scenario does not require the server to register any specific resource —
}); });
// Check 2: SHOULD return JSON-RPC error with code -32602 // Check 2: SHOULD return JSON-RPC error with code -32602
const errorCode = const errorCode = rpcError?.code;
caughtError instanceof McpError ? caughtError.code : undefined;
let errorCodeMessage: string | undefined; let errorCodeMessage: string | undefined;
if (result !== undefined) { if (result !== undefined) {
errorCodeMessage = `Server returned a result instead of an error (contents length: ${result.contents?.length ?? 'undefined'}). Servers SHOULD return a JSON-RPC error for non-existent resources.`; errorCodeMessage = `Server returned a result instead of an error (contents length: ${result.contents?.length ?? 'undefined'}). Servers SHOULD return a JSON-RPC error for non-existent resources.`;
} else if (!(caughtError instanceof McpError)) { } else if (!rpcError) {
errorCodeMessage = `Expected a JSON-RPC error, got: ${caughtError instanceof Error ? caughtError.message : String(caughtError)}`; errorCodeMessage = `Expected a JSON-RPC error, got HTTP ${response.status} with no JSON-RPC error in the body.`;
} else if (errorCode !== -32602) { } else if (errorCode !== -32602) {
errorCodeMessage = errorCodeMessage =
`Expected error code -32602 (Invalid Params), got ${errorCode}. ` + `Expected error code -32602 (Invalid Params), got ${errorCode}. ` +
@ -563,10 +563,7 @@ This scenario does not require the server to register any specific resource —
}); });
// Check 3: SHOULD include uri in error data field // Check 3: SHOULD include uri in error data field
const errorData = const errorData = rpcError?.data as { uri?: string } | undefined;
caughtError instanceof McpError
? (caughtError.data as { uri?: string } | undefined)
: undefined;
const dataUriMatches = errorData?.uri === nonexistentUri; const dataUriMatches = errorData?.uri === nonexistentUri;
checks.push({ checks.push({
@ -574,19 +571,13 @@ This scenario does not require the server to register any specific resource —
name: 'ResourcesNotFoundDataUri', name: 'ResourcesNotFoundDataUri',
description: description:
'Server includes the requested URI in the error data field (SHOULD)', 'Server includes the requested URI in the error data field (SHOULD)',
status: status: rpcError ? (dataUriMatches ? 'SUCCESS' : 'WARNING') : 'FAILURE',
caughtError instanceof McpError
? dataUriMatches
? 'SUCCESS'
: 'WARNING'
: 'FAILURE',
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
errorMessage: errorMessage: rpcError
caughtError instanceof McpError ? dataUriMatches
? dataUriMatches ? undefined
? undefined : `Error data.uri is ${JSON.stringify(errorData?.uri)}, expected "${nonexistentUri}". This is a SHOULD requirement.`
: `Error data.uri is ${JSON.stringify(errorData?.uri)}, expected "${nonexistentUri}". This is a SHOULD requirement.` : 'No JSON-RPC error received; cannot evaluate data field.',
: 'No JSON-RPC error received; cannot evaluate data field.',
specReferences, specReferences,
details: { details: {
requestedUri: nonexistentUri, requestedUri: nonexistentUri,
@ -594,7 +585,6 @@ This scenario does not require the server to register any specific resource —
} }
}); });
await connection.close();
return checks; return checks;
} }
} }

View file

@ -0,0 +1,107 @@
/**
* Unit tests for the shared stateless request helper (SEP-2575 + SEP-2243):
* standard header defaults/overrides, `_meta` injection, and JSON parsing.
*/
import http from 'http';
import { describe, test, expect } from 'vitest';
import {
buildStandardHeaders,
withRequestMeta,
sendStatelessRequest,
CONFORMANCE_CLIENT_INFO,
DEFAULT_CLIENT_CAPABILITIES
} from './stateless-client';
import { DRAFT_PROTOCOL_VERSION } from '../../types';
describe('buildStandardHeaders', () => {
test('sets the standard headers pinned to the draft protocol version', () => {
const headers = buildStandardHeaders('tools/list');
expect(headers['Mcp-Method']).toBe('tools/list');
expect(headers['MCP-Protocol-Version']).toBe(DRAFT_PROTOCOL_VERSION);
expect(headers['Content-Type']).toBe('application/json');
expect(headers.Accept).toContain('application/json');
expect(headers.Accept).toContain('text/event-stream');
expect(headers['Mcp-Name']).toBeUndefined();
});
test('sets Mcp-Name from params.name (tools/call) and params.uri (resources/read)', () => {
expect(
buildStandardHeaders('tools/call', { name: 'echo' })['Mcp-Name']
).toBe('echo');
expect(
buildStandardHeaders('resources/read', { uri: 'file:///a.txt' })[
'Mcp-Name'
]
).toBe('file:///a.txt');
});
test('overrides replace defaults case-insensitively', () => {
const headers = buildStandardHeaders('tools/list', undefined, {
headers: { 'mcp-protocol-version': '2025-06-18' }
});
expect(headers['MCP-Protocol-Version']).toBeUndefined();
expect(headers['mcp-protocol-version']).toBe('2025-06-18');
});
});
describe('withRequestMeta', () => {
test('injects the required _meta fields', () => {
const params = withRequestMeta({ name: 'echo' });
const meta = params._meta as Record<string, unknown>;
expect(meta['io.modelcontextprotocol/protocolVersion']).toBe(
DRAFT_PROTOCOL_VERSION
);
expect(meta['io.modelcontextprotocol/clientInfo']).toEqual(
CONFORMANCE_CLIENT_INFO
);
expect(meta['io.modelcontextprotocol/clientCapabilities']).toEqual(
DEFAULT_CLIENT_CAPABILITIES
);
expect(params.name).toBe('echo');
});
test('keys already present in params._meta win over the defaults', () => {
const params = withRequestMeta({
_meta: { 'io.modelcontextprotocol/protocolVersion': '2025-06-18' }
});
const meta = params._meta as Record<string, unknown>;
expect(meta['io.modelcontextprotocol/protocolVersion']).toBe('2025-06-18');
expect(meta['io.modelcontextprotocol/clientInfo']).toEqual(
CONFORMANCE_CLIENT_INFO
);
});
});
describe('sendStatelessRequest', () => {
test('parses a plain JSON response', async () => {
const server = http.createServer((req, res) => {
let body = '';
req.on('data', (chunk) => {
body += chunk;
});
req.on('end', () => {
const request = JSON.parse(body);
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(
JSON.stringify({
jsonrpc: '2.0',
id: request.id,
result: { tools: [] }
})
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const port = (server.address() as { port: number }).port;
try {
const response = await sendStatelessRequest(
`http://localhost:${port}/`,
'tools/list'
);
expect(response.status).toBe(200);
expect(response.body?.result).toEqual({ tools: [] });
} finally {
await new Promise<void>((resolve) => server.close(() => resolve()));
}
});
});

View file

@ -0,0 +1,274 @@
/**
* Stateless request helpers for server scenarios (SEP-2575 + SEP-2243).
*
* Every request the harness sends to a server under test carries the
* cross-cutting obligations regardless of what a scenario actually tests:
* the standard headers (MCP-Protocol-Version, Mcp-Method, Mcp-Name, Accept)
* and the `_meta` fields (protocolVersion, clientInfo, clientCapabilities),
* pinned to `DRAFT_PROTOCOL_VERSION`. Scenarios exercising these SEPs MUST
* build their requests through these helpers so a strictly-conformant server
* never rejects harness traffic for reasons unrelated to the behaviour under
* test (issues #311, #312, #315).
*/
import { DRAFT_PROTOCOL_VERSION } from '../../types';
export interface JsonRpcResponse {
jsonrpc: '2.0';
id: number | string | null;
result?: Record<string, unknown>;
error?: { code: number; message: string; data?: unknown };
}
export const CONFORMANCE_CLIENT_INFO = {
name: 'conformance-test-client',
version: '1.0.0'
} as const;
export const DEFAULT_CLIENT_CAPABILITIES = {
sampling: {},
elicitation: {},
roots: { listChanged: true }
} as const;
export interface StatelessResponse {
status: number;
headers: Headers;
contentType?: string;
/** The parsed JSON-RPC message (for SSE: the event matching the request id). */
body?: JsonRpcResponse;
/** All parsed events when the response was an SSE / chunked stream. */
events?: unknown[];
/** Raw response text when it could not be parsed as JSON. */
text?: string;
}
let nextRequestId = 1;
/**
* The `Mcp-Name` source field per SEP-2243: `params.name` for tools/call and
* prompts/get, `params.uri` for resources/read; absent otherwise.
*/
export function mcpNameForRequest(
method: string,
params?: Record<string, unknown>
): string | undefined {
if (method === 'tools/call' || method === 'prompts/get') {
return typeof params?.name === 'string' ? params.name : undefined;
}
if (method === 'resources/read') {
return typeof params?.uri === 'string' ? params.uri : undefined;
}
return undefined;
}
/**
* Build the conformant header set for a stateless request: Content-Type,
* Accept (both content types), MCP-Protocol-Version, Mcp-Method and (when the
* method carries one) Mcp-Name. `options.headers` overrides or extends the
* defaults, replacing any default whose name matches case-insensitively.
*/
export function buildStandardHeaders(
method: string,
params?: Record<string, unknown>,
options: { headers?: Record<string, string> } = {}
): Record<string, string> {
const headers: Record<string, string> = {
'Content-Type': 'application/json',
Accept: 'application/json, text/event-stream',
'MCP-Protocol-Version': DRAFT_PROTOCOL_VERSION,
'Mcp-Method': method
};
const name = mcpNameForRequest(method, params);
if (name !== undefined) {
headers['Mcp-Name'] = name;
}
if (options.headers) {
for (const [key, value] of Object.entries(options.headers)) {
// Replace any default that differs only by case, then set the override.
for (const existing of Object.keys(headers)) {
if (existing.toLowerCase() === key.toLowerCase()) {
delete headers[existing];
}
}
headers[key] = value;
}
}
return headers;
}
/**
* Merge params with the conformant `_meta` required on every stateless
* request. Keys already present in `params._meta` win over the defaults.
*/
export function withRequestMeta(
params?: Record<string, unknown>
): Record<string, unknown> {
return {
...params,
_meta: {
'io.modelcontextprotocol/protocolVersion': DRAFT_PROTOCOL_VERSION,
'io.modelcontextprotocol/clientInfo': CONFORMANCE_CLIENT_INFO,
'io.modelcontextprotocol/clientCapabilities': DEFAULT_CLIENT_CAPABILITIES,
...(params?._meta as Record<string, unknown> | undefined)
}
};
}
function isJsonRpcResponseShaped(event: unknown): event is JsonRpcResponse {
return (
typeof event === 'object' &&
event !== null &&
('result' in event || 'error' in event)
);
}
function parseSseLineInto(events: unknown[], rawLine: string): void {
const line = rawLine.trim();
if (!line) return;
const jsonText = line.startsWith('data:')
? line.replace(/^data:\s*/, '')
: line;
try {
events.push(JSON.parse(jsonText));
} catch {
// Non-JSON line (comments, partial frames) — ignore.
}
}
/**
* Read an SSE / chunked-stream response incrementally and resolve as soon as
* the JSON-RPC response matching `requestId` arrives, without waiting for the
* stream to close. If the stream ends (or is aborted) first, returns whatever
* events were parsed, with `body` set to the last response-shaped event.
*/
export async function readSseJsonRpcResponse(
res: Response,
requestId: number | string | null
): Promise<{ events: unknown[]; body?: JsonRpcResponse }> {
const events: unknown[] = [];
const matchesRequest = (event: unknown): event is JsonRpcResponse =>
isJsonRpcResponseShaped(event) && event.id === requestId;
const finish = (): { events: unknown[]; body?: JsonRpcResponse } => {
const match = events.find(matchesRequest);
const lastResponseShaped = [...events]
.reverse()
.find(isJsonRpcResponseShaped);
return { events, body: match ?? lastResponseShaped };
};
if (!res.body) return finish();
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
try {
for (;;) {
let value: Uint8Array | undefined;
let done = false;
try {
({ value, done } = await reader.read());
} catch {
// The stream was aborted (timeout) or dropped — return what arrived.
break;
}
if (value) {
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split(/\r?\n/);
buffer = lines.pop() ?? '';
for (const line of lines) parseSseLineInto(events, line);
if (events.some(matchesRequest)) {
// The response we were waiting for arrived; stop reading the stream.
await reader.cancel().catch(() => {});
break;
}
}
if (done) {
parseSseLineInto(events, buffer);
buffer = '';
break;
}
}
} finally {
try {
reader.releaseLock();
} catch {
// Lock already released (e.g. after cancel) — nothing to do.
}
}
return finish();
}
/**
* Send a single stateless JSON-RPC request with the full set of cross-cutting
* headers and `_meta`. Handles both JSON and SSE responses.
*/
export async function sendStatelessRequest(
serverUrl: string,
method: string,
params?: Record<string, unknown>,
options: { headers?: Record<string, string>; timeoutMs?: number } = {}
): Promise<StatelessResponse> {
const id = nextRequestId++;
const headers = buildStandardHeaders(method, params, {
headers: options.headers
});
const body = JSON.stringify({
jsonrpc: '2.0',
id,
method,
params: withRequestMeta(params)
});
const controller = new AbortController();
const timeout = setTimeout(
() => controller.abort(),
options.timeoutMs ?? 10000
);
try {
const res = await fetch(serverUrl, {
method: 'POST',
headers,
body,
signal: controller.signal
});
const contentType = res.headers.get('content-type') ?? undefined;
if (contentType?.includes('text/event-stream')) {
// Read the stream incrementally and resolve on the matching response —
// a server that keeps the stream open must not stall the harness.
const { events, body: matched } = await readSseJsonRpcResponse(res, id);
return {
status: res.status,
headers: res.headers,
contentType,
events,
body: matched
};
}
const text = await res.text();
try {
return {
status: res.status,
headers: res.headers,
contentType,
body: text ? (JSON.parse(text) as JsonRpcResponse) : undefined
};
} catch {
return { status: res.status, headers: res.headers, contentType, text };
}
} finally {
clearTimeout(timeout);
// Tear down any still-open SSE stream so sockets don't linger.
controller.abort();
}
}

View file

@ -7,6 +7,10 @@ import {
ConformanceCheck, ConformanceCheck,
DRAFT_PROTOCOL_VERSION DRAFT_PROTOCOL_VERSION
} from '../../types'; } from '../../types';
import {
buildStandardHeaders,
readSseJsonRpcResponse
} from './stateless-client';
const SPEC_REF = [ const SPEC_REF = [
{ {
@ -115,12 +119,13 @@ export class ServerStatelessScenario implements ClientScenario {
headersOverrides?: Record<string, string>, headersOverrides?: Record<string, string>,
id: string | number | null = 1 id: string | number | null = 1
) => { ) => {
const headers = { // The cross-cutting SEP-2243 headers (Mcp-Method, Mcp-Name, Accept,
'Content-Type': 'application/json', // MCP-Protocol-Version) are not what this scenario exercises, so they
Accept: 'application/json', // are always sent conformantly; overrides only alter the dimension a
'MCP-Protocol-Version': DRAFT_PROTOCOL_VERSION, // test case is about (issue #312).
...headersOverrides const headers = buildStandardHeaders(method, params, {
}; headers: headersOverrides
});
const body = JSON.stringify({ const body = JSON.stringify({
jsonrpc: '2.0', jsonrpc: '2.0',
@ -131,10 +136,22 @@ export class ServerStatelessScenario implements ClientScenario {
const res = await fetch(serverUrl, { method: 'POST', headers, body }); const res = await fetch(serverUrl, { method: 'POST', headers, body });
let data: any = null; let data: any = null;
try { // Servers may answer single requests over text/event-stream; pick the
data = await res.json(); // JSON-RPC message matching this request id instead of failing to parse
} catch { // the stream as JSON.
// Response might not be JSON const contentType =
typeof res.headers?.get === 'function'
? (res.headers.get('content-type') ?? '')
: '';
if (contentType.includes('text/event-stream')) {
const { body: matched } = await readSseJsonRpcResponse(res, id);
data = matched ?? null;
} else {
try {
data = await res.json();
} catch {
// Response might not be JSON
}
} }
return { res, data }; return { res, data };
}; };
@ -150,11 +167,7 @@ export class ServerStatelessScenario implements ClientScenario {
timeoutMs = 1000, timeoutMs = 1000,
onFirstFrame?: () => Promise<void> onFirstFrame?: () => Promise<void>
): Promise<any[]> => { ): Promise<any[]> => {
const headers = { const headers = buildStandardHeaders(method, params);
'Content-Type': 'application/json',
Accept: 'application/json',
'MCP-Protocol-Version': DRAFT_PROTOCOL_VERSION
};
const body = JSON.stringify({ const body = JSON.stringify({
jsonrpc: '2.0', jsonrpc: '2.0',