modelcontextprotocol/conformance
Watch
1
0
Fork
You've already forked conformance
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: version-aware Connection abstraction for server scenarios (#318)

Browse source 
* feat: vendor spec schema types per-version

Copies schema/{version}/schema.ts from the modelcontextprotocol spec
repo into src/spec-types/{version}.ts so the conformance suite can type
against draft spec versions before any SDK ships them.

npm run sync-schema -- <ref> refreshes the copies and records the spec
commit in src/spec-types/SOURCE.

* feat: add Connection abstraction and RunContext

Connection encapsulates how the conformance suite talks to a
server-under-test for a given spec version:

- connectStateful: 2025-x lifecycle. Thin adapter over the SDK Client
  (initialize handshake, session id, SSE handled by the SDK).
- connectStateless: 2026-x lifecycle (SEP-2575). Raw fetch with
  per-request _meta + MCP-Protocol-Version header. Decoupled from the
  SDK so the suite can test draft spec versions before the SDK
  implements them.

connectFor(specVersion) picks the implementation. RunContext bundles
serverUrl, specVersion and a bound connect() for the runner to hand to
each scenario.

Nothing uses this yet; wiring follows in the next commit.

* refactor: thread RunContext through ClientScenario.run

ClientScenario.run(serverUrl) becomes run(ctx: RunContext). The runner
builds the context from --spec-version and the server URL; scenarios
destructure ctx.serverUrl and otherwise behave identically.

No scenario uses ctx.connect() yet, so behaviour is unchanged: 214/214
tests pass, all-scenarios.test.ts still drives the everything-server
fixture exactly as before.

Test files use a testContext(url) helper to construct a RunContext.
The authorization-server scenario list is retyped to
ClientScenarioForAuthorizationServer since those scenarios test an
OAuth server, not an MCP server, and keep run(serverUrl).

* refactor: migrate server scenarios to ctx.connect() + conn.request()

22 carry-forward and lifecycle scenarios now go through the Connection
abstraction instead of connectToServer + SDK Client:

- tools.ts (8): list, call x6, with-progress
- prompts.ts (5)
- resources.ts (7): list, read x3, subscribe x2, not-found
- utils.ts (3): completion, ping, set-level
- json-schema-2020-12.ts, caching.ts, http-standard-headers.ts

Result types come from spec-types/{introducedIn}. Same scenario code
now passes under both --spec-version 2025-11-25 (SDK-backed stateful)
and --spec-version draft (raw stateless).

ToolsCallSampling/Elicitation/WithLogging and elicitation-* keep
connectToServer (need setRequestHandler/setLoggingLevel SDK surface);
they are tagged removedIn: DRAFT in the next commit. stateless.ts and
input-required-result.ts keep their sendRpc helper; migrating those is
deferred to the DRAFT-scenario coherence pass.

* fix: normalize Connection error to JsonRpcError; clean up RunContext import

connectStateful now catches the SDK's McpError and rethrows as
JsonRpcError so scenarios always see the same error class regardless
of which Connection impl ran. ResourcesNotFoundError uses
instanceof JsonRpcError instead of duck-typing.

types.ts uses a normal top-level import for RunContext instead of an
inline import() type.

* fix(dns-rebinding): use version-appropriate probe body

The 'valid Host accepted' check was sending an initialize body, which
a 2026 server returns 404 for. Probe body is now picked from
ctx.specVersion: initialize for the stateful lifecycle, server/discover
with _meta for the stateless lifecycle. The Host/Origin rejection check
is unchanged since rejection happens before body parsing.

* feat(everything-server): route stateless carry-forward methods to McpServer

Adds an in-memory dispatch client connected to the same McpServer the
stateful path uses. Stateless requests for tools/call, resources/*,
prompts/get and completion/complete that fall through the MRTR-specific
handlers are routed to it, so the fixture serves the carry-forward
scenarios under --spec-version draft without duplicating ~500 lines of
tool/resource/prompt registrations.

tools/list now merges the McpServer's tool list with the MRTR-only
stubs so json-schema-2020-12 finds its tool.

draft suite against the fixture: 36/39 (was 13/39). Remaining 3 are
fixture-side SEP gaps (no SSE forwarding for progress, no SEP-2549
ttlMs, no SEP-2243 Mcp-Method validation in the stateless path).

* refactor(connection): drop unused RequestOptions; move sdk-client; add tests

Addresses self-review findings on the new connection module:

- RequestOptions.handlers/.meta and ServerRequestHandler removed: zero
  callers (the scenarios that motivated them are deferred). They can be
  reintroduced when something actually uses them.
- scenarios/server/client-helper.ts moved to connection/sdk-client.ts so
  the connection module no longer depends on the scenarios module.
- connectStateless: handle CRLF SSE event separators; throw a useful
  error for non-JSON/non-SSE responses instead of a JSON parse error.
- JSONRPCNotification consistently imported from spec-types/2025-11-25.
- New connection.test.ts (8 tests) covering connectFor selection,
  _meta injection, error mapping, SSE LF/CRLF parsing, and the
  server-request-on-stream rejection.

* fix: address bughunt findings (response.ok check; targetVersion naming)

- connectStateless: throw on non-2xx responses that lack a JSON-RPC
  error envelope (e.g. gateway/framework error JSON), matching the
  stateful path's behavior. Previously such a response would return
  undefined as the result.
- json-schema-2020-12: rename negotiatedVersion to targetVersion and
  reword the skip message and details field. The value is ctx.specVersion
  (the run's --spec-version), not a negotiated version; the old wording
  was misleading. Drop dead 'unknown' fallback (specVersion is required)
  and the corresponding undefined test cases.

* fix(sse-multiple-streams): keep scenario in draft; version-aware request scaffolding

Concurrent POSTs each answered with JSON or their own SSE stream are
core transport behavior in the draft spec too, so removedIn was wrong
for this scenario. Only the request scaffolding is version-specific:
stateful runs keep the initialize/session-id setup, draft runs send
_meta + the MCP-Protocol-Version header on each request instead. The
two checks are unchanged.

* refactor: fold stateless-client helpers into connection/stateless

The two stateless request paths were ~90% duplicated:

- scenarios/server/stateless-client.ts (main #319): buildStandardHeaders,
  withRequestMeta, readSseJsonRpcResponse, sendStatelessRequest -> raw
  {status, body}; sends Mcp-Method/Mcp-Name (SEP-2243); has timeout
- connection/stateless.ts (this PR): connectStateless -> Connection.request
  -> throws JsonRpcError; missing Mcp-Method/Mcp-Name; no timeout

connection/stateless.ts now exports both layers:
- low-level sendStatelessRequest() (and the building blocks) moved verbatim
  from stateless-client.ts
- connectStateless() rebuilt as a thin wrapper over sendStatelessRequest():
  classifies SSE events into the notification sink, surfaces server->client
  requests on the response stream as a spec violation, throws JsonRpcError
  on error responses

So connectStateless() now also picks up the SEP-2243 headers and the 10s
timeout that the previous impl lacked.

stateless-client.ts deleted; its 4 importers + test file repointed at
'../../connection'. Test file moved to connection/stateless.test.ts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(everything-server): make stateless resources reach the carry-forward dispatch

The merge of main left two competing handler sets for resources/read,
resources/list and resources/templates/list: the inline stateless handlers
(test://stateless-static-text + a catch-all -32602) and the carry-forward
dispatch to the McpServer-registered resources. The catch-all made the
carry-forward unreachable, so test://static-text, test://static-binary and
the templates errored under --spec-version draft.

Now resources/list and resources/templates/list dispatch to the McpServer and
merge with the inline entries (same pattern as tools/list and prompts/list),
and resources/read keeps the inline static-text response but falls through to
the carry-forward for unknown URIs (preserving SEP-2164 data.uri on errors).

Also wrap the tools/list and prompts/list dispatch in try/finally so
dispatch.close() always runs, matching tools/call.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(scenarios): send SEP-2243 Mcp-Method header on hand-rolled stateless requests

sse-multiple-streams and dns-rebinding bypass sendStatelessRequest() because
they need raw fetch/undici for low-level inspection, but their hand-built
draft-path headers omitted Mcp-Method, so a strictly-conformant draft server
would 400 them before the behaviour under test runs. Build the headers via
buildStandardHeaders() instead, layering the scenario-specific overrides
(Accept ordering, Host/Origin) on top.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(runner): infer draft spec version for draft-only scenarios

When --spec-version is omitted, the runner defaulted to LATEST_SPEC_VERSION,
so draft-only scenarios (caching, resources-not-found, ...) silently ran over
the stateful connection instead of the stateless draft transport they were
written for; all-scenarios.test.ts had the same problem via testContext().
Infer the spec version from scenario.source.introducedIn (as negative.test.ts
already does) when no explicit version is given.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(connection): only throw JsonRpcError for properly-shaped error objects

connectStateless() destructured response.body.error unconditionally, so a
proxy returning {"error": "upstream timeout"} produced a JsonRpcError with
undefined code/message and lost the HTTP status. Require an object with a
numeric code before treating it as a JSON-RPC error; otherwise surface the
HTTP status and raw body in the thrown Error. Adds a unit test for the
non-conformant 502 case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(connection): align connectStateful normalization with connectStateless

Two consistency nits from automated review:
- Collected notifications were missing the jsonrpc field (the SDK's Zod
  parsing strips it), violating the JSONRPCNotification type contract.
- McpError messages kept the SDK's "MCP error <code>: " prefix, so
  JsonRpcError.message differed between the stateful and stateless impls.

Also restore progressToken filtering in tools-call-with-progress, which was
lost when the scenario migrated from the SDK onprogress callback to
conn.notifications: the server must echo the request's token back, and
notifications for other tokens are now ignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(runner): skip inapplicable scenario/spec-version combos; add --force; plumb spec version to the wire

Running a draft-only scenario with an explicit dated --spec-version (e.g.
server-stateless --spec-version 2025-11-25) used to silently pass: the flag
only affected ctx.connect(), which hand-rolling scenarios never call, so
they sent draft requests regardless and the run looked green while testing
nothing the flag claimed.

Three changes:
- buildStandardHeaders / withRequestMeta / sendStatelessRequest /
  connectStateless accept an optional specVersion (default: draft), and
  hand-rolling scenarios pass ctx.specVersion through, so requests declare
  the version the run was invoked with.
- The runner skips (exit 0) when an explicit --spec-version falls outside
  the scenario's introducedIn/removedIn window, with a message instead of
  a misleading green run.
- A new --force flag overrides the skip for deliberate mismatch probing;
  combined with the plumbing, a forced server-stateless run at 2025-11-25
  now genuinely sends that version (8/20 checks pass against the
  everything-server instead of a vacuous 26/26).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Paul Carleton <paulc@anthropic.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Felix Weinberger 2026-06-01 17:09:13 +01:00 • committed by GitHub
commit b1bb7298e8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
45 changed files with 10012 additions and 494 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -6,3 +6,4 @@ dist/
.idea/
.claude/settings.local.json
.sdk-under-test/
.sync-schema-tmp/

5
.prettierignore
View file

@ -4,6 +4,11 @@
# output (and the refresh workflow's `git diff` check).
src/seps/traceability.json
# Vendored verbatim from modelcontextprotocol/schema/{version}/schema.ts via
# `npm run sync-schema`. Keep byte-identical with upstream so the SOURCE pin
# is meaningful and re-syncing produces a clean diff.
src/spec-types/*.ts
# Local tooling workspaces (not part of the repo).
.claude/
.sdk-under-test/

176
examples/servers/typescript/everything-server.ts
View file

@ -21,6 +21,9 @@ import {
import { createMcpExpressApp } from '@modelcontextprotocol/sdk/server/express.js';
import {
ElicitResultSchema,
ResultSchema,
ProgressNotificationSchema,
LoggingMessageNotificationSchema,
ListToolsRequestSchema,
ListPromptsRequestSchema,
ListResourcesRequestSchema,
@ -30,6 +33,8 @@ import {
} from '@modelcontextprotocol/sdk/types.js';
import { z } from 'zod';
import { toJsonSchemaCompat } from '@modelcontextprotocol/sdk/server/zod-json-schema-compat.js';
import { InMemoryTransport } from '@modelcontextprotocol/sdk/inMemory.js';
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import cors from 'cors';
import { randomUUID, createHmac } from 'crypto';
@ -72,6 +77,49 @@ function getMrtInputText(inputResponse: unknown, field: string): string {
const transports: { [sessionId: string]: StreamableHTTPServerTransport } = {};
const servers: { [sessionId: string]: McpServer } = {};
// In-memory client connected to a fully-registered McpServer. Used by the
// stateless POST handler to serve carry-forward methods (tools/call,
// resources/*, prompts/get, completion/complete) without duplicating the
// registrations. The SDK doesn't yet support a stateless server natively,
// so this bridges via the in-memory transport after a one-time initialize.
//
// A fresh server+client pair is built per request so concurrent requests
// can't observe each other's notifications.
type DispatchClient = {
client: Client;
drainNotifications: () => unknown[];
close: () => Promise<void>;
};
async function getStatelessDispatchClient(): Promise<DispatchClient> {
const [clientT, serverT] = InMemoryTransport.createLinkedPair();
const server = createMcpServer();
await server.connect(serverT);
const client = new Client(
{ name: 'stateless-dispatch', version: '1.0.0' },
{ capabilities: { sampling: {}, elicitation: {} } }
);
await client.connect(clientT);
// Buffer notifications so the stateless handler can flush them to the SSE
// response after the request completes. The SDK pre-registers a handler for
// notifications/progress so a fallback alone would miss it.
const buffer: unknown[] = [];
const collect = async (n: unknown) =>
void buffer.push({ jsonrpc: '2.0', ...(n as object) });
client.setNotificationHandler(ProgressNotificationSchema, collect);
client.setNotificationHandler(LoggingMessageNotificationSchema, collect);
client.fallbackNotificationHandler = collect;
return {
client,
drainNotifications: () => buffer.splice(0, buffer.length),
close: async () => {
await client.close();
await server.close();
}
};
}
// In-memory event store for SEP-1699 resumability
const eventStoreData = new Map<
string,
@ -1307,11 +1355,19 @@ app.post('/mcp', async (req, res) => {
}
if (method === 'tools/list') {
const dispatch = await getStatelessDispatchClient();
try {
const fromServer = (await dispatch.client.request(
{ method: 'tools/list', params: {} },
ResultSchema as any
)) as { tools: any[]; [k: string]: unknown };
return res.json({
jsonrpc: '2.0',
id,
result: {
...fromServer,
tools: [
...fromServer.tools,
{
name: 'test_missing_capability',
description: 'Test tool requiring sampling',
@ -1380,15 +1436,26 @@ app.post('/mcp', async (req, res) => {
cacheScope: 'public'
}
});
} finally {
await dispatch.close();
}
}
// Mock fallbacks to answer prompts capability matches safely
if (method === 'prompts/list') {
const dispatch = await getStatelessDispatchClient();
try {
const fromServer = (await dispatch.client.request(
{ method: 'prompts/list', params: {} },
ResultSchema as any
)) as { prompts: any[]; [k: string]: unknown };
return res.json({
jsonrpc: '2.0',
id,
result: {
...fromServer,
prompts: [
...fromServer.prompts,
{
name: 'test_input_required_result_prompt',
description: 'MRTR: prompt that requires elicitation input'
@ -1398,6 +1465,9 @@ app.post('/mcp', async (req, res) => {
cacheScope: 'public'
}
});
} finally {
await dispatch.close();
}
}
// SEP-2322 MRTR: prompts/get handler
@ -1450,13 +1520,24 @@ app.post('/mcp', async (req, res) => {
}
}
// Resources on the stateless path (SEP-2575): SEP-2549 hints + SEP-2164 errors.
// Resources on the stateless path (SEP-2575): the McpServer-registered
// resources are merged with the stateless-only resource, mirroring the
// tools/list and prompts/list handlers above (SEP-2549 hints + SEP-2164
// errors via the carry-forward dispatch below).
if (method === 'resources/list') {
const dispatch = await getStatelessDispatchClient();
try {
const fromServer = (await dispatch.client.request(
{ method: 'resources/list', params: {} },
ResultSchema as any
)) as { resources: any[]; [k: string]: unknown };
return res.json({
jsonrpc: '2.0',
id,
result: {
...fromServer,
resources: [
...fromServer.resources,
{
uri: 'test://stateless-static-text',
name: 'Stateless Static Text',
@ -1468,18 +1549,30 @@ app.post('/mcp', async (req, res) => {
cacheScope: 'public'
}
});
} finally {
await dispatch.close();
}
}
if (method === 'resources/templates/list') {
const dispatch = await getStatelessDispatchClient();
try {
const fromServer = (await dispatch.client.request(
{ method: 'resources/templates/list', params: {} },
ResultSchema as any
)) as { resourceTemplates: any[]; [k: string]: unknown };
return res.json({
jsonrpc: '2.0',
id,
result: {
resourceTemplates: [],
...fromServer,
ttlMs: 300000,
cacheScope: 'public'
}
});
} finally {
await dispatch.close();
}
}
if (method === 'resources/read') {
@ -1501,16 +1594,8 @@ app.post('/mcp', async (req, res) => {
}
});
}
// SEP-2164: unknown resources get -32602 with the requested uri in data.
return res.status(200).json({
jsonrpc: '2.0',
id,
error: {
code: -32602,
message: 'Resource not found',
data: { uri }
}
});
// Other URIs (including unknown ones) fall through to the carry-forward
// dispatch below, which serves the McpServer-registered resources.
}
if (method === 'tools/call') {
@ -2065,6 +2150,73 @@ app.post('/mcp', async (req, res) => {
}
}
// Carry-forward methods that fell through the MRTR-specific handlers above
// (tools/call for non-MRTR tools, resources/*, prompts/get for non-MRTR
// prompts, completion/complete) are dispatched to the same McpServer the
// stateful path uses, via an in-memory client. This avoids duplicating the
// tool/resource/prompt registrations for the stateless path.
//
// tools/call is served as text/event-stream so progress and logging
// notifications from the underlying tool reach the conformance client.
if (method === 'tools/call') {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache'
});
const write = (msg: unknown) =>
res.write(`event: message\ndata: ${JSON.stringify(msg)}\n\n`);
const dispatch = await getStatelessDispatchClient();
try {
const result = await dispatch.client.request(
{ method, params },
ResultSchema as any
);
for (const n of dispatch.drainNotifications()) write(n);
write({ jsonrpc: '2.0', id, result });
} catch (e: any) {
for (const n of dispatch.drainNotifications()) write(n);
write({
jsonrpc: '2.0',
id,
error: { code: e.code ?? -32603, message: e.message, data: e.data }
});
} finally {
await dispatch.close();
}
return res.end();
}
if (
[
'resources/list',
'resources/read',
'resources/templates/list',
'prompts/get',
'completion/complete'
].includes(method)
) {
const dispatch = await getStatelessDispatchClient();
try {
const result = await dispatch.client.request(
{ method, params },
ResultSchema as any
);
return res.json({ jsonrpc: '2.0', id, result });
} catch (e: any) {
// SEP-2164: unknown resources get -32602 with the requested uri in
// data; the SDK's McpError does not populate data itself.
const data =
e.data ??
(method === 'resources/read' ? { uri: params.uri } : undefined);
return res.json({
jsonrpc: '2.0',
id,
error: { code: e.code ?? -32603, message: e.message, data }
});
} finally {
await dispatch.close();
}
}
// Removed Methods per SEP-2575 (Changed status from 200 to 400/404 per Transport Spec)
if (
[

1
package.json
View file

@ -20,6 +20,7 @@
"lint:fix_check": "npm run lint:fix && git diff --exit-code --quiet",
"tier-check": "node dist/index.js tier-check",
"traceability": "tsx src/index.ts traceability",
"sync-schema": "tsx scripts/sync-schema.ts",
"check": "npm run typecheck && npm run lint",
"typecheck": "tsgo --noEmit",
"prepack": "npm run build",

52
scripts/sync-schema.ts Normal file
View file

@ -0,0 +1,52 @@
#!/usr/bin/env -S npx tsx
/**
* Vendor schema/{version}/schema.ts from the modelcontextprotocol spec repo
* into src/spec-types/{version}.ts at a pinned SHA.
*
* Usage: npm run sync-schema -- <sha-or-ref>
*/
import { execFileSync } from 'node:child_process';
import { mkdirSync, writeFileSync, rmSync, copyFileSync } from 'node:fs';
import { join } from 'node:path';
const VERSIONS = ['2025-03-26', '2025-06-18', '2025-11-25', 'draft'] as const;
const SPEC_REPO =
'https://github.com/modelcontextprotocol/modelcontextprotocol.git';
const OUT_DIR = join(process.cwd(), 'src', 'spec-types');
const ref = process.argv[2];
if (!ref) {
console.error('Usage: npm run sync-schema -- <sha-or-ref>');
process.exit(1);
}
const tmp = join(process.cwd(), '.sync-schema-tmp');
rmSync(tmp, { recursive: true, force: true });
mkdirSync(tmp, { recursive: true });
mkdirSync(OUT_DIR, { recursive: true });
const git = (args: string[]) =>
execFileSync('git', args, { cwd: tmp, encoding: 'utf8' });
try {
console.log(`Fetching ${SPEC_REPO} @ ${ref} ...`);
git(['init', '-q']);
git(['remote', 'add', 'origin', SPEC_REPO]);
git(['fetch', '-q', '--depth', '1', 'origin', ref]);
git(['checkout', '-q', 'FETCH_HEAD']);
const sha = git(['rev-parse', 'HEAD']).trim();
for (const v of VERSIONS) {
copyFileSync(join(tmp, 'schema', v, 'schema.ts'), join(OUT_DIR, `${v}.ts`));
console.log(` ${v} -> src/spec-types/${v}.ts`);
}
writeFileSync(
join(OUT_DIR, 'SOURCE'),
`modelcontextprotocol@${sha}\n`,
'utf8'
);
console.log(`Pinned: modelcontextprotocol@${sha}`);
} finally {
rmSync(tmp, { recursive: true, force: true });
}

130
src/connection/connection.test.ts Normal file
View file

@ -0,0 +1,130 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { connectFor } from './select';
import { connectStateful } from './stateful';
import { connectStateless } from './stateless';
import { JsonRpcError } from './index';
describe('connectFor', () => {
it('returns stateful for dated 2025-x versions', () => {
expect(connectFor('2025-03-26')).toBe(connectStateful);
expect(connectFor('2025-06-18')).toBe(connectStateful);
expect(connectFor('2025-11-25')).toBe(connectStateful);
});
it('returns stateless for the draft version', () => {
// connectFor wraps connectStateless in a closure (to pass the spec
// version through), so identity with connectStateless no longer holds;
// assert it did not select the stateful implementation. The wire-level
// behaviour of the wrapper is covered in stateless.test.ts.
expect(connectFor('DRAFT-2026-v1')).not.toBe(connectStateful);
expect(connectFor('DRAFT-2026-v1')).not.toBe(connectStateless);
});
});
describe('connectStateless', () => {
const mockFetch = vi.fn();
vi.stubGlobal('fetch', mockFetch);
afterEach(() => mockFetch.mockReset());
function jsonResponse(body: unknown, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { 'content-type': 'application/json' }
});
}
function sseResponse(events: string[]) {
return new Response(events.join(''), {
status: 200,
headers: { 'content-type': 'text/event-stream' }
});
}
it('injects required _meta keys and MCP-Protocol-Version header', async () => {
mockFetch.mockResolvedValue(
jsonResponse({ jsonrpc: '2.0', id: 1, result: { ok: true } })
);
const conn = await connectStateless('http://test/mcp');
await conn.request('tools/list');
const [, init] = mockFetch.mock.calls[0];
expect(init.headers['MCP-Protocol-Version']).toBe('DRAFT-2026-v1');
const sent = JSON.parse(init.body);
expect(sent.params._meta['io.modelcontextprotocol/protocolVersion']).toBe(
'DRAFT-2026-v1'
);
expect(
sent.params._meta['io.modelcontextprotocol/clientInfo']
).toBeDefined();
expect(
sent.params._meta['io.modelcontextprotocol/clientCapabilities']
).toBeDefined();
});
it('throws JsonRpcError on JSON-RPC error responses', async () => {
mockFetch.mockResolvedValue(
jsonResponse({
jsonrpc: '2.0',
id: 1,
error: { code: -32601, message: 'Method not found' }
})
);
const conn = await connectStateless('http://test/mcp');
await expect(conn.request('nope')).rejects.toSatisfy(
(e) => e instanceof JsonRpcError && e.code === -32601
);
});
it('throws on non-2xx JSON without a JSON-RPC error envelope', async () => {
mockFetch.mockResolvedValue(
jsonResponse({ detail: 'gateway rejected' }, 502)
);
const conn = await connectStateless('http://test/mcp');
await expect(conn.request('tools/list')).rejects.toThrow(/HTTP 502/);
});
it('throws a useful error for non-JSON non-SSE responses', async () => {
mockFetch.mockResolvedValue(
new Response('<html>500</html>', {
status: 500,
headers: { 'content-type': 'text/html' }
})
);
const conn = await connectStateless('http://test/mcp');
await expect(conn.request('tools/list')).rejects.toThrow(/HTTP 500/);
});
it('parses SSE: collects notifications and returns final result (LF)', async () => {
mockFetch.mockResolvedValue(
sseResponse([
'event: message\ndata: {"jsonrpc":"2.0","method":"notifications/progress","params":{"progress":1}}\n\n',
'event: message\ndata: {"jsonrpc":"2.0","id":1,"result":{"done":true}}\n\n'
])
);
const conn = await connectStateless('http://test/mcp');
const result = await conn.request<{ done: boolean }>('tools/call', {});
expect(result.done).toBe(true);
expect(conn.notifications).toHaveLength(1);
expect(conn.notifications[0].method).toBe('notifications/progress');
});
it('parses SSE with CRLF line endings', async () => {
mockFetch.mockResolvedValue(
sseResponse([
'event: message\r\ndata: {"jsonrpc":"2.0","id":1,"result":{"ok":true}}\r\n\r\n'
])
);
const conn = await connectStateless('http://test/mcp');
const result = await conn.request<{ ok: boolean }>('tools/call', {});
expect(result.ok).toBe(true);
});
it('rejects server-to-client requests on the SSE stream', async () => {
mockFetch.mockResolvedValue(
sseResponse([
'event: message\ndata: {"jsonrpc":"2.0","id":99,"method":"elicitation/create","params":{}}\n\n'
])
);
const conn = await connectStateless('http://test/mcp');
await expect(conn.request('tools/call', {})).rejects.toThrow(/MRTR/);
});
});

78
src/connection/index.ts Normal file
View file

@ -0,0 +1,78 @@
/**
* Version-aware connection abstraction for server-conformance scenarios.
*
* A `Connection` knows how to send JSON-RPC requests to the server-under-test
* using the lifecycle appropriate for the spec version being tested:
*
* - 2025-x: stateful (initialize handshake, Mcp-Session-Id header)
* - 2026-x: stateless (no handshake, per-request _meta + MCP-Protocol-Version)
*
* Scenarios call `ctx.connect()` and then `conn.request(method, params)`; the
* runner picks the implementation based on `--spec-version`. Scenario code is
* the same regardless of which lifecycle is in use.
*/
import type { SpecVersion } from '../types';
import type { JSONRPCNotification } from '../spec-types/2025-11-25';
export interface Connection {
/**
* Send a JSON-RPC request and return its result.
* Throws `JsonRpcError` on JSON-RPC error responses.
*/
request<R = unknown>(
method: string,
params?: Record<string, unknown>
): Promise<R>;
/**
* All notifications received over this connection's lifetime, in arrival
* order. For the stateful impl this includes notifications from the
* standalone GET stream; for stateless it's only those on POST-response
* streams.
*/
readonly notifications: JSONRPCNotification[];
close(): Promise<void>;
}
/**
* Per-run context handed to `ClientScenario.run()`. The runner constructs this
* from the resolved `--spec-version` and server URL.
*/
export interface RunContext {
serverUrl: string;
specVersion: SpecVersion;
/**
* Open a version-appropriate connection to the server-under-test.
* Scenarios that test the connection mechanics themselves (initialize,
* GET-SSE, DNS rebinding) bypass this and use raw fetch.
*/
connect(): Promise<Connection>;
}
export class JsonRpcError extends Error {
constructor(
public readonly code: number,
message: string,
public readonly data?: unknown
) {
super(message);
this.name = 'JsonRpcError';
}
}
export { connectStateful } from './stateful';
export {
connectStateless,
sendStatelessRequest,
buildStandardHeaders,
withRequestMeta,
mcpNameForRequest,
readSseJsonRpcResponse,
CONFORMANCE_CLIENT_INFO,
DEFAULT_CLIENT_CAPABILITIES,
type JsonRpcResponse,
type StatelessResponse
} from './stateless';
export { connectFor } from './select';

94
src/connection/sdk-client.ts Normal file
View file

@ -0,0 +1,94 @@
/**
* Helper utilities for creating MCP clients to test servers
*/
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
import {
LoggingMessageNotificationSchema,
ProgressNotificationSchema
} from '@modelcontextprotocol/sdk/types.js';
export interface MCPClientConnection {
client: Client;
close: () => Promise<void>;
}
/**
* Create and connect an MCP client to a server
*/
export async function connectToServer(
serverUrl: string
): Promise<MCPClientConnection> {
const client = new Client(
{
name: 'conformance-test-client',
version: '1.0.0'
},
{
capabilities: {
// Client capabilities
sampling: {},
elicitation: {}
}
}
);
const transport = new StreamableHTTPClientTransport(new URL(serverUrl));
await client.connect(transport);
return {
client,
close: async () => {
await client.close();
}
};
}
/**
* Helper to collect notifications (logging and progress)
*/
export class NotificationCollector {
private loggingNotifications: any[] = [];
private progressNotifications: any[] = [];
constructor(client: Client) {
// Set up notification handler for logging messages
client.setNotificationHandler(
LoggingMessageNotificationSchema,
(notification) => {
this.loggingNotifications.push(notification);
}
);
// Set up notification handler for progress notifications
client.setNotificationHandler(
ProgressNotificationSchema,
(notification) => {
this.progressNotifications.push(notification);
}
);
}
/**
* Get all collected logging notifications
*/
getLoggingNotifications(): any[] {
return this.loggingNotifications;
}
/**
* Get all collected progress notifications
*/
getProgressNotifications(): any[] {
return this.progressNotifications;
}
/**
* Get all notifications (for backward compatibility)
*/
getNotifications(): any[] {
return this.loggingNotifications;
}
}

25
src/connection/select.ts Normal file
View file

@ -0,0 +1,25 @@
import type { SpecVersion } from '../types';
import type { Connection } from './index';
import { connectStateful } from './stateful';
import { connectStateless } from './stateless';
/**
* Spec versions that use the stateful lifecycle (initialize handshake,
* Mcp-Session-Id). Anything not in this list uses the stateless lifecycle.
*/
const STATEFUL_VERSIONS: ReadonlySet<string> = new Set([
'2024-11-05',
'2025-03-26',
'2025-06-18',
'2025-11-25'
]);
export function connectFor(
specVersion: SpecVersion
): (serverUrl: string) => Promise<Connection> {
return STATEFUL_VERSIONS.has(specVersion)
? connectStateful
: // Pass the version through so stateless requests declare the spec
// version the run was invoked with (matters under --force).
(serverUrl) => connectStateless(serverUrl, specVersion);
}

71
src/connection/stateful.ts Normal file
View file

@ -0,0 +1,71 @@
/**
* Stateful connection: 2025-x lifecycle (initialize handshake, session id).
*
* Backed by the SDK's `Client` so we don't reimplement the handshake, session
* header, or SSE response parsing. The SDK is the driver here, not the
* system-under-test; its own correctness is covered by the client-conformance
* scenarios.
*/
import {
ResultSchema,
McpError,
ProgressNotificationSchema,
LoggingMessageNotificationSchema
} from '@modelcontextprotocol/sdk/types.js';
import { connectToServer } from './sdk-client';
import type { JSONRPCNotification } from '../spec-types/2025-11-25';
import { JsonRpcError, type Connection } from './index';
export async function connectStateful(serverUrl: string): Promise<Connection> {
const { client, close } = await connectToServer(serverUrl);
const notifications: JSONRPCNotification[] = [];
const collect = (n: unknown) => {
// The SDK's Zod parsing strips the jsonrpc field; restore it so collected
// notifications match the JSONRPCNotification wire shape, as
// connectStateless provides.
notifications.push({
jsonrpc: '2.0',
...(n as object)
} as JSONRPCNotification);
};
// The SDK pre-registers a handler for notifications/progress (to drive the
// onprogress callback feature), so it never reaches the fallback. Register
// explicit collectors for the schemas the SDK claims, then a fallback for
// everything else.
client.setNotificationHandler(ProgressNotificationSchema, async (n) =>
collect(n)
);
client.setNotificationHandler(LoggingMessageNotificationSchema, async (n) =>
collect(n)
);
client.fallbackNotificationHandler = async (n) => collect(n);
return {
notifications,
async request<R>(
method: string,
params: Record<string, unknown> = {}
): Promise<R> {
try {
return (await client.request({ method, params }, ResultSchema)) as R;
} catch (e) {
// Normalize so scenarios always see JsonRpcError regardless of impl.
// The SDK prefixes messages with "MCP error <code>: "; strip it so
// the message matches what connectStateless surfaces.
if (e instanceof McpError) {
throw new JsonRpcError(
e.code,
e.message.replace(/^MCP error -?\d+: /, ''),
e.data
);
}
throw e;
}
},
close
};
}

158
src/connection/stateless.test.ts Normal file
View file

@ -0,0 +1,158 @@
/**
* Unit tests for the shared stateless request helper (SEP-2575 + SEP-2243):
* standard header defaults/overrides, `_meta` injection, and JSON parsing.
*/
import http from 'http';
import { describe, test, expect } from 'vitest';
import {
buildStandardHeaders,
withRequestMeta,
sendStatelessRequest,
connectStateless,
CONFORMANCE_CLIENT_INFO,
DEFAULT_CLIENT_CAPABILITIES
} from './stateless';
import { DRAFT_PROTOCOL_VERSION } from '../types';
describe('buildStandardHeaders', () => {
test('sets the standard headers pinned to the draft protocol version', () => {
const headers = buildStandardHeaders('tools/list');
expect(headers['Mcp-Method']).toBe('tools/list');
expect(headers['MCP-Protocol-Version']).toBe(DRAFT_PROTOCOL_VERSION);
expect(headers['Content-Type']).toBe('application/json');
expect(headers.Accept).toContain('application/json');
expect(headers.Accept).toContain('text/event-stream');
expect(headers['Mcp-Name']).toBeUndefined();
});
test('sets Mcp-Name from params.name (tools/call) and params.uri (resources/read)', () => {
expect(
buildStandardHeaders('tools/call', { name: 'echo' })['Mcp-Name']
).toBe('echo');
expect(
buildStandardHeaders('resources/read', { uri: 'file:///a.txt' })[
'Mcp-Name'
]
).toBe('file:///a.txt');
});
test('overrides replace defaults case-insensitively', () => {
const headers = buildStandardHeaders('tools/list', undefined, {
headers: { 'mcp-protocol-version': '2025-06-18' }
});
expect(headers['MCP-Protocol-Version']).toBeUndefined();
expect(headers['mcp-protocol-version']).toBe('2025-06-18');
});
});
describe('withRequestMeta', () => {
test('injects the required _meta fields', () => {
const params = withRequestMeta({ name: 'echo' });
const meta = params._meta as Record<string, unknown>;
expect(meta['io.modelcontextprotocol/protocolVersion']).toBe(
DRAFT_PROTOCOL_VERSION
);
expect(meta['io.modelcontextprotocol/clientInfo']).toEqual(
CONFORMANCE_CLIENT_INFO
);
expect(meta['io.modelcontextprotocol/clientCapabilities']).toEqual(
DEFAULT_CLIENT_CAPABILITIES
);
expect(params.name).toBe('echo');
});
test('keys already present in params._meta win over the defaults', () => {
const params = withRequestMeta({
_meta: { 'io.modelcontextprotocol/protocolVersion': '2025-06-18' }
});
const meta = params._meta as Record<string, unknown>;
expect(meta['io.modelcontextprotocol/protocolVersion']).toBe('2025-06-18');
expect(meta['io.modelcontextprotocol/clientInfo']).toEqual(
CONFORMANCE_CLIENT_INFO
);
});
});
describe('sendStatelessRequest', () => {
test('parses a plain JSON response', async () => {
const server = http.createServer((req, res) => {
let body = '';
req.on('data', (chunk) => {
body += chunk;
});
req.on('end', () => {
const request = JSON.parse(body);
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(
JSON.stringify({
jsonrpc: '2.0',
id: request.id,
result: { tools: [] }
})
);
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const port = (server.address() as { port: number }).port;
try {
const response = await sendStatelessRequest(
`http://localhost:${port}/`,
'tools/list'
);
expect(response.status).toBe(200);
expect(response.body?.result).toEqual({ tools: [] });
} finally {
await new Promise<void>((resolve) => server.close(() => resolve()));
}
});
});
describe('spec version plumbing', () => {
test('buildStandardHeaders sends the requested spec version', () => {
const headers = buildStandardHeaders('tools/list', undefined, {
specVersion: '2025-11-25'
});
expect(headers['MCP-Protocol-Version']).toBe('2025-11-25');
});
test('buildStandardHeaders defaults to the draft version', () => {
const headers = buildStandardHeaders('tools/list');
expect(headers['MCP-Protocol-Version']).toBe(DRAFT_PROTOCOL_VERSION);
});
test('withRequestMeta declares the requested spec version in _meta', () => {
const params = withRequestMeta({}, '2025-11-25');
const meta = params._meta as Record<string, unknown>;
expect(meta['io.modelcontextprotocol/protocolVersion']).toBe('2025-11-25');
});
test('withRequestMeta defaults to the draft version', () => {
const params = withRequestMeta({});
const meta = params._meta as Record<string, unknown>;
expect(meta['io.modelcontextprotocol/protocolVersion']).toBe(
DRAFT_PROTOCOL_VERSION
);
});
});
describe('connectStateless', () => {
test('surfaces HTTP status and body when the error field is not JSON-RPC shaped', async () => {
const server = http.createServer((req, res) => {
req.resume();
req.on('end', () => {
res.writeHead(502, { 'Content-Type': 'application/json' });
res.end(JSON.stringify({ error: 'upstream timeout' }));
});
});
await new Promise<void>((resolve) => server.listen(0, resolve));
const port = (server.address() as { port: number }).port;
try {
const conn = await connectStateless(`http://localhost:${port}/`);
await expect(conn.request('tools/list')).rejects.toThrow(
/HTTP 502.*upstream timeout/
);
} finally {
await new Promise<void>((resolve) => server.close(() => resolve()));
}
});
});

358
src/connection/stateless.ts Normal file
View file

@ -0,0 +1,358 @@
/**
* Stateless connection: 2026-x lifecycle (SEP-2575).
*
* No handshake. Every request carries `_meta` with protocolVersion, clientInfo,
* and clientCapabilities, plus the standard headers (MCP-Protocol-Version,
* Mcp-Method, Mcp-Name per SEP-2243). Implemented with raw fetch so the
* conformance suite can test draft spec versions before the SDK supports them.
*
* Exports two layers:
* - `sendStatelessRequest()` — low-level: returns `{status, headers, body,
* events}` and never throws on JSON-RPC errors. Scenarios that assert on
* HTTP status or error codes use this directly.
* - `connectStateless()` — high-level: a `Connection` whose `request()` calls
* `sendStatelessRequest()` and throws `JsonRpcError` on error responses.
* The runner picks this via `connectFor()` for `--spec-version draft`.
*
* Both build their requests through `buildStandardHeaders()` and
* `withRequestMeta()` so a strictly-conformant server never rejects harness
* traffic for reasons unrelated to the behaviour under test
* (issues #311, #312, #315).
*/
import { DRAFT_PROTOCOL_VERSION, type SpecVersion } from '../types';
import type { JSONRPCNotification } from '../spec-types/2025-11-25';
import { JsonRpcError, type Connection } from './index';
export interface JsonRpcResponse {
jsonrpc: '2.0';
id: number | string | null;
result?: Record<string, unknown>;
error?: { code: number; message: string; data?: unknown };
}
export const CONFORMANCE_CLIENT_INFO = {
name: 'conformance-test-client',
version: '1.0.0'
} as const;
export const DEFAULT_CLIENT_CAPABILITIES = {
sampling: {},
elicitation: {},
roots: { listChanged: true }
} as const;
export interface StatelessResponse {
status: number;
headers: Headers;
contentType?: string;
/** The parsed JSON-RPC message (for SSE: the event matching the request id). */
body?: JsonRpcResponse;
/** All parsed events when the response was an SSE / chunked stream. */
events?: unknown[];
/** Raw response text when it could not be parsed as JSON. */
text?: string;
}
let nextRequestId = 1;
/**
* The `Mcp-Name` source field per SEP-2243: `params.name` for tools/call and
* prompts/get, `params.uri` for resources/read; absent otherwise.
*/
export function mcpNameForRequest(
method: string,
params?: Record<string, unknown>
): string | undefined {
if (method === 'tools/call' || method === 'prompts/get') {
return typeof params?.name === 'string' ? params.name : undefined;
}
if (method === 'resources/read') {
return typeof params?.uri === 'string' ? params.uri : undefined;
}
return undefined;
}
/**
* Build the conformant header set for a stateless request: Content-Type,
* Accept (both content types), MCP-Protocol-Version, Mcp-Method and (when the
* method carries one) Mcp-Name. `options.headers` overrides or extends the
* defaults, replacing any default whose name matches case-insensitively.
* `options.specVersion` sets the MCP-Protocol-Version header (default: draft),
* so scenarios can send the spec version the run was invoked with.
*/
export function buildStandardHeaders(
method: string,
params?: Record<string, unknown>,
options: { headers?: Record<string, string>; specVersion?: SpecVersion } = {}
): Record<string, string> {
const headers: Record<string, string> = {
'Content-Type': 'application/json',
Accept: 'application/json, text/event-stream',
'MCP-Protocol-Version': options.specVersion ?? DRAFT_PROTOCOL_VERSION,
'Mcp-Method': method
};
const name = mcpNameForRequest(method, params);
if (name !== undefined) {
headers['Mcp-Name'] = name;
}
if (options.headers) {
for (const [key, value] of Object.entries(options.headers)) {
// Replace any default that differs only by case, then set the override.
for (const existing of Object.keys(headers)) {
if (existing.toLowerCase() === key.toLowerCase()) {
delete headers[existing];
}
}
headers[key] = value;
}
}
return headers;
}
/**
* Merge params with the conformant `_meta` required on every stateless
* request. Keys already present in `params._meta` win over the defaults.
* `specVersion` sets the declared protocolVersion (default: draft).
*/
export function withRequestMeta(
params?: Record<string, unknown>,
specVersion: SpecVersion = DRAFT_PROTOCOL_VERSION
): Record<string, unknown> {
return {
...params,
_meta: {
'io.modelcontextprotocol/protocolVersion': specVersion,
'io.modelcontextprotocol/clientInfo': CONFORMANCE_CLIENT_INFO,
'io.modelcontextprotocol/clientCapabilities': DEFAULT_CLIENT_CAPABILITIES,
...(params?._meta as Record<string, unknown> | undefined)
}
};
}
function isJsonRpcResponseShaped(event: unknown): event is JsonRpcResponse {
return (
typeof event === 'object' &&
event !== null &&
('result' in event || 'error' in event)
);
}
function parseSseLineInto(events: unknown[], rawLine: string): void {
const line = rawLine.trim();
if (!line) return;
const jsonText = line.startsWith('data:')
? line.replace(/^data:\s*/, '')
: line;
try {
events.push(JSON.parse(jsonText));
} catch {
// Non-JSON line (comments, partial frames) — ignore.
}
}
/**
* Read an SSE / chunked-stream response incrementally and resolve as soon as
* the JSON-RPC response matching `requestId` arrives, without waiting for the
* stream to close. If the stream ends (or is aborted) first, returns whatever
* events were parsed, with `body` set to the last response-shaped event.
*/
export async function readSseJsonRpcResponse(
res: Response,
requestId: number | string | null
): Promise<{ events: unknown[]; body?: JsonRpcResponse }> {
const events: unknown[] = [];
const matchesRequest = (event: unknown): event is JsonRpcResponse =>
isJsonRpcResponseShaped(event) && event.id === requestId;
const finish = (): { events: unknown[]; body?: JsonRpcResponse } => {
const match = events.find(matchesRequest);
const lastResponseShaped = [...events]
.reverse()
.find(isJsonRpcResponseShaped);
return { events, body: match ?? lastResponseShaped };
};
if (!res.body) return finish();
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
try {
for (;;) {
let value: Uint8Array | undefined;
let done = false;
try {
({ value, done } = await reader.read());
} catch {
// The stream was aborted (timeout) or dropped — return what arrived.
break;
}
if (value) {
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split(/\r?\n/);
buffer = lines.pop() ?? '';
for (const line of lines) parseSseLineInto(events, line);
if (events.some(matchesRequest)) {
// The response we were waiting for arrived; stop reading the stream.
await reader.cancel().catch(() => {});
break;
}
}
if (done) {
parseSseLineInto(events, buffer);
buffer = '';
break;
}
}
} finally {
try {
reader.releaseLock();
} catch {
// Lock already released (e.g. after cancel) — nothing to do.
}
}
return finish();
}
/**
* Send a single stateless JSON-RPC request with the full set of cross-cutting
* headers and `_meta`. Handles both JSON and SSE responses.
*/
export async function sendStatelessRequest(
serverUrl: string,
method: string,
params?: Record<string, unknown>,
options: {
headers?: Record<string, string>;
timeoutMs?: number;
specVersion?: SpecVersion;
} = {}
): Promise<StatelessResponse> {
const id = nextRequestId++;
const headers = buildStandardHeaders(method, params, {
headers: options.headers,
specVersion: options.specVersion
});
const body = JSON.stringify({
jsonrpc: '2.0',
id,
method,
params: withRequestMeta(params, options.specVersion)
});
const controller = new AbortController();
const timeout = setTimeout(
() => controller.abort(),
options.timeoutMs ?? 10000
);
try {
const res = await fetch(serverUrl, {
method: 'POST',
headers,
body,
signal: controller.signal
});
const contentType = res.headers.get('content-type') ?? undefined;
if (contentType?.includes('text/event-stream')) {
// Read the stream incrementally and resolve on the matching response —
// a server that keeps the stream open must not stall the harness.
const { events, body: matched } = await readSseJsonRpcResponse(res, id);
return {
status: res.status,
headers: res.headers,
contentType,
events,
body: matched
};
}
const text = await res.text();
try {
return {
status: res.status,
headers: res.headers,
contentType,
body: text ? (JSON.parse(text) as JsonRpcResponse) : undefined
};
} catch {
return { status: res.status, headers: res.headers, contentType, text };
}
} finally {
clearTimeout(timeout);
// Tear down any still-open SSE stream so sockets don't linger.
controller.abort();
}
}
/**
* `Connection` impl for the 2026-x stateless lifecycle. Thin wrapper over
* `sendStatelessRequest()`: classifies SSE-stream events into the notification
* sink, surfaces server→client *requests* on the response stream as a spec
* violation, and throws `JsonRpcError` on error responses.
*/
export async function connectStateless(
serverUrl: string,
specVersion: SpecVersion = DRAFT_PROTOCOL_VERSION
): Promise<Connection> {
const notifications: JSONRPCNotification[] = [];
async function request<R>(
method: string,
params?: Record<string, unknown>
): Promise<R> {
const response = await sendStatelessRequest(serverUrl, method, params, {
specVersion
});
for (const event of response.events ?? []) {
if (typeof event !== 'object' || event === null) continue;
if ('method' in event && !('id' in event)) {
notifications.push(event as JSONRPCNotification);
} else if ('method' in event && 'id' in event) {
throw new JsonRpcError(
-32600,
`Server sent request '${(event as { method: string }).method}' on response stream; stateless lifecycle forbids this (use MRTR)`
);
}
}
const rpcError = response.body?.error;
// Only a properly-shaped JSON-RPC error becomes a JsonRpcError; anything
// else (e.g. a proxy's `{"error": "upstream timeout"}`) falls through so
// the HTTP status and raw body are surfaced below.
if (
typeof rpcError === 'object' &&
rpcError !== null &&
typeof rpcError.code === 'number'
) {
throw new JsonRpcError(rpcError.code, rpcError.message, rpcError.data);
}
if (response.body?.result === undefined) {
throw new Error(
`HTTP ${response.status}: ` +
`expected a JSON-RPC result for '${method}', got ` +
(response.text !== undefined
? `non-JSON body (content-type ${response.contentType ?? '(none)'})`
: response.body !== undefined
? `unexpected body ${JSON.stringify(response.body)}`
: 'no result in response body')
);
}
return response.body.result as R;
}
return {
notifications,
request,
close: async () => {}
};
}

19
src/connection/testing.ts Normal file
View file

@ -0,0 +1,19 @@
import { LATEST_SPEC_VERSION, type SpecVersion } from '../types';
import { connectFor } from './select';
import type { RunContext } from './index';
/**
* Build a RunContext for unit tests that drive a scenario directly.
* Defaults to the latest dated spec version (stateful lifecycle) so existing
* tests keep their pre-RunContext behaviour.
*/
export function testContext(
serverUrl: string,
specVersion: SpecVersion = LATEST_SPEC_VERSION
): RunContext {
return {
serverUrl,
specVersion,
connect: () => connectFor(specVersion)(serverUrl)
};
}

17
src/index.ts
View file

@ -327,6 +327,10 @@ program
'--spec-version <version>',
'Filter scenarios by spec version (cumulative for date versions)'
)
.option(
'--force',
'Run a scenario even if it is not applicable at the requested --spec-version'
)
.option('--verbose', 'Show verbose output (JSON instead of pretty print)')
.action(async (options) => {
try {
@ -344,9 +348,17 @@ program
const result = await runServerConformanceTest(
validated.url,
validated.scenario,
outputDir
outputDir,
specVersionFilter,
options.force ?? false
);
// Inapplicable scenario/spec-version combination (already logged by
// the runner). Not a failure: exit 0.
if (result.skipped) {
process.exit(0);
}
const { failed } = printServerResults(
result.checks,
result.scenarioDescription,
@ -410,7 +422,8 @@ program
const result = await runServerConformanceTest(
validated.url,
scenarioName,
outputDir
outputDir,
specVersionFilter
);
allResults.push({ scenario: scenarioName, checks: result.checks });
} catch (error) {

52
src/runner/server.test.ts Normal file
View file

@ -0,0 +1,52 @@
/**
* Tests for the server-conformance runner: spec-version applicability
* skipping (an explicitly-requested version outside a scenario's window
* skips rather than silently testing something else; --force overrides).
*/
import { describe, test, expect } from 'vitest';
import { runServerConformanceTest } from './server';
import { DRAFT_PROTOCOL_VERSION, LATEST_SPEC_VERSION } from '../types';
// The skip decision happens before any network request, so an unreachable
// URL proves the scenario was not run.
const UNREACHABLE_URL = 'http://127.0.0.1:9/mcp';
describe('runServerConformanceTest spec-version applicability', () => {
test('skips a draft-only scenario at an explicit dated spec version', async () => {
const result = await runServerConformanceTest(
UNREACHABLE_URL,
'server-stateless',
undefined,
LATEST_SPEC_VERSION
);
expect(result.skipped).toBe(true);
expect(result.checks).toEqual([]);
});
test('skips a removed-in-draft scenario at the draft spec version', async () => {
// server-initialize tests the stateful handshake, which the draft
// (stateless) lifecycle removed.
const result = await runServerConformanceTest(
UNREACHABLE_URL,
'server-initialize',
undefined,
DRAFT_PROTOCOL_VERSION
);
expect(result.skipped).toBe(true);
expect(result.checks).toEqual([]);
});
test('does not skip an applicable scenario/spec-version combination', async () => {
// server-stateless at draft is applicable; the runner proceeds to run it
// (against an unreachable server, so checks exist and report failures —
// the point is only that it was not skipped).
const result = await runServerConformanceTest(
UNREACHABLE_URL,
'server-stateless',
undefined,
DRAFT_PROTOCOL_VERSION
);
expect(result.skipped).toBeUndefined();
expect(result.checks.length).toBeGreaterThan(0);
}, 60000);
});

59
src/runner/server.ts
View file

@ -1,7 +1,13 @@
import { promises as fs } from 'fs';
import path from 'path';
import { ConformanceCheck } from '../types';
import { getClientScenario } from '../scenarios';
import {
ConformanceCheck,
SpecVersion,
LATEST_SPEC_VERSION,
DRAFT_PROTOCOL_VERSION
} from '../types';
import { getClientScenario, isScenarioApplicableAt } from '../scenarios';
import { connectFor, type RunContext } from '../connection';
import { createResultDir, formatPrettyChecks } from './utils';
/**
@ -20,11 +26,14 @@ function formatMarkdown(text: string): string {
export async function runServerConformanceTest(
serverUrl: string,
scenarioName: string,
outputDir?: string
outputDir?: string,
specVersion?: SpecVersion,
force = false
): Promise<{
checks: ConformanceCheck[];
resultDir?: string;
scenarioDescription: string;
skipped?: boolean;
}> {
let resultDir: string | undefined;
@ -36,11 +45,53 @@ export async function runServerConformanceTest(
// Scenario is guaranteed to exist by CLI validation
const scenario = getClientScenario(scenarioName)!;
// An explicitly-requested spec version outside the scenario's applicability
// window is a contradiction: running anyway would test something other than
// what the flag claims. Skip (exit 0) unless --force.
if (
specVersion !== undefined &&
!force &&
!isScenarioApplicableAt(scenario.source, specVersion)
) {
const introduced =
'introducedIn' in scenario.source
? `introduced in ${scenario.source.introducedIn}` +
(scenario.source.removedIn !== undefined
? `, removed in ${scenario.source.removedIn}`
: '')
: 'extension scenario, not on the spec timeline';
console.log(
`SKIPPED: scenario '${scenarioName}' is not applicable at spec version ` +
`${specVersion} (${introduced}). Use --force to run it anyway.`
);
return {
checks: [],
resultDir,
scenarioDescription: scenario.description,
skipped: true
};
}
// When --spec-version is omitted, infer the version from the scenario's
// declared source so draft-only scenarios get the draft (stateless)
// connection rather than the stateful latest-spec default.
const resolvedSpecVersion =
specVersion ??
('introducedIn' in scenario.source &&
scenario.source.introducedIn === DRAFT_PROTOCOL_VERSION
? DRAFT_PROTOCOL_VERSION
: LATEST_SPEC_VERSION);
console.log(
`Running client scenario '${scenarioName}' against server: ${serverUrl}`
);
const checks = await scenario.run(serverUrl);
const ctx: RunContext = {
serverUrl,
specVersion: resolvedSpecVersion,
connect: () => connectFor(resolvedSpecVersion)(serverUrl)
};
const checks = await scenario.run(ctx);
if (resultDir) {
await fs.writeFile(

19
src/scenarios/index.ts
View file

@ -232,15 +232,16 @@ export const clientScenarios = new Map<string, ClientScenario>(
);
// All client scenarios for authorization server
const allClientScenariosListForAuthorizationServer: ClientScenario[] = [
const allClientScenariosListForAuthorizationServer: ClientScenarioForAuthorizationServer[] =
[
// Authorization server scenarios
new AuthorizationServerMetadataEndpointScenario()
];
];
// Client scenarios map for authorization server - built from list
export const clientScenariosForAuthorizationServer = new Map<
string,
ClientScenario
ClientScenarioForAuthorizationServer
>(
allClientScenariosListForAuthorizationServer.map((scenario) => [
scenario.name,
@ -398,6 +399,18 @@ function matchesSpecVersion(
);
}
/**
* Whether a scenario's applicability window covers `version`. Used by the
* runner to skip explicitly-requested scenario/spec-version combinations
* that contradict (e.g. a draft-only scenario at a dated spec version).
*/
export function isScenarioApplicableAt(
source: ScenarioSource,
version: SpecVersion
): boolean {
return matchesSpecVersion(source, version);
}
export function listScenariosForSpec(version: SpecVersion): string[] {
return scenariosList
.filter((s) => matchesSpecVersion(s.source, version))

12
src/scenarios/server/all-scenarios.test.ts
View file

@ -1,3 +1,4 @@
import { testContext } from '../../connection/testing';
import { spawn, ChildProcess } from 'child_process';
import { createServer } from 'net';
import {
@ -6,6 +7,7 @@ import {
listDraftClientScenarios,
listPendingClientScenarios
} from '../index';
import { DRAFT_PROTOCOL_VERSION, LATEST_SPEC_VERSION } from '../../types';
import path from 'path';
function getFreePort(): Promise<number> {
@ -139,7 +141,15 @@ describe('Server Scenarios', () => {
throw new Error(`Scenario ${scenarioName} not found`);
}
const checks = await scenario.run(serverUrl);
// Draft-only scenarios expect the draft (stateless) connection, so
// derive the spec version from the scenario's declared source.
const specVersion =
'introducedIn' in scenario.source &&
scenario.source.introducedIn === DRAFT_PROTOCOL_VERSION
? DRAFT_PROTOCOL_VERSION
: LATEST_SPEC_VERSION;
const checks = await scenario.run(testContext(serverUrl, specVersion));
// Verify checks were returned
expect(checks.length).toBeGreaterThan(0);

76
src/scenarios/server/caching.ts
View file

@ -10,7 +10,12 @@ import {
ConformanceCheck,
DRAFT_PROTOCOL_VERSION
} from '../../types';
import { sendStatelessRequest } from './stateless-client';
import {
JsonRpcError,
type Connection,
type RunContext
} from '../../connection';
import type { CacheableResult } from '../../spec-types/draft';
const SPEC_REFS = [
{
@ -30,12 +35,12 @@ interface CachingFields {
hasCacheScope: boolean;
}
function extractCachingFields(result: Record<string, unknown>): CachingFields {
function extractCachingFields(result: Partial<CacheableResult>): CachingFields {
const hasTtlMs = 'ttlMs' in result;
const hasCacheScope = 'cacheScope' in result;
return {
ttlMs: hasTtlMs ? result.ttlMs : undefined,
cacheScope: hasCacheScope ? result.cacheScope : undefined,
ttlMs: result.ttlMs,
cacheScope: result.cacheScope,
hasTtlMs,
hasCacheScope
};
@ -87,56 +92,61 @@ Servers MUST include \`ttlMs\` (integer >= 0) and \`cacheScope\` ("public" or "p
- \`resources/templates/list\`
- \`resources/read\``;
async run(serverUrl: string): Promise<ConformanceCheck[]> {
async run(ctx: RunContext): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
const allFields: Array<{ endpoint: string; fields: CachingFields }> = [];
// SEP-2549 only exists in the draft spec, so each cacheable endpoint is
// queried over the stateless path (SEP-2575): protocolVersion DRAFT-2026-v1
// plus the cross-cutting _meta and standard headers (issue #315).
// queried over the version-appropriate connection. Under --spec-version
// draft that resolves to the stateless impl (SEP-2575): protocolVersion
// DRAFT-2026-v1 plus the cross-cutting _meta and standard headers
// (issue #315).
let conn: Connection;
try {
conn = await ctx.connect();
} catch (error) {
checks.push({
id: 'sep-2549-caching-connection',
name: 'CachingConnection',
description: 'Caching hints scenario failed to connect',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Connection failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS
});
return checks;
}
const queryEndpoint = async (
checkId: string,
checkName: string,
endpoint: string,
params?: Record<string, unknown>
): Promise<Record<string, unknown> | undefined> => {
): Promise<(CacheableResult & Record<string, unknown>) | undefined> => {
const description = `${endpoint} response includes ttlMs and cacheScope caching hints`;
try {
const response = await sendStatelessRequest(
serverUrl,
endpoint,
params
);
const result = response.body?.result;
if (!result) {
const error = response.body?.error;
checks.push({
id: checkId,
name: checkName,
description,
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: error
? `${endpoint} returned JSON-RPC error ${error.code}: ${error.message}`
: `${endpoint} returned HTTP ${response.status} with no result`,
specReferences: SPEC_REFS,
details: { httpStatus: response.status, error }
});
return undefined;
}
const result = await conn.request<
CacheableResult & Record<string, unknown>
>(endpoint, params);
const fields = extractCachingFields(result);
allFields.push({ endpoint, fields });
checks.push(buildPresenceCheck(checkId, checkName, endpoint, fields));
return result;
} catch (error) {
const rpcError = error instanceof JsonRpcError ? error : undefined;
checks.push({
id: checkId,
name: checkName,
description,
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `${endpoint} request failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS
errorMessage: rpcError
? `${endpoint} returned JSON-RPC error ${rpcError.code}: ${rpcError.message}`
: `${endpoint} request failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: SPEC_REFS,
details: rpcError
? { error: { code: rpcError.code, message: rpcError.message } }
: undefined
});
return undefined;
}
@ -198,6 +208,8 @@ Servers MUST include \`ttlMs\` (integer >= 0) and \`cacheScope\` ("public" or "p
});
}
await conn.close();
// 6. Aggregate: ttlMs must be a non-negative integer
const ttlErrors: string[] = [];
const endpointsWithTtl = allFields.filter((f) => f.fields.hasTtlMs);

86
src/scenarios/server/dns-rebinding.ts
View file

@ -5,7 +5,12 @@
* to prevent DNS rebinding attacks. See GHSA-w48q-cv73-mx4w for details.
*/
import { ClientScenario, ConformanceCheck } from '../../types';
import {
ClientScenario,
ConformanceCheck,
DRAFT_PROTOCOL_VERSION
} from '../../types';
import { buildStandardHeaders, type RunContext } from '../../connection';
import { request } from 'undici';
const SPEC_REFERENCES = [
@ -42,32 +47,71 @@ function getHostFromUrl(serverUrl: string): string {
}
/**
* Send an MCP initialize request with custom Host and Origin headers.
* Build a request body that any server for the given spec version should
* accept without prior setup (initialize for the stateful lifecycle,
* server/discover with _meta for the stateless lifecycle).
*/
function probeBody(specVersion: string): {
jsonrpc: '2.0';
id: number;
method: string;
params: Record<string, unknown>;
} {
const clientInfo = {
name: 'conformance-dns-rebinding-test',
version: '1.0.0'
};
if (specVersion === DRAFT_PROTOCOL_VERSION) {
return {
jsonrpc: '2.0',
id: 1,
method: 'server/discover',
params: {
_meta: {
'io.modelcontextprotocol/protocolVersion': specVersion,
'io.modelcontextprotocol/clientInfo': clientInfo,
'io.modelcontextprotocol/clientCapabilities': {}
}
}
};
}
return {
jsonrpc: '2.0',
id: 1,
method: 'initialize',
params: {
protocolVersion: specVersion,
capabilities: {},
clientInfo
}
};
}
/**
* Send an MCP request with custom Host and Origin headers.
* Both headers are set to the same value so that servers checking either
* Host or Origin will properly detect the rebinding attempt.
*/
async function sendRequestWithHostAndOrigin(
serverUrl: string,
hostOrOrigin: string
hostOrOrigin: string,
specVersion: string
): Promise<{ statusCode: number; body: unknown }> {
// Build the SEP-2243 standard headers (Mcp-Method, Accept, ...) for the
// probe's JSON-RPC method so a strictly-conformant server only rejects the
// request for the Host/Origin values under test, then layer the
// scenario-specific headers on top.
const probe = probeBody(specVersion);
const response = await request(serverUrl, {
method: 'POST',
headers: buildStandardHeaders(probe.method, probe.params, {
headers: {
'Content-Type': 'application/json',
Host: hostOrOrigin,
Origin: `http://${hostOrOrigin}`,
Accept: 'application/json, text/event-stream'
},
body: JSON.stringify({
jsonrpc: '2.0',
id: 1,
method: 'initialize',
params: {
protocolVersion: '2025-11-25',
capabilities: {},
clientInfo: { name: 'conformance-dns-rebinding-test', version: '1.0.0' }
'MCP-Protocol-Version': specVersion
}
})
}),
body: JSON.stringify(probe)
});
let body: unknown;
@ -109,7 +153,8 @@ website tricks a user's browser into making requests to the local server.
See: https://github.com/modelcontextprotocol/typescript-sdk/security/advisories/GHSA-w48q-cv73-mx4w`;
async run(serverUrl: string): Promise<ConformanceCheck[]> {
async run(ctx: RunContext): Promise<ConformanceCheck[]> {
const { serverUrl, specVersion } = ctx;
const checks: ConformanceCheck[] = [];
const timestamp = new Date().toISOString();
@ -160,7 +205,8 @@ See: https://github.com/modelcontextprotocol/typescript-sdk/security/advisories/
try {
const response = await sendRequestWithHostAndOrigin(
serverUrl,
attackerHost
attackerHost,
specVersion
);
const isRejected =
response.statusCode >= 400 && response.statusCode < 500;
@ -200,7 +246,11 @@ See: https://github.com/modelcontextprotocol/typescript-sdk/security/advisories/
// Check 2: Valid localhost Host/Origin headers should be accepted (2xx response)
try {
const response = await sendRequestWithHostAndOrigin(serverUrl, validHost);
const response = await sendRequestWithHostAndOrigin(
serverUrl,
validHost,
specVersion
);
const isAccepted =
response.statusCode >= 200 && response.statusCode < 300;

Some files were not shown because too many files have changed in this diff Show more