feat: version-aware MockServer abstraction for client scenarios (#321)

* feat: vendor spec schema types per-version

Copies schema/{version}/schema.ts from the modelcontextprotocol spec
repo into src/spec-types/{version}.ts so the conformance suite can type
against draft spec versions before any SDK ships them.

npm run sync-schema -- <ref> refreshes the copies and records the spec
commit in src/spec-types/SOURCE.

* feat: add Connection abstraction and RunContext

Connection encapsulates how the conformance suite talks to a
server-under-test for a given spec version:

- connectStateful: 2025-x lifecycle. Thin adapter over the SDK Client
  (initialize handshake, session id, SSE handled by the SDK).
- connectStateless: 2026-x lifecycle (SEP-2575). Raw fetch with
  per-request _meta + MCP-Protocol-Version header. Decoupled from the
  SDK so the suite can test draft spec versions before the SDK
  implements them.

connectFor(specVersion) picks the implementation. RunContext bundles
serverUrl, specVersion and a bound connect() for the runner to hand to
each scenario.

Nothing uses this yet; wiring follows in the next commit.

* refactor: thread RunContext through ClientScenario.run

ClientScenario.run(serverUrl) becomes run(ctx: RunContext). The runner
builds the context from --spec-version and the server URL; scenarios
destructure ctx.serverUrl and otherwise behave identically.

No scenario uses ctx.connect() yet, so behaviour is unchanged: 214/214
tests pass, all-scenarios.test.ts still drives the everything-server
fixture exactly as before.

Test files use a testContext(url) helper to construct a RunContext.
The authorization-server scenario list is retyped to
ClientScenarioForAuthorizationServer since those scenarios test an
OAuth server, not an MCP server, and keep run(serverUrl).

* refactor: migrate server scenarios to ctx.connect() + conn.request()

22 carry-forward and lifecycle scenarios now go through the Connection
abstraction instead of connectToServer + SDK Client:

- tools.ts (8): list, call x6, with-progress
- prompts.ts (5)
- resources.ts (7): list, read x3, subscribe x2, not-found
- utils.ts (3): completion, ping, set-level
- json-schema-2020-12.ts, caching.ts, http-standard-headers.ts

Result types come from spec-types/{introducedIn}. Same scenario code
now passes under both --spec-version 2025-11-25 (SDK-backed stateful)
and --spec-version draft (raw stateless).

ToolsCallSampling/Elicitation/WithLogging and elicitation-* keep
connectToServer (need setRequestHandler/setLoggingLevel SDK surface);
they are tagged removedIn: DRAFT in the next commit. stateless.ts and
input-required-result.ts keep their sendRpc helper; migrating those is
deferred to the DRAFT-scenario coherence pass.

* fix: normalize Connection error to JsonRpcError; clean up RunContext import

connectStateful now catches the SDK's McpError and rethrows as
JsonRpcError so scenarios always see the same error class regardless
of which Connection impl ran. ResourcesNotFoundError uses
instanceof JsonRpcError instead of duck-typing.

types.ts uses a normal top-level import for RunContext instead of an
inline import() type.

* fix(dns-rebinding): use version-appropriate probe body

The 'valid Host accepted' check was sending an initialize body, which
a 2026 server returns 404 for. Probe body is now picked from
ctx.specVersion: initialize for the stateful lifecycle, server/discover
with _meta for the stateless lifecycle. The Host/Origin rejection check
is unchanged since rejection happens before body parsing.

* feat(everything-server): route stateless carry-forward methods to McpServer

Adds an in-memory dispatch client connected to the same McpServer the
stateful path uses. Stateless requests for tools/call, resources/*,
prompts/get and completion/complete that fall through the MRTR-specific
handlers are routed to it, so the fixture serves the carry-forward
scenarios under --spec-version draft without duplicating ~500 lines of
tool/resource/prompt registrations.

tools/list now merges the McpServer's tool list with the MRTR-only
stubs so json-schema-2020-12 finds its tool.

draft suite against the fixture: 36/39 (was 13/39). Remaining 3 are
fixture-side SEP gaps (no SSE forwarding for progress, no SEP-2549
ttlMs, no SEP-2243 Mcp-Method validation in the stateless path).

* refactor(connection): drop unused RequestOptions; move sdk-client; add tests

Addresses self-review findings on the new connection module:

- RequestOptions.handlers/.meta and ServerRequestHandler removed: zero
  callers (the scenarios that motivated them are deferred). They can be
  reintroduced when something actually uses them.
- scenarios/server/client-helper.ts moved to connection/sdk-client.ts so
  the connection module no longer depends on the scenarios module.
- connectStateless: handle CRLF SSE event separators; throw a useful
  error for non-JSON/non-SSE responses instead of a JSON parse error.
- JSONRPCNotification consistently imported from spec-types/2025-11-25.
- New connection.test.ts (8 tests) covering connectFor selection,
  _meta injection, error mapping, SSE LF/CRLF parsing, and the
  server-request-on-stream rejection.

* fix: address bughunt findings (response.ok check; targetVersion naming)

- connectStateless: throw on non-2xx responses that lack a JSON-RPC
  error envelope (e.g. gateway/framework error JSON), matching the
  stateful path's behavior. Previously such a response would return
  undefined as the result.
- json-schema-2020-12: rename negotiatedVersion to targetVersion and
  reword the skip message and details field. The value is ctx.specVersion
  (the run's --spec-version), not a negotiated version; the old wording
  was misleading. Drop dead 'unknown' fallback (specVersion is required)
  and the corresponding undefined test cases.

* fix(sse-multiple-streams): keep scenario in draft; version-aware request scaffolding

Concurrent POSTs each answered with JSON or their own SSE stream are
core transport behavior in the draft spec too, so removedIn was wrong
for this scenario. Only the request scaffolding is version-specific:
stateful runs keep the initialize/session-id setup, draft runs send
_meta + the MCP-Protocol-Version header on each request instead. The
two checks are unchanged.

* feat: add MockServer abstraction and ScenarioContext

MockServer encapsulates the lifecycle scaffold a client-conformance
scenario presents to the client-under-test:

- createServerStateful: 2025-x lifecycle. SDK Server +
  StreamableHTTPServerTransport (sessionless mode); the SDK handles the
  initialize handshake.
- createServerStateless: 2026-x lifecycle (SEP-2575). Raw express app
  that validates _meta + MCP-Protocol-Version on every request, serves
  server/discover, routes other methods to the supplied handlers.

createServerFor(specVersion) picks the implementation. ScenarioContext
bundles specVersion and a bound createServer() for the runner to hand
to each scenario.

This is the client-conformance mirror of src/connection (PR #318).
Nothing uses it yet; wiring follows in the next commit.

* refactor: thread ScenarioContext through Scenario.start()

Scenario.start() becomes start(ctx: ScenarioContext). The runner builds
the context from --spec-version (defaulting to LATEST_SPEC_VERSION) and
passes it through; scenarios receive it as _ctx and otherwise behave
identically.

No scenario uses ctx.createServer() yet, so behaviour is unchanged:
231/231 tests pass.

Test files use a testScenarioContext() helper. The runner already
threads MCP_CONFORMANCE_PROTOCOL_VERSION to the spawned client process,
so the fixture-side env wiring is unchanged.

* refactor: migrate tools_call to ctx.createServer; tag 2025-only client scenarios

ToolsCallScenario now goes through ctx.createServer() instead of an
inline express + SDK Server build. Same handlers, same checks; the
assertion now reads from srv.recorded so it works regardless of which
lifecycle scaffold the runner picked.

initialize, sse-retry, and elicitation-defaults are tagged
removedIn: DRAFT (initialize/GET-SSE/SSE-embedded-elicitation are gone
in the 2026 lifecycle; the MRTR sibling for elicitation-defaults is a
follow-up).

spec-version.test.ts: the 'draft is a superset of latest' invariant no
longer holds once removedIn: DRAFT exists; the test now asserts that
any scenario in latest-but-not-draft is explicitly removedIn.

* feat(auth): make createServer helper version-aware via ScenarioContext

The auth helper now takes ctx: ScenarioContext as its first argument
and branches on ctx.specVersion inside the /mcp route: the stateful
path (SDK Server + StreamableHTTPServerTransport) is unchanged; under
the draft version a raw stateless handler validates _meta + the
MCP-Protocol-Version header, serves server/discover, and routes the
same tools/list and tools/call responses.

The PRM endpoint, bearer-auth middleware, and request logger sit above
the branch and are version-independent.

All 25 call sites across the 12 auth scenario files pass ctx through;
ServerLifecycle and the express.Application return type are unchanged
so stop()/getChecks() are untouched.

Deviation from the MockServer wrapper approach: keeping the helper's
return type as express.Application avoids restructuring 25 call sites'
ServerLifecycle handling in this PR. Folding the auth seam onto
ctx.createServer() fully is a follow-up once the lifecycle ownership
moves into MockServer.

* feat(everything-client): pick stateless requester by MCP_CONFORMANCE_PROTOCOL_VERSION

Adds a statelessRequest(serverUrl, method, params) helper that POSTs
with _meta + MCP-Protocol-Version (the SEP-2575 lifecycle), shimming
around the SDK Client not yet supporting stateless mode. The
runRequestMetadataClient handler's meta constants are extracted to share
with the helper.

runBasicClient (initialize, tools_call, json-schema-ref-no-deref) now
branches on MCP_CONFORMANCE_PROTOCOL_VERSION: for the draft version it
uses statelessRequest to call tools/list then tools/call; for dated
versions it keeps the SDK Client path.

The runner already passes MCP_CONFORMANCE_PROTOCOL_VERSION to the
spawned client, so no runner change is needed.

* fix: address review findings on MockServer (dead opts, shared validator, capability derivation, recorded parity, specVersion threading)

- MockServerOptions removed (capabilities/configure had zero callers); opts
  param dropped from createServerStateful/Stateless/For and ScenarioContext.
- validateStatelessRequest extracted from mock-server/stateless and exported;
  both the stateless MockServer and auth/helpers/createServer.ts call it so
  _meta/header/version validation cannot drift.
- isStatefulVersion exported from connection/select; mock-server/select uses
  it instead of duplicating the version set.
- runner/client.ts: env MCP_CONFORMANCE_PROTOCOL_VERSION set unconditionally
  to the resolved version; runInteractiveMode now takes specVersion and the
  CLI passes it.
- createServerStateful: capabilities derived from handler method prefixes;
  newServer() moved inside the try so a capability mismatch surfaces as
  JSON-RPC -32603 instead of an HTML 500. Recording moved to the express
  layer so unregistered methods are captured (parity with stateless).
- readFinalSseMessage return type now declares error.data.
- Tests added for the capability derivation and unregistered-method recording.

* fix(mock-server): record stateless requests before validation; document preamble exclusions

The stateless impl pushed to recorded[] only after validateStatelessRequest
passed, so requests rejected for a missing header or _meta never appeared,
diverging from the stateful impl which records at the HTTP layer before SDK
dispatch. Record before validation, excluding only the server/discover
lifecycle preamble (mirroring the stateful initialize/notifications-initialized
exclusion), and fix the MockServer.recorded docstring which wrongly claimed
stateless excludes nothing.

* refactor(mock-server): tri-state result for validateStatelessRequest (reject/handled/route)

A successful server/discover was previously modeled as {ok: false},
which read as a validation failure but actually meant "already
answered, do not route". Replace the boolean discriminant with an
explicit kind: 'reject' (validation failed), 'handled' (valid and
already answered), or 'route' (dispatch to handlers).

Both consumers (the stateless /mcp route and the auth helper's
handleStateless) still write status/body for reject and handled and
route only on 'route', so responses are unchanged. Add direct unit
tests covering each branch of the new union.

* feat(runner): single-source the version→lifecycle mapping; export MCP_CONFORMANCE_LIFECYCLE

The stateless mock and the example client both hardcoded the draft
protocol version, so dating the draft (or adding a second stateless
version) would silently break them: the mock would keep rejecting
everything but the old constant, and the client would fall back to the
stateful path.

- Derive STATELESS_SPEC_VERSIONS and a lifecycleFor() helper from
  isStatefulVersion() in connection/select.ts, making it the single
  source of truth for the version→lifecycle mapping.
- Thread the resolved spec version into the stateless mock:
  createServerFor() binds it into createServerStateless(), and
  validateStatelessRequest() now takes the supported-versions list as a
  parameter. A mock bound to a version accepts exactly that version and
  echoes it in -32004 rejections and server/discover; unbound mocks
  accept every known stateless version.
- Export MCP_CONFORMANCE_LIFECYCLE (stateful|stateless) to client
  processes alongside MCP_CONFORMANCE_PROTOCOL_VERSION, in both the
  runner and the in-process test helper, so clients in any language can
  pick the lifecycle without keeping their own version map.
- everything-client: import DRAFT_PROTOCOL_VERSION from src/types
  instead of duplicating the string, branch on MCP_CONFORMANCE_LIFECYCLE
  (falling back to the version comparison for older runners), and send
  the runner-resolved version on stateless requests.
- Document MCP_CONFORMANCE_LIFECYCLE in the README and ignore the local
  .claude/ workspace dir in .prettierignore so prettier --check . is not
  tripped by untracked worktree checkouts.

* fix(everything-client): list-only flow for json-schema-ref-no-deref

runBasicClient now follows tools/list with a tools/call on the first
listed tool, but the json-schema-ref-no-deref scenario mock only serves
tools/list, so the call got -32601 and the client exited non-zero.

Register the scenario with a dedicated handler that stops after listing
tools, keeping the existing lifecycle branch (raw stateless request vs
SDK client). Listing is the whole flow for SEP-2106: the check is that
the client never fetches the network $ref canary while processing the
advertised schema.

* fix(tools_call): make getChecks() idempotent

getChecks() pushed into an instance array on every call, so calling it
more than once accumulated duplicate check entries. The runner is
allowed to call getChecks() repeatedly, and other scenarios already
build their result fresh per call for this reason.

Return a freshly built array instead and drop the now-unused instance
field. Add a unit test covering the not-called FAILURE case and
asserting repeated getChecks() calls do not grow the result.

* fix: ensure dispatch and transport cleanup runs when handlers throw

The stateless tools/list and prompts/list handlers in the everything
server awaited dispatch.close() after the in-memory request, so a throw
from dispatch.client.request() leaked the Client+Server+Transport pair.
Wrap both in try/finally, matching the tools/call and catch-all
handlers in the same file.

In the stateful mock server, the res 'close' listener that tears down
the per-request SDK transport and server was registered after the
handleRequest await; a throw there skipped registration and leaked the
pair. Register the listener before handing the request to the
transport.

* fix(everything-client): send Accept header on stateless requests

The draft transport spec requires POST requests to accept both
application/json and text/event-stream. Servers built on the SDK's
StreamableHTTPServerTransport already enforce this and reject
requests without it. Does not yet make json-schema-ref-no-deref pass
under --spec-version draft: the SDK transport also rejects the draft
protocol version itself, which is the known fixture gap until the
2026-native scenarios migrate onto MockServer.

* chore: exclude .claude/ from vitest globs

Local agent worktree checkouts under .claude/worktrees/ carry their
own copies of the test suite, which the bare **/*.test.ts include was
adding to every run (and to the pre-push hook). Mirrors the existing
.prettierignore entry.

* fix(request-metadata): use spec error code -32004 for unsupported protocol version

The draft spec defines -32004 (UnsupportedProtocolVersionError) with
data { supported, requested } and HTTP 400 for protocol version
rejections; -32001 (HeaderMismatch) is reserved for HTTP header
validation failures.

- request-metadata scenario: return -32004 with spec-shaped data for
  the simulated version-negotiation rejection (was -32001)
- everything-client: trigger negotiation on -32004 (was -32001)
- stateless mock server: include the requested version in the -32004
  error data alongside the supported list
- update tests to match and assert the new requested field

* fix(auth): register transport cleanup before handleRequest in createServer

The res 'close' listener that tears down the transport/server pair was
registered after awaiting handleRequest, so a throw mid-request skipped
cleanup registration entirely. Register it first, matching the stateful
mock server.

* fix(everything-server): return JSON-RPC errors from stateless list dispatch on failure

The stateless tools/list and prompts/list dispatch handlers had
try/finally but no catch, so a dispatch failure produced an unhandled
rejection instead of a response. Add the same catch-and-respond block
the carry-forward dispatch handler already uses.

* refactor(runner): drop MCP_CONFORMANCE_LIFECYCLE; derive lifecycle from protocol version

The protocol version fully determines the lifecycle, and deriving it is
part of being a conformant client. The in-repo example client imports
the stateless version set from src/, so there is no drift risk, and a
single env var cannot disagree with itself. README documents the
version→lifecycle mapping for clients in other languages.

* fix(everything-server): return JSON-RPC errors from stateless resource list dispatch on failure

resources/list and resources/templates/list arrived from main with
try/finally but no catch, unlike the sibling tools/list and
prompts/list handlers. A throwing dispatch would escape to Express's
HTML 500 instead of a JSON-RPC error. Mirrors the existing pattern.

* fix(everything-server): include requested field in -32004 error data

The spec's UnsupportedProtocolVersionError requires data to carry both
supported and requested. The mock-server and request-metadata -32004
producers already include it; this was the remaining producer.

* feat(runner): client-side spec-version inference, skip, and --force

Mirrors the server runner's policy from #318: an explicitly requested
--spec-version outside a scenario's applicability window skips it
(exit 0) unless --force; when --spec-version is omitted the version is
inferred from the scenario's applicability instead of defaulting to
the latest dated version.

* fix(auth): gate SEP-837 application_type check to spec versions that include it

The sep-837-application-type-present check fired on every Dynamic Client
Registration regardless of the targeted spec version, but SEP-837 only
exists in the draft spec. Clients conforming to 2025-11-25 or earlier were
failed on a requirement that is not part of the version under test.

Thread the ScenarioContext into createAuthServer (mirroring createServer)
and only emit the check when the resolved spec version includes SEP-837,
i.e. the stateless draft lifecycle. At dated versions the check is not
emitted at all. Audited the other auth helper checks (PKCE, metadata
discovery, token verification); all of them enforce requirements that are
already in the dated specs, so SEP-837 was the only draft-only check.

Unit tests cover both sides: the check fails an omitting client at the
draft version and is absent at 2025-11-25.

* feat(sdk-runner): --spec-version passthrough with per-SDK default

The sdk command forwarded --scenario/--suite/--timeout to the underlying
client/server invocation but had no way to set the spec version, so SDK
runs always used the suite default. Add a --spec-version flag that is
passed through to the inner run.

Also add an optional specVersion field to the per-SDK config, used as the
default when the flag isn't given; an explicit --spec-version overrides
it. Set typescript-sdk-v1 to 2025-11-25: the v1 line targets the latest
dated spec, which keeps draft-only scenarios out of its runs by default.

---------

Co-authored-by: Paul Carleton <paulc@anthropic.com>

README.md

@@ -64,12 +64,13 @@ npx @modelcontextprotocol/conformance client --command "<client-command>" --scen
 - `--command` - The command to run your MCP client (can include flags)
 - `--scenario` - The test scenario to run (e.g., "initialize")
 - `--suite` - Run a suite of tests in parallel: `all`, `core`, `extensions`, `backcompat`, `auth`, `metadata`, `draft` (scenarios targeting the in-progress draft spec), or `sep-835`
-- `--spec-version <version>` - Filter scenarios by spec version (e.g., `2025-11-25`, `DRAFT-2026-v1`; `draft` is accepted as an alias for the current draft identifier). The draft version selects the latest dated release plus any draft-only scenarios
+- `--spec-version <version>` - Filter scenarios by spec version (e.g., `2025-11-25`, `DRAFT-2026-v1`; `draft` is accepted as an alias for the current draft identifier). The draft version selects the latest dated release plus any draft-only scenarios. When omitted, the version is inferred from the scenario's spec applicability (draft-only scenarios run at the draft version, everything else at the latest dated release); an explicitly requested version outside a scenario's applicability window skips the scenario (exit 0) unless `--force` is passed
+- `--force` - Run a scenario even if it is not applicable at the requested `--spec-version`
 - `--expected-failures <path>` - Path to YAML baseline file of known failures (see [Expected Failures](#expected-failures))
 - `--timeout` - Timeout in milliseconds (default: 30000)
 - `--verbose` - Show verbose output
 
-The framework appends `<server-url>` as an argument to your command and sets the `MCP_CONFORMANCE_SCENARIO` environment variable to the scenario name. For scenarios that require additional context (e.g., client credentials), the `MCP_CONFORMANCE_CONTEXT` environment variable contains a JSON object with scenario-specific data. When `--spec-version` is passed, its resolved value is forwarded to the client process as `MCP_CONFORMANCE_PROTOCOL_VERSION`; example clients can use this value directly as their `protocolVersion`. SDKs that hard-code their protocol version can ignore it.
+The framework appends `<server-url>` as an argument to your command and sets the `MCP_CONFORMANCE_SCENARIO` environment variable to the scenario name. For scenarios that require additional context (e.g., client credentials), the `MCP_CONFORMANCE_CONTEXT` environment variable contains a JSON object with scenario-specific data. When `--spec-version` is passed, its resolved value is forwarded to the client process as `MCP_CONFORMANCE_PROTOCOL_VERSION`; example clients can use this value directly as their `protocolVersion`. SDKs that hard-code their protocol version can ignore it. Clients under test must derive the lifecycle from the protocol version they are asked to run: dated versions through `2025-11-25` use the stateful lifecycle (initialize handshake), while the 2026 draft (`DRAFT-2026-v1`) uses the stateless lifecycle (per-request `_meta`).
 
 ### Server Testing
 
@@ -237,6 +238,11 @@ npm start -- sdk --path ../typescript-sdk --skip-build --mode client
 # Narrow to one scenario / suite
 npm start -- sdk --path ../typescript-sdk --mode server --scenario server-initialize
 npm start -- sdk typescript-sdk --mode client --suite auth
+
+# Target a specific spec version (passed through to the underlying run).
+# When omitted, the SDK's `specVersion` from KNOWN_SDKS is used, if set —
+# e.g. typescript-sdk-v1 defaults to 2025-11-25.
+npm start -- sdk typescript-sdk --mode client --spec-version draft
 ```
 
 Build/run commands for each official SDK are looked up by name from [`src/sdk-runner/known-sdks.ts`](src/sdk-runner/known-sdks.ts) — no config file is required in the SDK repo. Resolution order is **CLI flag > built-in entry**, so any field can be overridden on the command line for refs that diverge from the built-in.