* feat(auth): add JWT-bearer helper for WIF client conformance (SEP-1933)
Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable,
tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario
(PR #2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates
cross-app-access.ts to use it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(auth): add JWT-bearer helper for WIF client conformance (SEP-1933)
Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable,
tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario
(PR #2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates
cross-app-access.ts to use it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(auth): add WIF JWT-bearer scenario and negative tests (SEP-1933)
Adds the auth/wif-jwt-bearer client conformance scenario using the RFC 7523
JWT-bearer grant (urn:ietf:params:oauth:grant-type:jwt-bearer).
The scenario pre-signs valid, wrong-audience, and expired JWTs on start()
to simulate cloud workload identity tokens. The conformance AS verifies the
assertion and emits per-class checks (wif-assertion-verified,
wif-assertion-missing, wif-assertion-audience, wif-assertion-expired,
wif-assertion-malformed). Broken example clients exercise the missing-assertion
and wrong-audience failure paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: apply prettier formatting to wif-jwt-bearer.ts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(auth): add expired-assertion negative test; clarify wif-jwt-bearer comments
- Add runWifJwtBearerExpiredAssertion + auth-test-wif-expired-assertion.ts
to exercise the wif-assertion-expired check path (was dead code)
- Clarify WifAssertionVerified description to note iss is not validated
- Add comment explaining clockTolerance: 5 is intentional (same-run keypair)
- Add comment explaining numeric expiresIn is absolute epoch seconds
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): disable DCR, detect retries, improve getChecks description
- Add WIF_CLIENT_ID constant; pre-seed provider's clientInformation() so
the SDK skips Dynamic Client Registration entirely (disableDynamicRegistration
on the auth server + pre-seeded client_id on the provider side)
- Add failedOnce/tokenRequestReceived tracking on the scenario; reject and
record wif-no-retry FAILURE if the client attempts a second token request
after the first fails
- Add hasAttempted guard in WifJwtBearerProvider.prepareTokenRequest() to
throw on retry from the client side
- Fix getChecks() sentinel description: distinguish "no request made" from
"request made but verification failed"
- Add client_id to context and Zod schema
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: prettier formatting on everything-client.ts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): let errors propagate from broken example clients
The try/catch blocks were silently swallowing auth failures, so the
negative tests passed purely because expectedFailureSlugs found the
AS-emitted check — not because client error-surfacing was verified.
allowClientError: true on the test cases handles the non-zero exit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): tag as draft, add scope-rejected check, add SEP traceability
- Change specVersions from ['extension'] to [DRAFT_PROTOCOL_VERSION] and add
source = { introducedIn: DRAFT_PROTOCOL_VERSION } so the scenario is
reachable via --spec-version draft (extension tag excluded it from all
spec-version runs)
- Move registration from extensionScenariosList to draftScenariosList in
index.ts to match the tag/list convention enforced by spec-version.test.ts
- Add wif-assertion-scope-rejected check: AS returns invalid_scope for a
valid JWT when the client requests the reserved 'wif.rejected' scope;
verify client surfaces the error and does not retry
- Add runWifJwtBearerScopeRejected to everything-client.ts; add optional
scope param to WifJwtBearerProvider; add CLI entry point and vitest case
- Add src/seps/sep-1933.yaml with requirements mapped to each check ID,
covering the three deferred checks (iss, sub, jti) with exclusion rationale
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: prettier formatting
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(wif-jwt-bearer): add wif-grant-fallback check for unauthorized_client fallback
Client bug class: WIF client receives unauthorized_client from JWT-bearer grant
and silently switches to authorization_code instead of surfacing the error.
The MCP SDK retries after UnauthorizedClientError (auth.js:152-154), calling
prepareTokenRequest() a second time. WifGrantFallbackProvider exploits this by
returning authorization_code params on the second call.
Spec anchor: RFC 7523 §2.1 — clients MUST use the JWT-bearer grant type;
silent grant-type switching hides authentication failures.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: rename unused scope param in WifGrantFallbackProvider
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(wif-jwt-bearer): use realistic SPIFFE and K8s PSAT token formats
valid_jwt uses a SPIFFE JWT-SVID subject (spiffe://conformance-test.local/...)
issued by a SPIRE-style issuer. wrong_audience_jwt uses a Kubernetes PSAT
subject and kubernetes.io claims, mirroring how K8s projected service-account
tokens look in production. Token verification and client behaviour are
unchanged; the formats ground the scenario in real workload identity platforms.
k8s_issuer and k8s_subject added to context and schema for external clients
that want to construct their own K8s-style assertions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: prettier formatting on wif-jwt-bearer.ts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore(wif-jwt-bearer): drop informational k8s_issuer/k8s_subject from context
These fields were not used by any client handler and are redundant since
the K8s claims are already baked into wrong_audience_jwt.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): address pre-merge review feedback
- Downgrade wif-no-retry, wif-grant-fallback, and wif-assertion-scope-rejected
to WARNING; RFC 7523 is silent on client retry and grant-type switching, so
FAILURE status was not traceable to spec text
- Add WifRetryProvider broken client that re-sends JWT-bearer after
unauthorized_client, making wif-no-retry actually exercisable as a WARNING
- Drop getChecks() sentinel (non-standard pattern that overloaded the
wif-assertion-verified check ID); test harness detects empty check sets
- Add comment on DRAFT_PROTOCOL_VERSION workaround for runner limitation
- Add comment explaining why both slash/no-slash audience forms are accepted
- Tighten schema comments for audience URL constraint and ES256 literal
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): drop decorative fields and neutralise JWT fixtures
- Remove issuer and subject from context schema and scenario return value;
neither is validated by the AS or used by the client, making them
misleading. The iss exclusion rationale is updated to explain why.
- Replace SPIFFE/K8s-PSAT constants and the kubernetes.io additionalClaims
with neutral values across all three JWT fixtures; no check inspects
token format, so format flavour was decorative and invited false
assumptions about fixture semantics.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): remove workaround comment, drop signing_algorithm, annotate retry provider
- Remove self-admitted workaround comment on DRAFT_PROTOCOL_VERSION; the
classification stands on its own and the comment was inviting challenge
- Drop signing_algorithm from context schema and scenario return; the field
was redundant once the value was fixed to ES256, and the assertion is opaque
to the client regardless
- Add comment to WifRetryProvider explaining it deliberately omits the
hasAttempted guard so the SDK retry reaches the AS
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): export scope constants, remove unused AS field, annotate state machine
- Export WIF_TRIGGER_UNAUTHORIZED_SCOPE and WIF_REJECTED_SCOPE from
createWorkloadJwt.ts and import them in everything-client.ts; removes
duplicate bare-string declarations that could silently diverge
- Remove tokenEndpointAuthSigningAlgValuesSupported from mock AS config;
the SDK does not consume this field in the JWT-bearer flow
- Add comment to onTokenRequest clarifying wif-no-retry and wif-grant-fallback
fire on any second request after any failure, not only post-unauthorized_client
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): address pcarleton review comments
- Move all broken WIF provider classes and runners to wif-broken-clients.ts;
everything-client.ts now contains only the conformant WifJwtBearerProvider
- Add sentinel in getChecks() for !tokenRequestReceived to prevent a silent
pass when the client never sends a request
- Drop wif-assertion-scope-rejected check; the no-retry behaviour after
invalid_scope is already covered by wif-no-retry, and sep-1933.yaml updated
accordingly
- Remove audience from context schema and scenario return; the field was
unused by any client implementation
- Expand scenario description to tell a client implementer what to implement
- Apply suggested description fix to wif-grant-fallback (does not assume
unauthorized_client was the specific error that preceded the fallback)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(wif-jwt-bearer): drop number type from expiresIn, use jose duration strings
Jose supports negative duration strings ('-60s', '60 seconds ago') that produce
already-expired tokens without needing absolute epoch arithmetic. Drop the
number overload, update the expired_jwt call site to '-60s', and update the
unit test accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The sep-2575-server-rejects-undeclared-capability check required
error.data.requiredCapabilities to be an array containing 'sampling'. The
schema's MissingRequiredClientCapabilityError defines it as a
ClientCapabilities object ({ "sampling": {} }), as do this repo's own
spec-types and the tasks/required-task-error scenario, so a spec-correct
server could not pass the check. It shipped because the positive -32021
branch had no self-test.
- stateless.ts: accept the object shape; each capability value must itself
be an object ({ "sampling": null } fails). State the shape in the
scenario's implementer-facing description.
- examples/servers/typescript/everything-server.ts: emit the object.
- stateless.test.ts: self-tests for object-passes / array-fails /
null-value-fails, sharing one capability-probe mock helper.
A check that cannot be exercised — the server under test lacks a
diagnostic fixture tool, or advertises a feature it then rejects — was
reported SKIPPED, which is excluded from pass/fail counts, exit codes,
and the expected-failures baseline. A server could dodge the entire
SEP-2575 capability-constraint and subscription surface (7 MUST-level
checks) and still read green, and a baselined failing scenario whose
checks degraded to SKIPPED flipped its baseline entry to stale, which
reads as fixed.
Such checks now fail with a 'Not testable:' message naming the missing
prerequisite and details.untestable for tooling, per the policy in
issue #248 (severity follows the requirement's keyword: MUST -> FAILURE,
SHOULD -> WARNING). Legitimate skips remain: optional capabilities the
server never declared, runner-level spec-version inapplicability, and
the documented tasks-status-notifications harness gap.
Also fixes a test-suite bug found en route: the stream mock's reader
lacked releaseLock, so listenToStream's teardown threw and every
stream-based assertion silently ran against zero frames.
* fix(http-custom-headers): use spec-permitted x-mcp-header types
The http-custom-headers fixture annotated three `number`-typed properties
(priority and float_val) with x-mcp-header. The SEP-2243 spec forbids this:
"MUST only be applied to parameters with primitive types (integer, string,
boolean). Parameters with type `number` are not permitted."
A spec-conformant client MUST reject (exclude from tools/list) any tool whose
x-mcp-header annotations violate the constraints, so it would drop both test
tools and never call them, failing ClientSupportsCustomHeaders through no fault
of its own.
- priority: number -> integer (its value, 42/1, and header are unchanged).
- float_val: drop the x-mcp-header annotation; a floating-point header param is
exactly what the spec prohibits and has no valid replacement. It stays in the
body and is now asserted NOT to be mirrored, like the unannotated query param.
- Treat 'integer' like 'number' in header-value validation and check-id
classification so Priority keeps its cross-SDK numeric tolerance.
* Reword float_val check to the rule actually under test
The new ClientCustomHeaderNoMirrorNumber check asserts absence of a header
for an *unannotated* parameter — the same designated-params-only rule the
query check covers. The previous description stated a 'MUST NOT mirror
number-typed params' requirement that SEP-2243 does not contain (the spec
forbids the annotation on number, with rejection as the consequence; it does
not define a client-side type filter). Reword the description and comment to
the rule actually under test, keeping the number/SEP-2243 context as the
explanatory why. Also trim the float_val schema description to match the
sibling query param style.
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
The scenario has introducedIn: '2025-11-25' and tests SEP-1699 priming events,
which were introduced in that version. Sending '2025-03-26' as the client's
protocol version makes a correctly-behaving server suppress the priming event
(older clients cannot handle empty-data SSE), so the scenario WARNs against a
conformant server.
Verified against typescript-sdk@v2-2026-07-28: with this fix the scenario goes
from 'WARNING: Server did not send priming event' to '3 passed, 0 failed'.
* feat: add positive tests for the Authorization Code Grant
* fix(auth): use import type for AuthorizationServerOptions in types.ts
* fix(auth): make client-id/client-secret optional; rename --secret→--client-secret
* fix(auth): generate PKCE verifier per-run; derive S256 challenge
* fix(auth): build authorization URL with URLSearchParams; drop hardcoded resource
* fix(auth): default token_endpoint_auth_methods_supported to client_secret_basic; add 'none' branch
* fix(auth): check auth-method support before starting callback server
* fix(auth): check error param first; treat state mismatch as fatal; drop fabricated code_challenge assertion
* fix(auth): redact tokens from check details
* fix(auth): harden callback server (path-scoped, close(), error handler, clear timeout)
* feat(auth): print redirect URI and timeout hint before browser prompt
---------
Co-authored-by: Michito Okai <michito.okai.zn@hitachi.com>
The ${{ github.action_path }} template expression resolves to a backslash
path on Windows (D:\a\_actions\...\v0.1.14). When inlined into a bash run
block, bash treats backslashes as escape sequences, mangling the path.
Switching to the auto-injected $GITHUB_ACTION_PATH env var routes through
shell variable expansion, which preserves the literal value.
Fixes#150.
Co-authored-by: aditya <aditya@rapidcircuitry.com>
* feat: add JSON-based setup option to authorization server conformance test
* refactor(auth): single-schema validation for --file; drop duplicate schema/merge
* chore: move authorization-server settings sample to examples/
* test(auth): cover --file merge precedence and validation
* fix(auth): validate --file standalone via .strict(); reject partial/typo'd files
The settings file must be a complete, valid AuthorizationServerOptions on
its own; CLI flags are optional overrides on top. .strict() rejects unknown
keys so a typo'd field surfaces as an error instead of being silently
ignored. ZodError from file parsing is reported with the file path.
* chore: drop placeholder --file merge test pending CLI test pattern
The test re-implemented the merge logic locally, which can drift from
src/index.ts. Proper coverage needs either programmatic commander.parse()
or an extracted loadAuthorizationServerOptions() helper; deferring that
decision to a follow-up rather than establishing the pattern here.
---------
Co-authored-by: Michito Okai <michito.okai.zn@hitachi.com>
#359 made the metadata `issuer` honor the metadataIssuer override so the
2025-03-26 backcompat scenario is RFC 8414 §3.3-compliant. The authorize
callback's `iss` parameter was still derived as `getAuthBaseUrl()+routePrefix`,
so when a scenario sets metadataIssuer (bare origin) but keeps a routePrefix,
the callback `iss` mismatches the metadata `issuer` and an RFC 9207-conforming
client correctly rejects it.
Extract `resolveIssuer()` and use it for both the metadata `issuer` and the
`'correct'`/`'normalized'` callback `iss` so they always match.
Updates SEP-2549 caching traceability text to match
modelcontextprotocol/modelcontextprotocol#2890:
- Scope the five *-caching-hints requirements to results with
resultType: "complete" (interim input_required results carry no
hints).
- Add two excluded client MUST NOT rows from the new Cache Key
section (cache-key mismatch, multi round-trip retry results).
Neither is wire-observable since a conformant client may simply
never cache.
No check IDs change; traceability.json will be refreshed by the
scheduled traceability workflow.
Claude-Session: https://claude.ai/code/session_01Gbqx5LodtAghT5fRYFxeja
* fix(auth): make 2025-03-26 backcompat mock RFC 8414 §3.3-compliant
Drop the routePrefix from auth/2025-03-26-oauth-metadata-backcompat. The
prefix caused createAuthServer to report issuer:'<origin>/oauth' while
serving the metadata at the root /.well-known/oauth-authorization-server —
an RFC 8414 §3.3 mismatch a conforming client must reject. The MCP server
in this scenario only mounts /mcp, so the unprefixed /authorize, /token,
/register routes do not collide (the sibling endpoint-fallback scenario
already mounts them at root).
The scenario continues to test exactly what it's meant to: 2025-03-26-spec
discovery with no PRM, OAuth metadata at the root well-known path.
* fix(auth): keep /oauth prefix in 2025-03-26 backcompat, override issuer instead
Dropping routePrefix made the metadata-advertised endpoints land on the
2025-03-26 hardcoded fallback paths, so a client that fetches the
well-known doc but ignores its body would now pass. Restore the prefix
and pin the metadata issuer to the bare origin via a lazy
metadataIssuer override so the mock is RFC 8414 §3.3-compliant without
losing the fallback-vs-metadata distinction.
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
The tasks extension scenarios hardcoded -32003 (Missing Required Client
Capability) and -32001 (Header Mismatch). The draft spec types renumbered
these to -32021 and -32020 per modelcontextprotocol#2907, so a server
built against current draft.ts now returns the new codes and these checks
would produce false failures.
Import MISSING_REQUIRED_CLIENT_CAPABILITY and HEADER_MISMATCH from
spec-types/draft instead of hardcoding, matching the existing
http-standard-headers scenario. The checks now assert whatever the schema
currently defines, so a future renumber propagates automatically.
Also drop the now-stale code number from the affected check-ID slugs
(for example sep-2663-server-returns-32003-when-required becomes
sep-2663-server-returns-missing-capability-when-required) and update the
sep-2663.yaml requirement text to -32021, keeping it consistent with the
already-updated sep-2575 and sep-2243 manifests.
Scenarios are still in the pending list so default CI is unaffected. The
traceability.json manifest already predates the upstream renumber and is
refreshed separately from a full suite run.
* feat(tasks): SEP-2663 lifecycle scenario (8 checks)
Adds the first scenario for the SEP-2663 io.modelcontextprotocol/tasks
extension — a single TasksLifecycleScenario covering sync vs async
dispatch, DetailedTask shape on tasks/get, tool errors vs protocol
errors, and cancellation semantics. 8 ConformanceCheck records, all
passing against a SEP-2663-conformant Go fixture.
Why "tasks" (not "tasks-v2"): SEP-2663 IS the tasks surface once it
lands; the v2 suffix is only meaningful in implementations that
maintain a v1 surface alongside, which the conformance suite does not.
Layout:
- src/scenarios/server/tasks/lifecycle.ts — scenario class
- src/scenarios/server/tasks/helpers.ts — raw-fetch escape hatch
(the SDK's typed schemas strip resultType/inputRequests/...)
- src/scenarios/server/tasks/lifecycle.test.ts — fork-local vitest
runner. Two modes: spawn a fixture binary via MCPKIT_TASKS_BINARY,
or point at an already-running server via MCPKIT_TASKS_SERVER_URL.
Skips when neither is set so it doesn't break upstream CI runs that
go through everything-server (which doesn't yet implement
io.modelcontextprotocol/tasks).
Scenario is registered in pendingClientScenariosList so
all-scenarios.test.ts skips it; promote to active once the upstream
fixture grows extension support.
Tagged ['extension', DRAFT_PROTOCOL_VERSION] — selectable via
--suite extensions and --spec-version draft.
* style(tasks): apply prettier formatting
* feat(tasks,mrtr): port full SEP-2663 + SEP-2322 scenario suite
Builds out the rest of the tasks scenarios (atop the lifecycle canary)
and adds the SEP-2322 ephemeral MRTR scenario in a sibling folder.
Both target their own fixtures; both runners are brand-neutral and
language-agnostic (TASKS_SERVER_URL / TASKS_SERVER_CMD,
MRTR_SERVER_URL / MRTR_SERVER_CMD; readiness via TCP polling).
Tasks ClientScenario classes:
- TasksLifecycleScenario (8 checks; v2-01..v2-08)
- TasksCapabilityNegotiationScenario (4 checks; v2-11/22/23/25, SEP-2575)
- TasksWireFieldsScenario (3 checks; v2-12/13/21)
- TasksRequestStateScenario (3 checks; v2-14/15/28)
- TasksMRTRInputScenario (3 checks; v2-16/17/29 partial fulfillment)
- TasksRequestHeadersScenario (3 checks; SEP-2243 request-header tolerance)
- TasksDispatchScenario (8 checks; v2-09/10/19/20/26/27/30/31)
- TasksStatusNotificationsScenario (1 check; SEP-2663 §notifications, optional)
MRTR ClientScenario class:
- MrtrEphemeralFlowScenario (7 checks + 1 SKIPPED; mrtr-01..07,
mrtr-08 deferred for spec terminology +
reference-impl reasons)
Both runners spawn the fixture via a shell command and detect readiness
by TCP-polling the URL's host/port — no log-line scanning, no
language-specific assumptions. The same env vars work for any server
implementation.
Scenarios are tagged ['extension', DRAFT_PROTOCOL_VERSION] and registered
in pendingClientScenariosList so all-scenarios.test.ts (which targets
the upstream everything-server) skips them until the fixture grows
SEP-2322 / SEP-2663 support.
* docs(tasks,mrtr): scenario READMEs for upstream porting
Restructured around ClientScenario classes (one row per class with
check-list under it) rather than per-numbered-test slugs. Documents
fixture requirements, env vars, open spec questions, and the
wire-format diff for each suite.
Per AGENTS.md, severity follows spec keyword (MUST/MUST NOT → FAILURE,
SHOULD/SHOULD NOT → WARNING). The READMEs explain why some checks emit
INFO rather than FAILURE (optional emission paths per SEP-2322).
* tasks: assert createdAt + lastUpdatedAt; factor _shared/ helpers
Two reviewer-driven additions:
1. SEP-2663 createdAt / lastUpdatedAt ISO-8601 assertion in
`tasks-server-task-creation` (per Luca's PR #262 review feedback).
The check now flags servers that emit non-ISO timestamps (epoch
seconds, RFC-2822, etc.) on TaskInfoV2 envelopes.
2. Factor cross-cutting test-harness helpers into _shared/:
- `_shared/test-runner.ts` — `waitForServerReady` (renamed from
`waitForTcpReady`; the call site cares about server readiness,
not the TCP-poll mechanism). Imported by tasks/ and mrtr/
all-scenarios.test.ts; replaces ~30 LOC of inline duplication
in each.
- `_shared/wire-format.ts` — `ISO_8601_PATTERN` constant +
`isIso8601(s)` predicate. Documented rationale for choosing a
regex over `Date.parse` (too permissive),
`new Date(s).toISOString()` (too strict), or
`Temporal.Instant.from` (Node 24+ experimental). Future
wire-shape predicates (data URI, percent-encoded filename,
etc.) can land here.
Cherry-pick footprint when graduating to upstream PR is the SEP
folder + the imported `_shared/` files. First PR through carries
them upstream; subsequent feat branches inherit via standard
upstream-sync flow.
All 9 scenario tests still pass against the Go reference fixtures.
* refactor(tasks,mrtr): use SDK Client + AnyResult instead of raw-fetch helpers
Drops initRawSession/rawRequest/rawRequestFull from tasks/helpers.ts in
favor of the SDK's Client + StreamableHTTPClientTransport, paired with
a Zod passthrough schema (AnyResult) that preserves SEP-2663 / SEP-2322
draft fields the SDK's typed schemas would strip.
headers.ts and notifications.ts keep a small inline fetch where the SDK
can't reach: per-request HTTP headers (SEP-2243) and SSE notification
observation. Both reuse the SDK session via transport.sessionId.
All SEP-2663 + MRTR ephemeral-flow scenarios pass against the Go fixture.
* style: prettier formatting on tasks/mrtr scenarios
* tasks: align ttlMs / pollIntervalMs assertions per 2026-05-07 SEP
PR 2663 commit 62758914 standardised every duration field on the Ms
suffix, integer milliseconds. wire-fields.ts now asserts ttlMs and
pollIntervalMs are present on CreateTaskResult, the legacy v1 ttl
and pollInterval keys are absent (already covered), and the interim
ttlSeconds / pollIntervalMilliseconds keys are also absent on a
post-2026-05-07 server. lifecycle.ts and the scenario README pick up
matching prose updates.
Verified by make testconf-tasks-v2 (8/8) against a renamed mcpkit
fixture, and make testconf-mrtr (7/7 + 1 SKIPPED) against the paired
MRTR surface.
* style: prettier column alignment on tasks README
* mrtr: flip MRTR_INPUT_REQUIRED_RESULT_TYPE per merged SEP-2322
SEP-2322 merged on 2026-05-06 with the variant renamed from
IncompleteResult to InputRequiredResult and the resultType discriminator
from "incomplete" to "input_required" (commit de6d76fb, per dsp-ant
request). The MRTR_INCOMPLETE_RESULT_TYPE constant was specifically
designed as a one-line flip point for this scenario.
Renames
- MRTR_INCOMPLETE_RESULT_TYPE = "incomplete" ->
MRTR_INPUT_REQUIRED_RESULT_TYPE = "input_required"
- isIncompleteResult -> isInputRequiredResult
- All "IncompleteResult" -> "InputRequiredResult" in scenario prose
and check descriptions (ephemeral-flow.ts, README.md)
SEP-2663 had not yet flipped its discriminator literal as of PR head
82fb2c4d (5/7 21:52 UTC). Caitie's 5/15 RC commitment (issue comment
4384052694 on PR 2322) tracks the alignment to "input_required" both
sides. The constant remains the one-line flip point in case the 2663
follow-up surprises us.
Tested via mcpkit's make testconf-mrtr (7/7 + 1 SKIPPED green against
a renamed mcpkit fixture) and make testconf-tasks-v2 (8/8 still
green, no regressions on the paired surface).
* mrtr: prettier formatting + fix README rename typo
Lefthook prettier reformatted column alignment on first push attempt;
README also had a stale "renamed from InputRequiredResult" — should
read "renamed from IncompleteResult". Fix both.
* mrtr-08: refresh SKIPPED prose post SEP-2322 merge
Two stale references in the mrtr-tasks-composition SKIPPED check:
- Comment block + errorMessage framed blocker (a) as "spec authors
disagree" / "input_required vs incomplete". SEP-2322 merged
2026-05-06 with "input_required" (commit de6d76fb). The blocker
now reads as "SEP-2663 has not yet aligned to the merged 2322
literal" — Caitie's 5/15 RC commitment (PR 2322 issue comment
4384052694) tracks the alignment.
- errorMessage referenced "IsIncomplete signal" — that field was
renamed to IsInputRequired on the mcpkit side in lockstep with
the SEP-2322 wire-variant rename. Updated to match.
Status stays SKIPPED because blocker (b) — the mcpkit middleware
refactor (issue 347) — is still open.
* mrtr-08: drop the resolved-discriminator framing from the SKIPPED message
After SEP-2322 merged with "input_required", the only blocker that
actually keeps mrtr-08 SKIPPED is the eager-task-creation pattern in
reference-server middleware (panyam/mcpkit issue 347). The earlier
two-blocker framing read as if the test were waiting on both, but
blocker (a) is effectively resolved for any server that emits the
merged-2322 literal — leaving (b) as the sole gate.
Tighten the comment block + description + errorMessage to lead with
the middleware refactor and demote the discriminator history to a
parenthetical aside.
* style: prettier line-break on mrtr-08 description string
* tasks-wire-fields: address PR 262 review feedback
Two requested changes from the SEP-2663 author's review pass on
modelcontextprotocol/conformance PR 262
(pullrequestreview-4254601106).
(1) Drop the interim-key absence checks. The transition window between
"ttlSeconds / pollIntervalMilliseconds" and the final
"ttlMs / pollIntervalMs" wording is over now that the merged
spec settled on the Ms suffix. Useful while the spec was in
flight, noise once it stabilized. Removes the absence checks
plus the surrounding comment + description + details fields.
(2) Add Number.isInteger() to ttlMs and pollIntervalMs validation.
Spec says integer milliseconds; the previous typeof + range check
would have allowed fractional values. Now both fields fail if
they're not integers.
README scenario table tightened: "ttlMs + pollIntervalMs present and
integer-valued; legacy ttl / pollInterval keys absent".
* style: prettier column alignment on wire-fields README row
* feat(tasks): align scenarios with the Final-merged SEP-2663 spec
SEP-2663 merged Final on 2026-05-15. Four normative wire changes baked in at merge time. This commit updates the tasks server-conformance scenarios so the suite validates each one as a MUST rather than tolerating divergence as INFO/MAY.
1. notifications/tasks/status renamed to notifications/tasks. The status-notifications scenario now FAILs on observing the legacy method on the v2 surface, and validates the new name's payload shape when emitted.
2. notifications/progress and notifications/message MUST NOT be sent for tasks. Two new absence-asserts on the status-notifications scenario, reusing the existing SSE-observation harness.
3. requestState removed from the tasks-v2 wire. The previous request-state scenario tested the deleted "Request State Management" section; replaced with absence-asserts on CreateTaskResult, DetailedTask, and notifications/tasks payloads. SEP-2322's InputRequiredResult on the MRTR surface still carries requestState and is unchanged.
4. -32003 (Missing Required Client Capability) for required-task tools when the client did not negotiate the extension. New scenario TasksRequiredTaskErrorScenario initializes without declaring the extension, calls a TaskSupport=required tool, and asserts the rejection code plus the structured requiredCapabilities payload. Documents `failing_job` as a fixture requirement.
Verified locally against panyam/mcpkit examples/tasks-v2: 8/9 scenarios pass; the request-state-removal scenario intentionally FAILs against the current mcpkit main because the implementation still emits requestState on DetailedTask. That gap is the next item on the mcpkit catch-up plan.
* chore(tasks): address PR 262 review feedback
- notifications.ts: skip the entire scenario; SEP-2663 routes
notifications/tasks over the SEP-2575 subscriptions/listen stream,
not the tools/call POST SSE the prior harness observed. Body
preserved in git blame for the rewrite follow-up.
- request-state.ts: keep the absence-asserts (per the asymmetric
recommendation in review) and reframe the doc block around the
SEP-2322 lexical-adjacency motivation rather than pre-merge SEP
history.
- required-task-error.ts: cite SEP-2575 §"Missing Required
Capabilities" as the canonical home of -32003; add SEP_2575_REF to
specReferences on the -32003 checks.
- Sweep decontextualized "merged spec dropped X" / "pre-merge SEP-2663
said Y" framing from tasks/* doc blocks and inline check
descriptions.
* chore(tasks): prettier — strip trailing blank line in notifications.ts
* chore(tasks): drop `requestState` from `tasks/update` params
`taskId` is the session-continuation handle on the tasks-v2 MRTR-resume
path; `requestState` lives only on the SEP-2322 ephemeral
InputRequiredResult surface, not on the tasks-v2 wire.
* chore(tasks): loosen strict ack equality on tasks/cancel + tasks/update
The cancel-empty-ack and tasks/update-ack checks were comparing the
whole response against the literal `{resultType:"complete"}` via
`JSON.stringify` equality. That rejected acks carrying `_meta` or any
other future result-shape metadata even though the spec permits them.
Now structural: assert `resultType === "complete"` plus absence of the
task-envelope fields (`taskId`, `status`, `result`, `error`,
`inputRequests`). `_meta` and other metadata pass through.
* chore(tasks): poll for terminal status after tasks/cancel
The cancel-empty-ack check observed status via an immediate tasks/get
after the cancel ack. Cancellation is eventually-consistent, so the
task could still be `working` (or transitioning) at the moment of
observation, making the assertion flaky against any reasonable server.
Now polls via `waitForTerminal()` and asserts the terminal status is
`cancelled`. README's `slow_compute` fixture row documents the
matching tool contract ("MUST settle to `cancelled` when
`tasks/cancel` arrives while running") so the test has a deterministic
anchor.
* chore(tasks): tasks/cancel on a terminal task is idempotent
The previous check asserted `-32602` when cancelling an already-terminal
task. That forces every client to handle the race between observing a
task's terminal status and racing in a cancel request — the task can
terminate between the two and the cancel then errors instead of acking.
Check 8 renamed to `tasks-cancel-terminal-idempotent-ack` and now
asserts the same empty-ack shape (`resultType:"complete"` + no task-
envelope fields) as on an active task. README's lifecycle table row
updated to match.
* docs(tasks): drop `requestState` from SEP-2663 wire-field listings
`requestState` belongs to the SEP-2322 ephemeral MRTR result shape
(`InputRequiredResult`), not to the tasks-v2 wire. The README listed
it under SEP-2663 wire fields in four spots — top blurb, the SEP-2663
row of the Specs table, the v1↔v2 wire-format diff, and the raw-fetch
design note. All four now describe only the actual SEP-2663 shapes;
the raw-fetch note still calls out the SEP-2322 ephemeral
`requestState` separately so the rationale for stripping it via raw
fetch stays intact. The SEP-2322 row of the Specs table is unchanged
since `requestState` is correctly listed there as an MRTR base type.
* docs(tasks): rewrite stale request-state table to match absence-asserts
The README's `tasks-request-state` table still described the prior
scenario shape (shape / echo / stale-tolerance checks), which had
assumed `requestState` was on the tasks-v2 wire. The scenario itself
was rewritten earlier into the two-check absence assertion
(`tasks-create-result-no-request-state` /
`tasks-get-detailed-no-request-state`); the README now matches.
A motivational paragraph above the table explains why a negative
test exists for a field the spec never defines (the SEP-2322
lexical-adjacency confusion vector).
* chore(tasks): assert -32003 (not -32601) for gated tasks methods
When a client calls tasks/get / tasks/update / tasks/cancel without
having negotiated the io.modelcontextprotocol/tasks extension, the
suite now asserts -32003 (Missing Required Client Capability,
SEP-2575 §"Missing Required Capabilities") instead of -32601.
The spec doesn't currently mandate this code for the gated-method
path, so the assertion is forward-looking: it follows the SEP-2575
pattern that already governs `required` tools (and that
subscriptions/listen for tasks is expected to use). README's
open-questions list calls this out so readers know it's a
spec-clarification ahead-of-the-curve.
Drops the now-obsolete "Invalid requestState — silent ack vs -32602"
open-questions item: the previous request-state scenario covered
that, but the rewritten absence-asserts scenario no longer exercises
that path.
* chore(tasks): prettier — realign README table pipes after row edits
* feat(tasks/mrtr): initRawSession helper for draft-version handshake
The SDK's Client.connect() pins protocolVersion to LATEST_PROTOCOL_VERSION
in the initialize body. Scenarios tagged DRAFT_PROTOCOL_VERSION would
still negotiate the previous stable on the wire, which means a strict
draft-only server rejects the handshake with unsupported_protocol_version.
initRawSession sidesteps the SDK: a raw fetch initialize that carries the
draft version, captures the session ID, sends notifications/initialized,
and exposes the small surface scenarios actually use - request,
requestFull, notification, close. Errors are thrown as McpError so
existing instanceof / .code checks keep working. SSE response parsing
folds in the per-scenario raw helper from headers.ts.
All 8 SEP-2663 tasks scenarios + the MRTR ephemeral-flow scenario migrate
to the raw session. tsc clean.
* feat(tasks/mrtr): wire-mode matrix; SEP-2575 _meta + SEP-2243 routing
headers
initRawSession grows a `stateless` option that picks between the
legacy session wire (initialize handshake + Mcp-Session-Id) and the
SEP-2575 stateless wire (no initialize; server/discover up front;
per-request `_meta.io.modelcontextprotocol/{protocolVersion,
clientInfo, clientCapabilities}` envelope; `MCP-Protocol-Version`
header on every call).
Tasks behavior is wire-independent in spec, so the harness can run
each scenario against both wires. To keep the legacy run green
through this PR's refactor (and to surface stateless-mode gaps
deliberately rather than as a regression), the default is legacy-
only — set TASKS_WIRE_MODES=legacy,stateless (or just stateless) to
opt the matrix on. Same toggle for MRTR via MRTR_WIRE_MODES.
Two follow-on cleanups in the same commit so the legacy run actually
passes against an SEP-2243-enforcing server:
- All request paths now auto-emit Mcp-Method (and Mcp-Name for
tools/call.params.name / resources/read.params.uri) on protocol
versions in `SEP_2243_ENFORCED_VERSIONS` (today DRAFT-2026-v1).
Scenarios that probe the mismatch path can still override via
`extraHeaders`. Empty record on older versions so the helper stays
silent when the spec doesn't require it.
- `tasks-headers-body-method-authoritative` was written under an
older SEP-2243 reading where Mcp-Method was tolerant. The current
spec (and mcpkit's PR 477 implementation, plus the upstream
http-header-validation scenario) says the server MUST reject
mismatches with -32001 HeaderMismatch. Renamed to
`tasks-headers-reject-mismatched-method` and the assertion flipped
to expect the rejection.
Verification: testconf-tasks-v2 + testconf-mrtr both green on
legacy wire against mcpkit's tasks-v2 fixture (9/9 + 1/1).
* rename TASKS_WIRE_MODES / MRTR_WIRE_MODES → MCP_WIRE_MODES
Wire-mode choice is a cross-cutting conformance concern, not a
per-suite knob. Operators wouldn't realistically toggle tasks and
mrtr to different wires (both fixtures speak the same MCP server,
either it supports both wires or one). Single env var read by both
harnesses; default unchanged at ['legacy'].
* default MCP_WIRE_MODES to legacy+stateless
The conformance suite is brand-neutral — every SDK consuming it
should validate both MCP wires for SEP-2663 / SEP-2322 behavior by
default. Pinning the default to legacy-only let mcpkit's CI stay
green while gaps in its stateless dispatcher pended, but quietly
under-tested any other SDK that adopts the suite.
Default now runs every tasks-v2 scenario × 2 wires and the mrtr
ephemeral-flow scenario × 2 wires. SDKs with only one wire
implemented pin via MCP_WIRE_MODES=legacy or =stateless. mcpkit's
own audit will need that pin (or to close panyam/mcpkit issue 480)
until the stateless dispatcher consumes per-request
_meta.clientCapabilities for the tasks extension declaration.
* helpers: send MCP-Protocol-Version on every post-initialize POST
Per 2025-11-25 transport spec §Protocol-Version Header, clients MUST
include MCP-Protocol-Version on every subsequent HTTP request after
initialize. Universal post-initialize requirement; applies to both
the legacy session wire and the SEP-2575 stateless wire.
The helper was dropping the header on legacy follow-up calls as a
workaround for our server's detectWireKind treating the header as a
stateless-wire signal. That workaround was non-spec-compliant; the
right fix is for our server to drop the header from its wire-detection
precedence list and rely on body-level _meta.protocolVersion as the
SEP-2575 signal. Filed separately.
Comment thread: PR 5 review (Luca, 2026-05-27).
* types/helpers: parameterize wire mode via ScenarioRunOptions
Drop the module-level defaultStateless flag + setDefaultWireStateless
shim that the harness was toggling in beforeEach. Wire mode now flows
as an explicit parameter through ClientScenario.run, so tests can be
parallelized safely (no shared mutable state between concurrent runs).
ClientScenario.run gains an optional ScenarioRunOptions second arg.
Existing scenarios that ignore the arg satisfy the interface as-is
(TypeScript permits an implementation to declare fewer parameters than
the interface). The 10 task/mrtr scenarios that consume wire mode add
one line each to thread `opts?.stateless` into initRawSession.
Harness loops call `scenario.run(SERVER_URL, { stateless: wire === 'stateless' })`
directly; no more beforeEach setter.
Comment thread: PR 5 review (Luca, 2026-05-27, comments 2 and 7 on parallelism).
* harness: use describe.each + it.each for the (wire x scenario) matrix
Replaces the nested for-loops in both the tasks and mrtr harnesses
with vitest's table-driven describe.each + it.each idioms. Same N x M
cardinality of tests, but vitest sees the parameter table directly:
- Reporter labels render the wire row inline:
`... > stateless wire > 'tasks-lifecycle' - all checks ...`
- Each (wire, scenario) becomes a leaf test vitest can schedule
independently, opening the door to test.concurrent / pool config
later without restructuring.
WIRE_TABLE pairs each WireMode with the boolean flag the harness
needs to pass via ScenarioRunOptions, so the `it.each` callback
receives the prepared `stateless` directly rather than deriving it
inside every test body.
No on-wire change. Behavior identical against the reference fixture
(9 task scenarios + 1 mrtr scenario, each per wire).
Comment thread: PR 5 review (Luca, 2026-05-27, comments 5, 6, 9, 10).
* helpers: extend Mcp-Name to prompts/get + tasks/* (SEP-2243 + SEP-2663)
routingHeaders now auto-emits Mcp-Name for the full standard-header
surface the spec defines:
- prompts/get -> params.name (SEP-2243 §Standard Headers)
- tasks/get -> params.taskId (SEP-2663 §Streamable HTTP routing headers)
- tasks/update -> params.taskId (SEP-2663)
- tasks/cancel -> params.taskId (SEP-2663)
tools/call (params.name) and resources/read (params.uri) were already
covered. Switch from if/else chain to a method-name switch since the
surface is now wider; cleaner to extend further.
Helper doc updated with the full surface list and a TODO for SEP-2243
§Custom Headers from Tool Parameters (Mcp-Param-* mirroring from
x-mcp-header annotations), which requires caching the tools/list
schema. Tracked as a separate follow-up; no current tasks/mrtr
scenario exercises a tool with x-mcp-header annotated inputs.
Comment thread: PR 5 review (Luca, 2026-05-27, comment 4).
* _shared/wire-mode: hoist parseWireModes out of the per-suite harnesses
Tasks and mrtr harnesses had byte-identical copies of WireMode type,
VALID_MODES, DEFAULT_WIRE_MODES, parseWireModes, and WIRE_MODES.
Hoisted to src/scenarios/server/_shared/wire-mode.ts so the single
parser drives both suites. Each harness now does:
import { parseWireModes, type WireMode } from '../_shared/wire-mode';
const WIRE_MODES: WireMode[] = parseWireModes();
No behavior change. MCP_WIRE_MODES env var semantics identical;
default still legacy+stateless.
Companion TODO landed in tasks/helpers.ts: most of that module
(initRawSession, RawSession, routingHeaders, SEP_2243_ENFORCED_VERSIONS,
readJsonRpcResponse, AnyResult, the failureCheck/skipCheck/errMsg test
helpers) isn't actually tasks-specific and belongs in _shared too.
Holding that file reshuffle for a separate follow-up PR so the move
gets its own focused review thread without crowding the SEP work.
Comment thread: PR 5 review (Luca, 2026-05-27, comment 8).
* refactor: hoist raw-session + sep-refs + checks helpers into _shared/
Picks up the TODO left in tasks/helpers.ts on PR 5: the raw fetch
session machinery, the SEP-2243 routing-header builder, the SEP_*
reference constants, and the failureCheck / skipCheck / errMsg /
AnyResult test scaffolding were never tasks-specific. They lived in
tasks/helpers.ts only because tasks/ was the first server suite to
need them. mrtr/helpers.ts re-declared SEP_2322_REF + errMsg +
failureCheck for the same reason.
Move them to src/scenarios/server/_shared/:
- sep-refs.ts: SEP_2243_REF, SEP_2322_REF, SEP_2575_REF, SEP_2663_REF
- checks.ts: errMsg, failureCheck, skipCheck, AnyResult
- raw-session.ts: JsonRpcResponse, RawSession, InitRawOptions,
initRawSession (legacy + stateless paths),
routingHeaders, SEP_2243_ENFORCED_VERSIONS,
readJsonRpcResponse, nextRawId
tasks/helpers.ts shrinks to the tasks-shaped pieces it actually owns:
TASKS_EXTENSION_ID and the waitForTerminal / waitForStatus polling
loops. mrtr/helpers.ts drops the duplicate SEP ref + the duplicate
test-scaffolding wrappers and re-imports from _shared/.
skipCheck and failureCheck in _shared/checks.ts make specReferences
required (a shared scaffold can't pick a sensible default), so all
call sites now pass the SEP they're grading against explicitly.
Pure file move + import-path churn. No behavioral change. Typecheck
+ lint clean.
* feat: emit SEP-2243 Mcp-Param-* headers from tool inputSchema annotations
Closes the TODO in routingHeaders. SEP-2243 §"Custom Headers from
Tool Parameters" requires the client to mirror primitive-typed
tools/call arguments as `Mcp-Param-<Suffix>` HTTP headers when the
tool's inputSchema annotates the property with
`x-mcp-header: "<Suffix>"`. Servers behind L7 routers / WAFs use
these headers to make routing or policy decisions without parsing
the JSON body.
Implementation:
- RawSession gains a `toolSchemas: Map<string, unknown>` cache.
- `initRawSession` issues a best-effort `tools/list` after the
handshake (after `notifications/initialized` for legacy; after
`server/discover` for SEP-2575 stateless) and populates the
cache from the response. Failures are swallowed — empty cache is
the safe default.
- `routingHeaders` takes an optional `toolSchemas` and walks the
cached schema for tools/call: for each primitive-typed property
(string / number / integer / boolean) with `x-mcp-header`, it
encodes the argument value per SEP-2243 §value-encoding (plain
printable ASCII verbatim; everything else wrapped as
`=?base64?{b64-utf8}?=`) and emits `Mcp-Param-<Suffix>`.
- Both wire flavours (legacy + SEP-2575) call routingHeaders the
same way, so the same Mcp-Param-* headers ship regardless of
transport mode.
Behavior wash for tools without `x-mcp-header` annotations — they
emit only Mcp-Method + Mcp-Name as before. Behavior wash for
non-tools/call methods. No new dependencies (uses
`globalThis.btoa` + `TextEncoder`, both standard in Node 20+).
Typecheck + lint + the existing vitest suite stay green.
* chore(_shared/raw-session): drop reference-impl naming from doc comments
Tightens three doc-comment passages that named or hinted at a
specific server-side implementation as the canonical reference. The
conformance suite is positioned brand-neutral; the doc comments now
describe the wire behavior in spec-anchored language only.
No behavioral change.
* sep-refs: point at rendered SEP pages on modelcontextprotocol.io
PR URLs freeze at merge commit; the rendered SEP pages on the spec
website absorb post-merge amendments. Switch all four refs:
SEP-2243 → /seps/2243-http-standardization
SEP-2322 → /seps/2322-MRTR
SEP-2575 → /seps/2575-stateless-mcp
SEP-2663 → /seps/2663-tasks-extension
* raw-session: use eventsource-parser for SSE parsing
Delegate the SSE branch of readJsonRpcResponse to eventsource-parser
(the same parser the MCP TS SDK uses). The hand-rolled splitter
worked for the single-frame JSON-RPC response we need but didn't
cover the full WHATWG SSE semantics — multi-line `data:`
continuation, CR/CRLF line endings, comment lines, the
`event:`/`id:`/`retry:` fields. The library covers all of it.
Already a direct dependency (3.x), so no new transitive footprint.
* test: pin readJsonRpcResponse contract for SSE + JSON branches
12 vitest cases against `_shared/raw-session.ts`'s content-type
dispatcher: application/json (id-match, mismatch, error frames) and
text/event-stream (single-frame extraction, id-filter across multi-
event streams, multi-line `data:` continuation, CRLF tolerance,
comment/retry/id field handling, error-frame return, empty body,
non-JSON data interleaving).
Builds synthetic `Response` objects locally; no fixture server
required, runs in every CI. Tests pin the contract the
eventsource-parser swap inherits — guards against future parser-
version regressions and against any subsequent rewrite that subtly
changes the SSE branch.
* tasks: isolate negative-path dispatch checks (conformance#262)
The removed-method (-32601) and unknown-taskId (-32602) checks
in tasks-dispatch-and-envelope previously sent requests missing
Mcp-Name, so strict servers validating routing headers per
SEP-2243 §Server Behavior could reject before reaching method
dispatch — making the asserted error code unreachable.
- routingHeaders(): emit Mcp-Name on tasks/result so the
removed-method check fails specifically on -32601, not
-32001 HeaderMismatch.
- validTasksParams() helper: baseline params per
tasks-namespace method; negative checks override one field
to express isolation intent declaratively.
- ISOLATION_HINT on wrong-code failures points at the most
likely cause (routing headers / _meta / params shape
short-circuit before dispatch).
Forward-safe under either reading of the open SEP-2663
client-MUST → server-MUST-reject elevation question: the added
Mcp-Name matches the body, so SEP-2243's universal mismatch
rule can't fire either way.
Verified: 18/18 tasks scenarios pass.
* tasks: rename scenario check IDs to sep-2663-* slugs
Aligns 13 tasks scenario check IDs with the SEP-2484 traceability
convention used by sep-2322.yaml and others: the slug emitted at
runtime matches the check: row in the SEP's requirement-traceability
yaml, so plan.modelcontextprotocol.io can link the SEP requirement
to its implementing scenario.
Rename map:
tasks-tools-call-without-extension-sync
→ sep-2663-server-rejects-undeclared-client
tasks-methods-gated-without-extension
→ sep-2663-tasks-methods-non-declaring-32003
tasks-server-task-creation
→ sep-2663-result-type-task-on-create
tasks-get-during-working
→ sep-2663-tasks-get-status-working
tasks-get-terminal-inlined-result
→ sep-2663-tasks-get-status-completed
tasks-tool-error-completed-iserror
→ sep-2663-tool-error-uses-completed-status
tasks-protocol-error-failed-shape
→ sep-2663-tasks-get-status-failed
tasks-cancel-empty-ack
→ sep-2663-cancel-ack-empty-result
tasks-removed-tasks-result
→ sep-2663-tasks-result-removed-method-not-found
tasks-legacy-task-param-ignored
→ sep-2663-legacy-task-param-ignored
tasks-strong-consistency-immediate-get
→ sep-2663-durable-create-strong-consistency
tasks-get-unknown-task-id-rejected
→ sep-2663-tasks-get-invalid-task-id-32602
tasks-mrtr-input-requests-on-tasks-get
→ sep-2663-tasks-get-status-input-required
Wire behavior unchanged — only the reported slugs change. Verified
18/18 tasks scenarios still pass.
Untouched (intentionally): scenario-flow gates like
tasks-extension-advertised, tasks-sync-tool-call, tasks-wire-field-renames,
tasks-immediate-result-shortcut etc., which test end-to-end flows
rather than a specific RFC-2119 sentence and surface in the
traceability manifest as untracked (same pattern as sep-2322.yaml).
* seps: add sep-2663.yaml requirement-traceability declaration
Declares the 37 conformance checks (and 12 excluded requirements)
the SEP-2663 Tasks Extension surface should produce, mapping each
normative MUST/SHOULD sentence on the SEP page to either a check ID
emitted by an existing scenario or a documented reason for exclusion.
Exclusions are all client-internal, host-internal, or architectural
requirements that aren't observable at the protocol level — confirmed
during the new-sep skill's exclusion round.
The 13 scenario checks renamed in the previous commit now match the
check: slugs declared here; the remaining ~12 yaml rows describe
requirements without a scenario emitter yet — those are gaps to fill
in follow-up work (notifications via subscriptions/listen, key
uniqueness across MRTR rounds, legacy-capability migration probes).
spec_url points at the SEP page itself (status: Final) — SEP-2663's
normative content lives on /seps/2663-tasks-extension and is mirrored
in modelcontextprotocol/experimental-ext-tasks for source-control
purposes. Every quoted text: field was verified copy-paste findable
against the rendered SEP page after stripping markdown formatting.
* tasks: enforce SEP-2663 routing-header validation on tasks surface
Two related fixes:
1. tasks/headers.ts description previously said servers "MUST NOT
require" the Mcp-Method / Mcp-Name routing headers on the tasks
surface and "MUST NOT change dispatch behavior based on them —
including when the headers disagree with the body." Both claims
are spec-incorrect. SEP-2663 author confirmed in the conformance
review thread that the client-MUST in §"Streamable HTTP: Routing
Headers" elevates Mcp-Name to "required standard header" status
for tasks/get / tasks/update / tasks/cancel, so SEP-2243's
§"Server Behavior" mismatch + missing-header rejection applies.
Description rewritten to match the spec: routing headers are
REQUIRED, server MUST reject missing or mismatched with -32001.
2. Add a negative-path check covering the most common failure mode
on the tasks surface: tasks/get with mismatched Mcp-Name MUST
return -32001 HeaderMismatch. Probes tasks/get specifically;
tasks/update / tasks/cancel follow by extension. Missing-header
case is deferred (requires raw-session enhancement to suppress
a header entirely; current extraHeaders mechanism only overrides).
3. New yaml row pinning the rule to the SEP-2243 §"Server Behavior"
sentence via row-level url override (the rule itself is SEP-2243;
the scope of application on the tasks surface is what's
distinctively SEP-2663).
Known impact: surfaces a real gap in SDKs that derived their
routing-header validation from the SEP-2243 base table only (which
lists tasks/* as NOT requiring Mcp-Name). Those SDKs will see this
check go FAILURE until they extend their validation to cover
tasks-namespace methods per SEP-2663 §540.
CI against everything-server is unaffected (skips the tasks suite
entirely; everything-server doesn't implement the extension).
* mrtr: activate mrtr-08 composition check + fold into sep-2663.yaml
Folds PR 7 (panyam/mcpconformance) into PR 262:
- Cherry-picks the mrtr-tasks-composition activation from
feat/sep-2663-mrtr-08-composition (ad4f267 upstream): real end-to-end
check driving the SEP-2663 commit 451f5e1 promotion flow
(InputRequiredResult rounds gather user_name via elicitation/create,
then CreateTaskResult on the final round escalates to async; the
final tasks/get result reflects the answer gathered during MRTR).
- Addresses LucaButBoring's PR 7 nit: replaces the inline SEP-2663
reference literal with the shared SEP_2663_REF constant (now also
pointing at the rendered SEP page).
- Renames the scenario's emitted check ID from `mrtr-tasks-composition`
to `sep-2663-mrtr-synchronous-before-task-creation` to match the
corresponding row in sep-2663.yaml — closes the gap between
declared requirement and implementing check.
- Updates mrtr/README.md: refreshes the Specs-covered table and the
per-check description table for the new slug, and drops the "Open
issues" section (both blockers are resolved — discriminator value
aligned upstream; mcpkit middleware refactored per panyam/mcpkit#347).
- Updates the sep-2663.yaml flow-gate comment block to note that
the ephemeral mrtr-* flow gates also fall into the untracked
category for the same reason as the tasks-* flow gates.
One fewer skipped check on PR 262. Verified 123/123 server-suite
tests pass against mcpkit's tasks-v2 + mrtr fixtures (post
panyam/mcpkit#499 + #501).
Closespanyam/mcpconformance#7.
* tasks/mrtr: drop legacy wire when targeting DRAFT-2026-v1
SEP-2575 (Accepted) removes the initialize handshake on DRAFT-2026-v1
and replaces it with self-contained per-request _meta. Driving the
legacy initialize+Mcp-Session-Id wire against a draft fixture is
therefore a logical contradiction: the harness asserts
`protocolVersion: "DRAFT-2026-v1"` while doing the handshake the
version removed. Servers that strictly enforce SEP-2575 correctly
reject these requests with -32602 (see Randgalt's report on
conformance PR 262, 2026-05-31); lenient servers silently accept
them and mask the spec gap.
Both effects are wrong for the harness — we should only emit
spec-permitted traffic.
Adds `effectiveWireModes(protocolVersion)` in _shared/wire-mode.ts.
It wraps `parseWireModes()` and filters `legacy` out when the target
version is in POST_SEP_2575_VERSIONS (currently just DRAFT-2026-v1;
widen when a future dated release picks SEP-2575 up). If the user
explicitly pinned MCP_WIRE_MODES=legacy against a post-SEP-2575
version, the helper warns and falls back to ['stateless'] so the
suite still runs.
Both draft-only suites (tasks, mrtr) switch to
`effectiveWireModes(DRAFT_PROTOCOL_VERSION)`. The MCP_WIRE_MODES
override remains honored within the spec-permitted set.
Verified end-to-end with a strict mcpkit fixture (panyam/mcpkit
fix/sep-2575-enforce-meta-on-draft):
- pre-fix harness → 7/18 fail with the spec-required error
- post-fix harness → 9/9 pass (stateless wire only)
* tasks/mrtr: derive wire from spec version, drop RunContext.wire
`RunContext.wire` was a parallel knob to `specVersion` that the tasks
and MRTR harnesses used to loop scenarios across both the legacy
session wire and the SEP-2575 stateless wire. After SEP-2575 collapsed
the legacy wire on DRAFT-2026-v1 (a6f7c27), the matrix shrank to a
single column on draft — and an explicit `wire` field was both
redundant and unsafe: it let callers go out of sync with the
`specVersion` they declared on the wire.
The CLI's `runServerConformanceTest` never set the field, so every
`npm start -- server --scenario tasks-* --url ...` invocation silently
emitted the legacy `initialize` handshake against a draft server,
producing requests with no `_meta.io.modelcontextprotocol/*` envelope.
Strict SEP-2575 servers correctly rejected the traffic; lenient ones
accepted it and masked the gap.
Changes:
- Drop `wire?` from `RunContext`.
- Add `isStateless(ctx)` in `_shared/wire-mode.ts`; scenarios derive
`stateless` from `POST_SEP_2575_VERSIONS.has(ctx.specVersion)`.
- Extend the CLI's spec-version inference to default extension
scenarios to draft (today every extension scenario in the repo lives
there). Without this, the inference stayed on `LATEST_SPEC_VERSION`
for tasks scenarios and kept emitting legacy traffic.
- Flatten the now-1-element wire loop in the tasks + mrtr harnesses;
remove the parseWireModes/effectiveWireModes/MCP_WIRE_MODES infra
that drove it (no remaining consumers).
Adds a vitest regression in `runner/server.test.ts` that spins up an
in-process recording HTTP server, drives `runServerConformanceTest`
at `tasks-lifecycle`, and asserts the first outgoing call is
`server/discover` carrying `_meta.io.modelcontextprotocol/*` — the
SEP-2575 fingerprint — so the CLI wire selection can't silently
regress.
* scenarios + connection: address conformance#262 layout review
Folds the parallel mini-framework the tasks/MRTR suites had grown into
the existing harness shape pcarleton called out in conformance PR 262.
No new spec coverage; existing scenarios still pass 241/241 unit tests
and the regression added for the wire-derivation fix still pins the CLI
fingerprint.
Layout
scenarios/server/_shared/ is gone. The helper bundle moves to
scenarios/server/tasks-mrtr-helpers.ts (matching the existing
input-required-result-helpers.ts precedent); raw-session.ts and its
test fold into connection/stateless.ts.
Per-suite runners + READMEs (tasks/all-scenarios.test.ts,
mrtr/all-scenarios.test.ts, _shared/test-runner.ts, both READMEs) are
removed. The scenarios are already registered in
scenarios/index.ts pendingClientScenariosList, so the CLI runner
(`npm start -- server --scenario tasks-lifecycle --url ...` or
`--suite pending --url ...`) is the supported entry point. Per
AGENTS.md: don't reimplement the runner.
Connection extension
Connection grows the three things tasks/MRTR scenarios actually need
beyond the existing surface:
- `request(method, params, extraHeaders?)` for the SEP-2243
header-mismatch tests.
- `discover(): Promise<Record<string, unknown>>` for
capability-negotiation scenarios. Resolved eagerly from the SDK
accessors on the stateful wire; issued lazily as `server/discover`
on the stateless wire and memoized.
- `ConnectOptions { capabilities?, clientInfo? }` on
RunContext.connect() so scenarios can negotiate extensions
(`extensions: { 'io.modelcontextprotocol/tasks': {} }`) at
bootstrap on either wire.
Crucially the stateless wire does NOT eagerly emit `server/discover`
when capabilities are declared. SEP-2575 says per-request
`_meta.io.modelcontextprotocol/clientCapabilities` is authoritative;
a server that gates extensions on a prior `server/discover` is
non-conformant, and the conformance suite's job is to surface that,
not paper over it.
Scenario migration
Every tasks/MRTR scenario switches from `initRawSession(...)` to
`await ctx.connect({ capabilities })`, and the capability scenario
switches from `session.initializeResult.capabilities` to
`await withExt.discover()`. The wire choice now flows entirely from
`specVersion` — `isStateless(ctx)` (moved to connection/select.ts
next to `connectFor`) is the single source of truth.
Description sweep
Each scenario's `description` string now spells out the required
server fixtures (tool names + behavior) directly. Previously that
content lived only in the file-header comment, so an implementer
reading the runner's emitted description was missing it.
Reviewer notes
- 23 files changed, +455 / -291 (net + because the Connection
abstraction now carries surface that used to live in raw-session;
the raw-session deletion was 692 lines on its own).
- Unit tests: 241 pass (the 12 deleted tests were redundant with
connection-level coverage; the genuinely-unique SSE multi-line
`data:` test pinned behavior the connection parser doesn't have).
- Reference fixture runs: tasks scenarios surface a fixture-side
spec gap (per-request `_meta` not honored without a prior
`server/discover`). That's the suite working; the gap will be
filed upstream against the fixture, not papered over here.
* tasks: dedupe MRTR ephemeral-flow against #188, fold composition into tasks/
Of the 8 checks in mrtr/ephemeral-flow.ts, 7 substantially overlapped
the SEP-2322 InputRequiredResult* scenarios that #188 already shipped
(basic-elicitation, sampling, list-roots, request-state, multiple-input-
requests, multi-round, wrong-input-key). Only sep-2663-mrtr-synchronous-
before-task-creation — the SEP-2663 commit 451f5e1 composition path
where the MRTR loop's final round returns a CreateTaskResult — is
genuinely new versus #188's coverage, and it's SEP-2663-specific, not
SEP-2322-specific.
Moves that one check to its own tasks scenario in tasks/composition.ts
(extensionId = io.modelcontextprotocol/tasks, registered alongside the
other tasks-* scenarios in scenarios/index.ts) and deletes the entire
mrtr/ directory.
Reuses the existing #188 helpers (isInputRequiredResult, mockElicitResponse
from input-required-result-helpers.ts) rather than reintroducing the
parallel mrtr/helpers.ts ones.
* address review on conformance#262 follow-up
- connectStateless.discover() now reuses the request() pipeline (send +
drainEvents + unwrap) instead of inlining a duplicate copy. The local
memoization stays via discoverPromise ??=.
- runner/server.test.ts mock server no longer silently swallows
unparseable bodies. They land in a parseFailures sidecar that the
test asserts is empty — a non-JSON body to this mock is exactly the
kind of regression the test exists to catch.
- Drop the tautological connectFor identity tests. They asserted
`typeof === 'function'` and `not.toBe(connectStateful)`, both of which
hold trivially once connectFor wraps in a closure. Wire-shape behavior
is already covered by the connectStateless mock-fetch suite and by
runner/server.test.ts.
* tasks/mrtr: address PR 262 audit before review
Pre-emptive cleanup against the upstream patterns established by recent
merges (#318 Connection abstraction, #319 draft-spec consistency, #303
SEP-2243 check IDs, #321 MockServer surface). No behavior change for
scenarios — every check is still emitted; only the shape of how they're
emitted moves toward upstream's house style.
Surface
- Revert withRequestMeta / sendStatelessRequest signatures to match
upstream byte-for-byte. The need that pushed me to widen those
(per-request _meta.clientCapabilities for extension declaration on
the SEP-2575 stateless wire) is now handled inside connectStateless:
ConnectOptions.capabilities / clientInfo are folded into params._meta
via the trailing _meta spread that withRequestMeta already supports.
This keeps the new "extension declaration" concept on the Connection
layer only — the lower-level helpers stay shared with every other
scenario in the suite.
- Document that Connection.request's extraHeaders throws on the
stateful wire instead of "ignored with a warning" — the SDK transport
manages headers internally, so silent drop in a conformance harness
would mask test correctness.
Layout
- src/scenarios/server/tasks-mrtr-helpers.ts moves under tasks/ as
tasks/mrtr-helpers.ts. All 10 importers live in tasks/; the file's
prior placement next to input-required-result-helpers.ts no longer
applies once those scenarios moved into tasks/.
- Drop the unused AnyResult export + zod import. The Connection wrapper
doesn't take a zod schema, so the helper was dead on arrival.
- SEP-2322 spec reference now re-exports MRTR_SPEC_REFERENCES[0] from
input-required-result-helpers so the codebase holds one URL per SEP
instead of two for SEP-2322.
Naming
- All 10 tasks/*.ts switch to `readonly source = { ... } as const` to
match the convention used by caching.ts, resources.ts,
http-standard-headers.ts. Drops the ScenarioSource import where now
unused.
- Scenario-scope each scenario's connect-failure check ID
(tasks-lifecycle-bootstrap, tasks-headers-bootstrap, …) instead of
sharing tasks-session-bootstrap. Matches the upstream caching.ts
pattern (sep-2549-caching-connection) where each scenario owns its
own bootstrap-failure ID.
Traceability
- sep-2663.yaml gains the positive-path counterpart to the existing
rejection requirement under Streamable HTTP: Routing Headers, so
every sep-2663-* check ID emitted by tasks/*.ts is declared in the
manifest.
Verification
- Conformance vitest: 270/270.
- Build: clean.
- mcpkit testconf-tasks-v2 (47 fork scenarios + 1 sentinel): pass.
- mcpkit testconf-mrtr (3 fork scenarios + 1 sentinel): pass.
* tasks/required-task-error: anchor -32003 checks to sep-2663.yaml rows
The two checks in required-task-error.ts each verified a distinct
normative requirement that sep-2663.yaml already declared, but the
check IDs (tasks-required-task-error-code, tasks-required-task-error-data-shape)
used the tasks-* flow-gate prefix and went untracked by the traceability
manifest.
Promote both check IDs to sep-2663-*:
- tasks-required-task-error-code → sep-2663-server-returns-32003-when-required
- tasks-required-task-error-data-shape → sep-2663-server-returns-32003-data-shape
Split the existing YAML entry to mirror the split. The spec sentence
glues two observable requirements with a trailing colon (the JSON-RPC
code, and the data-shape payload that names the missing extension);
each gets its own row so each is verifiable independently.
Remove the now-stale tasks-required-task-error-code reference from the
sep-2663.yaml flow-gate comment block — that check is no longer a
flow-gate.
No behavior change. 270/270 vitest still pass.
* runner: forward ConnectOptions from RunContext.connect
server.ts built the RunContext with `connect: () => connectFor(...)(serverUrl)`,
dropping the opts argument scenarios pass to ctx.connect(). TypeScript
doesn't catch this because `() => T` is assignable to `(opts?) => T`.
Result: scenarios declaring capabilities sent DEFAULT_CLIENT_CAPABILITIES
on the wire in CLI runs while testContext() forwarded opts correctly,
so unit tests passed but `npm start -- server --scenario tasks-*` against
a strict-draft server failed with missing-capability errors.
Tightens the regression test to assert scenario-passed capabilities reach
_meta.clientCapabilities.extensions, not just that defaults are present.
* tasks: align scenario descriptions and cancel-ack check with SEP-2663
capability.ts: description said non-declaring tasks/* requests get
-32601 MethodNotFound; the assertion checks MissingRequiredClientCapability
per SEP-2663 §Error Handling. Reworded to the named error type and
per-request _meta/server-discover framing (SEP-2575).
lifecycle.ts: description claimed cancel-on-terminal MUST return
-32602; Check 8 in the same file asserts an idempotent ack and the
SEP has no -32602 carve-out for terminal tasks. Removed the bullet.
lifecycle.ts Check 7: SEP-2663 §Task Cancellation says transition to
`cancelled` is not guaranteed; stop failing the ack-shape check on
the settled status and record it in details instead.
* connection: fix stale docstring and drop redundant eta-expansion
stateless.ts docstring referenced `connection.initializeResult`; the
implementation exposes `discover()`.
select.ts wrapped connectStateful in an identity arrow; pass it
directly now that signatures match.
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
The error-code allocation policy in modelcontextprotocol#2907 renumbers
the draft-only codes: HeaderMismatch -32001 -> -32020,
MissingRequiredClientCapability -32003 -> -32021,
UnsupportedProtocolVersion -32004 -> -32022. This updates every
reference in the suite (mocks, scenarios, fixtures, tests, traceability
yaml) and re-syncs the vendored draft schema to that commit.
src/seps/traceability.json is generated and intentionally left for the
scheduled refresh workflow.
Re-vendors schema/draft/schema.ts from the spec repo at the merge commit
of PR #2891 (2fb207da428f43a4981513ec2aef218b7533c5b3), which also
includes #2889 and #2890.
Type changes picked up:
- Add NotificationMetaObject (with io.modelcontextprotocol/subscriptionId)
- Add HEADER_MISMATCH (-32001) const and HeaderMismatchError interface
- CancelledNotificationParams.requestId is now required
- ClientNotification narrowed to CancelledNotification only
- ListRootsRequest.params narrowed (drops progressToken)
- Remove ElicitationCompleteNotification and elicitationId
Also replaces the local -32001 literal in
src/scenarios/server/http-standard-headers.ts with the new exported
HEADER_MISMATCH constant.
Claude-Session: https://claude.ai/code/session_01Gbqx5LodtAghT5fRYFxeja
The SEP-2243, SEP-2322, and SEP-2106 client-direction scenarios build
their own mock servers and did not answer server/discover, so a
2026-07-28 client that negotiates via discover before its scenario calls
hit a -32601 or generic fallback and had to synthesize the response
itself. The shared stateless mock and the request-metadata and auth
scenarios already served it.
BaseHttpScenario now answers server/discover with a valid DiscoverResult
(supportedVersions, capabilities from an overridable
discoverCapabilities(), serverInfo, plus the cacheable-result members)
before delegating to handlePost; http-standard-headers overrides the
capabilities to match the methods it serves. mrtr-client and
json-schema-ref-no-deref answer it directly. New tests pin the response
shape for all five scenarios.
The auth/scope-step-up scenario emitted sep-2350-scope-union-on-reauth at
every spec version, but the set-wise union requirement (SEP-2350) was only
introduced in 2026-07-28. This wrongly warned clients that are conformant at
2025-11-25 and drop the previously-granted scope on re-authorization (e.g.
python-sdk v1, surfaced while wiring up its runner in #329).
Gate the check behind specVersionAtLeast(specVersion, DRAFT_PROTOCOL_VERSION),
a small timeline-ordering helper added to types.ts (mirrors the ordering used
by scenario applicability). Add tests pinning both sides of the branch:
enforced at draft, silent at 2025-11-25.
Follow-up to #329.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The everything-server returned its -32602 missing-_meta rejection with
HTTP 200; the draft spec requires 400 Bad Request. It also applied the
per-request _meta validation to sessionless requests carrying a legacy
session-era version header, although per-request metadata is a
2026-07-28 requirement; such requests now fall through to the session
path.
The server-stateless scenario gains a companion check
(sep-2575-http-server-meta-invalid-400) requiring HTTP 400 on those
rejections, and sep-2575-server-unsupported-version-error now also
requires error.data.requested to echo the requested version. Two new
negative tests pin both.
At the 2026-07-28 revision every server result must include resultType,
and cacheable results (server/discover, tools/list, prompts/list,
resources/list, resources/templates/list, resources/read) must also
include ttlMs and cacheScope. The client-direction mock servers omitted
these, so a strictly-validating client was failed by the suite's own
non-conformant mock.
The shared stateless mock now stamps the missing members onto handler
results and its server/discover response (withRequiredDraftResultFields),
and the hand-rolled per-scenario servers set them explicitly or route
their fallbacks through the helper. The deliberately resultType-less
probe in sep-2322-client-request-state is unchanged. New tests pin the
response shapes.
* feat: add python-sdk as a built-in SDK runner target
Wire modelcontextprotocol/python-sdk into the `conformance sdk` runner
alongside typescript-sdk and go-sdk, using the SDK's existing conformance
fixtures for both client and server modes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: target python-sdk v1.x on spec 2025-11-25
Convert the built-in python-sdk entry from v2/main to python-sdk-v1,
mirroring typescript-sdk-v1: clone the v1.x branch, default specVersion
to 2025-11-25 so draft-only scenarios/checks are excluded, and point at
the SDK's own expected-failures baseline. v2/main is mid-refactor and
noisy; v1.x is the stable published line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: default sdk-runner server mode to the `active` suite
Server runs previously fell through to the underlying `server` command's
default; make it explicit and consistent with tiering, which runs the
`active` suite (excludes pending/draft). Avoids surfacing intentionally
-deferred `pending` scenarios (e.g. json-schema-2020-12, server-sse
-polling) as failures. Runner-wide default for all SDKs; callers can
still override with --suite.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: sync spec-types to modelcontextprotocol@0168c57f
Refreshes the vendored schema copies via npm run sync-schema. The draft
schema now declares LATEST_PROTOCOL_VERSION = "2026-07-28" (previously
the DRAFT-2026-v1 placeholder) and picks up the latest draft doc-comment
and type updates. The dated schemas are unchanged.
* fix: update the draft wire version to 2026-07-28
The spec repo's draft schema now declares
LATEST_PROTOCOL_VERSION = "2026-07-28", but the harness still asserted
the old DRAFT-2026-v1 placeholder, so an SDK that sends the version the
draft spec actually declares cannot pass the draft suite.
- bump DRAFT_PROTOCOL_VERSION in src/types.ts to 2026-07-28
- update the hard-coded literals in tests, example servers, and README
- add src/types.test.ts pinning DRAFT_PROTOCOL_VERSION to the vendored
draft schema's LATEST_PROTOCOL_VERSION so the two cannot drift, and
asserting the draft identifier stays distinct from the dated versions
* feat: vendor spec schema types per-version
Copies schema/{version}/schema.ts from the modelcontextprotocol spec
repo into src/spec-types/{version}.ts so the conformance suite can type
against draft spec versions before any SDK ships them.
npm run sync-schema -- <ref> refreshes the copies and records the spec
commit in src/spec-types/SOURCE.
* feat: add Connection abstraction and RunContext
Connection encapsulates how the conformance suite talks to a
server-under-test for a given spec version:
- connectStateful: 2025-x lifecycle. Thin adapter over the SDK Client
(initialize handshake, session id, SSE handled by the SDK).
- connectStateless: 2026-x lifecycle (SEP-2575). Raw fetch with
per-request _meta + MCP-Protocol-Version header. Decoupled from the
SDK so the suite can test draft spec versions before the SDK
implements them.
connectFor(specVersion) picks the implementation. RunContext bundles
serverUrl, specVersion and a bound connect() for the runner to hand to
each scenario.
Nothing uses this yet; wiring follows in the next commit.
* refactor: thread RunContext through ClientScenario.run
ClientScenario.run(serverUrl) becomes run(ctx: RunContext). The runner
builds the context from --spec-version and the server URL; scenarios
destructure ctx.serverUrl and otherwise behave identically.
No scenario uses ctx.connect() yet, so behaviour is unchanged: 214/214
tests pass, all-scenarios.test.ts still drives the everything-server
fixture exactly as before.
Test files use a testContext(url) helper to construct a RunContext.
The authorization-server scenario list is retyped to
ClientScenarioForAuthorizationServer since those scenarios test an
OAuth server, not an MCP server, and keep run(serverUrl).
* refactor: migrate server scenarios to ctx.connect() + conn.request()
22 carry-forward and lifecycle scenarios now go through the Connection
abstraction instead of connectToServer + SDK Client:
- tools.ts (8): list, call x6, with-progress
- prompts.ts (5)
- resources.ts (7): list, read x3, subscribe x2, not-found
- utils.ts (3): completion, ping, set-level
- json-schema-2020-12.ts, caching.ts, http-standard-headers.ts
Result types come from spec-types/{introducedIn}. Same scenario code
now passes under both --spec-version 2025-11-25 (SDK-backed stateful)
and --spec-version draft (raw stateless).
ToolsCallSampling/Elicitation/WithLogging and elicitation-* keep
connectToServer (need setRequestHandler/setLoggingLevel SDK surface);
they are tagged removedIn: DRAFT in the next commit. stateless.ts and
input-required-result.ts keep their sendRpc helper; migrating those is
deferred to the DRAFT-scenario coherence pass.
* fix: normalize Connection error to JsonRpcError; clean up RunContext import
connectStateful now catches the SDK's McpError and rethrows as
JsonRpcError so scenarios always see the same error class regardless
of which Connection impl ran. ResourcesNotFoundError uses
instanceof JsonRpcError instead of duck-typing.
types.ts uses a normal top-level import for RunContext instead of an
inline import() type.
* fix(dns-rebinding): use version-appropriate probe body
The 'valid Host accepted' check was sending an initialize body, which
a 2026 server returns 404 for. Probe body is now picked from
ctx.specVersion: initialize for the stateful lifecycle, server/discover
with _meta for the stateless lifecycle. The Host/Origin rejection check
is unchanged since rejection happens before body parsing.
* feat(everything-server): route stateless carry-forward methods to McpServer
Adds an in-memory dispatch client connected to the same McpServer the
stateful path uses. Stateless requests for tools/call, resources/*,
prompts/get and completion/complete that fall through the MRTR-specific
handlers are routed to it, so the fixture serves the carry-forward
scenarios under --spec-version draft without duplicating ~500 lines of
tool/resource/prompt registrations.
tools/list now merges the McpServer's tool list with the MRTR-only
stubs so json-schema-2020-12 finds its tool.
draft suite against the fixture: 36/39 (was 13/39). Remaining 3 are
fixture-side SEP gaps (no SSE forwarding for progress, no SEP-2549
ttlMs, no SEP-2243 Mcp-Method validation in the stateless path).
* refactor(connection): drop unused RequestOptions; move sdk-client; add tests
Addresses self-review findings on the new connection module:
- RequestOptions.handlers/.meta and ServerRequestHandler removed: zero
callers (the scenarios that motivated them are deferred). They can be
reintroduced when something actually uses them.
- scenarios/server/client-helper.ts moved to connection/sdk-client.ts so
the connection module no longer depends on the scenarios module.
- connectStateless: handle CRLF SSE event separators; throw a useful
error for non-JSON/non-SSE responses instead of a JSON parse error.
- JSONRPCNotification consistently imported from spec-types/2025-11-25.
- New connection.test.ts (8 tests) covering connectFor selection,
_meta injection, error mapping, SSE LF/CRLF parsing, and the
server-request-on-stream rejection.
* fix: address bughunt findings (response.ok check; targetVersion naming)
- connectStateless: throw on non-2xx responses that lack a JSON-RPC
error envelope (e.g. gateway/framework error JSON), matching the
stateful path's behavior. Previously such a response would return
undefined as the result.
- json-schema-2020-12: rename negotiatedVersion to targetVersion and
reword the skip message and details field. The value is ctx.specVersion
(the run's --spec-version), not a negotiated version; the old wording
was misleading. Drop dead 'unknown' fallback (specVersion is required)
and the corresponding undefined test cases.
* fix(sse-multiple-streams): keep scenario in draft; version-aware request scaffolding
Concurrent POSTs each answered with JSON or their own SSE stream are
core transport behavior in the draft spec too, so removedIn was wrong
for this scenario. Only the request scaffolding is version-specific:
stateful runs keep the initialize/session-id setup, draft runs send
_meta + the MCP-Protocol-Version header on each request instead. The
two checks are unchanged.
* feat: add MockServer abstraction and ScenarioContext
MockServer encapsulates the lifecycle scaffold a client-conformance
scenario presents to the client-under-test:
- createServerStateful: 2025-x lifecycle. SDK Server +
StreamableHTTPServerTransport (sessionless mode); the SDK handles the
initialize handshake.
- createServerStateless: 2026-x lifecycle (SEP-2575). Raw express app
that validates _meta + MCP-Protocol-Version on every request, serves
server/discover, routes other methods to the supplied handlers.
createServerFor(specVersion) picks the implementation. ScenarioContext
bundles specVersion and a bound createServer() for the runner to hand
to each scenario.
This is the client-conformance mirror of src/connection (PR #318).
Nothing uses it yet; wiring follows in the next commit.
* refactor: thread ScenarioContext through Scenario.start()
Scenario.start() becomes start(ctx: ScenarioContext). The runner builds
the context from --spec-version (defaulting to LATEST_SPEC_VERSION) and
passes it through; scenarios receive it as _ctx and otherwise behave
identically.
No scenario uses ctx.createServer() yet, so behaviour is unchanged:
231/231 tests pass.
Test files use a testScenarioContext() helper. The runner already
threads MCP_CONFORMANCE_PROTOCOL_VERSION to the spawned client process,
so the fixture-side env wiring is unchanged.
* refactor: migrate tools_call to ctx.createServer; tag 2025-only client scenarios
ToolsCallScenario now goes through ctx.createServer() instead of an
inline express + SDK Server build. Same handlers, same checks; the
assertion now reads from srv.recorded so it works regardless of which
lifecycle scaffold the runner picked.
initialize, sse-retry, and elicitation-defaults are tagged
removedIn: DRAFT (initialize/GET-SSE/SSE-embedded-elicitation are gone
in the 2026 lifecycle; the MRTR sibling for elicitation-defaults is a
follow-up).
spec-version.test.ts: the 'draft is a superset of latest' invariant no
longer holds once removedIn: DRAFT exists; the test now asserts that
any scenario in latest-but-not-draft is explicitly removedIn.
* feat(auth): make createServer helper version-aware via ScenarioContext
The auth helper now takes ctx: ScenarioContext as its first argument
and branches on ctx.specVersion inside the /mcp route: the stateful
path (SDK Server + StreamableHTTPServerTransport) is unchanged; under
the draft version a raw stateless handler validates _meta + the
MCP-Protocol-Version header, serves server/discover, and routes the
same tools/list and tools/call responses.
The PRM endpoint, bearer-auth middleware, and request logger sit above
the branch and are version-independent.
All 25 call sites across the 12 auth scenario files pass ctx through;
ServerLifecycle and the express.Application return type are unchanged
so stop()/getChecks() are untouched.
Deviation from the MockServer wrapper approach: keeping the helper's
return type as express.Application avoids restructuring 25 call sites'
ServerLifecycle handling in this PR. Folding the auth seam onto
ctx.createServer() fully is a follow-up once the lifecycle ownership
moves into MockServer.
* feat(everything-client): pick stateless requester by MCP_CONFORMANCE_PROTOCOL_VERSION
Adds a statelessRequest(serverUrl, method, params) helper that POSTs
with _meta + MCP-Protocol-Version (the SEP-2575 lifecycle), shimming
around the SDK Client not yet supporting stateless mode. The
runRequestMetadataClient handler's meta constants are extracted to share
with the helper.
runBasicClient (initialize, tools_call, json-schema-ref-no-deref) now
branches on MCP_CONFORMANCE_PROTOCOL_VERSION: for the draft version it
uses statelessRequest to call tools/list then tools/call; for dated
versions it keeps the SDK Client path.
The runner already passes MCP_CONFORMANCE_PROTOCOL_VERSION to the
spawned client, so no runner change is needed.
* fix: address review findings on MockServer (dead opts, shared validator, capability derivation, recorded parity, specVersion threading)
- MockServerOptions removed (capabilities/configure had zero callers); opts
param dropped from createServerStateful/Stateless/For and ScenarioContext.
- validateStatelessRequest extracted from mock-server/stateless and exported;
both the stateless MockServer and auth/helpers/createServer.ts call it so
_meta/header/version validation cannot drift.
- isStatefulVersion exported from connection/select; mock-server/select uses
it instead of duplicating the version set.
- runner/client.ts: env MCP_CONFORMANCE_PROTOCOL_VERSION set unconditionally
to the resolved version; runInteractiveMode now takes specVersion and the
CLI passes it.
- createServerStateful: capabilities derived from handler method prefixes;
newServer() moved inside the try so a capability mismatch surfaces as
JSON-RPC -32603 instead of an HTML 500. Recording moved to the express
layer so unregistered methods are captured (parity with stateless).
- readFinalSseMessage return type now declares error.data.
- Tests added for the capability derivation and unregistered-method recording.
* fix(mock-server): record stateless requests before validation; document preamble exclusions
The stateless impl pushed to recorded[] only after validateStatelessRequest
passed, so requests rejected for a missing header or _meta never appeared,
diverging from the stateful impl which records at the HTTP layer before SDK
dispatch. Record before validation, excluding only the server/discover
lifecycle preamble (mirroring the stateful initialize/notifications-initialized
exclusion), and fix the MockServer.recorded docstring which wrongly claimed
stateless excludes nothing.
* refactor(mock-server): tri-state result for validateStatelessRequest (reject/handled/route)
A successful server/discover was previously modeled as {ok: false},
which read as a validation failure but actually meant "already
answered, do not route". Replace the boolean discriminant with an
explicit kind: 'reject' (validation failed), 'handled' (valid and
already answered), or 'route' (dispatch to handlers).
Both consumers (the stateless /mcp route and the auth helper's
handleStateless) still write status/body for reject and handled and
route only on 'route', so responses are unchanged. Add direct unit
tests covering each branch of the new union.
* feat(runner): single-source the version→lifecycle mapping; export MCP_CONFORMANCE_LIFECYCLE
The stateless mock and the example client both hardcoded the draft
protocol version, so dating the draft (or adding a second stateless
version) would silently break them: the mock would keep rejecting
everything but the old constant, and the client would fall back to the
stateful path.
- Derive STATELESS_SPEC_VERSIONS and a lifecycleFor() helper from
isStatefulVersion() in connection/select.ts, making it the single
source of truth for the version→lifecycle mapping.
- Thread the resolved spec version into the stateless mock:
createServerFor() binds it into createServerStateless(), and
validateStatelessRequest() now takes the supported-versions list as a
parameter. A mock bound to a version accepts exactly that version and
echoes it in -32004 rejections and server/discover; unbound mocks
accept every known stateless version.
- Export MCP_CONFORMANCE_LIFECYCLE (stateful|stateless) to client
processes alongside MCP_CONFORMANCE_PROTOCOL_VERSION, in both the
runner and the in-process test helper, so clients in any language can
pick the lifecycle without keeping their own version map.
- everything-client: import DRAFT_PROTOCOL_VERSION from src/types
instead of duplicating the string, branch on MCP_CONFORMANCE_LIFECYCLE
(falling back to the version comparison for older runners), and send
the runner-resolved version on stateless requests.
- Document MCP_CONFORMANCE_LIFECYCLE in the README and ignore the local
.claude/ workspace dir in .prettierignore so prettier --check . is not
tripped by untracked worktree checkouts.
* fix(everything-client): list-only flow for json-schema-ref-no-deref
runBasicClient now follows tools/list with a tools/call on the first
listed tool, but the json-schema-ref-no-deref scenario mock only serves
tools/list, so the call got -32601 and the client exited non-zero.
Register the scenario with a dedicated handler that stops after listing
tools, keeping the existing lifecycle branch (raw stateless request vs
SDK client). Listing is the whole flow for SEP-2106: the check is that
the client never fetches the network $ref canary while processing the
advertised schema.
* fix(tools_call): make getChecks() idempotent
getChecks() pushed into an instance array on every call, so calling it
more than once accumulated duplicate check entries. The runner is
allowed to call getChecks() repeatedly, and other scenarios already
build their result fresh per call for this reason.
Return a freshly built array instead and drop the now-unused instance
field. Add a unit test covering the not-called FAILURE case and
asserting repeated getChecks() calls do not grow the result.
* fix: ensure dispatch and transport cleanup runs when handlers throw
The stateless tools/list and prompts/list handlers in the everything
server awaited dispatch.close() after the in-memory request, so a throw
from dispatch.client.request() leaked the Client+Server+Transport pair.
Wrap both in try/finally, matching the tools/call and catch-all
handlers in the same file.
In the stateful mock server, the res 'close' listener that tears down
the per-request SDK transport and server was registered after the
handleRequest await; a throw there skipped registration and leaked the
pair. Register the listener before handing the request to the
transport.
* fix(everything-client): send Accept header on stateless requests
The draft transport spec requires POST requests to accept both
application/json and text/event-stream. Servers built on the SDK's
StreamableHTTPServerTransport already enforce this and reject
requests without it. Does not yet make json-schema-ref-no-deref pass
under --spec-version draft: the SDK transport also rejects the draft
protocol version itself, which is the known fixture gap until the
2026-native scenarios migrate onto MockServer.
* chore: exclude .claude/ from vitest globs
Local agent worktree checkouts under .claude/worktrees/ carry their
own copies of the test suite, which the bare **/*.test.ts include was
adding to every run (and to the pre-push hook). Mirrors the existing
.prettierignore entry.
* fix(request-metadata): use spec error code -32004 for unsupported protocol version
The draft spec defines -32004 (UnsupportedProtocolVersionError) with
data { supported, requested } and HTTP 400 for protocol version
rejections; -32001 (HeaderMismatch) is reserved for HTTP header
validation failures.
- request-metadata scenario: return -32004 with spec-shaped data for
the simulated version-negotiation rejection (was -32001)
- everything-client: trigger negotiation on -32004 (was -32001)
- stateless mock server: include the requested version in the -32004
error data alongside the supported list
- update tests to match and assert the new requested field
* fix(auth): register transport cleanup before handleRequest in createServer
The res 'close' listener that tears down the transport/server pair was
registered after awaiting handleRequest, so a throw mid-request skipped
cleanup registration entirely. Register it first, matching the stateful
mock server.
* fix(everything-server): return JSON-RPC errors from stateless list dispatch on failure
The stateless tools/list and prompts/list dispatch handlers had
try/finally but no catch, so a dispatch failure produced an unhandled
rejection instead of a response. Add the same catch-and-respond block
the carry-forward dispatch handler already uses.
* refactor(runner): drop MCP_CONFORMANCE_LIFECYCLE; derive lifecycle from protocol version
The protocol version fully determines the lifecycle, and deriving it is
part of being a conformant client. The in-repo example client imports
the stateless version set from src/, so there is no drift risk, and a
single env var cannot disagree with itself. README documents the
version→lifecycle mapping for clients in other languages.
* fix(everything-server): return JSON-RPC errors from stateless resource list dispatch on failure
resources/list and resources/templates/list arrived from main with
try/finally but no catch, unlike the sibling tools/list and
prompts/list handlers. A throwing dispatch would escape to Express's
HTML 500 instead of a JSON-RPC error. Mirrors the existing pattern.
* fix(everything-server): include requested field in -32004 error data
The spec's UnsupportedProtocolVersionError requires data to carry both
supported and requested. The mock-server and request-metadata -32004
producers already include it; this was the remaining producer.
* feat(runner): client-side spec-version inference, skip, and --force
Mirrors the server runner's policy from #318: an explicitly requested
--spec-version outside a scenario's applicability window skips it
(exit 0) unless --force; when --spec-version is omitted the version is
inferred from the scenario's applicability instead of defaulting to
the latest dated version.
* fix(auth): gate SEP-837 application_type check to spec versions that include it
The sep-837-application-type-present check fired on every Dynamic Client
Registration regardless of the targeted spec version, but SEP-837 only
exists in the draft spec. Clients conforming to 2025-11-25 or earlier were
failed on a requirement that is not part of the version under test.
Thread the ScenarioContext into createAuthServer (mirroring createServer)
and only emit the check when the resolved spec version includes SEP-837,
i.e. the stateless draft lifecycle. At dated versions the check is not
emitted at all. Audited the other auth helper checks (PKCE, metadata
discovery, token verification); all of them enforce requirements that are
already in the dated specs, so SEP-837 was the only draft-only check.
Unit tests cover both sides: the check fails an omitting client at the
draft version and is absent at 2025-11-25.
* feat(sdk-runner): --spec-version passthrough with per-SDK default
The sdk command forwarded --scenario/--suite/--timeout to the underlying
client/server invocation but had no way to set the spec version, so SDK
runs always used the suite default. Add a --spec-version flag that is
passed through to the inner run.
Also add an optional specVersion field to the per-SDK config, used as the
default when the flag isn't given; an explicit --spec-version overrides
it. Set typescript-sdk-v1 to 2025-11-25: the v1 line targets the latest
dated spec, which keeps draft-only scenarios out of its runs by default.
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
* feat: vendor spec schema types per-version
Copies schema/{version}/schema.ts from the modelcontextprotocol spec
repo into src/spec-types/{version}.ts so the conformance suite can type
against draft spec versions before any SDK ships them.
npm run sync-schema -- <ref> refreshes the copies and records the spec
commit in src/spec-types/SOURCE.
* feat: add Connection abstraction and RunContext
Connection encapsulates how the conformance suite talks to a
server-under-test for a given spec version:
- connectStateful: 2025-x lifecycle. Thin adapter over the SDK Client
(initialize handshake, session id, SSE handled by the SDK).
- connectStateless: 2026-x lifecycle (SEP-2575). Raw fetch with
per-request _meta + MCP-Protocol-Version header. Decoupled from the
SDK so the suite can test draft spec versions before the SDK
implements them.
connectFor(specVersion) picks the implementation. RunContext bundles
serverUrl, specVersion and a bound connect() for the runner to hand to
each scenario.
Nothing uses this yet; wiring follows in the next commit.
* refactor: thread RunContext through ClientScenario.run
ClientScenario.run(serverUrl) becomes run(ctx: RunContext). The runner
builds the context from --spec-version and the server URL; scenarios
destructure ctx.serverUrl and otherwise behave identically.
No scenario uses ctx.connect() yet, so behaviour is unchanged: 214/214
tests pass, all-scenarios.test.ts still drives the everything-server
fixture exactly as before.
Test files use a testContext(url) helper to construct a RunContext.
The authorization-server scenario list is retyped to
ClientScenarioForAuthorizationServer since those scenarios test an
OAuth server, not an MCP server, and keep run(serverUrl).
* refactor: migrate server scenarios to ctx.connect() + conn.request()
22 carry-forward and lifecycle scenarios now go through the Connection
abstraction instead of connectToServer + SDK Client:
- tools.ts (8): list, call x6, with-progress
- prompts.ts (5)
- resources.ts (7): list, read x3, subscribe x2, not-found
- utils.ts (3): completion, ping, set-level
- json-schema-2020-12.ts, caching.ts, http-standard-headers.ts
Result types come from spec-types/{introducedIn}. Same scenario code
now passes under both --spec-version 2025-11-25 (SDK-backed stateful)
and --spec-version draft (raw stateless).
ToolsCallSampling/Elicitation/WithLogging and elicitation-* keep
connectToServer (need setRequestHandler/setLoggingLevel SDK surface);
they are tagged removedIn: DRAFT in the next commit. stateless.ts and
input-required-result.ts keep their sendRpc helper; migrating those is
deferred to the DRAFT-scenario coherence pass.
* fix: normalize Connection error to JsonRpcError; clean up RunContext import
connectStateful now catches the SDK's McpError and rethrows as
JsonRpcError so scenarios always see the same error class regardless
of which Connection impl ran. ResourcesNotFoundError uses
instanceof JsonRpcError instead of duck-typing.
types.ts uses a normal top-level import for RunContext instead of an
inline import() type.
* fix(dns-rebinding): use version-appropriate probe body
The 'valid Host accepted' check was sending an initialize body, which
a 2026 server returns 404 for. Probe body is now picked from
ctx.specVersion: initialize for the stateful lifecycle, server/discover
with _meta for the stateless lifecycle. The Host/Origin rejection check
is unchanged since rejection happens before body parsing.
* feat(everything-server): route stateless carry-forward methods to McpServer
Adds an in-memory dispatch client connected to the same McpServer the
stateful path uses. Stateless requests for tools/call, resources/*,
prompts/get and completion/complete that fall through the MRTR-specific
handlers are routed to it, so the fixture serves the carry-forward
scenarios under --spec-version draft without duplicating ~500 lines of
tool/resource/prompt registrations.
tools/list now merges the McpServer's tool list with the MRTR-only
stubs so json-schema-2020-12 finds its tool.
draft suite against the fixture: 36/39 (was 13/39). Remaining 3 are
fixture-side SEP gaps (no SSE forwarding for progress, no SEP-2549
ttlMs, no SEP-2243 Mcp-Method validation in the stateless path).
* refactor(connection): drop unused RequestOptions; move sdk-client; add tests
Addresses self-review findings on the new connection module:
- RequestOptions.handlers/.meta and ServerRequestHandler removed: zero
callers (the scenarios that motivated them are deferred). They can be
reintroduced when something actually uses them.
- scenarios/server/client-helper.ts moved to connection/sdk-client.ts so
the connection module no longer depends on the scenarios module.
- connectStateless: handle CRLF SSE event separators; throw a useful
error for non-JSON/non-SSE responses instead of a JSON parse error.
- JSONRPCNotification consistently imported from spec-types/2025-11-25.
- New connection.test.ts (8 tests) covering connectFor selection,
_meta injection, error mapping, SSE LF/CRLF parsing, and the
server-request-on-stream rejection.
* fix: address bughunt findings (response.ok check; targetVersion naming)
- connectStateless: throw on non-2xx responses that lack a JSON-RPC
error envelope (e.g. gateway/framework error JSON), matching the
stateful path's behavior. Previously such a response would return
undefined as the result.
- json-schema-2020-12: rename negotiatedVersion to targetVersion and
reword the skip message and details field. The value is ctx.specVersion
(the run's --spec-version), not a negotiated version; the old wording
was misleading. Drop dead 'unknown' fallback (specVersion is required)
and the corresponding undefined test cases.
* fix(sse-multiple-streams): keep scenario in draft; version-aware request scaffolding
Concurrent POSTs each answered with JSON or their own SSE stream are
core transport behavior in the draft spec too, so removedIn was wrong
for this scenario. Only the request scaffolding is version-specific:
stateful runs keep the initialize/session-id setup, draft runs send
_meta + the MCP-Protocol-Version header on each request instead. The
two checks are unchanged.
* refactor: fold stateless-client helpers into connection/stateless
The two stateless request paths were ~90% duplicated:
- scenarios/server/stateless-client.ts (main #319): buildStandardHeaders,
withRequestMeta, readSseJsonRpcResponse, sendStatelessRequest -> raw
{status, body}; sends Mcp-Method/Mcp-Name (SEP-2243); has timeout
- connection/stateless.ts (this PR): connectStateless -> Connection.request
-> throws JsonRpcError; missing Mcp-Method/Mcp-Name; no timeout
connection/stateless.ts now exports both layers:
- low-level sendStatelessRequest() (and the building blocks) moved verbatim
from stateless-client.ts
- connectStateless() rebuilt as a thin wrapper over sendStatelessRequest():
classifies SSE events into the notification sink, surfaces server->client
requests on the response stream as a spec violation, throws JsonRpcError
on error responses
So connectStateless() now also picks up the SEP-2243 headers and the 10s
timeout that the previous impl lacked.
stateless-client.ts deleted; its 4 importers + test file repointed at
'../../connection'. Test file moved to connection/stateless.test.ts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(everything-server): make stateless resources reach the carry-forward dispatch
The merge of main left two competing handler sets for resources/read,
resources/list and resources/templates/list: the inline stateless handlers
(test://stateless-static-text + a catch-all -32602) and the carry-forward
dispatch to the McpServer-registered resources. The catch-all made the
carry-forward unreachable, so test://static-text, test://static-binary and
the templates errored under --spec-version draft.
Now resources/list and resources/templates/list dispatch to the McpServer and
merge with the inline entries (same pattern as tools/list and prompts/list),
and resources/read keeps the inline static-text response but falls through to
the carry-forward for unknown URIs (preserving SEP-2164 data.uri on errors).
Also wrap the tools/list and prompts/list dispatch in try/finally so
dispatch.close() always runs, matching tools/call.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(scenarios): send SEP-2243 Mcp-Method header on hand-rolled stateless requests
sse-multiple-streams and dns-rebinding bypass sendStatelessRequest() because
they need raw fetch/undici for low-level inspection, but their hand-built
draft-path headers omitted Mcp-Method, so a strictly-conformant draft server
would 400 them before the behaviour under test runs. Build the headers via
buildStandardHeaders() instead, layering the scenario-specific overrides
(Accept ordering, Host/Origin) on top.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(runner): infer draft spec version for draft-only scenarios
When --spec-version is omitted, the runner defaulted to LATEST_SPEC_VERSION,
so draft-only scenarios (caching, resources-not-found, ...) silently ran over
the stateful connection instead of the stateless draft transport they were
written for; all-scenarios.test.ts had the same problem via testContext().
Infer the spec version from scenario.source.introducedIn (as negative.test.ts
already does) when no explicit version is given.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(connection): only throw JsonRpcError for properly-shaped error objects
connectStateless() destructured response.body.error unconditionally, so a
proxy returning {"error": "upstream timeout"} produced a JsonRpcError with
undefined code/message and lost the HTTP status. Require an object with a
numeric code before treating it as a JSON-RPC error; otherwise surface the
HTTP status and raw body in the thrown Error. Adds a unit test for the
non-conformant 502 case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(connection): align connectStateful normalization with connectStateless
Two consistency nits from automated review:
- Collected notifications were missing the jsonrpc field (the SDK's Zod
parsing strips it), violating the JSONRPCNotification type contract.
- McpError messages kept the SDK's "MCP error <code>: " prefix, so
JsonRpcError.message differed between the stateful and stateless impls.
Also restore progressToken filtering in tools-call-with-progress, which was
lost when the scenario migrated from the SDK onprogress callback to
conn.notifications: the server must echo the request's token back, and
notifications for other tokens are now ignored.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(runner): skip inapplicable scenario/spec-version combos; add --force; plumb spec version to the wire
Running a draft-only scenario with an explicit dated --spec-version (e.g.
server-stateless --spec-version 2025-11-25) used to silently pass: the flag
only affected ctx.connect(), which hand-rolling scenarios never call, so
they sent draft requests regardless and the run looked green while testing
nothing the flag claimed.
Three changes:
- buildStandardHeaders / withRequestMeta / sendStatelessRequest /
connectStateless accept an optional specVersion (default: draft), and
hand-rolling scenarios pass ctx.specVersion through, so requests declare
the version the run was invoked with.
- The runner skips (exit 0) when an explicit --spec-version falls outside
the scenario's introducedIn/removedIn window, with a message instead of
a misleading green run.
- A new --force flag overrides the skip for deliberate mismatch probing;
combined with the plumbing, a forced server-stateless run at 2025-11-25
now genuinely sends that version (8/20 checks pass against the
everything-server instead of a vacuous 26/26).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(everything-server): serve resources and caching hints on the stateless draft path
The stateless (SEP-2575) request path now answers resources/list,
resources/templates/list and resources/read — including the SEP-2164
-32602 + data.uri error for unknown URIs — and includes the SEP-2549
ttlMs/cacheScope hints on its cacheable list results, matching what the
draft-spec scenarios exercise.
* feat: make the harness's own draft-spec traffic conformant
Fixes the class of issues where a strictly-conformant SDK rejects the
harness's requests for reasons unrelated to the behaviour under test
(#311, #312, #315).
- add a shared stateless draft request helper (draft-client.ts) that puts
the cross-cutting SEP-2243 headers (Mcp-Method, Mcp-Name,
MCP-Protocol-Version, Accept) and SEP-2575 _meta (protocolVersion,
clientInfo, clientCapabilities) on every request, with overrides for
negative cases and the SHOULD-level retry on
UnsupportedProtocolVersionError
- migrate the draft scenarios onto it: the SEP-2322 helpers and
server-stateless now send Mcp-Method/Mcp-Name (#312); the SEP-2243
server scenarios drop the legacy initialize handshake and carry _meta on
every raw request (#311); caching and sep-2164-resource-not-found are
stateless DRAFT-2026-v1 requests instead of an SDK 2025-11-25 session
(#315)
- add a draft self-conformance gate (draft-self-conformance.test.ts) that
points the harness's own draft drivers at the existing client-testing
judge scenarios (request-metadata, http-standard-headers) and requires
zero failures or warnings
- align the request-metadata mock's unsupported-version rejection with the
draft schema (-32004) and teach the everything-client to recognize it
- rewrite the SEP-2549/SEP-2164 negative example servers as stateless
draft servers so they keep exercising the violation, not the transport
* refactor: rename the stateless request helpers and harden them per review
- rename draft-client.ts to stateless-client.ts and name the helpers by
feature (sendStatelessRequest, buildStandardHeaders, buildRequestMeta,
withRequestMeta, CONFORMANCE_CLIENT_INFO); 'draft' now only appears in
DRAFT_PROTOCOL_VERSION, the one name that tracks the moving target
- retry only genuine UnsupportedProtocolVersionError rejections (-32004,
tolerating -32001/-32602) and only with recognized versions; record the
retry on the response
- parse text/event-stream responses incrementally and resolve on the
matching event instead of blocking until the timeout
- keep the MCP-Protocol-Version header and _meta protocolVersion in sync
when the version is overridden via meta overrides
- make server-stateless's sendRpc SSE-tolerant
- emit explicit setup failures in the SEP-2243 server scenarios when the
tools/list discovery fails instead of misleading per-case results
- latch the worst status per check in the request-metadata judge so an
early violation cannot be masked by a later conformant request
- rename the gate to harness-traffic-conformance.test.ts and derive its
driver list from the draft suite with documented per-judge exclusions
(35 pairings)
- everything-server: advertise resources in stateless server/discover and
use -32004 for unsupported protocol version
- add unit tests for the helper (retry guard, SSE handling, header/_meta
version sync, default headers)
* refactor: slim the stateless client helper for the initial merge
* feat: split draft-spec scenarios into their own suite
- add --suite draft to the server command, selecting scenarios whose
source.introducedIn is the draft protocol version
- exclude draft scenarios from the default active suite until the draft
spec is published as a dated release
- derive the client command's draft suite from source.introducedIn so the
non-auth draft scenarios (SEP-2243/2575/2322/2106) are included alongside
the auth ones
- keep everything-server self-test coverage for draft scenarios in
all-scenarios.test.ts (active + non-pending draft)
- document available suites in the README and add suite-membership tests
* fix: repair broken specReferences URLs
- SEP-986: SEP/SEP-986.md no longer exists in the spec repo; point at the
Tool Names section of the published 2025-11-25 tools page instead
- SEP-990: the ext-auth spec file is enterprise-managed-authorization.mdx,
not enterprise-oauth.mdx
Fixes#313
* chore: refresh SEP traceability manifest after #303 and #304
Regenerated from a full client+server suite run against
typescript-sdk@258f1a04a5d3 on current main:
- sep-2468: the two new client checks are emitted (tested) and the two
authorization-server rows move to excluded (#304)
- sep-2243: 12 rows flip to tested now that the custom-header check IDs
match their requirements (#303)
* chore: regenerate the manifest on top of SEP-2106 (#295)
Re-ran both suites after rebasing onto main so the manifest also picks
up the new sep-2106 block (1 tested, 4 excluded, 3 untracked check IDs
emitted by the server scenario with no yaml row yet).
* chore: add SEP-2106 requirement-traceability YAML (tools JSON Schema 2020-12)
* Drop structured-content text-fallback row from SEP-2106
That SHOULD predates SEP-2106 (pre-existing context line in tools.mdx); 2106
only relaxes structuredContent to allow non-object values (permissive). Point
spec_url at the SEP doc security section where the remaining obligations live.
* feat: SEP-2106 JSON Schema 2020-12 vocabulary preservation checks
Extend the json-schema-2020-12 server scenario (previously SEP-1613 only) with
three checks that verify the broader JSON Schema 2020-12 vocabulary SEP-2106
permits in a tool inputSchema survives tools/list rather than being stripped:
composition (allOf/anyOf), conditional (if/then/else), and reference ($anchor)
keywords.
- everything-server: add these keywords to json_schema_2020_12_tool's raw schema
- add a negative-test server (sep-2106-stripped-schema.ts) plus a negative test
asserting the checks emit FAILURE when the vocabulary is stripped
* feat: SEP-2106 client scenario for network $ref dereferencing
Add json-schema-ref-no-deref, a client-suite scenario that advertises a tool
whose inputSchema contains a $ref pointing at a canary URL hosted by the
scenario. The sep-2106-no-network-ref-deref check fails if the client fetches
the canary while processing the schema, covering the SEP-2106 requirement that
implementations MUST NOT automatically dereference network-URI $ref values.
- register the scenario in the client scenarios list
- everything-client: handle the scenario with the basic connect+listTools flow
- tests: compliant client passes, a deliberately dereferencing client is
caught, and a client that never lists tools cannot pass
* Address review findings in SEP-2106 scenarios
- json-schema-ref-deref: make getChecks() idempotent (fresh array per call,
matching the http-standard-headers convention) and guard the async /mcp
handler like the example servers do
- json-schema-2020-12: include the SEP-2106 keywords ($anchor, allOf/anyOf,
if/then/else) in the description's example inputSchema so a server that
copies it verbatim passes the new checks
- sep-2106-stripped-schema: default to port 3007 to match negative.test.ts and
avoid sharing a default with sep-2549-no-caching-hints
* Soft-gate SEP-2106 keyword checks on the negotiated protocol version
The json-schema-2020-12 scenario is introducedIn 2025-11-25 (SEP-1613) but the
three SEP-2106 keyword-preservation checks test draft-era behavior. Rather than
failing servers that only negotiated 2025-11-25, gate the failure on the wire
protocolVersion from the initialize handshake: preserved keywords are SUCCESS at
any version, stripped keywords are FAILURE only when the server negotiated the
draft version and SKIPPED otherwise.
The published SDK does not advertise the draft version yet, so the negative test
asserts SKIPPED for the stripped-schema server; the FAILURE branch is covered by
unit tests on sep2106KeywordCheckStatus.
Add two client scenarios for the SEP-2468 requirements that had no
conformance coverage:
- auth/metadata-issuer-mismatch: the AS metadata document declares an
issuer that differs from the issuer identifier used to construct the
well-known URL; the client must not use the metadata
(sep-2468-client-validate-metadata-issuer).
- auth/iss-normalized: the authorization redirect carries an iss value
that only matches the recorded issuer after URL normalization; the
client must compare with simple string comparison and reject
(sep-2468-client-no-normalization).
The two authorization-server-side rows (AS SHOULD include iss, AS MUST
advertise authorization_response_iss_parameter_supported) are excluded:
observing iss in an authorization response requires driving an
Authorization Code Grant against the AS under test, which the
authorization-server suite does not implement yet (#208).
The x-mcp-header / Mcp-Param / Base64 half of SEP-2243 was already tested,
but the scenarios emitted one check ID per test case while sep-2243.yaml
declares one per normative requirement, so the traceability join (exact ID
match) reported 14 of 20 requirements as untested. #287 collapsed the
standard-header IDs but left the custom-header ones per-case.
This finishes the rename and adds DECLARED_CHECK_IDS backfill (the
request-metadata.ts pattern) so every requirement-level ID is emitted on
every run — exercised, backfilled as FAILURE, or SKIPPED when the server
under test exposes no x-mcp-header tool. SEP-2243 traceability goes from
6/20 to 18/20 tested; the remaining 2 (server-not-expect-null,
server-reject-missing-required) have no test case yet.
Client scenarios:
- param-header-{name} -> client-mirrors-designated-params /
client-encode-values / client-base64-unsafe, picked by what the
parameter exercises
- no-mirror-unannotated -> client-mirrors-designated-params
- param-header-tool-call-gate -> client-supports-custom-headers, now also
emitted SUCCESS when the annotated tool is called with Mcp-Param headers
- the invalid-tool loop emits the specific x-mcp-header constraint each
tool violates (not-empty / charset / unique / primitive-only)
Server scenarios:
- accepts-valid-base64 -> server-decode-base64
- rejects-invalid-base64-{padding,chars} -> server-reject-invalid-param-chars
- literal-missing-base64-{prefix,suffix} -> server-validate-param-match
- rejects-missing-custom-header -> server-validate-param-match
- every custom-header rejection's -32001 check -> server-reject-param-mismatch
- standard-header reject cases pass the coarse ID through the catch path too
'All scopes required for the current operation' has no harness-observable
ground truth: the WWW-Authenticate challenge is the only place the server
declares what an operation requires, so there is nothing independent to
compare it against. Detecting the negative (incremental challenging) would
need server-side auth scenarios that do not exist yet and would exercise the
example app's scope configuration rather than SDK behavior, and the spec
explicitly permits dynamic per-request scope determination, so a second
challenge is not conclusively non-conformant.
The client-side half of SEP-2350 (scope union on re-authorization) remains
tested by auth/scope-step-up.
* chore: refresh SEP traceability manifest (typescript-sdk@main)
Regenerated from a client+server suite run against typescript-sdk@5fc42e9be115
following the recipe in .github/workflows/traceability.yml.
New entries since the last refresh (typescript-sdk@22595b96):
- SEP-2322 (MRTR, #188): 17 tested, 0 untested, 16 excluded, 3 untracked
- SEP-2549 (TTL for list results, #275): 7 tested, 0 untested, 13 excluded
- SEP-2260: 12 excluded rows, no checks
- SEP-2207: yaml rows added since the last refresh now appear
(1 tested, 1 untested: sep-2207-server-no-offline-access)
No previously-tested requirement regressed.
* Exclude sep-2207 server offline_access guidance until RS auth scenarios exist
sep-2207-server-no-offline-access was declared in the yaml but no scenario
emits it, so it surfaced as the only untested requirement in the refreshed
manifest. The check needs to probe the SDK server's Protected Resource
Metadata scopes_supported and WWW-Authenticate challenge scope, and the
server suite does not yet exercise the SDK server as an OAuth protected
resource at all.
Mark the requirement excluded with a pointer to #116 (server-side
authorization baseline) rather than leaving it as a permanently-untested
row; revisit when server-side authorization scenarios land.
* chore: additional server conformance tests for SEP-2575
* fix(sep-2575): use the spec's notifications filter object in subscriptions/listen
The scenario sent params.subscriptions: [{type: 'tools/list-changed'}], a
shape that does not exist in the SEP. SubscriptionsListenRequestParams
requires params.notifications: {toolsListChanged?, promptsListChanged?, ...}
(see schema.ts SubscriptionFilter and the canonical
SubscriptionsListenRequest example). A compliant server would either reject
the old request as invalid params or treat it as subscribing to nothing,
failing the ack/list-changed checks.
Also makes the everything-server's acknowledgment echo the agreed
notifications filter, as the SubscriptionsAcknowledgedNotification schema
requires.
* fix(sep-2575): match the spec's notifications/*/list_changed method names
The list-changed checks and the everything-server used
notifications/tools/list-changed (hyphen); the spec's method is
notifications/tools/list_changed (underscore) per ToolListChangedNotification
/ PromptListChangedNotification in schema.ts. With the hyphen form the
notification-filter check cannot catch a real leak, and the list-changed
checks fail a server that emits the correct name.
Also drops the f.params.toolsListChanged === true fallback match: that field
is the subscriptions/listen request filter, not a notification parameter, so
it never appears on a compliant server's notification.
* fix(sep-2575): keep collected frames when the listen stream read times out
listenToStream aborts the fetch after timeoutMs. If the abort fired while
blocked in reader.read(), the rejection propagated to the outer catch and the
helper returned [] — discarding every frame already received. A compliant
server holds a subscriptions/listen stream open indefinitely, so the timeout
is the *normal* way these reads end; previously every subscription check
would report a false 'no frames received' FAILURE against such a server.
The current everything-server only avoided this by res.end()ing immediately
after the ack.
* fix(sep-2575): open the listen stream before triggering list mutations
The notification-filter and list-changed checks fired the mutation trigger
*before* opening the subscriptions/listen stream. A compliant server only
delivers list-changed notifications to streams that are open at the time of
the change, so the SHOULD checks would fail any server that does not replay
past changes to new subscribers, and the filter check could never observe a
real leak.
listenToStream now takes an onFirstFrame hook; the three trigger-based
checks open the stream, wait for the acknowledgment, then fire the mutation
on a separate connection and keep reading.
The everything-server now keeps subscriptions/listen streams open, registers
them with their filters, and fans the list_changed notification out to open
streams when a trigger tool runs - instead of unconditionally emitting a
list-changed frame on every stream open (which itself violated the
notification-filter requirement in spirit: it announced a change when
nothing had changed).
* fix(sep-2575): check the subscription id on every listen-stream notification
server-tags-subscription-id only inspected the acknowledgment frame, so a
server that tags the ack but omits io.modelcontextprotocol/subscriptionId
from subsequent notifications passed. The check now triggers a tool-list
change once the stream is acknowledged and asserts every notification frame
on the stream carries the id in params._meta.
Also drops the f.body.method / f.params.method fallbacks when resolving a
frame's method - those shapes don't exist in JSON-RPC and only masked
malformed frames.
* fix(sep-2575): report WARNING, not FAILURE, for the SHOULD-level list-changed checks
The spec text behind server-sends-{tools,prompts}-list-changed-on-subscription
is a SHOULD ('servers that declared the listChanged capability SHOULD send a
notification...'). Severity follows the spec keyword: MUST -> FAILURE,
SHOULD -> WARNING. runCheck now accepts a warning flag and the two
list-changed checks use it on their failure paths.
* fix(sep-2575): skip stream checks when the diagnostic tool is not exposed
http-server-no-independent-requests-on-stream and
server-no-log-without-loglevel reported SUCCESS when the server rejected the
tools/call outright (e.g. test_streaming_elicitation / test_logging_tool do
not exist): a single error frame contains no independent request and no log
notification, so the check passed without exercising anything. The sibling
list-changed checks already SKIP when their trigger tool is missing; these
now do the same.
* chore(sep-2575): exclude the four unobservable server requirements from traceability
Marks the section-C rows of #296 as excluded rather than untested:
no-prior-context, no-connection-reuse-required, disconnect-is-cancel and
stops-on-cancel describe internal server state that a black-box harness
cannot observe on the wire. With these excluded and the part-B checks landed,
every testable server-side SEP-2575 requirement is covered; the remaining
untested rows are all client-side and blocked on a SEP-2575-aware reference
client.
* fix(sep-2575): emit the request-metadata checks even when the client never connects
The reference SDK conformance client has no handler registered for the
request-metadata scenario, so it exits without sending a request, the
scenario emits zero checks, the suite reports "0 passed, 0 failed", and the
traceability manifest records all seven client-side requirements as
untested.
Declare the scenario check IDs up front and backfill any that were never
emitted as FAILURE from getChecks(): the emitted ID set is now the same for
every client, and a client that never connects fails loudly. A check that
is legitimately not applicable opts out by being emitted as SKIPPED
explicitly (version-header-matches-meta now does this when either version
field is absent). The positive-path test pins the declared list against the
reference client in both directions.
Also refresh the traceability manifest from a run against
typescript-sdk@22595b96: the seven client rows flip to tested and the
manifest catches up with the server-side scenarios from #297.
---------
Co-authored-by: Yuan Teoh <yuanteoh@google.com>
* chore: additional server conformance tests for SEP-2575
* fix(sep-2575): use the spec's notifications filter object in subscriptions/listen
The scenario sent params.subscriptions: [{type: 'tools/list-changed'}], a
shape that does not exist in the SEP. SubscriptionsListenRequestParams
requires params.notifications: {toolsListChanged?, promptsListChanged?, ...}
(see schema.ts SubscriptionFilter and the canonical
SubscriptionsListenRequest example). A compliant server would either reject
the old request as invalid params or treat it as subscribing to nothing,
failing the ack/list-changed checks.
Also makes the everything-server's acknowledgment echo the agreed
notifications filter, as the SubscriptionsAcknowledgedNotification schema
requires.
* fix(sep-2575): match the spec's notifications/*/list_changed method names
The list-changed checks and the everything-server used
notifications/tools/list-changed (hyphen); the spec's method is
notifications/tools/list_changed (underscore) per ToolListChangedNotification
/ PromptListChangedNotification in schema.ts. With the hyphen form the
notification-filter check cannot catch a real leak, and the list-changed
checks fail a server that emits the correct name.
Also drops the f.params.toolsListChanged === true fallback match: that field
is the subscriptions/listen request filter, not a notification parameter, so
it never appears on a compliant server's notification.
* fix(sep-2575): keep collected frames when the listen stream read times out
listenToStream aborts the fetch after timeoutMs. If the abort fired while
blocked in reader.read(), the rejection propagated to the outer catch and the
helper returned [] — discarding every frame already received. A compliant
server holds a subscriptions/listen stream open indefinitely, so the timeout
is the *normal* way these reads end; previously every subscription check
would report a false 'no frames received' FAILURE against such a server.
The current everything-server only avoided this by res.end()ing immediately
after the ack.
* fix(sep-2575): open the listen stream before triggering list mutations
The notification-filter and list-changed checks fired the mutation trigger
*before* opening the subscriptions/listen stream. A compliant server only
delivers list-changed notifications to streams that are open at the time of
the change, so the SHOULD checks would fail any server that does not replay
past changes to new subscribers, and the filter check could never observe a
real leak.
listenToStream now takes an onFirstFrame hook; the three trigger-based
checks open the stream, wait for the acknowledgment, then fire the mutation
on a separate connection and keep reading.
The everything-server now keeps subscriptions/listen streams open, registers
them with their filters, and fans the list_changed notification out to open
streams when a trigger tool runs - instead of unconditionally emitting a
list-changed frame on every stream open (which itself violated the
notification-filter requirement in spirit: it announced a change when
nothing had changed).
* fix(sep-2575): check the subscription id on every listen-stream notification
server-tags-subscription-id only inspected the acknowledgment frame, so a
server that tags the ack but omits io.modelcontextprotocol/subscriptionId
from subsequent notifications passed. The check now triggers a tool-list
change once the stream is acknowledged and asserts every notification frame
on the stream carries the id in params._meta.
Also drops the f.body.method / f.params.method fallbacks when resolving a
frame's method - those shapes don't exist in JSON-RPC and only masked
malformed frames.
* fix(sep-2575): report WARNING, not FAILURE, for the SHOULD-level list-changed checks
The spec text behind server-sends-{tools,prompts}-list-changed-on-subscription
is a SHOULD ('servers that declared the listChanged capability SHOULD send a
notification...'). Severity follows the spec keyword: MUST -> FAILURE,
SHOULD -> WARNING. runCheck now accepts a warning flag and the two
list-changed checks use it on their failure paths.
* fix(sep-2575): skip stream checks when the diagnostic tool is not exposed
http-server-no-independent-requests-on-stream and
server-no-log-without-loglevel reported SUCCESS when the server rejected the
tools/call outright (e.g. test_streaming_elicitation / test_logging_tool do
not exist): a single error frame contains no independent request and no log
notification, so the check passed without exercising anything. The sibling
list-changed checks already SKIP when their trigger tool is missing; these
now do the same.
* chore(sep-2575): exclude the four unobservable server requirements from traceability
Marks the section-C rows of #296 as excluded rather than untested:
no-prior-context, no-connection-reuse-required, disconnect-is-cancel and
stops-on-cancel describe internal server state that a black-box harness
cannot observe on the wire. With these excluded and the part-B checks landed,
every testable server-side SEP-2575 requirement is covered; the remaining
untested rows are all client-side and blocked on a SEP-2575-aware reference
client.
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
Robin Otto
Felix Weinberger
Paul Carleton
Marcelo Trylesinski
Aditya Chilka
Peter Alexander
Takashi Norimatsu
Sri Panyam
Bill Prin
Caitie McCaffrey
Yuan Teoh