Make the draft-spec scenarios internally consistent (fixes #311, #312, #315) (#319)

* feat(everything-server): serve resources and caching hints on the stateless draft path

The stateless (SEP-2575) request path now answers resources/list,
resources/templates/list and resources/read — including the SEP-2164
-32602 + data.uri error for unknown URIs — and includes the SEP-2549
ttlMs/cacheScope hints on its cacheable list results, matching what the
draft-spec scenarios exercise.

* feat: make the harness's own draft-spec traffic conformant

Fixes the class of issues where a strictly-conformant SDK rejects the
harness's requests for reasons unrelated to the behaviour under test
(#311, #312, #315).

- add a shared stateless draft request helper (draft-client.ts) that puts
  the cross-cutting SEP-2243 headers (Mcp-Method, Mcp-Name,
  MCP-Protocol-Version, Accept) and SEP-2575 _meta (protocolVersion,
  clientInfo, clientCapabilities) on every request, with overrides for
  negative cases and the SHOULD-level retry on
  UnsupportedProtocolVersionError
- migrate the draft scenarios onto it: the SEP-2322 helpers and
  server-stateless now send Mcp-Method/Mcp-Name (#312); the SEP-2243
  server scenarios drop the legacy initialize handshake and carry _meta on
  every raw request (#311); caching and sep-2164-resource-not-found are
  stateless DRAFT-2026-v1 requests instead of an SDK 2025-11-25 session
  (#315)
- add a draft self-conformance gate (draft-self-conformance.test.ts) that
  points the harness's own draft drivers at the existing client-testing
  judge scenarios (request-metadata, http-standard-headers) and requires
  zero failures or warnings
- align the request-metadata mock's unsupported-version rejection with the
  draft schema (-32004) and teach the everything-client to recognize it
- rewrite the SEP-2549/SEP-2164 negative example servers as stateless
  draft servers so they keep exercising the violation, not the transport

* refactor: rename the stateless request helpers and harden them per review

- rename draft-client.ts to stateless-client.ts and name the helpers by
  feature (sendStatelessRequest, buildStandardHeaders, buildRequestMeta,
  withRequestMeta, CONFORMANCE_CLIENT_INFO); 'draft' now only appears in
  DRAFT_PROTOCOL_VERSION, the one name that tracks the moving target
- retry only genuine UnsupportedProtocolVersionError rejections (-32004,
  tolerating -32001/-32602) and only with recognized versions; record the
  retry on the response
- parse text/event-stream responses incrementally and resolve on the
  matching event instead of blocking until the timeout
- keep the MCP-Protocol-Version header and _meta protocolVersion in sync
  when the version is overridden via meta overrides
- make server-stateless's sendRpc SSE-tolerant
- emit explicit setup failures in the SEP-2243 server scenarios when the
  tools/list discovery fails instead of misleading per-case results
- latch the worst status per check in the request-metadata judge so an
  early violation cannot be masked by a later conformant request
- rename the gate to harness-traffic-conformance.test.ts and derive its
  driver list from the draft suite with documented per-judge exclusions
  (35 pairings)
- everything-server: advertise resources in stateless server/discover and
  use -32004 for unsupported protocol version
- add unit tests for the helper (retry guard, SSE handling, header/_meta
  version sync, default headers)

* refactor: slim the stateless client helper for the initial merge
This commit is contained in:
Paul Carleton 2026-05-26 21:10:23 +01:00 • committed by GitHub
commit 076b417b23
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 894 additions and 596 deletions

View file

@ -104,9 +104,10 @@ async function incompatibleVersionClient(serverUrl: string) {
if (response.status === 400) {
const body = await response.json();
if (body.error?.code === -32001) {
if (body.error?.code === -32004 || body.error?.code === -32001) {
return body; // Abort cleanly
}
return body;
}
return response.json();
}

View file

@ -3,9 +3,23 @@ import {
Scenario,
ScenarioUrls,
ConformanceCheck,
CheckStatus,
DRAFT_PROTOCOL_VERSION
} from '../../types';
/**
* Severity ranking used to latch per-id check results: a single
* non-conformant request is a violation even if later requests are
* conformant, so a later better status must never overwrite a worse one.
*/
const STATUS_SEVERITY: Record<CheckStatus, number> = {
FAILURE: 3,
WARNING: 2,
SUCCESS: 1,
INFO: 1,
SKIPPED: 0
};
/**
* Every check ID this scenario can emit. Declared-but-unemitted checks are
* backfilled as FAILURE by getChecks(), so the emitted ID set is the same for
@ -92,10 +106,17 @@ export class RequestMetadataScenario implements Scenario {
private addOrUpdateCheck(check: ConformanceCheck): void {
const index = this.checks.findIndex((c) => c.id === check.id);
if (index !== -1) {
this.checks[index] = check;
} else {
if (index === -1) {
this.checks.push(check);
return;
}
// Keep the worst status observed for this id (FAILURE > WARNING > SUCCESS
// > SKIPPED): an equal-or-worse result replaces the stored check (so its
// details stay fresh), but a better result must not erase a violation
// recorded from an earlier request.
const existing = this.checks[index];
if (STATUS_SEVERITY[check.status] >= STATUS_SEVERITY[existing.status]) {
this.checks[index] = check;
}
}
@ -256,7 +277,8 @@ export class RequestMetadataScenario implements Scenario {
jsonrpc: '2.0',
id: request.id ?? null,
error: {
code: -32001,
// UnsupportedProtocolVersionError per the draft schema.
code: -32004,
message: 'Unsupported protocol version',
data: {
supported: [DRAFT_PROTOCOL_VERSION]