modelcontextprotocol/conformance
Watch
1
0
Fork
You've already forked conformance
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: add client-side JSON Schema 2020-12 preservation scenario (#335)

Browse source 
* feat: add client-side JSON Schema 2020-12 preservation scenario

The mock server, using the version-aware `MockServer` abstraction from #321,
first advertises a focal tool with a detailed inputSchema and also offers a
permissive `json_schema_echo` tool. The client under test retrieves the
inputSchema via `tools/list`, then sends the same inputSchema back to the server
using the echo tool, completing the round-trip.

The scenario diffs from the echoed schema against the canonical fixture. It
emits eight per-keyword conformance checks. SEP-2106 vocabulary checks are
soft-gated using the existing sep2106KeywordCheckStatus helper. They only flip
to FAILURE during DRAFT-2026-v1 runs.

Signed-off-by: Satoshi Ito <satoshi.ito.tf@hitachi.com>

* Drive draft-target preservation tests through the stateless lifecycle

The stateful test helper passed createServerStateful bare, which no longer
compiles now that it takes a second specVersion argument, and forcing the
stateful mock at the draft version trips wire-schema validation (draft
results require the SEP-2549 resultType/ttlMs/cacheScope members). Use
testScenarioContext(DRAFT_PROTOCOL_VERSION), which selects the stateless
mock server, and drive it with sendStatelessRequest, matching the pattern
in json-schema-ref-deref.test.ts.

* Reword client keyword-preservation rows as issue-derived expectations

SEP-2106 §1 contains no normative preservation sentence, so the six new
sep-2106.yaml rows should not attribute a spec MUST to it. Reword the
remaining rows as expectations derived from issue #101. Also drop the
three rows whose check IDs are not sep-prefixed: the traceability
manifest only collects IDs matching /^sep-\d+-/, so those rows could
never be marked tested.

* Replace stale DRAFT-2026-v1 references with the 2026-07-28 draft name

The draft protocol identifier was renamed in #331. Interpolate
DRAFT_PROTOCOL_VERSION into the SKIPPED errorMessage suffix (matching
the server-side scenario) and update the header comment.

---------

Signed-off-by: Satoshi Ito <satoshi.ito.tf@hitachi.com>
Co-authored-by: Paul Carleton <paulc@anthropic.com>
This commit is contained in:
Satoshi Ito 2026-08-01 02:24:05 +09:00 • committed by GitHub
commit 81eb1c3eda
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 812 additions and 38 deletions
Download patch file Download diff file Expand all files Collapse all files

63
examples/clients/typescript/everything-client.ts
View file

@ -228,6 +228,69 @@ async function runListToolsOnlyClient(serverUrl: string): Promise<void> {
registerScenario('json-schema-ref-no-deref', runListToolsOnlyClient);
// ============================================================================
// json-schema-2020-12-preservation (SEP-1613, SEP-2106, Issue #101)
//
// Scenario contract:
// 1. tools/list — observe `json_schema_2020_12_tool` and its inputSchema
// 2. tools/call json_schema_echo with `{ schema: <observed inputSchema> }`
// The scenario diffs the echoed schema against its fixture to detect
// client-side keyword stripping; this handler just round-trips the schema
// verbatim, which is the compliant behavior.
// ============================================================================
const FOCAL_TOOL_NAME = 'json_schema_2020_12_tool';
const ECHO_TOOL_NAME = 'json_schema_echo';
async function runJsonSchema2020_12PreservationClient(
serverUrl: string
): Promise<void> {
if (USE_STATELESS_LIFECYCLE) {
logger.debug(
'Stateless lifecycle: listing tools and echoing observed schema'
);
const list = await statelessRequest(serverUrl, 'tools/list');
const focal = list?.tools?.find(
(t: { name: string }) => t.name === FOCAL_TOOL_NAME
);
if (!focal) {
throw new Error(`Focal tool '${FOCAL_TOOL_NAME}' not advertised`);
}
await statelessRequest(serverUrl, 'tools/call', {
name: ECHO_TOOL_NAME,
arguments: { schema: focal.inputSchema }
});
logger.debug('Successfully echoed observed inputSchema');
return;
}
const client = new Client(
{ name: 'test-client', version: '1.0.0' },
{ capabilities: {} }
);
const transport = new StreamableHTTPClientTransport(new URL(serverUrl));
await client.connect(transport);
try {
const list = await client.listTools();
const focal = list.tools.find((t) => t.name === FOCAL_TOOL_NAME);
if (!focal) {
throw new Error(`Focal tool '${FOCAL_TOOL_NAME}' not advertised`);
}
await client.callTool({
name: ECHO_TOOL_NAME,
arguments: { schema: focal.inputSchema as Record<string, unknown> }
});
logger.debug('Successfully echoed observed inputSchema');
} finally {
await transport.close();
}
}
registerScenario(
'json-schema-2020-12-preservation',
runJsonSchema2020_12PreservationClient
);
// ============================================================================
// request-metadata scenario (SEP-2575)
// ============================================================================