modelcontextprotocol/conformance
Watch
1
0
Fork
You've already forked conformance
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: refresh SEP traceability manifest after #295, #303, and #304 (#305)

Browse source 
* chore: refresh SEP traceability manifest after #303 and #304

Regenerated from a full client+server suite run against
typescript-sdk@258f1a04a5d3 on current main:

- sep-2468: the two new client checks are emitted (tested) and the two
  authorization-server rows move to excluded (#304)
- sep-2243: 12 rows flip to tested now that the custom-header check IDs
  match their requirements (#303)

* chore: regenerate the manifest on top of SEP-2106 (#295)

Re-ran both suites after rebasing onto main so the manifest also picks
up the new sep-2106 block (1 tested, 4 excluded, 3 untracked check IDs
emitted by the server scenario with no yaml row yet).
This commit is contained in:
Paul Carleton 2026-05-23 00:28:09 +01:00 • committed by GitHub
commit 276c6805a8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 73 additions and 32 deletions
Download patch file Download diff file Expand all files Collapse all files

105
src/seps/traceability.json
View file

@ -1,7 +1,7 @@
{
"schemaVersion": 1,
"docs": "https://github.com/modelcontextprotocol/conformance/blob/main/AGENTS.md#traceability-manifest",
"source": "typescript-sdk@5fc42e9be115",
"source": "typescript-sdk@258f1a04a5d3",
"seps": {
"837": {
"yaml": "src/seps/sep-837.yaml",
@ -41,6 +41,48 @@
"unkeyed": 0
}
},
"2106": {
"yaml": "src/seps/sep-2106.yaml",
"specUrl": "https://modelcontextprotocol.io/seps/2106-json-schema-2020-12#security-implications",
"requirements": [
{
"check": "sep-2106-no-network-ref-deref",
"status": "tested",
"text": "Implementations MUST NOT automatically dereference `$ref` values that resolve to a network URI (i.e. anything that is not a same-document JSON Pointer such as `#/$defs/Foo` or an internal `$anchor`); an opt-in mode that fetches non-local `$ref`s MUST be disabled by default / Schemas that fail to validate due to an unresolved external `$ref` SHOULD be rejected rather than silently treated as permissive"
}
],
"excluded": [
{
"text": "SDK maintainers SHOULD: Document the migration in SDK release notes; Where ergonomic, provide typed helpers (e.g. generics over a tool's `outputSchema`) so consumers do not need to write narrowing guards by hand",
"reason": "Migration/deprecation guidance for SDK maintainers; about SDK source code, not protocol-observable wire behavior"
},
{
"text": "JSON Schema validation already handles type checking, value constraints, and required field validation, and implementations MUST continue to validate all inputs and outputs against declared schemas",
"reason": "Restates pre-existing schema-validation behavior and is too broad to attribute to a specific observable SEP-2106 check; input/output validation overlaps existing tool scenarios"
},
{
"text": "An opt-in mode that fetches non-local `$ref`s SHOULD enforce an allowlist of hosts (or at minimum reject loopback, link-local, and private network addresses), apply timeouts and size limits, and log dereferenced URIs",
"reason": "Applies only when the non-default opt-in network-$ref fetch mode is enabled; not observable in default conformance runs"
},
{
"text": "Implementations SHOULD apply reasonable bounds — for example, a maximum schema depth, a cap on the total number of subschemas, or a per-validation time budget — to prevent a malicious tool definition from acting as a CPU DoS vector against the validator",
"reason": "Internal validator resource limits (max schema depth, subschema cap, per-validation time budget); a defensive measure not observable on the wire"
}
],
"unkeyed": [],
"untracked": [
"sep-2106-anchor-keyword-preserved",
"sep-2106-composition-keywords-preserved",
"sep-2106-conditional-keywords-preserved"
],
"summary": {
"tested": 1,
"untested": 0,
"excluded": 4,
"untracked": 3,
"unkeyed": 0
}
},
"2164": {
"yaml": "src/seps/sep-2164.yaml",
"specUrl": "https://modelcontextprotocol.io/specification/draft/server/resources#error-handling",
@ -138,35 +180,35 @@
},
{
"check": "sep-2243-client-supports-custom-headers",
"status": "untested",
"status": "tested",
"text": "MCP clients MUST support this feature [custom headers via x-mcp-header]."
},
{
"check": "sep-2243-client-mirrors-designated-params",
"status": "untested",
"status": "tested",
"text": "When a client invokes a tool whose definition includes such designations, conforming clients MUST mirror the designated parameter values into HTTP headers as described below."
},
{
"check": "sep-2243-x-mcp-header-not-empty",
"status": "untested",
"status": "tested",
"text": "The x-mcp-header value MUST NOT be empty.",
"url": "https://modelcontextprotocol.io/specification/draft/server/tools#custom-headers"
},
{
"check": "sep-2243-x-mcp-header-charset",
"status": "untested",
"status": "tested",
"text": "The x-mcp-header value MUST contain only ASCII characters (excluding space and `:`).",
"url": "https://modelcontextprotocol.io/specification/draft/server/tools#custom-headers"
},
{
"check": "sep-2243-x-mcp-header-unique",
"status": "untested",
"status": "tested",
"text": "The x-mcp-header value MUST be case-insensitively unique within a single tool definition.",
"url": "https://modelcontextprotocol.io/specification/draft/server/tools#custom-headers"
},
{
"check": "sep-2243-x-mcp-header-primitive-only",
"status": "untested",
"status": "tested",
"text": "x-mcp-header MUST only be applied to parameters with primitive types (number, string, or boolean).",
"url": "https://modelcontextprotocol.io/specification/draft/server/tools#custom-headers"
},
@ -178,17 +220,17 @@
},
{
"check": "sep-2243-client-encode-values",
"status": "untested",
"status": "tested",
"text": "Clients MUST encode parameter values before including them in HTTP headers: number values MUST be converted to their decimal string representation; boolean values MUST be converted to the lowercase strings \"true\" or \"false\"."
},
{
"check": "sep-2243-client-base64-unsafe",
"status": "untested",
"status": "tested",
"text": "When a value cannot be safely represented as plain ASCII (e.g., contains non-ASCII characters, control characters, or leading/trailing whitespace), clients MUST use Base64 encoding of the UTF-8 representation, wrapped as =?base64?{encoded}?=."
},
{
"check": "sep-2243-server-decode-base64",
"status": "untested",
"status": "tested",
"text": "Servers and intermediaries that need to inspect these values MUST decode them accordingly."
},
{
@ -208,17 +250,17 @@
},
{
"check": "sep-2243-server-reject-invalid-param-chars",
"status": "untested",
"status": "tested",
"text": "Servers MUST reject requests with a recognized Mcp-Param-{Name} header that contain invalid characters."
},
{
"check": "sep-2243-server-validate-param-match",
"status": "untested",
"status": "tested",
"text": "Any server that processes the message body MUST validate that encoded header values, after decoding if Base64-encoded, match the corresponding parameter values in the body."
},
{
"check": "sep-2243-server-reject-param-mismatch",
"status": "untested",
"status": "tested",
"text": "Servers MUST reject requests with a 400 Bad Request HTTP status and JSON-RPC error code -32001 if any validation fails."
}
],
@ -243,15 +285,14 @@
"unkeyed": [],
"untracked": [
"sep-2243-invalid-tool-tools-list-gate",
"sep-2243-param-header-tool-call-gate",
"sep-2243-server-accepts-whitespace-header-value",
"sep-2243-server-no-xmcp-tool"
],
"summary": {
"tested": 6,
"untested": 14,
"tested": 18,
"untested": 2,
"excluded": 4,
"untracked": 4,
"untracked": 3,
"unkeyed": 0
}
},
@ -573,20 +614,10 @@
"requirements": [
{
"check": "sep-2468-client-validate-metadata-issuer",
"status": "untested",
"status": "tested",
"text": "After retrieving a metadata document, MCP clients MUST validate it as required by RFC8414 Section 3.3 or OpenID Connect Discovery Section 4.3: the issuer value in the document MUST be identical to the issuer identifier used to construct the well-known URL. If they differ, the client MUST NOT use the metadata.",
"url": "https://modelcontextprotocol.io/specification/draft/basic/authorization#authorization-server-metadata-discovery"
},
{
"check": "sep-2468-as-include-iss",
"status": "untested",
"text": "MCP authorization servers SHOULD include the iss parameter in authorization responses, including error responses, as defined in RFC9207 Section 2."
},
{
"check": "sep-2468-as-advertise-iss-supported",
"status": "untested",
"text": "Authorization servers that include the iss parameter MUST advertise this by setting authorization_response_iss_parameter_supported to true in their metadata (RFC9207 Section 2.3)."
},
{
"check": "sep-2468-client-compare-iss-supported",
"status": "tested",
@ -609,11 +640,21 @@
},
{
"check": "sep-2468-client-no-normalization",
"status": "untested",
"status": "tested",
"text": "After decoding the iss value from the application/x-www-form-urlencoded response per RFC 9207 Section 2.4, clients MUST NOT apply scheme or host case folding, default-port elision, trailing-slash, or percent-encoding normalization (RFC 3986 Sections 6.2.2-6.2.3) before comparison."
}
],
"excluded": [
{
"text": "MCP authorization servers SHOULD include the iss parameter in authorization responses, including error responses, as defined in RFC9207 Section 2.",
"reason": "Targets the authorization server under test; observing iss in an authorization response requires driving an Authorization Code Grant against the AS, which the authorization-server suite does not implement yet (it only probes the metadata endpoint).",
"issue": "https://github.com/modelcontextprotocol/conformance/issues/208"
},
{
"text": "Authorization servers that include the iss parameter MUST advertise this by setting authorization_response_iss_parameter_supported to true in their metadata (RFC9207 Section 2.3).",
"reason": "Conditional on the AS actually including iss in an authorization response, which requires driving an Authorization Code Grant against the AS under test; the authorization-server suite does not implement that yet.",
"issue": "https://github.com/modelcontextprotocol/conformance/issues/208"
},
{
"text": "This validation applies equally to error responses - on mismatch the client MUST NOT act on or display error, error_description, or error_uri.",
"reason": "display is UI-facing; act-on has no protocol-observable signal beyond the existing reject-on-mismatch checks"
@ -622,9 +663,9 @@
"unkeyed": [],
"untracked": [],
"summary": {
"tested": 4,
"untested": 4,
"excluded": 1,
"tested": 6,
"untested": 0,
"excluded": 3,
"untracked": 0,
"unkeyed": 0
}