# Conformance scenarios not yet passing against the Python SDK on main. # CI exits 0 if only these fail, exits 1 on unexpected failures or stale entries. # # Baseline established against @modelcontextprotocol/conformance pinned in # .github/workflows/conformance.yml (CONFORMANCE_VERSION = 0.2.0-alpha.3). # New conformance releases are adopted by deliberately bumping that pin and # reconciling this file in the same change. # # Entries are grouped by SEP. As each SEP lands in the SDK the corresponding # scenarios start passing and MUST be removed from this list (the runner fails # on stale entries), so the baseline burns down per milestone. client: # --- Draft-spec scenarios (in `--suite draft`, also part of `--suite all`) --- # SEP-2575 (request metadata / _meta envelope): client does not populate the # _meta envelope or the MCP-Protocol-Version header semantics yet. - request-metadata # SEP-2322 (multi-round-trip requests): client does not echo requestState / # handle IncompleteResult yet. - sep-2322-client-request-state # SEP-2243 (HTTP standardization): no fixture handler / client header support yet. - http-custom-headers - http-invalid-tool-headers # SEP-2106 (JSON Schema $ref handling): client still dereferences network $refs. - json-schema-ref-no-deref # SEP-2468 (authorization response iss parameter): not implemented in the client. - auth/iss-supported - auth/iss-not-advertised - auth/iss-supported-missing - auth/iss-wrong-issuer - auth/iss-unexpected - auth/iss-normalized - auth/metadata-issuer-mismatch # SEP-2352 (authorization server migration): client does not re-register when # PRM authorization_servers changes. - auth/authorization-server-migration # SEP-837 (application_type during DCR): the check only fires on draft-version # runs; this draft scenario is the one place the client still hits it. - auth/offline-access-not-supported # --- Pre-existing scenarios that fail on checks added after conformance 0.1.15 --- # SEP-2350 (scope step-up): WARNING-only; the expected-failures evaluator # counts WARNINGs as failures. - auth/scope-step-up # SEP-990 (enterprise-managed authorization extension): no fixture handler / # client support for the token-exchange + JWT bearer flow. - auth/enterprise-managed-authorization server: # --- Draft-spec scenarios (in `--suite draft`; the `active` suite is green) --- # SEP-2575 (stateless HTTP / _meta envelope): server has no stateless mode, # _meta-derived capabilities, error-code mappings, or server/discover yet. - server-stateless # SEP-2322 (multi-round-trip requests / IncompleteResult): not implemented; # most scenarios currently fail early with "Missing session ID" because # mcp-everything-server only runs in stateful mode. - input-required-result-basic-elicitation - input-required-result-basic-sampling - input-required-result-basic-list-roots - input-required-result-request-state - input-required-result-multiple-input-requests - input-required-result-multi-round - input-required-result-non-tool-request - input-required-result-result-type - input-required-result-tampered-state - input-required-result-capability-check # SEP-2549 (caching): no ttlMs/cacheScope support; scenario also hits the # stateful-mode "Missing session ID" error. - caching # SEP-2243 (HTTP header standardization): -32001 HeaderMismatch handling and # case-insensitive/whitespace-trimmed header validation not implemented. - http-header-validation - http-custom-header-server-validation # WARNING-only entries: these scenarios emit no FAILURE checks, only SHOULD-level # WARNINGs, but the expected-failures evaluator counts WARNINGs as failures. # SEP-2164: server returns -32600 (not -32602) and omits error.data.uri. - sep-2164-resource-not-found # SEP-2322 SHOULD-level behaviours (re-request missing inputResponses, ignore # unrecognized inputResponses keys). - input-required-result-missing-input-response - input-required-result-ignore-extra-params # Intentionally NOT baselined (2 of 19 draft scenarios): the SEP-2322 # negative-case scenarios input-required-result-unsupported-methods and # input-required-result-validate-input pass today only because the stateful # server's -32600 "Missing session ID" satisfies their assertions. They will # start failing for real once stateless mode lands; add them then.