php-sdk/tests/Conformance/conformance-baseline-2026-07-28.yml
Christopher Hertel 3d596ca75b
[Server][Tests] Serve the modern (SEP-2575) lifecycle (#444)
* [Server][Tests] Serve the modern (SEP-2575) lifecycle

The revision removed the initialize handshake: every request carries its own
_meta - protocol version, client capabilities, client info - and answers
server/discover for what initialize used to return. There is no session to
keep, so there is nothing for the handshake-era dispatcher to do.

Builder::buildStateless() produces StatelessProtocol, and
StatelessHttpTransport serves it over one POST-only endpoint - GET and
DELETE are 405, and no Mcp-Session-Id exists to hand out.

Results go through a per-era wire codec rather than the handlers, so
resultType and the caching hints are stamped for 2026-07-28 and left off the
revisions that do not define them.

Covered by the modern conformance suite (make conformance-draft-server)
against its own baseline; unit coverage arrives with the fixes stacked on
this.

* Apply suggestions from code review

Co-authored-by: Christopher Hertel <mail@christopher-hertel.de>

* Fix critical stateless-lifecycle bugs from review

- Don't leak internal exception messages to clients on uncaught
  \Throwable; use a generic message like the handshake-era protocol
  and every built-in handler already do.
- Default a missing/invalid request id to null instead of "", and
  stop answering notifications and malformed messages as if they
  were "method not found" 404s.
- Read the POST body in a loop instead of a single read() call,
  which PSR-7 allows to return short.
2026-08-18 22:08:54 +02:00

71 lines
4 KiB
YAML

# Expected conformance failures for spec revision 2026-07-28.
#
# Separate from conformance-baseline.yml because the two revisions exercise
# different lifecycles against different endpoints: the dated baseline covers
# the handshake era served at /, this one covers the modern (SEP-2575) era
# served at /stateless.
#
# Entries are per-check (scenario:check-id) rather than whole scenarios, so a
# scenario that is mostly conformant keeps reporting the parts that work and a
# new regression inside it still fails the run.
#
# Run with:
# make conformance-draft-server
#
# Absent by design, because they pass in full: server-stateless (the SEP-2575
# lifecycle), caching (SEP-2549 hints), and every wire-schema-valid check bar
# one noted below.
server:
# --- SEP-2243: standard HTTP headers ---------------------------------------
# Mcp-Method / Mcp-Name validation and the X-MCP-* parameter rules are not
# implemented. Self-contained and the largest remaining block; a good next
# chunk after MRTR.
- http-header-validation:sep-2243-server-reject-invalid-headers
- http-custom-header-server-validation:sep-2243-server-no-xmcp-tool
- http-custom-header-server-validation:sep-2243-server-decode-base64
- http-custom-header-server-validation:sep-2243-server-validate-param-match
- http-custom-header-server-validation:sep-2243-server-reject-invalid-param-chars
- http-custom-header-server-validation:sep-2243-server-reject-param-mismatch
# --- SEP-2322: the MRTR retry half -----------------------------------------
# InputRequiredResult exists and tools/call can answer with one — that is what
# makes server-stateless green. What is missing is the second round trip: the
# SDK does not read inputResponses, verify requestState integrity, or resume a
# call from it. Until it does, every scenario that retries fails at the retry.
- input-required-result-basic-elicitation:sep-2322-elicitation-incomplete
- input-required-result-basic-list-roots:sep-2322-list-roots-incomplete
- input-required-result-basic-sampling:sep-2322-sampling-incomplete
- input-required-result-capability-check:sep-2322-respect-client-capabilities
- input-required-result-multi-round:sep-2322-multi-round-r1
- input-required-result-multiple-input-requests:sep-2322-multiple-inputs-incomplete
- input-required-result-non-tool-request:sep-2322-non-tool-incomplete
- input-required-result-request-state:sep-2322-request-state-incomplete
- input-required-result-result-type:sep-2322-result-type-included
- input-required-result-tampered-state:sep-2322-reject-tampered-state
# --- SHOULD-level warnings -------------------------------------------------
# These report WARNING rather than FAILURE (the severity follows the spec
# keyword), so the run summary counts them as passed — but the baseline
# evaluator still wants them declared, and declaring them is right: each is a
# SHOULD the SDK does not yet honour.
- http-header-validation:sep-2243-server-reject-error-code
- input-required-result-ignore-extra-params:sep-2322-ignore-unexpected-params
- input-required-result-missing-input-response:sep-2322-missing-response-rerequests
- sep-2164-resource-not-found:sep-2164-data-uri
- sep-2164-resource-not-found:sep-2164-error-code
# --- Pre-existing, not lifecycle-specific ----------------------------------
# Each of these fails identically on the handshake endpoint at /, so none is
# caused by (or fixable in) the modern lifecycle. Listed so the draft run is
# honest, not to excuse them — resources-templates-read in particular is the
# single wire-schema failure left, a resource-template URI the fixture emits
# in a shape the schema rejects on both revisions.
- dns-rebinding-protection:localhost-host-rebinding-rejected
- json-schema-2020-12:json-schema-2020-12-tool-found
- prompts-get-embedded-resource:prompts-get-embedded-resource
- prompts-get-with-image:prompts-get-with-image
- resources-templates-read:wire-schema-valid
- tools-call-with-progress:tools-call-with-progress
client: []