Pin each conformance leg to a spec-revision wire (#3304)

.github/actions/conformance/expected-failures.2025-11-25.yml

@@ -0,0 +1,26 @@
+# Expected failures at the 2025-11-25 wire, read by the `--suite all
+# --spec-version 2025-11-25` server and client legs: every scenario the pinned
+# harness marks applicable at 2025-11-25, run over the stateful handshake.
+#
+# That selection is a superset of the frozen 2025-11-25 requirement set (`npx
+# $CONFORMANCE_PKG list --requirements 2025-11-25`), so an entry here for a
+# scenario in that set means `conformance tier-check` scores the SDK below 100%
+# for 2025-11-25 even though CI is green; link the tracking issue next to any
+# such entry.
+#
+# Kept separate from expected-failures.yml even though the bare `--suite all`
+# legs currently run these same scenarios at 2025-11-25 too: the harness's
+# default wire is due to move (modelcontextprotocol/conformance#426) and this
+# file must keep meaning 2025-11-25. Until then a failure here also needs an
+# entry there. The file is passed even while empty: run-client.sh keys its solo
+# re-verification off the harness's baseline output.
+#
+# Baseline established against the harness pinned via CONFORMANCE_PKG in
+# .github/workflows/conformance.yml. New conformance releases are adopted by
+# deliberately bumping that pin and reconciling all three expected-failures
+# files in the same change. Entries MUST be removed as they start passing (the
+# runner fails on stale entries).
+
+client: []
+
+server: []

.github/actions/conformance/expected-failures.2026-07-28.yml

@@ -1,19 +1,23 @@
-# Expected failures for the carried-forward 2026-07-28 legs
-# (`--suite all --spec-version 2026-07-28` for both server and client).
+# Expected failures at the 2026-07-28 wire, read by the `--suite all
+# --spec-version 2026-07-28` server and client legs: every scenario the pinned
+# harness marks applicable at 2026-07-28, run stateless with per-request _meta.
+#
+# That selection is a superset of the frozen 2026-07-28 requirement set (`npx
+# $CONFORMANCE_PKG list --requirements 2026-07-28`), so an entry here for a
+# scenario in that set means `conformance tier-check` scores the SDK below 100%
+# for 2026-07-28 even though CI is green; link the tracking issue next to any
+# such entry.
 #
 # This baseline is separate from expected-failures.yml because entries are
-# keyed by scenario name only: a scenario that passes at its default version
-# in the 2025 legs but fails when forced to 2026-07-28 (or vice versa) cannot
-# be expressed in a shared file (the passing leg would flag the entry as
-# stale). Like expected-failures.yml, this single file covers both
-# directions: the client 2026 leg reads the `client:` section and the server
-# 2026 leg reads the `server:` section. Both burn down independently of the
-# 2025 legs.
+# keyed by scenario name only: a scenario that passes at the harness default
+# wire in the bare `--suite all` legs but fails when forced to 2026-07-28 (or
+# vice versa) cannot be expressed in a shared file. Where the bare legs already
+# default a scenario to 2026-07-28, a failure here also needs an entry there.
 #
 # Baseline established against the harness pinned via CONFORMANCE_PKG in
 # .github/workflows/conformance.yml. New conformance releases are adopted by
-# deliberately bumping that pin and reconciling both this file and
-# expected-failures.yml in the same change.
+# deliberately bumping that pin and reconciling all three expected-failures
+# files in the same change.
 #
 # Entries are grouped by what unblocks them. As each gap closes the
 # corresponding scenarios start passing and MUST be removed from this list

.github/actions/conformance/expected-failures.yml

@@ -1,10 +1,16 @@
-# Conformance scenarios not yet passing against the Python SDK on main.
-# CI exits 0 if only these fail, exits 1 on unexpected failures or stale entries.
+# Conformance scenarios not yet passing against the Python SDK on main, read by
+# the bare `--suite all` server and client legs (every scenario shipped with the
+# pinned harness, each at the harness's default wire for it). CI exits 0 if only
+# these fail, exits 1 on unexpected failures or stale entries.
+#
+# The wire-pinned legs read expected-failures.2025-11-25.yml and
+# expected-failures.2026-07-28.yml instead; see the header of
+# .github/workflows/conformance.yml for the leg layout.
 #
 # Baseline established against the harness pinned via CONFORMANCE_PKG in
 # .github/workflows/conformance.yml. New conformance releases are adopted by
-# deliberately bumping that pin and reconciling both this file and
-# expected-failures.2026-07-28.yml in the same change.
+# deliberately bumping that pin and reconciling all three expected-failures
+# files in the same change.
 #
 # Entries are grouped by SEP. As each SEP lands in the SDK the corresponding
 # scenarios start passing and MUST be removed from this list (the runner fails
@@ -30,10 +36,9 @@ client:
 
 server:
   # SEP-2663 (io.modelcontextprotocol/tasks): the SDK does not implement the
-  # tasks extension yet. These extension-tagged scenarios are selected only by
-  # the bare `--suite all` leg — extension scenarios never match a
-  # --spec-version filter and the active/draft suites exclude them — so these
-  # entries are inert for the other legs that read this file.
+  # tasks extension yet. The bare `--suite all` leg is the only one that
+  # reaches these extension-tagged scenarios (they never match a
+  # --spec-version filter), so these entries live here only.
   #
   # `tasks-status-notifications` is intentionally NOT listed: the harness
   # skips it unconditionally (pending its rewrite against subscriptions/

.github/actions/conformance/run-client.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 # Run a client conformance suite, re-verifying unexpected failures solo.
-# Concurrent suite runs on a 2-vCPU runner can push scenarios with real-time
+# Concurrent suite runs on a hosted runner can push scenarios with real-time
 # waits past tolerance; solo, a real failure fails again while a contention
 # artifact passes. Failures that only reproduce under concurrency are excused.
 set -uo pipefail
@@ -101,4 +101,8 @@ if [ -n "$output_dir" ]; then
     printf '%s\n' "${scenarios[@]}" > "$output_dir/FLAKE_RESCUED"
 fi
 echo "All ${#scenarios[@]} unexpected failure(s) passed when re-run solo; the suite failures were parallel-run contention."
+# Surface the rescue on the run summary; the log line above and the
+# FLAKE_RESCUED marker in the uploaded results are otherwise easy to miss on a
+# green run.
+echo "::warning title=Conformance flake rescued::${scenarios[*]} failed under suite concurrency and passed solo${output_dir:+ (see ${output_dir}-solo and FLAKE_RESCUED in the uploaded results)}."
 exit 0