Pin each conformance leg to a spec-revision wire (#3304)

This commit is contained in:
Max 2026-08-14 13:39:06 +01:00 • committed by GitHub
commit 14aa889992
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -1,19 +1,23 @@
# Expected failures for the carried-forward 2026-07-28 legs
# (`--suite all --spec-version 2026-07-28` for both server and client).
# Expected failures at the 2026-07-28 wire, read by the `--suite all
# --spec-version 2026-07-28` server and client legs: every scenario the pinned
# harness marks applicable at 2026-07-28, run stateless with per-request _meta.
#
# That selection is a superset of the frozen 2026-07-28 requirement set (`npx
# $CONFORMANCE_PKG list --requirements 2026-07-28`), so an entry here for a
# scenario in that set means `conformance tier-check` scores the SDK below 100%
# for 2026-07-28 even though CI is green; link the tracking issue next to any
# such entry.
#
# This baseline is separate from expected-failures.yml because entries are
# keyed by scenario name only: a scenario that passes at its default version
# in the 2025 legs but fails when forced to 2026-07-28 (or vice versa) cannot
# be expressed in a shared file (the passing leg would flag the entry as
# stale). Like expected-failures.yml, this single file covers both
# directions: the client 2026 leg reads the `client:` section and the server
# 2026 leg reads the `server:` section. Both burn down independently of the
# 2025 legs.
# keyed by scenario name only: a scenario that passes at the harness default
# wire in the bare `--suite all` legs but fails when forced to 2026-07-28 (or
# vice versa) cannot be expressed in a shared file. Where the bare legs already
# default a scenario to 2026-07-28, a failure here also needs an entry there.
#
# Baseline established against the harness pinned via CONFORMANCE_PKG in
# .github/workflows/conformance.yml. New conformance releases are adopted by
# deliberately bumping that pin and reconciling both this file and
# expected-failures.yml in the same change.
# deliberately bumping that pin and reconciling all three expected-failures
# files in the same change.
#
# Entries are grouped by what unblocks them. As each gap closes the
# corresponding scenarios start passing and MUST be removed from this list