feat(sep-2640): skills server conformance against the Accepted SEP, enumeration + manifest + directory (#330)

* chore: add SEP-2640 requirement-traceability YAML (Skills Extension)

* style: apply yaml formatter pass

* chore(sep-2640): record spec source provenance

* chore(sep-2640): re-extract against SEP HEAD 556154c (drops mcp-resource-template)

Re-extract sep-2640.yaml against current SEP-2640 HEAD. The SEP removed
the `mcp-resource-template` index entry type in two commits on
2026-06-04 (fd50cc91 "Remove mcp-resource-template entries from skill
index", 556154c0 "Remove remaining resource template mentions from SDK
and rationale sections"), after this extraction was first captured on
2026-06-03 at b77fdfe8. Provenance comment bumped accordingly.

Resulting changes:

- sep-2640-index-entry-type-enum: drop `"mcp-resource-template"` from
  the enum.
- sep-2640-index-name-required: drop the "is Required for ... Omitted
  for mcp-resource-template" conditional language; the column is now
  unconditional Required=Yes at SEP HEAD.
- sep-2640-index-digest-required: same simplification; also align
  "hexadecimal" -> "hex" with the HEAD table text.
- sep-2640-template-resource-template-registered: removed. The SEP no
  longer defines a template entry type, so the SHOULD on registering an
  MCP resource template is gone.
- Excluded "Hosts SHOULD surface template entries..." excerpt: removed.
  The sentence no longer exists in the SEP.

Sweep verified: zero "template" mentions in the SEP at HEAD. Security
Implications section sentences referenced by the remaining excluded
entries are intact. No other check rows touched.

* feat(sep-2640): ResourcesDirectoryReadScenario for resources/directory/read

Adds the conformance scenario for the SEP-2640 directoryRead surface that
landed in spec commit 2e04c48d (2026-06-09). Per AGENTS.md "fewer scenarios,
more checks", a single ResourcesDirectoryReadScenario emits 6
ConformanceChecks, one per new sep-2640.yaml requirement row.

Class named for the wire method (resources/directory/read), matching the
existing ResourcesListScenario / ResourcesReadTextScenario / etc. family in
src/scenarios/server/resources.ts. The runner-facing name field stays as
'sep-2640-skills' (umbrella) so mcpkit's conformance/Makefile entry
--scenario sep-2640-skills keeps working without a cross-repo race.

src/seps/sep-2640.yaml
- File-level provenance held at 556154c (the existing PR 330 baseline).
- 6 new check rows appended for the directoryRead additions, each
  carrying a verbatim, grep-F-searchable excerpt from the SEP at
  2e04c48da90224000e750ffd54a3611f2824fbc0:
    - sep-2640-capability-directory-read-flag
    - sep-2640-directory-read-method-registered
    - sep-2640-directory-read-subdir-mimetype
    - sep-2640-directory-read-result-resources-shape
    - sep-2640-directory-read-invalid-params
    - sep-2640-directory-read-pagination
- forward_reference header comment notes the asymmetry: PR 97 schema
  rewrite (360123d0, 2026-06-08) made 3 existing rows stale and drifted
  ~11 others' wording; full re-extraction at SEP HEAD is mcpkit#780's
  lifecycle. Provenance deliberately holds at 556154c until that lands.

src/types.ts
- Adds io.modelcontextprotocol/skills to EXTENSION_IDS so the scenario's
  source: { extensionId: ... } tag type-checks.

src/scenarios/server/directory.ts (new)
- Capability discovery via wire-observable signal: -32601 method-not-found
  is the only definitive "server did not declare directoryRead" signal.
- Fixture assumption: server exposes skill://acme/billing/refunds/templates
  with at least one subdirectory child (mcpkit examples/skills layout).
  No skill:// resources at all -> every check emits SKIPPED so the
  scenario stays green against upstream's everything-server fixture.
- 6 checks (1:1 with the YAML rows above):
  1. directoryRead declared (derived from method registration)
  2. method registered (happy-path call succeeds)
  3. result.resources shape matches resources/list
  4. subdir child carries mimeType: "inode/directory"
  5. non-directory URI returns -32602
  6. nextCursor round-trips (single-page is conformant)

src/scenarios/index.ts
- Imports + registers ResourcesDirectoryReadScenario in
  allClientScenariosList, matching the registration name 'sep-2640-skills'
  that mcpkit's conformance/Makefile already passes to --scenario.

Verified end-to-end against mcpkit examples/skills:
  cd ~/newstack/mcpkit/main && \
    MCPCONFORMANCE_SKILLS_PATH=~/newstack/mcpkit/conf-skills \
    make -C conformance testconf-skills
  -> 6/6 SUCCESS, 0 failed, 0 warnings.

Out of scope:
- PR 97 schema rewrite YAML refresh (separate ticket, mcpkit#780 lifecycle).
- Negative-capability fixture (server without directoryRead) - mcpkit
  examples/skills has no flag for it; ext/skills/client_directory_test.go
  already covers the SDK pre-call guard.

* feat(sep-2640): expand skills conformance — index + manifest scenarios, drift fixes

Builds on the directory-read scenario with the server-observable skills surface
and brings the whole set up to the current framework conventions.

New scenarios (src/scenarios/server/skills/):
- SkillsIndexScenario (sep-2640-skills-index): skill://index.json is exposed and
  readable; every skills[].type is in the SEP enum; every entry has a non-empty
  name; a present digest matches sha256:{64hex}; entry URLs use skill://.
- SkillsManifestScenario (sep-2640-skills-manifest): a skill-md SKILL.md is
  markdown-typed; frontmatter name/description surface on the resource; the
  final skill-path segment equals the frontmatter name; skill _meta keys use the
  reverse-domain prefix.

directory.ts (moved into skills/): capability is now read from server/discover
instead of inferred from -32601; an undeclared optional capability SKIPs rather
than FAILs; the untestable.ts helpers back the not-exercisable paths; all
mcpkit-specific fixture URIs are gone — the directory to exercise is discovered
dynamically, so the scenario is brand-neutral.

Capability gating accepts directoryRead at either extensions[id].directoryRead
(SEP-2640's inline example) or extensions[id].config.directoryRead (the SEP-2133
negotiation envelope SEP-2640 defers to); the two SEPs are inconsistent on
nesting, so the check accepts either rather than privileging one reading.

All three scenarios verified green against a live SEP-2640 server (6/6, 6/6, 7/7);
typecheck, eslint, prettier, and the 504-test vitest suite pass. sep-2640.yaml
gains a header note mapping each scenario to the rows it now backs.

* feat(sep-2640): re-extract against the 2026-08-21 rewrite

The prior extraction held at SEP commit 556154c (2026-06-05) by an explicit
in-file decision to wait for the index-schema refactor. The 08-21 core-maintainer
rewrite made that pin actively wrong rather than merely behind.

Removed as no longer normative: the skill://index.json well-known resource
(absent from the current text), the entry type enum, the no-nested-skills rule
(now reversed - skills MAY nest), and the five archive rows (archives are
deferred to an appendix). Reworded the two index-field rows, the empty-listing
row and the verify-digest row.

Added the new surface: skills/list and skills/get with their pagination,
atomicity and caching attributes; the {uri, frontmatter, resources} entry
schema; resources completeness, the "dynamic" sentinel and per-file size; the
512-entry and 16 MiB limits; and the host obligations around lazy retrieval,
content-bound approval and frontmatter re-verification.

index.ts is replaced by enumeration.ts, which drives both new methods. manifest.ts
and directory.ts swap their skill://index.json discovery fallback for skills/list.

The capability helper no longer accepts a config envelope. SEP-2133 is Final and
defines the extensions field as a map of identifiers to settings objects, with no
envelope and no slot for id/specVersion/stability; SEP-2640's capability block
matches. The earlier leniency was based on a misreading.

77 rows: 65 checks (39 wire-tested by the three server scenarios) and 12 excluded
as host-internal.

* feat(sep-2640): close the extraction gaps found by a normative-sentence audit

The first pass extracted the Specification sections but under-covered Security
Implications, which carries 21 of the SEP's 97 RFC-2119 sentences. Sweeping
every normative sentence in the source surfaced rows that were missing entirely:

- origin tagging at the point content enters model context (the L179 host MUST)
- allowed-tools MUST be ignored for MCP-origin skills
- reads bound to the originating server, so a skill from A cannot drive a read
  against B, and any cross-origin read gated behind per-call approval
- servers identified by a host-assigned label, never serverInfo.name
- the three disk-cache rules: write-isolation or re-hash, exclusion from
  filesystem-skill discovery paths, and removal when a server is removed
- a resources/read of a SKILL.md by another route is not a load
- a nested SKILL.md read as supporting content is ordinary markdown
- the registry-assembly and virtual-mount no-prefetch restatements
- the SDK affordances guidance, which is the row mcpkit itself is measured by

Five rows previously filed as excluded are now declared instead. They are real
requirements that a host-side harness could check; calling them unobservable was
a limitation of this scenario set, not of the requirement.

Adds one wire check, names-should-be-unique, since a within-listing collision is
observable and tells an operator the listing needs disambiguating.

96 rows: 89 checks (40 wire-tested) and 7 excluded.

* style: apply prettier to the SEP-2640 scenarios and yaml

Quote-style normalisation in the yaml (escaped single quotes become
double-quoted scalars) and one return-type wrap in helpers.ts. The parsed
yaml is byte-identical in content; verified by comparing the parse before
and after.

* docs(sep-2640): note why resultType is not declared in this yaml

It appears in all three result examples but is a base-protocol field on the
common Result interface, not a SEP-2640 requirement; its caching-hint
obligations are already owned by sep-2549.yaml.

* fix(sep-2640): follow nextCursor on resources/directory/read

The directory scenario issued one resources/directory/read and every check
below it inspected that single page: the result-shape check, the
subdirectory-mimeType check, and the target selection for the -32602 probe.
SEP-2640 says directory-read pagination mirrors resources/list, so a
conformant server MAY split a directory across pages, and reading only the
first made "no subdirectory here" indistinguishable from "the subdirectory
is on page two".

Reported by Sam Bloomberg against the go-sdk implementation, which exposes
DirectoryOptions.PageSize. Rebuilding that server with PageSize 1 fails
sep-2640-directory-read-subdir-mimetype; with this change it passes 7/7,
and both servers still pass at their default page size.

Only findable against a second implementation. mcpkit hardcoded its page
size to zero with no override, so it never emitted a cursor and the defect
was invisible to every run against it.

listAllResources already did this for resources/list. directoryReadAll
applies the same shape to directory reads, with a page bound and a
repeated-cursor guard.

* feat(sep-2640): client scenario for the no-prefetch MUST NOT

The traceability file filed retrieval policy under "host obligations a
harness cannot observe", alongside 48 other rows. That was wrong for this
one: "hosts MUST NOT retrieve a skill's files ahead of need" reduces to
whether a request arrived, which makes it among the most wire-observable
rules in the SEP.

Adds a Scenario (harness stands up the server, client is the SUT) that
serves a listing naming a SKILL.md and a supporting file, then fails if
either is read. The client's contract, keyed on MCP_CONFORMANCE_SCENARIO,
is to connect, enumerate, and stop.

SKIPs rather than passes when the client never calls skills/list, since
without a listing there is no window in which prefetching is possible and
a pass would prove nothing.

Verified both directions against mcpkit's client: SUCCESS as written, and
FAILURE with a deliberate ReadFromEntry added to the driver. A check that
has never been shown to fail is the same trap as the directory-pagination
false negative this suite shipped for months.

* feat(sep-2640): client scenarios for the four read-time verification MUSTs

Adds digest, size, frontmatter and unlisted-URI variants alongside the
no-prefetch scenario. Each serves an internally honest listing, tampers with
exactly one thing on the wire, and watches whether the client notices.

Rejection is detected by absence, following auth/resource-mismatch: the
client's contract is "load the skill, then read its supporting file", and a
client that verifies aborts on the load and never reaches the supporting
file. Reaching it is the violation. The unlisted variant is proven by the
read never appearing on the wire at all, since a conformant client refuses
locally.

The frontmatter variant recomputes digest and size for the tampered body, so
the frontmatter comparison is the only thing left that can fail. Without
that the digest check would mask it and the scenario would pass for the
wrong reason.

allowClientError is set on all four: a verifying client exits non-zero
because it correctly aborted, and that is the pass.

Verified both directions against mcpkit's client. All four SUCCESS as
written; all four FAILURE against a driver rewritten to read through
ReadSkillURI, which skips verification entirely.

* fix(sep-2640): gate the cache-attributes check on the negotiated version

The check graded ttlMs and cacheScope without looking at the protocol
version, which made it wrong in both directions. Below 2026-07-28 those
fields are not defined by the negotiated schema at all, so a server that
omits them is correct and the WARNING was a false positive. From 2026-07-28
they are required, so a missing one is a failure rather than a warning.

Now SKIPPED with a version reason below 2026-07-28, and SUCCESS or FAILURE
above it.

Reported by Sam Bloomberg against the Go SDK, which was correct on both
wires while this check mis-graded it on both. Second finding from the same
reviewer against this branch, and the second that only a second
implementation could surface: mcpkit emits the attributes on every version
it supports, so it could never have produced the false positive.

Verified three ways: mcpkit at 2026-07-28 SUCCESS, the Go SDK at 2025-11-25
SKIPPED where it previously warned, and a deliberately stripped mcpkit build
at 2026-07-28 FAILURE.

* docs(sep-2640): how to run the skills scenarios against any implementation

Three implementations have now been cross-checked and every one needed a
different invocation, so the knowledge was living in chat rather than the
repo. Records the two flags and why both are needed, how to read totals that
include wire-schema-valid, and the per-SDK setup.

The two traps are worth writing down because both have already cost someone
time. Extension scenarios do not match --spec-version alone, so --force is
always needed. And --spec-version selects the wire lifecycle rather than
filtering, so omitting it against a pre-2026-07-28 server produces a -32022
that reads like a missing method.

Also records that the go-sdk serves 2026-07-28 only when its streamable
transport is Stateless, which is correct per SEP-2575 and which I briefly
mistook for a defect.

* fix(sep-2640): advertise skills on server/discover, not only initialize

Both client scenarios declared the skills extension and `resources` in their
`initialize` reply, but `BaseHttpScenario` intercepts `server/discover` before
`handlePost` runs. On the 2026-07-28 lifecycle a discover-first client
therefore saw the base class default of `{tools: {}}` and had nothing to gate
a `skills/list` call on.

The scenarios still passed, because `server/discover` is optional for clients
and neither client we ran used it. That is the failure mode where a check goes
green because its condition never arose, so both lifecycles are now asserted
in capabilities.test.ts rather than left to the next implementation to find.

Overriding `discoverCapabilities()` covers both paths at once, since
`sendInitialize` already defaults to it.

* fix(sep-2640): give the capability and naming checks something to fail on

`sep-2640-capability-empty-object` reported a hardcoded SUCCESS, so it could
not fail and was checking nothing. It now asserts the observable half of "an
empty object indicates support for the extension with no optional features":
the declared value must be an object at all.

That also closes a false SKIP. `skillsCapability` folded a malformed
declaration into `undefined`, which downstream read as "extension not
declared" and skipped the whole suite, so a server declaring the extension as
`true` produced a clean run. `declaredSkillsCapability` separates absent from
malformed; the inline/envelope check now SKIPs on a malformed value, since
there are no settings to place either way. The base `DiscoverResult` schema
rejects a non-object too, so both halves are asserted.

`SKILL_NAME_PATTERN` accepted consecutive hyphens. The Agent Skills
specification that SEP-2640 defers to for naming lists four rules and we
encoded three, so `pdf--processing` passed. Its own valid and invalid examples
are now the test cases.

Negative controls recorded for all three: with the check back to a hardcoded
SUCCESS the four malformed-declaration cases fail, and with the previous regex
the consecutive-hyphen case fails.

* docs(sep-2640): name skills/list in the discovery prose, not index.json

The 2026-08-21 revision replaced the `skill://index.json` well-known resource
with `skills/list`, and the code follows it — `discoverDirectory` and the
manifest fallback both call `skillsListAll`. Only the prose lagged, including
three strings a user actually sees: two untestable-check reasons and the
directory scenario's description.

The remaining mentions in helpers.ts, enumeration.ts and sep-2640.yaml are
deliberate history, describing what was replaced.

* fix(sep-2640): grade unlisted reads against the entry, not one fixture URI

The `unlisted` client scenario asserted `reads.includes(UNLISTED_URI)`, so it
only caught a client that probed the exact path this fixture happens to serve.
Any client choosing a different unlisted path passed without being tested, and
the check would have reported SUCCESS for a client that reads unlisted files
freely.

The requirement is "hosts MUST resolve reads of the skill's files only to URIs
listed in that entry's resources", so the permitted set now comes from the
entry the client was handed and any read outside it is the violation. That is
the sentence itself rather than a proxy for it, and it lets a SUT derive its
own probe instead of hardcoding ours.

Negative control: a client reading a synthesized `mcpkit-unlisted-probe-0.md`
is now a FAILURE, where the previous check reported SUCCESS.

* fix(sep-2640): require the manifest read, and retire the unlisted scenario

Two changes to the client-side verification scenarios, one of which fixes a
hole the other one shares.

**The manifest read is now a prerequisite.** Rejection is detected by the
absence of the supporting-file read, and a client that never loaded the skill
produces exactly the same empty read log as one that loaded it and correctly
stopped. Both reported SUCCESS, so `listCalled` was the only gate and a client
implementing `skills/list` and nothing else passed every variant here. It
scored 5/5 on this suite while doing nothing.

Absence now means something only after the client has read the tampered
`SKILL.md`. A missing prerequisite reports untestable, which per #248 is a
FAILURE rather than a SKIP: SKIPPED is excluded from pass/fail counts and exit
codes, so it would read as green and hide the gap.

Negative control: a driver that lists and exits goes from 1/1 to 0/1 on all
three variants.

**`-verify-unlisted` is retired**, on Peter Alexander's call. Nothing was
tampered in it, so the only thing under test was whether the driver asked for
a URI it should not have, which grades the driver rather than the client. It
is also the rule an inspection tool answers differently from a host, since
refusing a read requires having a load to refuse, which MCP Inspector 2.6.0
reported as "surfaced" rather than passed.

The requirement is not deleted. It moves to an `excluded:` row alongside the
acting-on window that scopes it, and the reason records that it is excluded by
construction rather than in principle: a `SKILL.md` body linking to a file
absent from `resources` would make the harness adversarial again, since a host
that follows the link fetches an unlisted file on its own initiative. The
scenario name and check id are named there so a future reintroduction lines up
with historical reports rather than reading as a new requirement.

Check rows go from 89 to 88.

* feat(sep-2640): check skills/get cache attributes and the resources capability

Both requirements landed when the stable spec page merged ext-skills#139
today, and both are checkable on the wire.

**`sep-2640-skills-get-cache-attributes`.** `GetSkillResult extends
CacheableResult`, so `ttlMs` and `cacheScope` are REQUIRED as they are on
`resources/read`. Held out of this PR until the page merged rather than
grading against text that was still open: SEP-2640 says of these fields that
"whether the result should also carry the base protocol's caching attributes
... is left open", so this is a behaviour change and a server conformant to
the SEP text alone omits both. It is a reading not everyone shares. The MCP
Inspector's SEP-2640 writeup argues the opposite, that the SEP forecloses it
and inventing the requirement would report a reasonable server as broken.

Gated the same way as the skills/list check: below 2026-07-28 `CacheableResult`
is undefined by the negotiated schema, so omission is correct and the check
SKIPs with a version reason rather than failing a conformant legacy server.

**`sep-2640-capability-requires-resources`.** A server declaring the extension
MUST also declare the base `resources` capability, since skill files are served
through `resources/read`. The page states it as a consequence of the base
Resources specification rather than a free-standing new rule, and the
declaration is on the wire either way.

Verified against mcpkit: 32/32 on 2026-07-28 and 30/30 on 2025-11-25, where the
get-cache check SKIPs below the floor as designed. Negative control recorded: a
build with the attributes stripped from `handleSkillsGet` reports 31/32 with
`sep-2640-skills-get-cache-attributes` FAILURE.

Check rows go from 88 to 90.

* fix(sep-2640): compare frontmatter by content, not key order

entry-frontmatter-identical compared each top-level key with JSON.stringify,
which is key-order sensitive for nested objects. A server whose encoder sorts
map keys (Go's encoding/json does) failed this MUST for any SKILL.md whose
metadata keys are not in alphabetical order, although the content is
identical.

Compare structurally instead. A real difference still fails; both cases are
pinned in negative.test.ts.

* fix(sep-2640): make the client-scenario mocks answer like a conformant server

The skills/list and resources/read replies carried only resultType, so on the
2026-07-28 wire they lacked the ttlMs/cacheScope a ReadResourceResult
requires, and skills/list lacked the attributes the enumeration scenario
requires of servers.

initialize always answered 2026-07-28, while the runner hands these extension
scenarios' clients 2025-11-25 by default. A client that checks the negotiated
version disconnected before listing, and every verify variant reported
SKIPPED whether or not the client verified anything.

The mocks now stamp the caching members and answer initialize at the
requested version when the suite can speak it. The runner default is left
alone because it also applies to the auth extension scenarios. Pinned in
draft-result-fields.test.ts next to the other hand-rolled mocks.

* docs(sep-2640): run from the published package; --force only with --spec-version

The run notes pointed at a fork branch that goes stale on merge, and said
--force is always needed. Extension scenarios only SKIP when an explicit
--spec-version is passed; without it they run on the draft wire.

* docs(sep-2640): drop RUNNING_SEP2640.md

Per-run notes and SDK result tables belong in the PR description, not a
root-level doc that goes stale on merge.

---------

Co-authored-by: Paul Carleton <paulc@anthropic.com>

.claude/skills/mcp-sdk-tier-audit/README.md

@@ -9,6 +9,15 @@ Two components work together:
 
 ## Quick Start: CLI
 
+For an SDK listed in `src/sdk-runner/known-sdks.ts`, tier-check manages the whole conformance side itself — clone/build, then for each shipped revision the server invocation that SDK declares for that revision:
+
+```bash
+gh auth login
+npx @modelcontextprotocol/conformance tier-check --sdk go-sdk
+```
+
+The per-SDK sections below are for the manual path: driving a server you started yourself.
+
 The CLI is a subcommand of the [MCP Conformance](https://github.com/modelcontextprotocol/conformance) tool.
 
 ```bash
@@ -95,56 +104,33 @@ The skill lives in `.claude/skills/` in this repo, so if you open [Claude Code](
 3. Run the skill:
 
 ```
-/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url> [client-cmd]
+/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url> [client-cmd] [--requirements <revision>]
 ```
 
 Pass the client command as the third argument to include client conformance testing. If omitted, client conformance is skipped and noted as a gap in the report.
 
-**TypeScript SDK example:**
+**Pass `--requirements` with every revision the SDK claims**, comma-separated. Each revision's scenarios run at that revision's own wire version, and all of them must pass for Tier 1: the dated revisions through `2025-11-25` use the stateful initialize handshake while `2026-07-28` is stateless, so a scenario belonging to both has to work on both and one run does not cover the other. It also means a scenario added to the suite after a revision shipped cannot fail an SDK that had no opportunity to adopt it. Without the flag, scoring uses the suite as it stands today, which is not a tier claim. See [Conformance Requirements](../../../README.md#conformance-requirements), and run `conformance list --requirements 2025-11-25,2026-07-28` to see both sets.
 
-```bash
-# Terminal 1: start the everything server (build first: npm run build)
-cd ~/src/mcp/typescript-sdk && npm run test:conformance:server:run
+**Known SDKs — no server to start, no commands to look up:**
 
-# Terminal 2: run the audit (from the conformance repo)
-/mcp-sdk-tier-audit ~/src/mcp/typescript-sdk http://localhost:3000/mcp "npx tsx ~/src/mcp/typescript-sdk/test/conformance/src/everythingClient.ts"
+```
+/mcp-sdk-tier-audit <sdk checkout> --requirements 2025-11-25,2026-07-28
 ```
 
-**Python SDK example:**
+The skill drives `tier-check --sdk-path`, which builds the SDK and starts the
+server invocation its config declares **per revision** — the per-SDK commands
+live in [`src/sdk-runner/known-sdks.ts`](../../../src/sdk-runner/known-sdks.ts),
+not here. That matters because a hand-run example cannot be correct for every
+SDK: go-sdk needs a different server process per revision (`-stateless` defaults
+to true, so a bare invocation mis-measures 2025-11-25 — [#446](https://github.com/modelcontextprotocol/conformance/issues/446)),
+and csharp-sdk serves the two revisions at different endpoints. Prose copies of
+those invocations drift; the config is tested.
 
-```bash
-# Terminal 1: install and start the everything server
-cd ~/src/mcp/python-sdk && uv sync --frozen --all-extras --package mcp-everything-server
-uv run mcp-everything-server --port 3001
+**An SDK not in `known-sdks.ts`:** start its everything server yourself and pass
+the URL and client command explicitly — see [Other SDKs](#running-conformance-tests)
+below. The one endpoint you give must serve every claimed revision at its own
+wire, or the numbers will be wrong for the revisions it does not speak.
 
-# Terminal 2: run the audit (from the conformance repo)
-/mcp-sdk-tier-audit ~/src/mcp/python-sdk http://localhost:3001/mcp "uv run python ~/src/mcp/python-sdk/.github/actions/conformance/client.py"
-```
-
-**Go SDK example:**
-
-```bash
-# Terminal 1: build and start the everything server
-cd ~/src/mcp/go-sdk && go build -o /tmp/go-conformance-server ./conformance/everything-server
-go build -o /tmp/go-conformance-client ./conformance/everything-client
-/tmp/go-conformance-server -http="localhost:3002"
-
-# Terminal 2: run the audit (from the conformance repo)
-/mcp-sdk-tier-audit ~/src/mcp/go-sdk http://localhost:3002 "/tmp/go-conformance-client"
-```
-
-**C# SDK example:**
-
-```bash
-# Terminal 1: start the everything server (requires .NET SDK)
-cd ~/src/mcp/csharp-sdk
-dotnet run --project tests/ModelContextProtocol.ConformanceServer --framework net9.0 -- --urls http://localhost:3003
-
-# Terminal 2: run the audit (from the conformance repo)
-/mcp-sdk-tier-audit ~/src/mcp/csharp-sdk http://localhost:3003 "dotnet run --project ~/src/mcp/csharp-sdk/tests/ModelContextProtocol.ConformanceClient"
-```
-
-The skill derives `owner/repo` from git remote, runs the CLI, launches parallel evaluations for docs and policy, and writes detailed reports to `results/`.
 
 ### Any Other AI Coding Agent
 
@@ -181,66 +167,17 @@ Run the CLI for the scorecard, then review docs and policies yourself using the
 
 ## Running Conformance Tests
 
-To include conformance test results, start the SDK's everything server first, then pass the URL to the CLI. To also run client conformance tests, pass `--client-cmd` with the command to launch the SDK's conformance client.
-
-**TypeScript SDK**:
+For any SDK in [`src/sdk-runner/known-sdks.ts`](../../../src/sdk-runner/known-sdks.ts), don't start servers by hand — the config runs the right invocation per revision:
 
 ```bash
-# Terminal 1: start the server (SDK must be built first)
-cd ~/src/mcp/typescript-sdk && npm run build
-npm run test:conformance:server:run   # starts on port 3000
+# both legs, every revision, one verdict
+npx @modelcontextprotocol/conformance tier-check --sdk go-sdk
 
-# Terminal 2: run tier-check (server + client conformance)
-npm run --silent tier-check -- \
-  --repo modelcontextprotocol/typescript-sdk \
-  --conformance-server-url http://localhost:3000/mcp \
-  --client-cmd 'npx tsx ~/src/mcp/typescript-sdk/test/conformance/src/everythingClient.ts'
+# a single leg at a single revision, for debugging
+npx @modelcontextprotocol/conformance sdk --path <sdk checkout> --mode server --requirements 2026-07-28
 ```
 
-**Python SDK**:
-
-```bash
-# Terminal 1: install and start the server
-cd ~/src/mcp/python-sdk
-uv sync --frozen --all-extras --package mcp-everything-server
-uv run mcp-everything-server --port 3001   # specify port to avoid conflicts
-
-# Terminal 2: run tier-check (server + client conformance)
-npm run --silent tier-check -- \
-  --repo modelcontextprotocol/python-sdk \
-  --conformance-server-url http://localhost:3001/mcp \
-  --client-cmd 'uv run python ~/src/mcp/python-sdk/.github/actions/conformance/client.py'
-```
-
-**Go SDK**:
-
-```bash
-# Terminal 1: build and start the server
-cd ~/src/mcp/go-sdk
-go build -o /tmp/go-conformance-server ./conformance/everything-server
-go build -o /tmp/go-conformance-client ./conformance/everything-client
-/tmp/go-conformance-server -http="localhost:3002"
-
-# Terminal 2: run tier-check (server + client conformance)
-npm run --silent tier-check -- \
-  --repo modelcontextprotocol/go-sdk \
-  --conformance-server-url http://localhost:3002 \
-  --client-cmd '/tmp/go-conformance-client'
-```
-
-**C# SDK**:
-
-```bash
-# Terminal 1: start the server (requires .NET SDK)
-cd ~/src/mcp/csharp-sdk
-dotnet run --project tests/ModelContextProtocol.ConformanceServer --framework net9.0 -- --urls http://localhost:3003
-
-# Terminal 2: run tier-check (server + client conformance)
-npm run --silent tier-check -- \
-  --repo modelcontextprotocol/csharp-sdk \
-  --conformance-server-url http://localhost:3003 \
-  --client-cmd 'dotnet run --project ~/src/mcp/csharp-sdk/tests/ModelContextProtocol.ConformanceClient'
-```
+To reproduce one leg fully by hand, copy the build/server/client commands from that SDK's `known-sdks.ts` entry (including its `specOverrides` for the revision you're running) rather than from a README: the entries are exercised by the test suite and per-revision, prose examples are neither.
 
 **Other SDKs:** Your SDK needs an "everything server" — an HTTP server implementing the [Streamable HTTP transport](https://modelcontextprotocol.io/specification/draft/basic/transports.md) with all MCP features (tools, resources, prompts, etc.). See the implementations above as reference.
 

.claude/skills/mcp-sdk-tier-audit/SKILL.md

@@ -5,7 +5,7 @@ description: >-
   Produces tier classification (1/2/3) with evidence table, gap list, and
   remediation guide. Works for any official MCP SDK (TypeScript, Python, Go,
   C#, Java, Kotlin, PHP, Swift, Rust, Ruby).
-argument-hint: '<local-path> <conformance-server-url> [client-cmd] [--branch <branch>]'
+argument-hint: '<local-path> [conformance-server-url] [client-cmd] [--requirements <revisions>] [--branch <branch>]'
 ---
 
 # MCP SDK Tier Audit
@@ -26,7 +26,9 @@ If this fails (exit code non-zero or shows "not logged in"), stop immediately an
 
 Do NOT proceed to any other step if this check fails.
 
-After parsing arguments (Step 1), also verify the conformance server is reachable:
+**Known SDKs need no server from you.** When the SDK is in `src/sdk-runner/known-sdks.ts` (typescript-sdk, python-sdk, go-sdk, rust-sdk, csharp-sdk, ruby-sdk, and variants), skip the server preflight entirely: Step 2's `--sdk-path` mode builds the SDK and starts the right server per revision itself. The reachability check below applies only to the URL fallback for unknown SDKs.
+
+After parsing arguments (Step 1), for the URL fallback only, verify the conformance server is reachable:
 
 ```bash
 curl -sf <conformance-server-url> -o /dev/null -w '%{http_code}' 2>&1 || true
@@ -41,8 +43,9 @@ If the server is not reachable, stop and tell the user:
 Extract from the user's input:
 
 - **local-path**: absolute path to the SDK checkout (e.g. `~/src/mcp/typescript-sdk`)
-- **conformance-server-url**: URL where the SDK's everything server is already running (e.g. `http://localhost:3000/mcp`)
+- **conformance-server-url** (only for SDKs not in `known-sdks.ts`): URL where the SDK's everything server is already running (e.g. `http://localhost:3000/mcp`). That one endpoint must serve every claimed revision at its own wire version; SDKs that vary the server per revision cannot be driven this way
 - **client-cmd** (optional): command to run the SDK's conformance client (e.g. `npx tsx test/conformance/src/everythingClient.ts`). If not provided, client conformance tests are skipped and noted as a gap in the report.
+- **requirements** (optional): spec revisions to score against, comma-separated, e.g. `--requirements 2025-11-25,2026-07-28`. Each revision's scenarios run at that revision's wire version, and every listed revision must pass for Tier 1. Scores the SDK against exactly the scenarios that revision required when it shipped, rather than everything the suite carries today. Prefer it whenever the question is "does this SDK conform to revision X". Without it, scoring uses today's suite, which can fail an SDK for a scenario added after it shipped. Run `conformance list --requirements <revision>` to see the set.
 - **branch** (optional): Git branch to check on GitHub (e.g. `--branch fweinberger/v1x-governance-docs`). If not provided, derive from the local checkout's current branch: `cd <local-path> && git rev-parse --abbrev-ref HEAD`. This is passed to the tier-check CLI so that policy signal file checks use the correct branch instead of the repo's default branch.
 
 The first two arguments are required. If either is missing, ask the user to provide it.
@@ -57,15 +60,35 @@ cd <local-path> && git remote get-url origin | sed 's#.*github.com[:/]##; s#\.gi
 
 The `tier-check` CLI handles all deterministic checks — server conformance, client conformance, labels, triage, P0 resolution, releases, policy signals, and spec tracking. You are already in the conformance repo, so run it directly.
 
+For a known SDK, this manages everything — build, per-revision server, both legs:
+
+```bash
+npm run --silent tier-check -- \
+  --sdk-path <local-path> \
+  --requirements <revisions> \
+  --output json
+```
+
+`--repo` and `--branch` are derived from the SDK config and checkout; pass them only to override. For an SDK not in `known-sdks.ts`, fall back to driving it directly:
+
 ```bash
 npm run --silent tier-check -- \
   --repo <owner/repo> \
   --branch <branch> \
+  --requirements <revisions> \
   --conformance-server-url <conformance-server-url> \
   --client-cmd '<client-cmd>' \
   --output json
 ```
 
+Omit `--requirements` only if the user did not name a revision. When it is set the
+scorecard reports `requirements_revision`, both pass rates count exactly the
+scenarios that revision requires, and anything run but not scored carries a
+`notScoredReason` of `extension` or `added-after-release`. `requirements_revisions`
+lists every revision scored, and each detail carries the `revision` it came from. Quote the revision
+alongside any conformance number, and report the not-scored failures separately
+rather than folding them into the score or omitting them.
+
 If no client-cmd was detected, omit the `--client-cmd` flag (client conformance will be skipped). The `--branch` flag should always be included (derived from the local checkout if not explicitly provided).
 
 The CLI output includes server conformance pass rate, client conformance pass rate (with per-spec-version breakdown), issue triage compliance, P0 resolution times, label taxonomy, stable release status, policy signal files, and spec tracking gap. Parse the JSON output to feed into Step 4.
@@ -117,8 +140,8 @@ Combine the deterministic scorecard (from the CLI) with the evaluation results (
 
 ### Tier 1 requires ALL of:
 
-- Server conformance test pass rate == 100% (date-versioned scenarios only; `draft` and `extension` are informational and not scored)
-- Client conformance test pass rate == 100% (date-versioned scenarios only; `draft` and `extension` are informational and not scored)
+- Server conformance test pass rate == 100% across every requirement set given to `--requirements` (each run at its own wire), otherwise of date-versioned scenarios only
+- Client conformance test pass rate == 100%, on the same basis
 - Issue triage compliance >= 90% within 2 business days
 - All P0 bugs resolved within 7 days
 - Stable release >= 1.0.0 with no pre-release suffix

.claude/skills/new-sep/SKILL.md

@@ -0,0 +1,201 @@
+---
+name: new-sep
+description: >-
+  Scaffold a sep-NNNN.yaml requirement-traceability file for the MCP
+  conformance repo from a SEP PR's spec diff. Runs the new-sep CLI, then
+  parses the modelcontextprotocol/modelcontextprotocol spec diff to populate
+  `requirements[]` with the RFC 2119 sentences and proposed check IDs.
+argument-hint: '<sep-number>'
+---
+
+# new-sep: SEP traceability YAML scaffolding
+
+You are bootstrapping a `sep-NNNN.yaml` file for a new SEP in the MCP conformance repo. The output is the requirement-traceability file specified by SEP-2484: a YAML that maps each normative sentence from the SEP's spec diff to a `check:` ID (testable) or an `excluded:` reason (not testable). The CLI gets the skeleton; you fill in the rows by reading the spec diff.
+
+## Step 0: Pre-flight checks
+
+Before doing anything else, verify GitHub CLI authentication:
+
+```bash
+gh auth status 2>&1
+```
+
+If this fails, stop immediately and tell the user:
+
+> GitHub authentication is required for this skill. Please run `gh auth login` first, then re-run.
+
+Verify you're running inside the conformance repo:
+
+```bash
+test -f package.json && jq -r '.name' package.json
+```
+
+The name should be `@modelcontextprotocol/conformance`. If not, stop and ask the user to `cd` into the conformance repo first.
+
+## Step 1: Parse arguments
+
+Extract from the user's input:
+
+- **sep-number** (required): the SEP number, e.g. `2164`. This is also the PR number in `modelcontextprotocol/modelcontextprotocol` by convention.
+
+## Step 2: Generate the skeleton
+
+Run the CLI:
+
+```bash
+npm run --silent build
+node dist/index.js new-sep <NNNN>
+```
+
+(For development against a non-built source tree: `npx tsx src/index.ts new-sep ...`.)
+
+The CLI writes `src/seps/sep-<NNNN>.yaml` with `sep`, `spec_url`, and two TODO `requirements[]` rows. Capture the output path from the CLI's `Wrote …` line and remember it as `$YAML`.
+
+If the CLI errors with "does not change any docs/specification/draft/\*.mdx", the SEP's spec changes landed in a separate PR — ask the user for the spec file path and rerun with `--spec-path docs/specification/draft/<path>`. Do not guess.
+
+## Step 3: Fetch the spec diff
+
+`AGENTS.md` (lines 64–72) is explicit that severity must come from the spec text itself, not the SEP markdown or the conformance PR description:
+
+```bash
+gh api "repos/modelcontextprotocol/modelcontextprotocol/pulls/<NNNN>/files" \
+  --jq '.[] | select(.filename | test("^docs/specification/draft/.*\\.mdx$")) | {filename, patch}'
+```
+
+For each file, pull the added (`+`-prefixed) lines from `patch`. If `patch` is truncated for a large file, fall back to fetching the whole file at the PR's head ref:
+
+```bash
+gh api "repos/modelcontextprotocol/modelcontextprotocol/contents/<path>?ref=<sep-branch>" \
+  --jq '.content' | base64 -d
+```
+
+## Step 4: Extract RFC 2119 requirements
+
+Walk the added lines and identify sentences containing the keywords: **MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, **MAY**, **OPTIONAL**.
+
+**Quote the whole sentence**, not just the matched line. The matched word may sit inside a bullet point whose lead-in sentence supplies the keyword by inheritance — e.g.:
+
+> Servers SHOULD return standard JSON-RPC errors for common failure cases:
+>
+> - Resource not found: -32602 (Invalid Params)
+
+The bullet inherits `SHOULD`. The yaml row should quote the _combined_ obligation: `'Servers SHOULD return standard JSON-RPC errors for common failure cases: Resource not found: -32602 (Invalid Params)'` — see `src/seps/sep-2164.yaml` for the canonical example.
+
+**Regex alone is insufficient** (this is called out in Issue #243). Read for context: pronouns, "the server", and "such cases" all refer back to the lead-in.
+
+## Step 5: Map severity → check vs. excluded
+
+From `AGENTS.md:50-56`:
+
+| Keyword                                        | Severity | YAML field                 |
+| ---------------------------------------------- | -------- | -------------------------- |
+| MUST / MUST NOT / SHALL / SHALL NOT / REQUIRED | FAILURE  | `check: sep-<NNNN>-<slug>` |
+| SHOULD / SHOULD NOT                            | WARNING  | `check: sep-<NNNN>-<slug>` |
+| MAY / OPTIONAL                                 | —        | _no row — skip entirely_   |
+
+MAY / OPTIONAL sentences are noted in Step 4 only so you consciously skip them — they never produce a yaml row.
+
+A row is `excluded:` when a MUST/SHOULD requirement can't be protocol-observed by the harness. Do **not** write any `excluded:` row on your own authority — every exclusion goes through Step 6.
+
+While classifying, sort each MUST/SHOULD row into one of three buckets:
+
+- **`check:`** — observably testable on the wire.
+- **clearly-excluded** — you're confident it can't be observed (e.g. "clients SHOULD also accept -32002" when the harness only drives servers).
+- **borderline** — you'd default to `check:` but observability is questionable. Markers:
+  - _Internal state_ — verbs like _record_, _store_, _associate_, _track_, _cache_. The harness sees wire traffic, not memory; usually only observable via a downstream row already in your list.
+  - _UI / human-facing_ — _display_, _show_, _render_, _prompt the user_.
+  - _Precondition phrasing_ — "Before doing X, the implementation MUST Y" where X is itself another row.
+
+Slug convention: lowercase-kebab, derived from the verb phrase. Examples from `sep-2164.yaml`: `no-empty-contents`, `error-code`. Same `id` is used for SUCCESS and FAILURE (`AGENTS.md:52`).
+
+## Step 6: Confirm exclusions with the user
+
+Nothing becomes `excluded:` without sign-off. Two rounds:
+
+**Round 1 — clearly-excluded, single batch question.** One `AskUserQuestion` listing all clearly-excluded rows in the question body (slug + one-line reason each). Options:
+
+- `Exclude all as listed (Recommended)`
+- `Flip all to check:`
+- `Let me adjust per-row` — if chosen, append these rows to round 2.
+
+Skip this round if the bucket is empty.
+
+**Round 2 — borderline, one question per row.** One `AskUserQuestion` call with a question per borderline row (loop in batches of 4 if needed). For each:
+
+- header: the proposed slug
+- question: quote the requirement sentence + your one-line observability concern
+- options (list `check:` first — it's the default for borderline):
+  - `check:` — keep as a testable check
+  - `excluded: <reason>` — drop to excluded with your stated reason
+  - `merge into <other-slug>` — offer when the row is a precondition for another row already in the list
+
+Apply the answers before writing. For any `excluded:` outcome, write the reason verbatim into the yaml and add an `issue:` URL if the user supplies one. A `merge` outcome means: drop this row, and append its `text:` to the surviving row's `text:` separated by `/` so the traceability isn't lost.
+
+## Step 7: Rewrite the YAML
+
+Replace the two TODO rows the CLI generated with one row per extracted requirement. Preserve the CLI's quoting style (single quotes, two-space indent — see `src/seps/sep-2164.yaml`).
+
+**Key order within each row** — for `check:` rows the **`check:` key comes first**, then `text:`, then any optional `url:`. Scanning the left margin should reveal every check ID without reading the quoted sentences. For `excluded:` rows the order is **`text:` first**, then `excluded:`, then optional `issue:` — there's no ID to scan for, so lead with the requirement.
+
+**Row order in the file** — all `check:` rows first (in spec-diff order), then **all `excluded:` rows grouped at the bottom**, separated from the checks by **one blank line**. Do not interleave.
+
+```yaml
+requirements:
+  - check: sep-NNNN-first-slug
+    text: '...'
+  - check: sep-NNNN-second-slug
+    text: '...'
+
+  - text: '...'
+    excluded: 'reason'
+    issue: https://github.com/modelcontextprotocol/conformance/issues/<NNNN>
+```
+
+If a requirement is ambiguous or you're not confident, leave it as a `TODO:` row rather than guessing — humans review this yaml before scenarios get written.
+
+Also fix the `spec_url`: the CLI emits the page URL with no anchor. If the requirements you extracted live under a specific spec subsection (e.g. `#error-handling`), append it.
+
+If a requirement comes from a **different spec page** than `spec_url` (the SEP touched multiple `.mdx` files — the CLI prints these as "PR also changes N other spec file(s)"), give that row a full `url:` override:
+
+```yaml
+- check: sep-NNNN-slug
+  text: '...'
+  url: https://modelcontextprotocol.io/specification/draft/other/page#anchor
+```
+
+A row's effective spec reference is `row.url ?? file.spec_url`.
+
+Write the result back to `$YAML`.
+
+## Step 8: Suggest a host scenario
+
+`AGENTS.md` prefers **fewer scenarios with more checks** over one-scenario-per-check. Before telling the user to write a new scenario, look for an existing one the new checks could be folded into.
+
+Determine the suite directory from the requirement subjects ("MCP clients MUST…" → `client/`, "Servers MUST…" → `server/`, "authorization servers MUST…" → `authorization-server/`; a SEP may map to more than one). Then search that directory for scenarios touching the same spec area:
+
+```bash
+rg -l -i '<domain-term>|<domain-term-2>' src/scenarios/<suite>/ --type ts
+```
+
+Pick 2–3 domain terms from the SEP's subject matter (for a discovery SEP: `metadata`, `well-known`; for an auth-response SEP: `redirect`, `callback`, `pkce`). For each hit, pull the scenario's `name`/`description` to confirm relevance:
+
+```bash
+rg -A1 'name:|description:' <hit.ts>
+```
+
+If you find a plausible host, recommend it by path. If nothing fits, say so explicitly — a new scenario file is then the right call.
+
+## Step 9: Hand-off
+
+Report to the user, in this order:
+
+1. Path to the generated yaml.
+2. Row counts: "`N check:` rows, `M excluded:` rows" — and note which exclusions the user signed off in Step 6.
+3. Any requirements you left as `TODO:` and why.
+4. **Host-scenario recommendation** from Step 8 — either "consider adding these checks to `src/scenarios/<suite>/<file>.ts` (it already exercises _X_)" or "no existing scenario covers this area; a new file is appropriate".
+5. Remaining next steps the user owns:
+   - add the checks to the host scenario (or create one) under `src/scenarios/{client,server,authorization-server}/`,
+   - register any new scenario in `src/scenarios/index.ts` (`AGENTS.md:48`),
+   - add a passing example to the everything-client/server and a negative test, per `AGENTS.md:74-81`.
+
+Do **not** generate or edit scenario `.ts` files or touch `src/scenarios/index.ts`. The skill's scope ends at the yaml plus the recommendation.

.claude/skills/review-scenario/SKILL.md

@@ -0,0 +1,53 @@
+---
+name: review-scenario
+description: Use when reviewing a conformance PR that adds or changes scenario .ts files for a SEP — before approving, before requesting changes, or as a self-check before opening one.
+---
+
+# review-scenario
+
+## What to check
+
+**Spec diff is ground truth.** Pull the SEP's actual spec changes and read the RFC-2119 sentences yourself — don't trust the PR description or SEP summary for keyword levels:
+
+```bash
+gh api "repos/modelcontextprotocol/modelcontextprotocol/pulls/<SEP>/files" \
+  --jq '.[] | select(.filename | test("^docs/specification/draft/.*\\.mdx$")) | {filename, patch}'
+```
+
+If the SEP includes a conformance-test-case table, that table is authoritative for the cases it lists. A table/prose mismatch is a spec gap to flag, not something to silently resolve either way.
+
+**Traceability YAML.** `src/seps/sep-<SEP>.yaml` should exist (run `/new-sep <SEP>` first if not). Diff its rows against the spec sentences you extracted; flag rows that paraphrase rather than quote, claim a keyword level the spec doesn't, or assert something the spec never says. Check IDs follow `sep-<NNNN>-<kebab-slug>`.
+
+**Per-scenario-file:**
+
+- **Spec backing** — would a fully spec-compliant implementation FAIL this check? If yes — or if two compliant SDKs in different languages would get different results — the spec hasn't pinned the behavior; note it as a gap rather than enforce it.
+- **Dead checks** — emits FAILURE with no reachable SUCCESS counterpart, or sits behind an always-false guard.
+- **Logic** — does a missing/malformed input silently pass? Does the assertion distinguish "rejected for the right reason" from "rejected at all"?
+
+**Coverage.** Count YAML `check:` rows vs how many the PR's scenarios actually exercise; list the gaps.
+
+**Proof it runs.** The PR should reference at least one real implementation the scenario ran green against — the in-repo everything-client/server, or an external SDK via `npx https://pkg.pr.new/@modelcontextprotocol/conformance@<PR>`. No run referenced → ask for one before approving.
+
+**Negative test.** Pins the specific failing slugs, not just `failures.length > 0` (AGENTS.md §Examples: prove it passes and fails).
+
+## Output
+
+This is a first pass for a human reviewer — give them what they need to verify each finding without re-deriving it.
+
+**Open with a summary:** N scenarios added/changed, M distinct check IDs emitted, X/Y YAML `check:` rows covered, and which implementation it was run against.
+
+**Then one bullet per finding.** Each bullet makes its own case — the reviewer should be able to confirm or refute it from the bullet alone:
+
+> **`<check-id>`** — [`file.ts:Lnn`](https://github.com/modelcontextprotocol/conformance/blob/<HEAD-SHA>/path/file.ts#Lnn) — claim. Spec: _"quoted normative sentence"_ ([page#anchor](https://modelcontextprotocol.io/specification/draft/...#anchor)). Consequence: what a compliant impl would do and how this check would mis-report it.
+
+e.g.
+
+> **`client-consistent-version`** — [`stateless.ts:86`](…/blob/abc123/src/scenarios/client/stateless.ts#L86) — no spec backing. Spec: _"Servers MUST NOT rely on prior requests over the same connection to establish context (e.g., capabilities, protocol version)"_ ([lifecycle#stateless](…)). A compliant client may change `protocolVersion` per request; this check FAILs it. The `flippingVersionClient` negative test enforces a non-requirement.
+
+Get `<HEAD-SHA>` once via `gh pr view <PR> --json headRefOid -q .headRefOid` and use it for all permalinks so they don't drift on rebase.
+
+Order: spec-backing → logic/dead → coverage (gap list) → conventions. Put spec gaps in a separate trailing list — those go upstream, not to the PR author.
+
+**Self-review:** fix in place and re-run.
+
+**If asked to push fixes** (stacked diff on top of the PR head): one commit per finding, commit message is the finding. Leave design-level items (scenario count, refactors) as prose.

.github/CODEOWNERS

@@ -0,0 +1,3 @@
+# Authorization server conformance area — team managed in modelcontextprotocol/access
+/src/scenarios/authorization-server/ @modelcontextprotocol/conformance-auth
+/src/runner/authorization-server.ts @modelcontextprotocol/conformance-auth

.github/dependabot.yml

@@ -1,42 +0,0 @@
-version: 2
-
-# Security updates always come through regardless of these settings.
-# Scheduled (non-security) updates are limited to minor/patch only —
-# major bumps of devDeps like eslint/typescript are manual decisions.
-
-updates:
-  - package-ecosystem: npm
-    directory: /
-    schedule:
-      interval: monthly
-    groups:
-      all-dependencies:
-        patterns:
-          - '*'
-    ignore:
-      - dependency-name: '*'
-        update-types: ['version-update:semver-major']
-
-  - package-ecosystem: npm
-    directory: /examples/servers/typescript
-    schedule:
-      interval: monthly
-    groups:
-      all-dependencies:
-        patterns:
-          - '*'
-    ignore:
-      - dependency-name: '*'
-        update-types: ['version-update:semver-major']
-
-  - package-ecosystem: github-actions
-    directory: /
-    schedule:
-      interval: monthly
-    groups:
-      all-actions:
-        patterns:
-          - '*'
-    ignore:
-      - dependency-name: '*'
-        update-types: ['version-update:semver-major']

.github/workflows/traceability.yml

@@ -0,0 +1,130 @@
+name: Refresh SEP traceability manifest
+
+# Regenerates src/seps/traceability.json by running the conformance suite against
+# the reference SDK and recording which check IDs were emitted, then opens a PR
+# with the diff. NOT a PR gate — runs on demand / on a schedule and proposes an
+# update for review. plan.modelcontextprotocol.io reads the committed file from
+# main.
+#
+# Depends on the `conformance sdk` subcommand (#277), which clones+builds the SDK
+# and runs the client+server suites. The `run` job executes third-party SDK code,
+# so it has NO repo write token (read-only perms, persist-credentials: false) and
+# only uploads results as an artifact; the separate `propose` job holds the
+# write/PR permissions and never executes SDK code.
+
+on:
+  workflow_dispatch:
+    inputs:
+      sdk:
+        description: 'SDK ref to run against (e.g. typescript-sdk@<sha>)'
+        default: 'typescript-sdk@main'
+  schedule:
+    - cron: '0 6 * * 1' # Weekly, Monday 06:00 UTC.
+
+concurrency:
+  group: traceability-refresh
+  cancel-in-progress: true
+
+jobs:
+  run:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    env:
+      SDK_REF: ${{ inputs.sdk || 'typescript-sdk@main' }}
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          persist-credentials: false # no git token while SDK code runs
+
+      - uses: actions/setup-node@v6
+        with:
+          node-version: 24
+          cache: npm
+
+      # The SDK's own build (e.g. typescript-sdk uses `pnpm install && pnpm run
+      # build:all`) needs pnpm on PATH; corepack provides it.
+      - run: corepack enable
+
+      - run: npm ci
+      - run: npm run build
+
+      - name: Run conformance suites against the reference SDK
+        # `sdk` requires --mode client|server; run both into the same results dir
+        # (the second reuses the cached checkout + build via --skip-build).
+        # `|| true`: the manifest only needs the emitted check IDs (written
+        # regardless of pass/fail), so SDK conformance failures must not fail
+        # this step. The "no results produced" guard below is the real safety net.
+        run: |
+          node dist/index.js sdk "$SDK_REF" --mode client --suite all -o results || true
+          node dist/index.js sdk "$SDK_REF" --mode server --suite all --skip-build -o results || true
+
+      - name: Fail if no results were produced
+        run: |
+          if [ -z "$(find results -name checks.json -print -quit 2>/dev/null)" ]; then
+            echo "No checks.json produced — the suite run failed; not proposing a manifest."
+            exit 1
+          fi
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: conformance-results
+          path: results
+          retention-days: 7
+
+  propose:
+    needs: run
+    runs-on: ubuntu-latest
+    # Requires the repo/org setting "Allow GitHub Actions to create and approve
+    # pull requests" to be enabled, otherwise `gh pr create` fails.
+    permissions:
+      contents: write
+      pull-requests: write
+    env:
+      SDK_REF: ${{ inputs.sdk || 'typescript-sdk@main' }}
+    steps:
+      - uses: actions/checkout@v6
+      - uses: actions/setup-node@v6
+        with:
+          node-version: 24
+          cache: npm
+      - run: npm ci
+      - run: npm run build
+
+      - uses: actions/download-artifact@v4
+        with:
+          name: conformance-results
+          path: results
+
+      - name: Regenerate manifest
+        run: |
+          set -euo pipefail
+          # Record the resolved sha (stable per SDK commit) so the manifest's
+          # `source` only changes when the SDK actually advances — no per-run noise.
+          ref="${SDK_REF#*@}"
+          sha="$(git ls-remote https://github.com/modelcontextprotocol/typescript-sdk.git "$ref" | cut -f1)"
+          node dist/index.js traceability --results results \
+            --source "typescript-sdk@${sha:0:12}"
+
+      - name: Open/update the rolling refresh PR
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          set -euo pipefail
+          if git diff --quiet -- src/seps/traceability.json; then
+            echo "traceability.json unchanged"
+            exit 0
+          fi
+          # One rolling branch/PR, force-updated each run, so the schedule does
+          # not accrue a new PR every week.
+          branch="traceability-refresh"
+          git config user.name 'github-actions[bot]'
+          git config user.email 'github-actions[bot]@users.noreply.github.com'
+          git checkout -B "$branch"
+          git add src/seps/traceability.json
+          git commit -m "chore: refresh SEP traceability manifest ($SDK_REF)"
+          git push --force origin "$branch"
+          gh pr view "$branch" >/dev/null 2>&1 || gh pr create \
+            --head "$branch" \
+            --title 'chore: refresh SEP traceability manifest' \
+            --body 'Automated refresh from a conformance run against the reference SDK. Review the coverage diff before merging.'

.gitignore

@@ -4,3 +4,6 @@ lefthook-local.yml
 dist/
 .vscode/
 .idea/
+.claude/settings.local.json
+.sdk-under-test/
+.sync-schema-tmp/

.prettierignore

@@ -0,0 +1,15 @@
+# Generated by `conformance traceability` — formatting is owned by the
+# generator (deterministic JSON.stringify), not Prettier. Without this, the
+# repo's `prettier --check .` would reformat the file and fight the generator's
+# output (and the refresh workflow's `git diff` check).
+src/seps/traceability.json
+
+# Vendored verbatim from modelcontextprotocol/schema/{version}/schema.ts and
+# schema.json via `npm run sync-schema`. Keep byte-identical with upstream so
+# the SOURCE pin is meaningful and re-syncing produces a clean diff.
+src/spec-types/*.ts
+src/spec-types/*.schema.json
+
+# Local tooling workspaces (not part of the repo).
+.claude/
+.sdk-under-test/

AGENTS.md

@@ -52,7 +52,10 @@ Keep scenarios separate when they're genuinely independent features or when they
 - **Same `id` for SUCCESS and FAIL.** A check should use one slug and flip `status` + `errorMessage`, not branch into `foo-success` vs `foo-failure` slugs.
 - **Optimize for Ctrl+F on the slug.** Repetitive check blocks are fine — easier to find the failing one than to unwind a clever helper.
 - Reuse `ConformanceCheck` and other types from `src/types.ts` rather than defining parallel shapes.
+- **Don't reimplement the runner.** New subcommands that need to "select scenarios → run them → print summary → compute exit code" must go through the existing `client` / `server` commands (subprocess via `process.execPath` like `tier-check` and `sdk` do) or call shared helpers — never a parallel suite-map / summary loop.
 - Include `specReferences` pointing to the relevant spec section.
+- **Severity follows the spec keyword:** MUST / MUST NOT → `FAILURE`; SHOULD / SHOULD NOT → `WARNING`. (CI treats WARNING as a failure, so Tier-1 SDKs still need to satisfy SHOULDs — see #245.)
+- **A missing prerequisite is a failure, not a skip** (#248). When a check cannot be exercised — the server under test lacks a diagnostic fixture tool, rejects the probe, or advertises a feature it does not serve — report it failed via `notTestable()` / `untestableCheck()` from `src/scenarios/untestable.ts`, with the missing prerequisite named in `errorMessage` and `details.untestable: true`. SKIPPED reads as green in every consumer (pass counts, exit codes, expected-failures), so it is reserved for checks that are legitimately not applicable: an optional capability the server never declared, spec-version inapplicability handled by the runner, or a documented harness gap the suite itself tracks (e.g. `tasks-status-notifications` pending its subscriptions/listen rewrite). The untestable severity follows the underlying requirement's keyword (MUST → `FAILURE`, SHOULD → `WARNING`); the expected-failures baseline is the escape hatch for SDKs that haven't built the fixture yet.
 
 ## Descriptions and wording
 
@@ -60,12 +63,53 @@ Be precise about what's **required** vs **optional**. A scenario description tha
 
 When in doubt about spec details (OAuth parameters, audiences, grant types), check the actual spec in `modelcontextprotocol` rather than guessing.
 
+## Reviewing PRs
+
+### SEP scenarios
+
+Verify requirement levels against the SEP's **spec diff** — the change to `docs/specification/draft/` in the SEP's PR — not the SEP markdown summary or the conformance PR's description. The keyword that governs check severity is the one in the spec text; a bullet under a "Servers SHOULD…" sentence is SHOULD-level even if the SEP's title says "standardize."
+
+```sh
+gh api "repos/modelcontextprotocol/modelcontextprotocol/contents/docs/specification/draft/<path>?ref=<sep-branch>" --jq '.content' | base64 -d
+```
+
+### Adding a new SEP
+
+Scaffold the requirement-traceability YAML with:
+
+```sh
+npx @modelcontextprotocol/conformance new-sep <NNNN>
+```
+
+The command looks up PR #`<NNNN>` in `modelcontextprotocol/modelcontextprotocol` (SEP numbers are PR numbers), derives `spec_url` from the `docs/specification/draft/*.mdx` file it changes, and writes `src/seps/sep-<NNNN>.yaml` with TODO `requirements[]` rows. Use `--spec-path` or `--spec-url` to skip the lookup. The `new-sep` Claude Code skill drives the same flow end-to-end, parses the spec diff, and fills in the requirement rows.
+
+### Traceability manifest
+
+`src/seps/traceability.json` is a generated map of, per SEP, which declared `check:` IDs are actually emitted when the conformance suite runs against the reference SDK. It is consumed by plan.modelcontextprotocol.io to track SEP-2484 progress.
+
+The emitted check IDs come from a real suite run (not a source scan), so dynamic (template-literal) IDs resolve to their concrete values. Generate the manifest from a results directory:
+
+```sh
+# 1. Run the suite against the reference SDK, collecting checks.json files:
+node dist/index.js client --command '<sdk conformance client>' --suite all -o results
+node dist/index.js server --url '<sdk conformance server url>' --suite all -o results
+# 2. Build the manifest from those results:
+npm run traceability -- --results results
+npm run traceability -- --results results --strict   # exit 1 on any untested (advisory)
+```
+
+Manifest shape: `{ schemaVersion, docs, source, seps }`, where `seps` is keyed by SEP number. Each requirement is `tested` (its check ID was emitted) or `untested` (declared but never emitted — a real gap, or a check that only fires against a deliberately-broken impl, i.e. it needs a negative test). `"tested" means a scenario emitted the check ID, NOT that any SDK passes it` — per-SDK results live in `tier-check`. Matching is exact, so a scenario's emitted check IDs must match the requirement slugs in the yaml (one check ID per MUST/SHOULD, emitted once per case). `source` records what was run against (e.g. `typescript-sdk@<sha>`); the `docs` field points back here.
+
+Contract for consumers (plan.mcp.io): a SEP appears only if it has a traceability yaml or emits `sep-NNNN-*` check IDs. **A SEP absent from the manifest has no conformance artifacts — treat it as not-started** (diff against your own SEP list to find them). `untracked` lists emitted IDs with no yaml row (usually scenario gates).
+
+The manifest is refreshed by `.github/workflows/traceability.yml` (manual/scheduled), which runs the suite against typescript-sdk and opens a PR with the diff — it is **not** a PR gate. Untested checks are advisory for now; the intended future policy is that an untested check must be backed by a negative test.
+
 ## Examples: prove it passes and fails
 
 A new scenario should come with:
 
 1. **A passing example** — usually by extending `examples/clients/typescript/everything-client.ts` or the everything-server, not a new file.
-2. **Evidence it fails when it should** — ideally a negative example (a deliberately broken client), or at minimum a manual run showing the failure mode.
+2. **A negative test** — a deliberately-broken implementation in `examples/{clients,servers}/typescript/` plus a vitest case asserting the check emits `FAILURE`/`WARNING` against it. See `src/scenarios/client/auth/index.test.ts` and `src/scenarios/server/negative.test.ts` for the pattern. A passing run against the everything-server proves the check doesn't false-positive, but not that it catches anything.
 
 Delete unused example scenarios. If a scenario key in the everything-client has no corresponding test, remove it.
 
@@ -79,3 +123,4 @@ Use the existing CLI runner (`npx @modelcontextprotocol/conformance client|serve
 - `npm test` passes
 - For non-trivial scenario changes, run against at least one real SDK (typescript-sdk or python-sdk) to see actual output. For changes to shared infrastructure (runner, tier-check), test against go-sdk or csharp-sdk too.
 - Scenario is registered in the right suite in `src/scenarios/index.ts`
+- If you changed a `sep-*.yaml` or scenario check IDs, `src/seps/traceability.json` will drift; the traceability workflow refreshes it via PR (or regenerate locally with `--results` from a suite run)

CONTRIBUTING.md

@@ -28,19 +28,44 @@ This repo uses **npm** — don't commit `pnpm-lock.yaml` or `yarn.lock`.
 
 ## Running your scenario
 
+Run a client scenario against the bundled example client:
+
 ```sh
-# Against the bundled TypeScript example
 npm run build
 node dist/index.js client --command "tsx examples/clients/typescript/everything-client.ts" --scenario <your-scenario>
+```
 
+Run a server scenario against any server running locally (replace 3000 with your server's port, if necessary):
+
+```sh
 # Against a server
 node dist/index.js server --url http://localhost:3000/mcp --scenario <your-scenario>
 ```
 
-See the [README](./README.md) for full CLI options and the [SDK Integration Guide](./SDK_INTEGRATION.md) for testing against a real SDK.
+Run a server scenario against the latest TypeScript SDK:
+
+```sh
+# In another directory, clone and prepare the SDK once
+cd ..
+git clone https://github.com/modelcontextprotocol/typescript-sdk.git
+cd typescript-sdk
+pnpm install
+pnpm run build:all
+
+# Start the SDK's conformance server in one terminal
+cd test/conformance
+PORT=3100 pnpm run test:conformance:server:run
+
+# Back in this repo, run your local conformance build against it
+cd /path/to/conformance
+npm run build
+node dist/index.js server --url http://localhost:3100/mcp --scenario <your-scenario>
+```
+
+See the [README](./README.md) for full CLI options and the [SDK Integration Guide](./SDK_INTEGRATION.md) for more on testing against a real SDK.
 
 ## Pull requests
 
 - Register your scenario in the right suite in `src/scenarios/index.ts`
-- Run against at least one real SDK before opening the PR — we'll ask what the output looked like
+- Run against at least one real SDK (see above) before opening the PR — we'll ask what the output looked like
 - Keep PRs focused; one feature or scenario group at a time

MAINTAINERS.md

@@ -0,0 +1,10 @@
+# MCP Maintainers
+
+This document lists current maintainers of the Model Context Protocol conformance test framework.
+
+**Last updated:** March 27, 2026
+
+## Maintainers
+
+- [Paul Carleton](https://github.com/pcarleton)
+- [Felix Weinberger](https://github.com/felixweinberger)

README.md

@@ -2,9 +2,6 @@
 
 A framework for testing MCP (Model Context Protocol) client and server implementations against the specification.
 
-> [!WARNING]
-> This repository is a work in progress and is unstable. Join the conversation in the #conformance-testing-wg in the MCP Contributors discord.
-
 **For SDK maintainers:** See [SDK Integration Guide](./SDK_INTEGRATION.md) for a streamlined guide on integrating conformance tests into your SDK repository.
 
 ## Quick Start
@@ -66,12 +63,15 @@ npx @modelcontextprotocol/conformance client --command "<client-command>" --scen
 
 - `--command` - The command to run your MCP client (can include flags)
 - `--scenario` - The test scenario to run (e.g., "initialize")
-- `--suite` - Run a suite of tests in parallel (e.g., "auth")
+- `--suite` - Run a suite of tests in parallel: `all`, `core`, `extensions`, `backcompat`, `auth`, `metadata`, `draft` (scenarios targeting the in-progress draft spec), or `sep-835`
+- `--spec-version <version>` - Filter scenarios by spec version (e.g., `2025-11-25`, `2026-07-28`; `draft` is accepted as an alias for the current draft identifier). The draft version selects the latest dated release plus any draft-only scenarios. When omitted, the version is inferred from the scenario's spec applicability (draft-only scenarios run at the draft version, everything else at the latest dated release); an explicitly requested version outside a scenario's applicability window skips the scenario (exit 0) unless `--force` is passed
+- `--force` - Run a scenario even if it is not applicable at the requested `--spec-version`
+- `--requirements <revision>` - Run exactly what a spec revision requires, frozen at its release (see [Conformance Requirements](#conformance-requirements))
 - `--expected-failures <path>` - Path to YAML baseline file of known failures (see [Expected Failures](#expected-failures))
 - `--timeout` - Timeout in milliseconds (default: 30000)
 - `--verbose` - Show verbose output
 
-The framework appends `<server-url>` as an argument to your command and sets the `MCP_CONFORMANCE_SCENARIO` environment variable to the scenario name. For scenarios that require additional context (e.g., client credentials), the `MCP_CONFORMANCE_CONTEXT` environment variable contains a JSON object with scenario-specific data.
+The framework appends `<server-url>` as an argument to your command and sets the `MCP_CONFORMANCE_SCENARIO` environment variable to the scenario name. For scenarios that require additional context (e.g., client credentials), the `MCP_CONFORMANCE_CONTEXT` environment variable contains a JSON object with scenario-specific data. When `--spec-version` is passed, its resolved value is forwarded to the client process as `MCP_CONFORMANCE_PROTOCOL_VERSION`; example clients can use this value directly as their `protocolVersion`. SDKs that hard-code their protocol version can ignore it. Clients under test must derive the lifecycle from the protocol version they are asked to run: dated versions through `2025-11-25` use the stateful lifecycle (initialize handshake), while the 2026 draft (`2026-07-28`) uses the stateless lifecycle (per-request `_meta`).
 
 ### Server Testing
 
@@ -83,7 +83,8 @@ npx @modelcontextprotocol/conformance server --url <url> [--scenario <scenario>]
 
 - `--url` - URL of the server to test
 - `--scenario <scenario>` - Test scenario to run (e.g., "server-initialize"). Runs all available scenarios by default
-- `--suite <suite>` - Suite to run: "active" (default), "all", or "pending"
+- `--suite <suite>` - Suite to run: "active" (default; excludes pending and draft-spec scenarios), "all", "draft" (scenarios targeting the in-progress draft spec), or "pending"
+- `--requirements <revision>` - Run exactly what a spec revision requires, frozen at its release (see [Conformance Requirements](#conformance-requirements))
 - `--expected-failures <path>` - Path to YAML baseline file of known failures (see [Expected Failures](#expected-failures))
 - `--verbose` - Show verbose output
 
@@ -99,6 +100,92 @@ npx @modelcontextprotocol/conformance server --url <url> [--scenario <scenario>]
 
 - `checks.json` - Array of conformance check results with pass/fail status
 
+### Wire-schema checks
+
+Every scenario also validates each JSON-RPC message on the wire against the
+spec's JSON schema for the negotiated spec version, and emits up to two
+synthetic checks alongside the scenario's own:
+
+- `wire-schema-valid` - fails when a message _the implementation under test
+  sent_ violates the spec JSON schema. The failure details include every
+  violating message and its schema errors.
+- `wire-schema-harness-error` - fails when the _harness itself_ sent an
+  invalid message. This indicates a bug in the conformance suite (or a
+  deliberately nonconformant fixture), not in the implementation under test;
+  please report it.
+
+Scenarios that exchange no instrumented wire traffic (see issue #418) emit
+neither check. Like any other check, `wire-schema-valid` can be baselined via
+the expected-failures file.
+
+## Conformance Requirements
+
+`--suite` and `--spec-version` describe the suite as it is today. Neither answers
+"which scenarios did I need to pass to conform to the spec released on
+2026-07-28", because the suite keeps growing: a scenario merged after a revision
+ships still carries that revision's applicability tag, so it is
+indistinguishable from one that existed at release.
+
+A requirement set answers that question. Each `requirements/<revision>.yaml`
+names the scenarios a revision requires, for the two roles the specification
+defines: an MCP server acting as an OAuth resource server, and an MCP client
+acting as an OAuth client. It deliberately covers no authorization-server
+scenarios, because the specification puts authorization server implementation
+beyond its own scope, so those scenarios serve people deploying an authorization
+server rather than implementations of MCP itself.
+
+**Scenarios run at their revision's wire version.** That is the point of a
+per-revision set, not a label on it: the dated revisions through `2025-11-25`
+use the stateful initialize handshake and `2026-07-28` is stateless with
+per-request `_meta`, and a scenario emits different checks under each. A
+scenario belonging to both revisions must therefore be run twice, once under
+each set. Passing it on one wire says nothing about the other:
+
+```bash
+# what does conforming to 2026-07-28 actually require?
+npx @modelcontextprotocol/conformance list --requirements 2026-07-28
+
+# run exactly that
+npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp --requirements 2026-07-28
+```
+
+`--requirements` replaces `--suite`, `--spec-version` and `--scenario`, since the
+set already names every scenario that runs and the revision fixes the wire they
+run at. Without it nothing changes: the default is still to run everything, which
+is where completeness lives.
+
+`tier-check` takes several at once, and every one of them must pass for Tier 1:
+
+```bash
+npx @modelcontextprotocol/conformance tier-check --repo <owner/repo> \
+  --conformance-server-url http://localhost:3000/mcp \
+  --requirements 2025-11-25,2026-07-28
+```
+
+Only scenarios a revision actually requires decide the exit code and the pass
+rate. Anything run without being scored is reported separately and cannot fail
+the run.
+
+Requirement sets are frozen, and `not_scored` holds what a revision runs and
+reports without counting. Two reasons qualify, and the report names which
+applies:
+
+| Reason                | Meaning                                                                                                                                                    |
+| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `extension`           | Optional by definition. SEP-1730: "Experimental features and protocol extensions (such as Tasks and MCP Apps) are not required for any tier."              |
+| `added-after-release` | The scenario did not exist when the revision shipped, so no implementation could have been passing it.                                                     |
+| `pending`             | The suite's own reference fixture cannot pass it yet, so it cannot be required — but the implementation under test may pass it, so it runs for visibility. |
+
+Both still run, so a failing extension stays visible in the report; neither moves
+the pass rate. Promoting an entry into the required lists is a deliberate,
+reviewable change, which is how the suite grows without retroactively failing
+anyone.
+
+This is the project's contract and lives in this repository. It is the opposite
+of an [expected-failures](#expected-failures) baseline, which lives in an
+implementation's own repository and records what that implementation knows it
+fails. A baselined failure is still a failure against a requirement set.
+
 ## Expected Failures
 
 SDKs that don't yet pass all conformance tests can specify a baseline of known failures. This allows running conformance tests in CI without failing, while still catching regressions.
@@ -135,6 +222,40 @@ This ensures:
 - CI fails on new regressions (unexpected failures)
 - CI fails when a fix lands but the baseline isn't updated (stale entries)
 
+### Baselining a single check
+
+A scenario is many checks — `server-stateless` alone is over twenty — so baselining the
+whole scenario to excuse one of them stops enforcing the other nineteen. An entry can
+instead name a single check, as `<scenario>:<check-id>`:
+
+```yaml
+server:
+  - tasks-lifecycle # whole scenario may fail
+  - server-stateless:sep-2575-server-implements-discover # only this check may fail
+```
+
+The check id is the left-hand column the runner already prints for each check, so it can
+be copied straight out of a failing run.
+
+With a per-check entry, every failing check in that scenario is judged on its own: the
+named one is excused, and any other failure is still an unexpected regression. The four
+exit-code rules above apply per check rather than per scenario, with one addition — a
+baselined check that is absent or skipped is tolerated, because a scenario that bails on
+a failed prerequisite legitimately never reaches its later checks, and the prerequisite
+reports its own failure anyway.
+
+Two things to know:
+
+- **A check id addresses every occurrence of that id.** Ids repeat within a run (a loop,
+  a retried flow), and the occurrences collapse to one verdict, most-severe first. So
+  baselining a repeated id excuses all of its occurrences — coarser than ideal, still far
+  narrower than baselining the scenario.
+- **Mind the space.** `- scenario:check-id` is a string; `- scenario: check-id` is YAML
+  for a mapping and is rejected with an error.
+
+A scenario cannot be listed both wholesale and per-check — the wholesale entry already
+excuses everything, so the pair is contradictory and is rejected.
+
 ## GitHub Action
 
 This repo provides a composite GitHub Action so SDK repos don't need to write their own conformance scripts.
@@ -212,28 +333,149 @@ Run `npx @modelcontextprotocol/conformance list --server` to see all available s
 - **resources-\*** - Resource management scenarios
 - **prompts-\*** - Prompt management scenarios
 
-## SDK Tier Assessment
+## Running Against an SDK at a Specific Ref
 
-The `tier-check` subcommand evaluates an MCP SDK repository against [SEP-1730](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1730) (the SDK Tiering System):
+The `sdk` subcommand clones an SDK repository at a given ref, builds it, and runs the **local** conformance build against it. This is the inner-loop tool for scenario authors and the basis for cross-SDK CI. Examples below use `npm start --` so they run from source — no `npm run build` between edits.
+
+`--mode client` or `--mode server` is required — each invocation tests exactly one side, so client and server are run (and pass/fail) independently.
+
+```bash
+# Run the client conformance suite against typescript-sdk @main (v2)
+npm start -- sdk typescript-sdk --mode client
+
+# Run the server conformance suite (separate invocation)
+npm start -- sdk typescript-sdk --mode server
+
+# A specific main-line SHA or branch (v2 monorepo)
+npm start -- sdk typescript-sdk@abc123f --mode client
+npm start -- sdk typescript-sdk@some-branch --mode server
+
+# The published v1.x line — separate entry (npm build), defaults to the v1.x branch
+npm start -- sdk typescript-sdk-v1 --mode client
+npm start -- sdk typescript-sdk-v1@v1.29.0 --mode server
+
+# Use an existing local checkout (no clone, no fetch)
+npm start -- sdk --path ../typescript-sdk --skip-build --mode client
+
+# Narrow to one scenario / suite
+npm start -- sdk --path ../typescript-sdk --mode server --scenario server-initialize
+npm start -- sdk typescript-sdk --mode client --suite auth
+
+# Target a specific spec version (passed through to the underlying run).
+# When omitted, the SDK's `specVersion` from KNOWN_SDKS is used, if set —
+# e.g. typescript-sdk-v1 defaults to 2025-11-25.
+npm start -- sdk typescript-sdk --mode client --spec-version draft
+```
+
+Build/run commands for each official SDK are looked up by name from [`src/sdk-runner/known-sdks.ts`](src/sdk-runner/known-sdks.ts) — no config file is required in the SDK repo. Resolution order is **CLI flag > built-in entry**, so any field can be overridden on the command line for refs that diverge from the built-in.
+
+An SDK can have more than one entry when its layout differs across major versions — e.g. `typescript-sdk` (v2, the `main` monorepo) and `typescript-sdk-v1` (the published npm v1.x line). An entry may set `defaultRef` (the branch used when you don't pass `@<ref>`) and `repo` (the real clone target when the entry name is an alias).
+
+When the right invocation depends on the spec version being targeted, the entry carries it in `specOverrides` instead of a comment to copy from. The matching entry is merged over the base config when you pass `--spec-version` (or when the entry's own `specVersion` default applies), so version-specific runs need no extra flags:
+
+```bash
+# go-sdk's server pins -stateless=false for the dated-spec suites; its
+# specOverrides['2026-07-28'] entry swaps in the stateless invocation, so
+# this is the whole command:
+npm start -- sdk go-sdk --mode server --suite all --spec-version 2026-07-28
+
+# same for csharp-sdk (stateless URL), rust-sdk (STATELESS=1 env), and
+# python-sdk (per-revision expected-failures baseline)
+```
+
+Explicit CLI flags still beat everything, config included — overriding a field for a one-off run:
+
+```bash
+npm start -- sdk go-sdk@my-fork-branch --mode server \
+  --build-cmd 'go build -o ./.conformance-server ./experimental/server'
+```
+
+To add a new SDK to the matrix, add an entry to `KNOWN_SDKS`.
+
+Clones are cached under `.sdk-under-test/` and reused (fetched) on subsequent runs.
+
+## SDK Tier Assessment
+
+The `tier-check` subcommand evaluates an MCP SDK repository against [SEP-1730](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1730) (the SDK Tiering System). There are two ways to run it, and they answer different questions.
+
+### 1. The CLI, for the deterministic half
+
+Conformance pass rates, issue triage, P0 resolution, labels, releases, policy files. No AI, no judgment, reproducible.
+
+For a known SDK (see `src/sdk-runner/known-sdks.ts`), one command does the whole
+thing — clone, build, and for **each** shipped revision start the server
+invocation that SDK's config declares for that revision, run both legs at that
+revision's wire, and merge:
 
 ```bash
-# Without conformance tests (fastest)
 gh auth login
-npm run --silent tier-check -- --repo modelcontextprotocol/typescript-sdk --skip-conformance
+npx @modelcontextprotocol/conformance tier-check --sdk go-sdk
+# or against a checkout you already have:
+npx @modelcontextprotocol/conformance tier-check --sdk-path ~/src/go-sdk
+```
 
-# With conformance tests (start the everything server first)
-npm run --silent tier-check -- \
+Per-revision resolution is the point, not a convenience. Some SDKs fix their
+wire era per server process (go-sdk only speaks 2026-07-28 when started
+stateless) or per endpoint (csharp-sdk serves it at `/stateless`), so no single
+server URL can be measured against two revisions. The per-SDK, per-revision
+invocations live in `known-sdks.ts` `specOverrides`; adding an SDK there is how
+it joins the matrix.
+
+For an SDK not in the config, drive it directly — but then the one endpoint you
+give must serve every claimed revision at its own wire, and the server must
+already be running:
+
+```bash
+npx @modelcontextprotocol/conformance tier-check \
   --repo modelcontextprotocol/typescript-sdk \
-  --conformance-server-url http://localhost:3000/mcp
+  --conformance-server-url http://localhost:3000/mcp \
+  --client-cmd '<command that runs the SDK conformance client>' \
+  --requirements 2025-11-25,2026-07-28
 ```
 
-For a full AI-assisted assessment with remediation guide, use Claude Code:
+Omit `--client-cmd` and the client leg is skipped and reported as a gap. Omit
+`--requirements` and scoring falls back to the suite as it stands today, which is
+not what you want for a tier claim; see [Conformance Requirements](#conformance-requirements).
+
+The exit code is a CI verdict on the machine-checkable half: nonzero when any
+scored conformance scenario fails or a leg could not be measured. Governance
+findings (triage, P0s, policy files) shape the tier but never the exit code.
+
+### 2. The skill, for the whole assessment
+
+The CLI cannot judge documentation coverage, dependency policy or roadmap quality, and those decide the tier as much as conformance does. The [`mcp-sdk-tier-audit`](.claude/skills/mcp-sdk-tier-audit/README.md) skill runs the CLI, adds those evaluations, and writes a full report with a remediation plan. In Claude Code, from a checkout of this repo:
 
 ```
-/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url>
+/mcp-sdk-tier-audit <local-sdk-path> <conformance-server-url> '<client-cmd>' --requirements 2025-11-25,2026-07-28
 ```
 
-See [`.claude/skills/mcp-sdk-tier-audit/README.md`](.claude/skills/mcp-sdk-tier-audit/README.md) for full documentation.
+The server must already be running and stay up for the whole audit. Expect a few minutes.
+
+### Reading the result
+
+```
+Scored against 2025-11-25 and 2026-07-28, each run at its own wire version.
+
+    Server   67/67 required scenarios (100%)
+    Client   50/50 required scenarios (100%)
+
+  Not scored (8 run, 4 failing, no effect on tier):
+    ✗ auth/dpop (extension)
+    ✗ json-schema-2020-12-preservation (added-after-release)
+
+Tier 1 Blockers:
+  • triage
+  • p0_resolution
+```
+
+- **Required scenarios** are the only ones that move the number. `67/67` spans every revision listed: a scenario belonging to both runs once per revision, on that revision's wire, and both have to pass.
+- **Not scored** ran and is reported so you can see it, but cannot fail the tier. `extension` means optional by definition; `added-after-release` means the scenario did not exist when that revision shipped. A failure here is information, not a blocker.
+- **Not measured** is different from either, and means the run could not happen at all, e.g. a requirement set naming a scenario this build no longer has. Treat it as a broken invocation, never as an SDK failure.
+- **Tier 1 blockers** lists every requirement short of Tier 1. Conformance absent from that list means the SDK met every requirement each listed revision imposes.
+
+The exit code follows the same rule: it reflects required scenarios only, so an implementation that meets a revision's requirements exits 0 even with failing extensions.
+
+In `--output json`, `passed` / `failed` / `total` describe the scored set and so always agree with `pass_rate`; anything run without being scored is counted separately under `not_scored`.
 
 ## Architecture
 

action.yml

@@ -42,14 +42,14 @@ runs:
     - name: Build conformance tests
       shell: bash
       run: |
-        cd "${{ github.action_path }}"
+        cd "$GITHUB_ACTION_PATH"
         npm ci
         npm run build
 
     - name: Run conformance tests
       shell: bash
       run: |
-        CONFORMANCE="${{ github.action_path }}/dist/index.js"
+        CONFORMANCE="$GITHUB_ACTION_PATH/dist/index.js"
 
         # Build the command arguments
         ARGS="${{ inputs.mode }}"

examples/authorization-server-settings.example.json

@@ -0,0 +1,3 @@
+{
+  "url": "https://auth.example.com"
+}

examples/clients/typescript/auth-test-dpop-bearer.ts

@@ -0,0 +1,21 @@
+#!/usr/bin/env node
+
+import { runDpopClient } from './helpers/dpopClientFlow';
+import { runAsCli } from './helpers/cliRunner';
+
+/**
+ * Broken DPoP client: builds correct per-request proofs but presents the
+ * DPoP-bound token with the `Bearer` scheme instead of `DPoP`. Isolates a
+ * failure of sep-1932-client-dpop-auth-scheme (RFC 9449 §7.1).
+ */
+export async function runClient(serverUrl: string): Promise<void> {
+  await runDpopClient(serverUrl, {
+    scheme: 'Bearer',
+    freshProofPerRequest: true,
+    sendTokenRequestProof: true,
+    handleAsNonce: true,
+    handleRsNonce: true
+  });
+}
+
+runAsCli(runClient, import.meta.url, 'auth-test-dpop-bearer <server-url>');

examples/clients/typescript/auth-test-dpop-no-as-nonce.ts

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+
+import { runDpopClient } from './helpers/dpopClientFlow';
+import { runAsCli } from './helpers/cliRunner';
+
+/**
+ * Broken DPoP client: ignores the authorization server's `use_dpop_nonce`
+ * challenge at the token endpoint (RFC 9449 §8) — it does not retry the token
+ * request with the supplied nonce. Isolates a failure of
+ * sep-1932-client-as-nonce.
+ */
+export async function runClient(serverUrl: string): Promise<void> {
+  await runDpopClient(serverUrl, {
+    scheme: 'DPoP',
+    freshProofPerRequest: true,
+    sendTokenRequestProof: true,
+    handleAsNonce: false,
+    handleRsNonce: true
+  });
+}
+
+runAsCli(runClient, import.meta.url, 'auth-test-dpop-no-as-nonce <server-url>');

examples/clients/typescript/auth-test-dpop-no-nonce.ts

@@ -0,0 +1,25 @@
+#!/usr/bin/env node
+
+import { runDpopClient } from './helpers/dpopClientFlow';
+import { runAsCli } from './helpers/cliRunner';
+
+/**
+ * Nonce-incapable but otherwise compliant DPoP client (SEP-1932 / RFC 9449):
+ * presents the DPoP-bound token with the `DPoP` Authorization scheme and a fresh
+ * proof on every request, but implements NO `use_dpop_nonce` handling. Against
+ * the nonce-less `auth/dpop` scenario (where neither server issues a challenge)
+ * it completes the flow successfully — proving that a client which does not
+ * support nonces still passes when the server does not require one (the common
+ * case, since server nonces are OPTIONAL in RFC 9449 §8/§9).
+ */
+export async function runClient(serverUrl: string): Promise<void> {
+  await runDpopClient(serverUrl, {
+    scheme: 'DPoP',
+    freshProofPerRequest: true,
+    sendTokenRequestProof: true,
+    handleAsNonce: false,
+    handleRsNonce: false
+  });
+}
+
+runAsCli(runClient, import.meta.url, 'auth-test-dpop-no-nonce <server-url>');

examples/clients/typescript/auth-test-dpop-no-rs-nonce.ts

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+
+import { runDpopClient } from './helpers/dpopClientFlow';
+import { runAsCli } from './helpers/cliRunner';
+
+/**
+ * Broken DPoP client: obtains the token correctly (handles the AS nonce) but
+ * ignores the MCP server's `use_dpop_nonce` challenge at the resource
+ * (RFC 9449 §9) — it does not retry the request with the supplied nonce.
+ * Isolates a failure of sep-1932-client-rs-nonce.
+ */
+export async function runClient(serverUrl: string): Promise<void> {
+  await runDpopClient(serverUrl, {
+    scheme: 'DPoP',
+    freshProofPerRequest: true,
+    sendTokenRequestProof: true,
+    handleAsNonce: true,
+    handleRsNonce: false
+  });
+}
+
+runAsCli(runClient, import.meta.url, 'auth-test-dpop-no-rs-nonce <server-url>');

examples/clients/typescript/auth-test-dpop-no-token-proof.ts

@@ -0,0 +1,27 @@
+#!/usr/bin/env node
+
+import { runDpopClient } from './helpers/dpopClientFlow';
+import { runAsCli } from './helpers/cliRunner';
+
+/**
+ * Broken DPoP client: never sends a DPoP proof at the token endpoint, so it
+ * never asks for a sender-constrained token — the AS falls back to an unbound
+ * Bearer token. Isolates a failure of sep-1932-client-token-request-proof
+ * (RFC 9449 §5). Because the resulting token is unbound, the resource-side
+ * binding check (sep-1932-client-fresh-proof) also fails as a consequence.
+ */
+export async function runClient(serverUrl: string): Promise<void> {
+  await runDpopClient(serverUrl, {
+    scheme: 'DPoP',
+    freshProofPerRequest: true,
+    sendTokenRequestProof: false,
+    handleAsNonce: true,
+    handleRsNonce: true
+  });
+}
+
+runAsCli(
+  runClient,
+  import.meta.url,
+  'auth-test-dpop-no-token-proof <server-url>'
+);