Compare commits

..
Author SHA1 Message Date
Owen Devereaux
6524782667
[v1.x] fix: handle ClosedResourceError when transport closes mid-request (#2334)
Some checks failed
Main branch checks / checks (push) Has been cancelled
Co-authored-by: Owen Devereaux <owendevereaux@users.noreply.github.com>
Co-authored-by: Max Isbey <224885523+maxisbey@users.noreply.github.com>
2026-03-24 22:24:47 +00:00
Max Isbey
2e9897e2b9
[v1.x] fix: handle non-UTF-8 bytes in stdio server stdin (#2303) 2026-03-17 18:40:43 +00:00
BabyChrist666
f8d98b63a7
Backport: Add missing TasksCallCapability to v1.x (#2137)
Co-authored-by: Max Isbey <224885523+maxisbey@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-03-02 12:56:24 +00:00
Felix Weinberger
c68e254bad
docs: add server-side tool error handling documentation (#2129) 2026-02-24 11:31:00 +00:00
Felix Weinberger
1ef124e4a8
docs: add snippet verification for docs/ pages (#2115) 2026-02-23 15:38:15 +00:00
Felix Weinberger
cfbbd7d71a
docs: fix GitHub links to point to v1.x branch (#2102) 2026-02-23 15:26:21 +00:00
Felix Weinberger
1f9fb348f3
docs: fix stub pages and improve docs structure (#2101)
Co-authored-by: Max Isbey <224885523+maxisbey@users.noreply.github.com>
2026-02-23 15:18:24 +00:00
Felix Weinberger
c86477c7b7
docs: comprehensive feature documentation for SEP-1730 Tier 1 (#2090)
Co-authored-by: Max Isbey <224885523+maxisbey@users.noreply.github.com>
2026-02-23 14:57:16 +00:00
Felix Weinberger
a77462b64d
docs: restructure README into docs/ pages (#2091)
Co-authored-by: Max Isbey <224885523+maxisbey@users.noreply.github.com>
2026-02-23 14:17:12 +00:00
Felix Weinberger
b1adfcd885
Add VERSIONING.md, ROADMAP.md, and DEPENDENCY_POLICY.md (#2084) 2026-02-19 17:23:58 +00:00
Max Isbey
66aaf93bcd
[v1.x] fix: prevent command injection in example URL opening (#2085) 2026-02-18 19:04:18 +00:00
Felix Weinberger
23a615783e
feat: add idle timeout for StreamableHTTP sessions (#1994) 2026-02-18 10:34:18 +00:00
Felix Weinberger
67458948ae
fix: add RFC 8707 resource validation to OAuth client (#2069) 2026-02-17 14:35:42 +00:00
Felix Weinberger
78ddff022b
ci: backport conformance tests from main to v1.x (#2068) 2026-02-16 19:27:57 +00:00
Max Isbey
bac2789e09
fix: remove unused requests dependency from simple-chatbot example (#1959) 2026-01-26 14:03:06 +00:00
Luca Chang
3d9d34552a
[v1.x] fix: return HTTP 404 for unknown session IDs instead of 400 (#1945)
Some checks failed
Main branch checks / checks (push) Has been cancelled
Co-authored-by: Maxime <67350340+max-rousseau@users.noreply.github.com>
Co-authored-by: Max Isbey <224885523+maxisbey@users.noreply.github.com>
2026-01-24 19:09:14 +00:00
Max Isbey
d891525958
Backport: Support for Resource and ResourceTemplate metadata (#1928)
Co-authored-by: Jacem Elwaar <jacem@mcpappsbuilders.com>
2026-01-22 14:02:25 +00:00
Max Isbey
d3787c96b4
ci: add all-green job to pull-request-checks workflow (#1929) 2026-01-22 13:56:22 +00:00
1746 changed files with 43339 additions and 236353 deletions

View file

@ -1,118 +0,0 @@
Review the pull request: $ARGUMENTS
Follow these steps carefully. Use the `gh` CLI for all GitHub interactions.
## Step 1: Resolve the PR
Parse `$ARGUMENTS` to determine the PR. It can be:
- A full URL like `https://github.com/owner/repo/pull/123`
- A `owner/repo#123` reference
- A bare number like `123` (use the current repo)
- A description — search for it with `gh pr list --search "<description>" --limit 5` and pick the best match
Once resolved, fetch the PR metadata:
```bash
gh pr view <PR> --json number,title,body,author,state,baseRefName,headRefName,url,labels,milestone,additions,deletions,changedFiles,createdAt,updatedAt,mergedAt,reviewDecision,reviews,assignees
```
## Step 2: Gather the diff
Get the full diff of the PR:
```bash
gh pr diff <PR>
```
If the diff is very large (>3000 lines), focus on the most important files first and summarize the rest.
## Step 3: Collect PR discussion context
Fetch all comments and review threads:
```bash
gh api repos/{owner}/{repo}/pulls/{number}/comments --paginate
gh api repos/{owner}/{repo}/issues/{number}/comments --paginate
gh api repos/{owner}/{repo}/pulls/{number}/reviews --paginate
```
Pay attention to:
- Reviewer feedback and requested changes
- Author responses and explanations
- Any unresolved conversations
- Approval or rejection status
## Step 4: Find and read linked issues
Look for issue references in:
- The PR body (patterns like `#123`, `fixes #123`, `closes #123`, `resolves #123`)
- The PR branch name (patterns like `issue-123`, `fix/123`)
- Commit messages
For each linked issue, fetch its content:
```bash
gh issue view <number> --json title,body,comments,labels,state
```
Read through issue comments to understand the original problem, user reports, and any discussed solutions.
## Step 5: Analyze and validate
With all context gathered, analyze the PR critically:
1. **Intent alignment**: Does the code change actually solve the problem described in the PR and/or linked issues?
2. **Completeness**: Are there aspects of the issue or requested feature that the PR doesn't address?
3. **Scope**: Does the PR include changes unrelated to the stated goal? Are there unnecessary modifications?
4. **Correctness**: Based on the diff, are there obvious bugs, edge cases, or logic errors?
5. **Testing**: Does the PR include tests? Are they meaningful and do they cover the important cases?
6. **Breaking changes**: Could this PR break existing functionality or APIs?
7. **Unresolved feedback**: Are there reviewer comments that haven't been addressed?
## Step 6: Produce the review summary
Present the summary in this format:
---
### PR Review: `<title>` (<url>)
**Author:** <author> | **Status:** <state> | **Review decision:** <decision>
**Base:** `<base>` ← `<head>` | **Changed files:** <n> | **+<additions> / -<deletions>**
#### Problem
<1-3 sentences describing what problem this PR is trying to solve, based on the PR description and linked issues>
#### Solution
<1-3 sentences describing the approach taken in the code>
#### Key changes
<Bulleted list of the most important changes, grouped by theme. Include file paths.>
#### Linked issues
<List of linked issues with their title, state, and a one-line summary of the discussion>
#### Discussion highlights
<Summary of important comments from reviewers and the author. Flag any unresolved threads.>
#### Concerns
<List any issues found during validation: bugs, missing tests, scope creep, unaddressed feedback, etc. If none, say "No concerns found.">
#### Verdict
<One of: APPROVE / REQUEST CHANGES / NEEDS DISCUSSION, with a brief justification>
#### Suggested action
<Clear recommendation for the reviewer: what to approve, what to push back on, what to ask about>
---

View file

@ -1,150 +0,0 @@
---
name: test-quality
description: Test quality bar for this repo. Read when writing, reviewing, or designing tests — covers naming, abstraction level, assertions, determinism, and the process for agent-driven test work.
---
# Test & code quality guide
What "best practice for new work" means in this repo. Each rule carries its recorded reasoning
where one exists; a rule with no stated why is a convention — follow it anyway.
## Naming & shape
- **Test names are behaviour sentences** stating the observable outcome, not the feature being
poked: `test_elicit_form_decline_returns_no_content`, never `test_elicit_form_decline`.
- Plain top-level `test_*` functions; no `Test` classes (legacy files have them — don't copy).
- **Docstrings: 1–2 sentences of behaviour, honest about provenance** — spec-mandated,
SDK-defined, or pinning a known gap? Say which: provenance is the triage key when the test
later fails. A pinned-gap assertion breaking usually means a change *fixed* the gap; a
spec-mandated assertion breaking means a regression.
- Define things in dependency order; nothing forward-references. For client↔server tests:
handlers → server construction → client setup → act → assert — the test reads in the order
the conversation happens.
- Inline the server (or equivalent setup) in the test, so the whole observable behaviour fits
on one screen. Lift to a file-level fixture only when several tests in *that file* genuinely
share it; never share across files.
- A big multi-step test is fine when the property is irreducibly multi-step (e.g. resumability).
Split when a failure wouldn't tell you which claim broke — not for shortness. Compensate with
a numbered "Steps:" docstring so a reader sees the choreography before the body.
## Level of abstraction
- **Drive through the highest-level public API that can observe the property.** Hand-built wire
requests are brittle and don't prove the user-facing contract; tests that stay above the
internals keep working when the internals change. Drop to raw HTTP only when the assertion is
about something the high-level API *cannot* observe (status codes, headers, wire framing).
- **Scripting a peer over raw streams is a last resort**, reserved for behaviour the typed API
cannot *produce* (malformed input, an impossible peer response). First ask what it would take
for the public API to express it — often a small helper suffices. Every such test's docstring
states why the public API couldn't do it.
- **In-memory / in-process first.** HTTP-, SSE-, and auth-shaped behaviour can all be driven
through an in-process ASGI transport; threads only when necessary, subprocesses only when the
process boundary is itself the thing under test. In-process isn't just faster — it surfaces
bugs (a real stream leak was found this way) that subprocess indirection masks.
- **Tests read like real user code**: no aliasing shims in conftest, no walls of suppressions,
no private imports unless that is genuinely the documented way to do the thing.
- The assertion must prove the round trip — no side-channel state. What the server saw comes
back through the protocol, or via a closure-captured list asserted after the call. Handlers
assert their dispatch identity first (`assert params.name == "add"`), proving the request
that arrived is the request the test sent.
## Assertions
- **Transformations** (input → output the SDK produced) → full-object `snapshot(...)` equality,
so an added or dropped field fails. Regenerate with `--inline-snapshot=fix` so intentional
changes arrive as a reviewable diff; never hand-edit snapshot literals.
- **Pass-through values** (opaque tokens, `_meta`, cursors) → identity against the same
variable you sent. A snapshot of a pass-through value only "matches" because a human checked
two literals correspond — it proves nothing.
- **Errors** → `pytest.raises` + `.code` against the named constant; snapshot SDK-authored
messages; never `match=` on message text. Third-party text (pydantic, jsonschema) → stable
prefix only — never pin text that changes with a dependency upgrade — with a comment saying so.
## Determinism
- **No sleeps, ever.** A sleep guesses at timing instead of waiting on the condition, so it
either flakes or pads the run — sleeps head the list of the older test code's failure modes.
Coordinate with `anyio.Event` so the wait ends exactly when the condition holds (that
discipline is why 529 e2e tests run in ~10 s). Sole exception: tests *of* time-based
features, with a comment.
- Bound every indefinite wait with `anyio.fail_after(5)`. **5 is the standard** — widening it
needs an articulable reason; "10 to be safe" is covering up a flake, not fixing one, and
unexplained widenings propagate (one agent used 10 and every later one copied it).
- **Never assert wall-clock time, even with huge margins.** `elapsed < 0.9` on a ~0.01 s
operation — a 90× margin — was still rejected in review. `fail_after` bounding a hang is the
only timing primitive allowed.
- **Concurrency tests must prove genuine interleaving.** Without barriers the scheduler is free
to serialize — "a" can start, finish, and return before "b" even begins — so two `start_soon`
calls prove nothing. Gate with events so all parties are mid-flight before any proceeds, emit
interleaved, then assert the demux.
- **Don't over-synchronize either.** When delivery ordering is guaranteed (notifications
emitted during a request you're awaiting, over a single ordered in-memory stream), a plain
collected list asserted after the call is correct; events are for messages not tied to an
awaited operation. Verify the ordering guarantee actually holds for your transport first.
- Async tests use anyio, not asyncio.
## Behaviour philosophy
- **Pin current behaviour; never xfail.** A green suite asserting what actually happens is a
regression bar for any refactor; an xfail proves nothing about it. Where current behaviour
falls short of spec, pin the divergent output and record the gap as data — a tracking issue,
with the docstring naming the known gap (suites with a requirements manifest record it as a
divergence entry). Not hidden, not skipped.
- **Hollow-proof check**: before claiming a test covers a behaviour, re-read the claim and ask
"which assertion proves *this*?" A passing test near a behaviour is not proof of it — a full
review of the e2e suite found two such cases even under this discipline.
## Hygiene
- No new `# pragma`, `# type: ignore`, `# noqa` by default — restructure first; a suppression
usually means a test or a type is missing. The narrow sanctioned escape hatches (and the
audit to run before pushing) are in AGENTS.md. In tests, narrow types with `assert
isinstance`; never `Any`/`object` when a real type exists.
- **Warnings raised during tests are findings, not noise** (the repo runs
`filterwarnings = error`). Fix the cause; if the fix can't land in the same change, scope the
suppression to the one fixture that needs it and track the real fix explicitly.
- Registered-but-never-invoked handler bodies are `raise NotImplementedError`, so they cannot
silently become load-bearing.
- Comments live next to the line they explain, not in docstrings; single backticks for code
refs; match the surrounding comment density (one-liners next to one-liners). No
`from __future__ import annotations` (py310+ repo).
- Test work doesn't change `src/` as a side effect — the one mechanical exception is deleting a
pragma a new test now covers. If a test can't be written without a library change, raise it
as a finding or defer the test; don't quietly edit `src/`.
## Process (for agent-driven work)
- **Small chunks.** ≤10 tests per human-reviewed batch; when fanning out to multiple agents,
≤5 per agent. Review quality scales inversely with batch size: the observed shortcuts
(over-stacked tests, a timing assertion, wrong abstraction level) all surfaced in one
oversized 27-test batch.
- **Design → review → implement, as separate steps.** The design deliverable is not just the
plan — it's the judgement calls (abstraction level, deferrals, contested assertions) stated
explicitly, so the reviewer vetoes them before implementation rather than discovering them
in the diff.
- **High-stakes areas get fresh adversarial reviewers** on both the design and the
implementation — fresh, because whoever designed it (or saw the proposed fix) anchors on the
same layer. SERIOUS findings (wrong assertion, missed MUST/SHOULD, an unrecorded known gap)
re-loop; style doesn't. Reserve the full panel for areas where being wrong is worse than
being slow.
- **Investigations pair a full-context look with a fresh agent given only the problem** —
never the proposed fix — and compare conclusions. The unbiased read regularly catches
anchoring on the wrong layer.
- **When review questions a decision, reconsider genuinely**: re-derive the tradeoff, state the
options, recommend with reasons. Defending the original choice is a valid outcome;
reflexively agreeing with every challenge is as bad as ignoring it.
- **Validate the reference artifact before building on it.** Whatever you treat as ground truth
(a spec import, a baseline, a generated list), check it *first* — discovering it was
incomplete after five batches costs far more than before batch one.
- **Verify as you go**: per-file `uv run --frozen pytest <file> -q` + pyright + ruff while
iterating; full suite, coverage, and `./scripts/test` (when `src/` was touched) at
integration.
- **Every agent writes notes**: what it did, what broke, and — most importantly — what it
couldn't decide, so open questions surface instead of being silently resolved by whoever hit
them.
- **Quality over speed is a stated goal, not a preference** ("rushing something out the door is
not the goal here, explicitly so"). Don't quietly de-scope agreed work mid-stream;
renegotiate scope explicitly.
- **Don't copy patterns from existing test code by default** — much of the repo's older test
code is below the current bar (sleeps, mocks, `Test` classes, raciness). These rules define
the bar for new work.

2
.gitattribute Normal file
View file

@ -0,0 +1,2 @@
# Generated
uv.lock linguist-generated=true

View file

@ -11,26 +11,13 @@ body:
id: checks id: checks
attributes: attributes:
label: Initial Checks label: Initial Checks
description: > description: Just making sure you're using the latest version of MCP Python SDK.
Both the 2.x stable line and the 1.x maintenance line are supported, and only
the newest release of each line receives fixes.
options: options:
- label: I confirm that I'm using the newest release of my line (the latest 2.x, or the latest 1.x if I'm still on v1) - label: I confirm that I'm using the latest version of MCP Python SDK
required: true required: true
- label: I confirm that I searched for my issue in https://github.com/modelcontextprotocol/python-sdk/issues before opening this issue - label: I confirm that I searched for my issue in https://github.com/modelcontextprotocol/python-sdk/issues before opening this issue
required: true required: true
- type: dropdown
id: release-line
attributes:
label: Release line
description: Which major version of the SDK are you using?
options:
- 2.x (current stable)
- 1.x (maintenance line, v1.x branch)
validations:
required: true
- type: textarea - type: textarea
id: description id: description
attributes: attributes:
@ -52,7 +39,7 @@ body:
demonstrating the bug. demonstrating the bug.
placeholder: | placeholder: |
from mcp.server.mcpserver import MCPServer from mcp.server.fastmcp import FastMCP
... ...
render: Python render: Python

View file

@ -1,59 +0,0 @@
name: v2 feedback
description: Bugs, API friction, or docs gaps in v2 of the SDK
title: "[v2] "
labels: ["v2"]
body:
- type: markdown
attributes:
value: |
Anything in v2 that broke, surprised you, or slowed you down is useful, including API friction and docs gaps.
Docs: https://py.sdk.modelcontextprotocol.io/ · Migration from v1: https://py.sdk.modelcontextprotocol.io/migration/
- type: textarea
id: what
attributes:
label: What happened?
description: What did you do, and what went wrong (or felt wrong)? Paste error output verbatim if there is any.
validations:
required: true
- type: textarea
id: expected
attributes:
label: What did you expect?
validations:
required: false
- type: textarea
id: repro
attributes:
label: Code to reproduce
description: The smallest snippet or repository that shows it. For docs feedback, link the page instead.
render: Python
validations:
required: false
- type: input
id: version
attributes:
label: SDK version
description: The published version (`pip show mcp`) or commit.
validations:
required: false
- type: dropdown
id: area
attributes:
label: Area
options:
- Server
- Client
- Transports
- Auth
- Documentation
- Migration
- Other
validations:
required: false

View file

@ -6,28 +6,16 @@ It handles all conformance test scenarios via environment variables and CLI argu
Contract: Contract:
- MCP_CONFORMANCE_SCENARIO env var -> scenario name - MCP_CONFORMANCE_SCENARIO env var -> scenario name
- MCP_CONFORMANCE_CONTEXT env var -> optional JSON (for client-credentials scenarios) - MCP_CONFORMANCE_CONTEXT env var -> optional JSON (for client-credentials scenarios)
- MCP_CONFORMANCE_PROTOCOL_VERSION env var -> spec version the harness mock
server is speaking (e.g. "2025-11-25", "2026-07-28"). Always set; when
--spec-version is omitted the harness picks per-scenario (LATEST_SPEC_VERSION
for active scenarios, DRAFT_PROTOCOL_VERSION for draft-only ones).
- Server URL as last CLI argument (sys.argv[1]) - Server URL as last CLI argument (sys.argv[1])
- Must exit 0 within the harness --timeout (CI passes 60s; the default is 30s) - Must exit 0 within 30 seconds
Scenarios: Scenarios:
initialize - Connect, initialize, list tools, close initialize - Connect, initialize, list tools, close
tools_call - Connect, call add_numbers(a=5, b=3), close tools_call - Connect, call add_numbers(a=5, b=3), close
sse-retry - Connect, call test_reconnection, close sse-retry - Connect, call test_reconnection, close
json-schema-ref-no-deref - Connect, list tools (no $ref deref)
json-schema-2020-12-preservation - List tools, echo the focal inputSchema back verbatim
request-metadata - Connect with all callbacks; client stamps _meta
http-standard-headers - Tool, resource and prompt round-trips (Mcp-* headers checked)
http-invalid-tool-headers - List tools, call every surfaced tool (x-mcp-header filter)
http-custom-headers - Replay the harness's toolCalls (x-mcp-header -> Mcp-Param-*)
elicitation-sep1034-client-defaults - Elicitation with default accept callback elicitation-sep1034-client-defaults - Elicitation with default accept callback
sep-2322-client-request-state - Drive the MRTR auto-loop (SEP-2322)
auth/client-credentials-jwt - Client credentials with private_key_jwt auth/client-credentials-jwt - Client credentials with private_key_jwt
auth/client-credentials-basic - Client credentials with client_secret_basic auth/client-credentials-basic - Client credentials with client_secret_basic
auth/enterprise-managed-authorization - SEP-990 ID-JAG (RFC 8693 + RFC 7523 jwt-bearer)
auth/* - Authorization code flow (default for auth scenarios) auth/* - Authorization code flow (default for auth scenarios)
""" """
@ -40,23 +28,19 @@ from collections.abc import Callable, Coroutine
from typing import Any, cast from typing import Any, cast
from urllib.parse import parse_qs, urlparse from urllib.parse import parse_qs, urlparse
import httpx2 import httpx
import mcp_types as types
from mcp_types.version import MODERN_PROTOCOL_VERSIONS
from pydantic import AnyUrl from pydantic import AnyUrl
from mcp import ClientSession, types
from mcp.client.auth import OAuthClientProvider, TokenStorage from mcp.client.auth import OAuthClientProvider, TokenStorage
from mcp.client.auth.extensions.client_credentials import ( from mcp.client.auth.extensions.client_credentials import (
ClientCredentialsOAuthProvider, ClientCredentialsOAuthProvider,
PrivateKeyJWTOAuthProvider, PrivateKeyJWTOAuthProvider,
SignedJWTParameters, SignedJWTParameters,
) )
from mcp.client.auth.extensions.identity_assertion import IdentityAssertionOAuthProvider
from mcp.client.auth.utils import build_protected_resource_metadata_discovery_urls
from mcp.client.client import Client
from mcp.client.context import ClientRequestContext
from mcp.client.streamable_http import streamable_http_client from mcp.client.streamable_http import streamable_http_client
from mcp.shared.auth import AuthorizationCodeResult, OAuthClientInformationFull, OAuthClientMetadata, OAuthToken from mcp.shared.auth import OAuthClientInformationFull, OAuthClientMetadata, OAuthToken
from mcp.shared.context import RequestContext
# Set up logging to stderr (stdout is for conformance test output) # Set up logging to stderr (stdout is for conformance test output)
logging.basicConfig( logging.basicConfig(
@ -66,28 +50,6 @@ logging.basicConfig(
) )
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
#: Spec version the harness is running this scenario at (e.g. "2025-11-25",
#: "2026-07-28"). The harness always sets this (when --spec-version is omitted
#: it picks per-scenario: LATEST_SPEC_VERSION for active scenarios,
#: DRAFT_PROTOCOL_VERSION for draft-only ones), so None means we were invoked
#: outside the harness.
PROTOCOL_VERSION: str | None = os.environ.get("MCP_CONFORMANCE_PROTOCOL_VERSION")
def client_mode() -> str:
"""Pick the Client(mode=) for the harness leg.
On a modern leg (2026-07-28+) -> 'auto' so Client.discover() runs and the
_meta envelope + MCP-Protocol-Version header are stamped on every request.
On a handshake-era leg -> 'legacy' so the initialize handshake runs exactly
as before (no server/discover probe is sent against a mock that would 400 it).
Outside the harness -> 'auto' (probe + fallback).
"""
if PROTOCOL_VERSION is None or PROTOCOL_VERSION in MODERN_PROTOCOL_VERSIONS:
return "auto"
return "legacy"
# Type for async scenario handler functions # Type for async scenario handler functions
ScenarioHandler = Callable[[str], Coroutine[Any, None, None]] ScenarioHandler = Callable[[str], Coroutine[Any, None, None]]
@ -147,13 +109,12 @@ class ConformanceOAuthCallbackHandler:
def __init__(self) -> None: def __init__(self) -> None:
self._auth_code: str | None = None self._auth_code: str | None = None
self._state: str | None = None self._state: str | None = None
self._iss: str | None = None
async def handle_redirect(self, authorization_url: str) -> None: async def handle_redirect(self, authorization_url: str) -> None:
"""Fetch the authorization URL and extract the auth code from the redirect.""" """Fetch the authorization URL and extract the auth code from the redirect."""
logger.debug(f"Fetching authorization URL: {authorization_url}") logger.debug(f"Fetching authorization URL: {authorization_url}")
async with httpx2.AsyncClient() as client: async with httpx.AsyncClient() as client:
response = await client.get( response = await client.get(
authorization_url, authorization_url,
follow_redirects=False, follow_redirects=False,
@ -169,8 +130,6 @@ class ConformanceOAuthCallbackHandler:
self._auth_code = query_params["code"][0] self._auth_code = query_params["code"][0]
state_values = query_params.get("state") state_values = query_params.get("state")
self._state = state_values[0] if state_values else None self._state = state_values[0] if state_values else None
iss_values = query_params.get("iss")
self._iss = iss_values[0] if iss_values else None
logger.debug(f"Got auth code from redirect: {self._auth_code[:10]}...") logger.debug(f"Got auth code from redirect: {self._auth_code[:10]}...")
return return
else: else:
@ -180,37 +139,55 @@ class ConformanceOAuthCallbackHandler:
else: else:
raise RuntimeError(f"Expected redirect response, got {response.status_code} from {authorization_url}") raise RuntimeError(f"Expected redirect response, got {response.status_code} from {authorization_url}")
async def handle_callback(self) -> AuthorizationCodeResult: async def handle_callback(self) -> tuple[str, str | None]:
"""Return the captured auth code, state, and iss.""" """Return the captured auth code and state."""
if self._auth_code is None: if self._auth_code is None:
raise RuntimeError("No authorization code available - was handle_redirect called?") raise RuntimeError("No authorization code available - was handle_redirect called?")
result = AuthorizationCodeResult(code=self._auth_code, state=self._state, iss=self._iss) auth_code = self._auth_code
state = self._state
self._auth_code = None self._auth_code = None
self._state = None self._state = None
self._iss = None return auth_code, state
return result
# --- Stub callbacks (declare capabilities in _meta without doing real work) --- # --- Scenario Handlers ---
async def stub_sampling_callback( @register("initialize")
context: ClientRequestContext, async def run_initialize(server_url: str) -> None:
params: types.CreateMessageRequestParams, """Connect, initialize, list tools, close."""
) -> types.CreateMessageResult | types.ErrorData: async with streamable_http_client(url=server_url) as (read_stream, write_stream, _):
return types.CreateMessageResult( async with ClientSession(read_stream, write_stream) as session:
role="assistant", await session.initialize()
content=types.TextContent(type="text", text=""), logger.debug("Initialized successfully")
model="conformance-stub", await session.list_tools()
) logger.debug("Listed tools successfully")
async def stub_list_roots_callback(context: ClientRequestContext) -> types.ListRootsResult | types.ErrorData: @register("tools_call")
return types.ListRootsResult(roots=[]) async def run_tools_call(server_url: str) -> None:
"""Connect, initialize, list tools, call add_numbers(a=5, b=3), close."""
async with streamable_http_client(url=server_url) as (read_stream, write_stream, _):
async with ClientSession(read_stream, write_stream) as session:
await session.initialize()
await session.list_tools()
result = await session.call_tool("add_numbers", {"a": 5, "b": 3})
logger.debug(f"add_numbers result: {result}")
@register("sse-retry")
async def run_sse_retry(server_url: str) -> None:
"""Connect, initialize, list tools, call test_reconnection, close."""
async with streamable_http_client(url=server_url) as (read_stream, write_stream, _):
async with ClientSession(read_stream, write_stream) as session:
await session.initialize()
await session.list_tools()
result = await session.call_tool("test_reconnection", {})
logger.debug(f"test_reconnection result: {result}")
async def default_elicitation_callback( async def default_elicitation_callback(
context: ClientRequestContext, context: RequestContext["ClientSession", Any],
params: types.ElicitRequestParams, params: types.ElicitRequestParams,
) -> types.ElicitResult | types.ErrorData: ) -> types.ElicitResult | types.ErrorData:
"""Accept elicitation and apply defaults from the schema (SEP-1034).""" """Accept elicitation and apply defaults from the schema (SEP-1034)."""
@ -218,7 +195,7 @@ async def default_elicitation_callback(
# For form mode, extract defaults from the requested_schema # For form mode, extract defaults from the requested_schema
if isinstance(params, types.ElicitRequestFormParams): if isinstance(params, types.ElicitRequestFormParams):
schema = params.requested_schema schema = params.requestedSchema
logger.debug(f"Elicitation schema: {schema}") logger.debug(f"Elicitation schema: {schema}")
properties = schema.get("properties", {}) properties = schema.get("properties", {})
for prop_name, prop_schema in properties.items(): for prop_name, prop_schema in properties.items():
@ -229,211 +206,19 @@ async def default_elicitation_callback(
return types.ElicitResult(action="accept", content=content) return types.ElicitResult(action="accept", content=content)
# --- Scenario Handlers ---
@register("initialize")
async def run_initialize(server_url: str) -> None:
"""Connect, initialize, list tools, close."""
async with Client(server_url, mode=client_mode()) as client:
logger.debug("Initialized successfully")
await client.list_tools()
logger.debug("Listed tools successfully")
@register("json-schema-ref-no-deref")
async def run_json_schema_ref_no_deref(server_url: str) -> None:
"""Initialize and list tools; the scenario fails only if the client fetches a network $ref.
The client never walks inputSchema or resolves $refs, so listing is enough (SEP-2106).
Pinned to mode='legacy': the harness reports PROTOCOL_VERSION=2026-07-28 for this
scenario but its mock server only speaks the handshake-era lifecycle and 400s a
modern-stamped tools/list. The check is lifecycle-agnostic so this is harmless.
"""
async with Client(server_url, mode="legacy") as client:
await client.list_tools()
@register("json-schema-2020-12-preservation")
async def run_json_schema_2020_12_preservation(server_url: str) -> None:
"""List tools, then echo the focal tool's inputSchema back verbatim (SEP-1613 / SEP-2106).
The harness diffs what the client round-trips through `json_schema_echo` against its
fixture to detect 2020-12 keywords ($schema, $defs, $anchor, additionalProperties,
allOf/anyOf, if/then/else) being stripped while parsing tools/list. Unlike
json-schema-ref-no-deref, this mock is version-aware, so client_mode() applies.
"""
async with Client(server_url, mode=client_mode()) as client:
listed = await client.list_tools()
focal = next(tool for tool in listed.tools if tool.name == "json_schema_2020_12_tool")
await client.call_tool("json_schema_echo", {"schema": focal.input_schema})
@register("tools_call")
async def run_tools_call(server_url: str) -> None:
"""Connect, list tools, call add_numbers(a=5, b=3), close."""
async with Client(server_url, mode=client_mode()) as client:
await client.list_tools()
result = await client.call_tool("add_numbers", {"a": 5, "b": 3})
logger.debug(f"add_numbers result: {result}")
@register("sse-retry")
async def run_sse_retry(server_url: str) -> None:
"""Connect, list tools, call test_reconnection, close."""
async with Client(server_url, mode=client_mode()) as client:
await client.list_tools()
result = await client.call_tool("test_reconnection", {})
logger.debug(f"test_reconnection result: {result}")
@register("request-metadata")
async def run_request_metadata(server_url: str) -> None:
"""Connect on the modern path with every client capability declared.
The scenario inspects every request's `_meta` envelope (SEP-2575) for
protocolVersion / clientInfo / clientCapabilities, and the matching
MCP-Protocol-Version header. mode='auto' makes the SDK send
server/discover (covering the unsupported-version retry check), then adopt
and stamp the envelope on the follow-up requests.
"""
async with Client(
server_url,
mode=client_mode(),
sampling_callback=stub_sampling_callback,
list_roots_callback=stub_list_roots_callback,
elicitation_callback=default_elicitation_callback,
) as client:
await client.list_tools()
result = await client.call_tool("add_numbers", {"a": 5, "b": 3})
logger.debug(f"add_numbers result: {result}")
@register("http-standard-headers")
async def run_http_standard_headers(server_url: str) -> None:
"""Touch tools, resources and prompts on the modern path so each standard header is checked (SEP-2243).
The scenario inspects Mcp-Method on each request and Mcp-Name on tools/call,
resources/read and prompts/get, and reports methods the client never sent as
SKIPPED rather than failed, so exercise one of each. initialize and
notifications/initialized stay SKIPPED: the modern path discovers via
server/discover and never sends them.
"""
async with Client(server_url, mode=client_mode()) as client:
tools = await client.list_tools()
if tools.tools:
await client.call_tool(tools.tools[0].name, _stub_required_args(tools.tools[0].input_schema))
resources = await client.list_resources()
if resources.resources:
await client.read_resource(resources.resources[0].uri)
prompts = await client.list_prompts()
if prompts.prompts:
await client.get_prompt(prompts.prompts[0].name)
def _stub_required_args(input_schema: dict[str, Any]) -> dict[str, Any]:
"""Minimal arguments satisfying a tool inputSchema's required list."""
by_type: dict[str, Any] = {
"string": "x",
"integer": 0,
"number": 0,
"boolean": False,
"object": {},
"array": [],
"null": None,
}
properties = input_schema.get("properties", {})
return {name: by_type.get(properties.get(name, {}).get("type"), "x") for name in input_schema.get("required", [])}
@register("http-invalid-tool-headers")
async def run_http_invalid_tool_headers(server_url: str) -> None:
"""List tools, then call every tool the SDK surfaces (SEP-2243).
The harness mock advertises one valid tool plus several with malformed
x-mcp-header annotations (empty, non-primitive type, duplicate, invalid
chars). The scenario passes if valid_tool is called and the malformed
ones are not -- so a conforming client filters them out of the list_tools
result and the loop below never sees them. The scenario sets
allowClientError, so a per-call failure is logged and skipped rather
than aborting the whole run.
"""
async with Client(server_url, mode=client_mode()) as client:
listed = await client.list_tools()
logger.debug(f"Surfaced tools: {[t.name for t in listed.tools]}")
for tool in listed.tools:
try:
await client.call_tool(tool.name, _stub_required_args(tool.input_schema))
except Exception:
logger.exception(f"call_tool({tool.name!r}) failed")
@register("http-custom-headers")
async def run_http_custom_headers(server_url: str) -> None:
"""List tools, then replay the harness's `toolCalls` so x-mcp-header args mirror into headers (SEP-2243).
The scenario supplies the exact arguments to send (including the null/edge-case values that
exercise omission and Base64 encoding) via the context `toolCalls`; using them verbatim is
what drives every per-parameter check. `list_tools` first so the SDK caches each tool's
annotations; a tool the SDK dropped (invalid annotations) is skipped. Per-call failures are
logged and skipped rather than aborting the run.
"""
tool_calls: list[dict[str, Any]] = []
if os.environ.get("MCP_CONFORMANCE_CONTEXT"):
tool_calls = get_conformance_context().get("toolCalls", [])
async with Client(server_url, mode=client_mode()) as client:
listed = await client.list_tools()
surfaced = {tool.name for tool in listed.tools}
logger.debug(f"Surfaced tools: {sorted(surfaced)}")
for call in tool_calls:
name = call["name"]
if name not in surfaced:
logger.debug(f"skipping {name!r}: not surfaced by list_tools")
continue
try:
await client.call_tool(name, call.get("arguments") or {})
except Exception:
logger.exception(f"call_tool({name!r}) failed")
@register("elicitation-sep1034-client-defaults") @register("elicitation-sep1034-client-defaults")
async def run_elicitation_defaults(server_url: str) -> None: async def run_elicitation_defaults(server_url: str) -> None:
"""Connect with elicitation callback that applies schema defaults.""" """Connect with elicitation callback that applies schema defaults."""
async with Client(server_url, mode=client_mode(), elicitation_callback=default_elicitation_callback) as client: async with streamable_http_client(url=server_url) as (read_stream, write_stream, _):
await client.list_tools() async with ClientSession(
result = await client.call_tool("test_client_elicitation_defaults", {}) read_stream, write_stream, elicitation_callback=default_elicitation_callback
) as session:
await session.initialize()
await session.list_tools()
result = await session.call_tool("test_client_elicitation_defaults", {})
logger.debug(f"test_client_elicitation_defaults result: {result}") logger.debug(f"test_client_elicitation_defaults result: {result}")
@register("sep-2322-client-request-state")
async def run_mrtr_client(server_url: str) -> None:
"""Drive the SEP-2322 client mock through `Client.call_tool`'s auto-loop.
The mock inspects raw `tools/call` params, so registering an
`elicitation_callback` and letting the driver run is enough to satisfy
all five wire-shape checks: the driver echoes `request_state` byte-exact
and omits it when the server sent none, every retry mints a fresh
JSON-RPC id, the unrelated call between auto-loops carries no MRTR
params, and the no-`resultType` response parses as a terminal
`CallToolResult` so the driver never retries it.
"""
async def confirm(
context: ClientRequestContext, params: types.ElicitRequestParams
) -> types.ElicitResult | types.ErrorData:
return types.ElicitResult(action="accept", content={"confirmed": True})
async with Client(server_url, mode=client_mode(), elicitation_callback=confirm) as client:
await client.list_tools()
await client.call_tool("test_mrtr_echo_state", {})
await client.call_tool("test_mrtr_unrelated", {})
await client.call_tool("test_mrtr_no_state", {})
result = await client.call_tool("test_mrtr_no_result_type", {})
assert isinstance(result, types.CallToolResult)
@register("auth/client-credentials-jwt") @register("auth/client-credentials-jwt")
async def run_client_credentials_jwt(server_url: str) -> None: async def run_client_credentials_jwt(server_url: str) -> None:
"""Client credentials flow with private_key_jwt authentication.""" """Client credentials flow with private_key_jwt authentication."""
@ -487,70 +272,6 @@ async def run_client_credentials_basic(server_url: str) -> None:
await _run_auth_session(server_url, oauth_auth) await _run_auth_session(server_url, oauth_auth)
@register("auth/enterprise-managed-authorization")
async def run_enterprise_managed_authorization(server_url: str) -> None:
"""SEP-990 enterprise-managed authorization: RFC 8693 token-exchange at the
enterprise IdP for an ID-JAG, then RFC 7523 jwt-bearer at the MCP
authorization server."""
context = get_conformance_context()
client_id = context.get("client_id")
client_secret = context.get("client_secret")
idp_client_id = context.get("idp_client_id")
idp_id_token = context.get("idp_id_token")
idp_token_endpoint = context.get("idp_token_endpoint")
if not client_id:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'client_id'")
if not client_secret:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'client_secret'")
if not idp_client_id:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'idp_client_id'")
if not idp_id_token:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'idp_id_token'")
if not idp_token_endpoint:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'idp_token_endpoint'")
# IdentityAssertionOAuthProvider takes the AS issuer as configuration (the
# SEP-990 trust model: the resource server is never asked which AS to use).
# The harness does not put the issuer in context, so for conformance we
# learn it from the harness's PRM document (RFC 9728); production
# deployments would supply it as static configuration instead.
prm_url = build_protected_resource_metadata_discovery_urls(None, server_url)[0]
async with httpx2.AsyncClient(timeout=30.0) as http:
prm = (await http.get(prm_url)).raise_for_status().json()
as_issuer = prm["authorization_servers"][0]
async def fetch_id_jag(audience: str, resource: str) -> str:
"""Leg 1 - RFC 8693 token-exchange at the enterprise IdP."""
async with httpx2.AsyncClient(timeout=30.0) as http:
resp = await http.post(
idp_token_endpoint,
data={
"grant_type": "urn:ietf:params:oauth:grant-type:token-exchange",
"requested_token_type": "urn:ietf:params:oauth:token-type:id-jag",
"subject_token": idp_id_token,
"subject_token_type": "urn:ietf:params:oauth:token-type:id_token",
"audience": audience,
"resource": resource,
"client_id": idp_client_id,
},
)
resp.raise_for_status()
return resp.json()["access_token"]
oauth_auth = IdentityAssertionOAuthProvider(
server_url=server_url,
storage=InMemoryTokenStorage(),
client_id=client_id,
client_secret=client_secret,
issuer=as_issuer,
assertion_provider=fetch_id_jag,
token_endpoint_auth_method="client_secret_basic",
)
await _run_auth_session(server_url, oauth_auth)
async def run_auth_code_client(server_url: str) -> None: async def run_auth_code_client(server_url: str) -> None:
"""Authorization code flow (default for auth/* scenarios).""" """Authorization code flow (default for auth/* scenarios)."""
callback_handler = ConformanceOAuthCallbackHandler() callback_handler = ConformanceOAuthCallbackHandler()
@ -593,21 +314,24 @@ async def run_auth_code_client(server_url: str) -> None:
await _run_auth_session(server_url, oauth_auth) await _run_auth_session(server_url, oauth_auth)
async def _run_auth_session(server_url: str, oauth_auth: httpx2.Auth) -> None: async def _run_auth_session(server_url: str, oauth_auth: OAuthClientProvider) -> None:
"""Common session logic for all OAuth flows.""" """Common session logic for all OAuth flows."""
http_client = httpx2.AsyncClient(auth=oauth_auth, timeout=30.0) client = httpx.AsyncClient(auth=oauth_auth, timeout=30.0)
transport = streamable_http_client(url=server_url, http_client=http_client) async with streamable_http_client(url=server_url, http_client=client) as (read_stream, write_stream, _):
async with Client(transport, mode=client_mode(), elicitation_callback=default_elicitation_callback) as client: async with ClientSession(
read_stream, write_stream, elicitation_callback=default_elicitation_callback
) as session:
await session.initialize()
logger.debug("Initialized successfully") logger.debug("Initialized successfully")
tools_result = await client.list_tools() tools_result = await session.list_tools()
logger.debug(f"Listed tools: {[t.name for t in tools_result.tools]}") logger.debug(f"Listed tools: {[t.name for t in tools_result.tools]}")
# Call the first available tool (different tests have different tools) # Call the first available tool (different tests have different tools)
if tools_result.tools: if tools_result.tools:
tool_name = tools_result.tools[0].name tool_name = tools_result.tools[0].name
try: try:
result = await client.call_tool(tool_name, {}) result = await session.call_tool(tool_name, {})
logger.debug(f"Called {tool_name}, result: {result}") logger.debug(f"Called {tool_name}, result: {result}")
except Exception as e: except Exception as e:
logger.debug(f"Tool call result/error: {e}") logger.debug(f"Tool call result/error: {e}")
@ -623,7 +347,6 @@ def main() -> None:
server_url = sys.argv[1] server_url = sys.argv[1]
scenario = os.environ.get("MCP_CONFORMANCE_SCENARIO") scenario = os.environ.get("MCP_CONFORMANCE_SCENARIO")
logger.debug(f"Conformance protocol version: {PROTOCOL_VERSION!r} -> mode={client_mode()!r}")
if scenario: if scenario:
logger.debug(f"Running explicit scenario '{scenario}' against {server_url}") logger.debug(f"Running explicit scenario '{scenario}' against {server_url}")

View file

@ -1,26 +0,0 @@
# Expected failures at the 2025-11-25 wire, read by the `--suite all
# --spec-version 2025-11-25` server and client legs: every scenario the pinned
# harness marks applicable at 2025-11-25, run over the stateful handshake.
#
# That selection is a superset of the frozen 2025-11-25 requirement set (`npx
# $CONFORMANCE_PKG list --requirements 2025-11-25`), so an entry here for a
# scenario in that set means `conformance tier-check` scores the SDK below 100%
# for 2025-11-25 even though CI is green; link the tracking issue next to any
# such entry.
#
# Kept separate from expected-failures.yml even though the bare `--suite all`
# legs currently run these same scenarios at 2025-11-25 too: the harness's
# default wire is due to move (modelcontextprotocol/conformance#426) and this
# file must keep meaning 2025-11-25. Until then a failure here also needs an
# entry there. The file is passed even while empty: run-client.sh keys its solo
# re-verification off the harness's baseline output.
#
# Baseline established against the harness pinned via CONFORMANCE_PKG in
# .github/workflows/conformance.yml. New conformance releases are adopted by
# deliberately bumping that pin and reconciling all three expected-failures
# files in the same change. Entries MUST be removed as they start passing (the
# runner fails on stale entries).
client: []
server: []

View file

@ -1,29 +0,0 @@
# Expected failures at the 2026-07-28 wire, read by the `--suite all
# --spec-version 2026-07-28` server and client legs: every scenario the pinned
# harness marks applicable at 2026-07-28, run stateless with per-request _meta.
#
# That selection is a superset of the frozen 2026-07-28 requirement set (`npx
# $CONFORMANCE_PKG list --requirements 2026-07-28`), so an entry here for a
# scenario in that set means `conformance tier-check` scores the SDK below 100%
# for 2026-07-28 even though CI is green; link the tracking issue next to any
# such entry.
#
# This baseline is separate from expected-failures.yml because entries are
# keyed by scenario name only: a scenario that passes at the harness default
# wire in the bare `--suite all` legs but fails when forced to 2026-07-28 (or
# vice versa) cannot be expressed in a shared file. Where the bare legs already
# default a scenario to 2026-07-28, a failure here also needs an entry there.
#
# Baseline established against the harness pinned via CONFORMANCE_PKG in
# .github/workflows/conformance.yml. New conformance releases are adopted by
# deliberately bumping that pin and reconciling all three expected-failures
# files in the same change.
#
# Entries are grouped by what unblocks them. As each gap closes the
# corresponding scenarios start passing and MUST be removed from this list
# (the runner fails on stale entries), so the baseline burns down per
# milestone.
client: []
server: []

View file

@ -1,55 +1,4 @@
# Conformance scenarios not yet passing against the Python SDK on main, read by # Known conformance test failures for v1.x
# the bare `--suite all` server and client legs (every scenario shipped with the # These are tracked and should be removed as they're fixed.
# pinned harness, each at the harness's default wire for it). CI exits 0 if only server: []
# these fail, exits 1 on unexpected failures or stale entries. client: []
#
# The wire-pinned legs read expected-failures.2025-11-25.yml and
# expected-failures.2026-07-28.yml instead; see the header of
# .github/workflows/conformance.yml for the leg layout.
#
# Baseline established against the harness pinned via CONFORMANCE_PKG in
# .github/workflows/conformance.yml. New conformance releases are adopted by
# deliberately bumping that pin and reconciling all three expected-failures
# files in the same change.
#
# Entries are grouped by SEP. As each SEP lands in the SDK the corresponding
# scenarios start passing and MUST be removed from this list (the runner fails
# on stale entries), so the baseline burns down per milestone.
client:
# SEP-1932 (DPoP): the SDK's OAuth client does not implement DPoP proofs.
# The entries are per-check (conformance #406) because both scenarios
# pass their non-DPoP checks (discovery, token acquisition, request
# flow) live.
- auth/dpop:sep-1932-client-token-request-proof
- auth/dpop:sep-1932-client-dpop-auth-scheme
- auth/dpop:sep-1932-client-fresh-proof
- auth/dpop-nonce:sep-1932-client-token-request-proof
- auth/dpop-nonce:sep-1932-client-dpop-auth-scheme
- auth/dpop-nonce:sep-1932-client-fresh-proof
- auth/dpop-nonce:sep-1932-client-as-nonce
- auth/dpop-nonce:sep-1932-client-rs-nonce
# Workload identity federation: the OAuth client does not implement the
# urn:ietf:params:oauth:grant-type:jwt-bearer grant (it answers with
# authorization_code). Per-check for the same reason.
- auth/wif-jwt-bearer:wif-grant-type
server:
# SEP-2663 (io.modelcontextprotocol/tasks): the SDK does not implement the
# tasks extension yet. The bare `--suite all` leg is the only one that
# reaches these extension-tagged scenarios (they never match a
# --spec-version filter), so these entries live here only.
#
# `tasks-status-notifications` is intentionally NOT listed: the harness
# skips it unconditionally (pending its rewrite against subscriptions/
# listen), and a baseline entry for a scenario with no failing checks is
# flagged stale.
- tasks-lifecycle
- tasks-capability-negotiation
- tasks-wire-fields
- tasks-request-state-removal
- tasks-mrtr-input
- tasks-request-headers
- tasks-dispatch-and-envelope
- tasks-required-task-error
- tasks-mrtr-composition

View file

@ -1,108 +0,0 @@
#!/bin/bash
# Run a client conformance suite, re-verifying unexpected failures solo.
# Concurrent suite runs on a hosted runner can push scenarios with real-time
# waits past tolerance; solo, a real failure fails again while a contention
# artifact passes. Failures that only reproduce under concurrency are excused.
set -uo pipefail
: "${CONFORMANCE_PKG:?set CONFORMANCE_PKG (pinned in .github/workflows/conformance.yml)}"
# One attempt: a solo failure on the quiet runner disproves the contention
# hypothesis; a second try would be the blind retry this script avoids.
SOLO_ATTEMPTS="${CONFORMANCE_SOLO_ATTEMPTS:-1}"
# Relative args resolve from the repo root; same contract as run-server.sh.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR/../../.." || exit 1
log="$(mktemp)"
trap 'rm -f "$log"' EXIT
npx --yes "$CONFORMANCE_PKG" client "$@" 2>&1 | tee "$log"
rc=${PIPESTATUS[0]}
if [ "$rc" -eq 0 ]; then
exit 0
fi
plain="$(sed 's/\x1b\[[0-9;]*m//g' "$log")"
# If the harness's summary wording changes, the list comes up empty and the
# original exit code passes through - never a false green.
mapfile -t scenarios < <(
printf '%s\n' "$plain" |
sed -n '/^Unexpected failures (not in baseline):$/,/^$/p' |
sed -n 's/^ ✗ //p'
)
if [ "${#scenarios[@]}" -eq 0 ]; then
exit "$rc"
fi
for scenario in "${scenarios[@]}"; do
if ! [[ "$scenario" =~ ^[A-Za-z0-9/_-]+$ ]]; then
echo "Extracted unexpected-failure name '${scenario}' does not look like a scenario name; passing the suite failure through." >&2
exit "$rc"
fi
done
# A stale baseline entry is a configuration error a solo rerun cannot excuse.
# Here-string, not a pipe: grep -q quitting early would SIGPIPE printf and,
# under pipefail, skip this guard exactly when the pattern is present.
if grep -q '^Stale baseline entries' <<<"$plain"; then
echo "Suite also reported stale baseline entries; not retrying." >&2
exit "$rc"
fi
# Drop the suite-only flags: --scenario replaces --suite, and solo runs are
# judged directly rather than against the baseline.
rerun_args=()
output_dir=""
skip_next=0
expect_output_dir=0
for arg in "$@"; do
if [ "$skip_next" -eq 1 ]; then
if [ "$expect_output_dir" -eq 1 ]; then
output_dir="$arg"
fi
skip_next=0
expect_output_dir=0
continue
fi
case "$arg" in
--output-dir)
skip_next=1
expect_output_dir=1
;;
--suite | --expected-failures) skip_next=1 ;;
--output-dir=*) output_dir="${arg#--output-dir=}" ;;
--suite=* | --expected-failures=*) ;;
*) rerun_args+=("$arg") ;;
esac
done
if [ -n "$output_dir" ]; then
rerun_args+=(--output-dir "${output_dir}-solo")
fi
for scenario in "${scenarios[@]}"; do
passed=0
for attempt in $(seq 1 "$SOLO_ATTEMPTS"); do
echo ""
echo "Re-running '${scenario}' solo (attempt ${attempt}/${SOLO_ATTEMPTS})..."
if npx --yes "$CONFORMANCE_PKG" client --scenario "$scenario" "${rerun_args[@]}"; then
passed=1
break
fi
done
if [ "$passed" -ne 1 ]; then
echo "'${scenario}' still fails when run alone: real failure, not suite contention." >&2
exit 1
fi
done
if [ -n "$output_dir" ]; then
mkdir -p "$output_dir"
printf '%s\n' "${scenarios[@]}" > "$output_dir/FLAKE_RESCUED"
fi
echo "All ${#scenarios[@]} unexpected failure(s) passed when re-run solo; the suite failures were parallel-run contention."
# Surface the rescue on the run summary; the log line above and the
# FLAKE_RESCUED marker in the uploaded results are otherwise easy to miss on a
# green run.
echo "::warning title=Conformance flake rescued::${scenarios[*]} failed under suite concurrency and passed solo${output_dir:+ (see ${output_dir}-solo and FLAKE_RESCUED in the uploaded results)}."
exit 0

View file

@ -7,36 +7,15 @@ SERVER_URL="http://localhost:${PORT}/mcp"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR/../../.." cd "$SCRIPT_DIR/../../.."
# Refuse to start if something is already listening on the port. The readiness # Start everything-server
# check below cannot tell our server apart from a stale one, so a leftover
# listener would mean silently running conformance against old code.
if (: > "/dev/tcp/localhost/${PORT}") 2>/dev/null; then
echo "Error: port ${PORT} is already in use." >&2
echo "Stop the stale process first (lsof -ti:${PORT} -sTCP:LISTEN | xargs kill) or set PORT to a free port." >&2
exit 1
fi
echo "Starting mcp-everything-server on port ${PORT}..."
uv run --frozen mcp-everything-server --port "$PORT" & uv run --frozen mcp-everything-server --port "$PORT" &
SERVER_PID=$! SERVER_PID=$!
trap "kill $SERVER_PID 2>/dev/null || true; wait $SERVER_PID 2>/dev/null || true" EXIT
cleanup() { # Wait for server to be ready
echo "Stopping server (PID: ${SERVER_PID})..."
kill $SERVER_PID 2>/dev/null || true
wait $SERVER_PID 2>/dev/null || true
}
trap cleanup EXIT
# Wait for server to be ready. --max-time keeps a hung listener from wedging
# the loop, and a dead server process fails fast instead of retrying.
echo "Waiting for server to be ready..."
MAX_RETRIES=30 MAX_RETRIES=30
RETRY_COUNT=0 RETRY_COUNT=0
while ! curl -s --max-time 2 "$SERVER_URL" > /dev/null 2>&1; do while ! curl -s "$SERVER_URL" > /dev/null 2>&1; do
if ! kill -0 $SERVER_PID 2>/dev/null; then
echo "Server process exited unexpectedly" >&2
exit 1
fi
RETRY_COUNT=$((RETRY_COUNT + 1)) RETRY_COUNT=$((RETRY_COUNT + 1))
if [ $RETRY_COUNT -ge $MAX_RETRIES ]; then if [ $RETRY_COUNT -ge $MAX_RETRIES ]; then
echo "Server failed to start after ${MAX_RETRIES} retries" >&2 echo "Server failed to start after ${MAX_RETRIES} retries" >&2
@ -47,5 +26,5 @@ done
echo "Server ready at $SERVER_URL" echo "Server ready at $SERVER_URL"
npx --yes "${CONFORMANCE_PKG:?set CONFORMANCE_PKG (pinned in .github/workflows/conformance.yml)}" \ # Run conformance tests
server --url "$SERVER_URL" "$@" npx @modelcontextprotocol/conformance@0.1.13 server --url "$SERVER_URL" "$@"

View file

@ -1,21 +1,9 @@
version: 2 version: 2
updates: updates:
- package-ecosystem: "uv"
directory: "/"
schedule:
interval: monthly
cooldown:
default-days: 14
groups:
python-packages:
patterns:
- "*"
- package-ecosystem: "github-actions" - package-ecosystem: "github-actions"
directory: "/" directory: "/"
schedule: schedule:
interval: monthly interval: monthly
cooldown:
default-days: 14
groups: groups:
github-actions: github-actions:
patterns: patterns:

View file

@ -1,38 +0,0 @@
<!--
Pull requests from outside the maintainer team need to link an open issue that
a maintainer has assigned to you (or one labeled `help wanted`); others are
closed automatically until that's in place. See CONTRIBUTING.md for details.
-->
Fixes #
<!-- Provide a brief summary of your changes -->
## Motivation and Context
<!-- Why is this change needed? What problem does it solve? -->
## How Has This Been Tested?
<!-- Have you tested this in a real application? Which scenarios were tested? -->
## Breaking Changes
<!-- Will users need to update their code or configurations? -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Documentation update
## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
- [ ] I am assigned to the linked issue (or it is labeled `help wanted`, or I'm a maintainer)
- [ ] I have disclosed any AI assistance and can explain the change in my own words
- [ ] I have read the [MCP Documentation](https://modelcontextprotocol.io)
- [ ] My code follows the repository's style guidelines
- [ ] New and existing tests pass locally
- [ ] I have added appropriate error handling
- [ ] I have added or updated documentation as needed
## Additional context
<!-- Add any other context, implementation notes, or design decisions -->

View file

@ -1,117 +0,0 @@
// Docs preview gate and PR comment. .github/workflows/docs-preview.yml wires
// these up: `authorize` decides whether a run may build and deploy a preview
// (and for which commit), `comment` reports the outcome on the pull request.
// The security model is described in the workflow's header.
'use strict';
const MARKER = '<!-- docs-preview -->';
const BOT_LOGIN = 'github-actions[bot]';
// Sets the job outputs `authorized`, `pr_number`, `head_sha` and
// `slash_attempt` for a pull_request_target or /preview-docs issue_comment run.
async function authorize({ github, context, core }) {
const { owner, repo } = context.repo;
async function permissionFor(username) {
const { data } = await github.rest.repos.getCollaboratorPermissionLevel({ owner, repo, username });
return { level: data.permission, role: data.role_name };
}
let authorized = false;
let prNumber = '';
let headSha = '';
let slashAttempt = false;
if (context.eventName === 'pull_request_target') {
const pr = context.payload.pull_request;
prNumber = String(pr.number);
headSha = pr.head.sha;
// No automatic preview for fork PRs: actions/checkout refuses to fetch a
// fork's head in a pull_request_target run. A maintainer can still request
// one with /preview-docs. (head.repo is null once the fork is deleted.)
const headRepo = pr.head.repo;
if (!headRepo || headRepo.id !== context.payload.repository.id) {
core.info(`PR #${prNumber} head is on ${headRepo ? headRepo.full_name : 'a deleted fork'}; fork PRs are previewed via /preview-docs only.`);
} else {
// Gate on the *sender* (whoever caused this run — on synchronize that
// is the pusher), not the PR author, so a non-admin pushing to an
// admin-opened branch does not get an automatic build.
const actor = context.payload.sender.login;
const perm = await permissionFor(actor);
authorized = perm.level === 'admin';
core.info(`pull_request_target by ${actor} (level=${perm.level}, role=${perm.role}) → authorized=${authorized}`);
}
} else {
// issue_comment: the job-level `if:` already guarantees this is a PR
// comment starting with /preview-docs.
slashAttempt = true;
const actor = context.payload.comment.user.login;
prNumber = String(context.payload.issue.number);
const perm = await permissionFor(actor);
authorized = perm.level === 'admin' || perm.role === 'maintain';
if (authorized) {
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: Number(prNumber) });
if (pr.state !== 'open') {
authorized = false;
core.info(`PR #${prNumber} is ${pr.state}; refusing to preview.`);
} else {
headSha = pr.head.sha;
}
}
core.info(`/preview-docs by ${actor} (level=${perm.level}, role=${perm.role}) → authorized=${authorized}`);
}
core.setOutput('authorized', String(authorized));
core.setOutput('pr_number', prNumber);
core.setOutput('head_sha', headSha);
core.setOutput('slash_attempt', String(slashAttempt));
}
// Posts or updates the preview comment on the PR. Reads the outcome of the
// earlier jobs from the step's env: AUTHORIZED, PR_NUMBER, HEAD_SHA,
// DEPLOY_RESULT, DEPLOYMENT_URL, ALIAS_URL, RUN_URL.
async function comment({ github, context }) {
const { owner, repo } = context.repo;
const env = process.env;
const issue_number = Number(env.PR_NUMBER);
async function upsert(body) {
const comments = await github.paginate(github.rest.issues.listComments, { owner, repo, issue_number, per_page: 100 });
const existing = comments.find((c) => c.user?.login === BOT_LOGIN && c.body?.includes(MARKER));
if (existing) {
await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body });
} else {
await github.rest.issues.createComment({ owner, repo, issue_number, body });
}
}
if (env.AUTHORIZED !== 'true') {
await github.rest.issues.createComment({
owner, repo, issue_number,
body: `@${context.actor} — only repository admins or maintainers can run \`/preview-docs\` (and the PR must be open).`,
});
return;
}
if (env.DEPLOY_RESULT !== 'success') {
await upsert(
`${MARKER}\n### 📚 Documentation preview\n\n` +
`❌ Preview build **failed** for \`${env.HEAD_SHA.slice(0, 7)}\` — [workflow logs](${env.RUN_URL}).`
);
return;
}
const previewUrl = env.ALIAS_URL || env.DEPLOYMENT_URL;
const ts = new Date().toISOString().replace('T', ' ').replace(/\.\d+Z$/, ' UTC');
await upsert(
`${MARKER}\n### 📚 Documentation preview\n\n` +
`| | |\n|---|---|\n` +
`| **Preview** | ${previewUrl} |\n` +
`| **Deployment** | ${env.DEPLOYMENT_URL} |\n` +
`| **Commit** | \`${env.HEAD_SHA.slice(0, 7)}\` |\n` +
`| **Triggered by** | @${context.actor} |\n` +
`| **Updated** | ${ts} |\n`
);
}
module.exports = { authorize, comment };

View file

@ -1,241 +0,0 @@
// Scenario tests for docs_preview.js: who gets a preview build (`authorize`)
// and what the pull request shows afterwards (`comment`).
//
// node --test .github/scripts/docs_preview.test.js
//
// No dependencies; the GitHub client is a small fake defined at the bottom.
// CI runs it in the checks job (.github/workflows/shared.yml).
'use strict';
const test = require('node:test');
const assert = require('node:assert/strict');
const { authorize, comment } = require('./docs_preview.js');
const REPO = { owner: 'modelcontextprotocol', repo: 'python-sdk' };
const BASE_REPO = { id: 1, full_name: 'modelcontextprotocol/python-sdk' };
const HEAD = 'e4dfda7baa127ab00ebcd1d5324560cbe3cdfe42';
const MARKER = '<!-- docs-preview -->';
// `permission` / `role_name` as the collaborators API reports them.
const PEOPLE = {
admin: { permission: 'admin', role_name: 'admin' },
maintainer: { permission: 'write', role_name: 'maintain' },
writer: { permission: 'write', role_name: 'write' },
outsider: { permission: 'read', role_name: 'read' },
};
// ── authorize ──────────────────────────────────────────────────────────────
// `expect` is the full set of job outputs the step writes.
const authorizeScenarios = [
{
name: 'admin pushes to (or opens) a PR → automatic preview of that head',
event: pushed(7, 'admin'),
expect: { authorized: 'true', pr_number: '7', head_sha: HEAD, slash_attempt: 'false' },
},
{
name: 'someone with write but not admin pushes → no automatic preview',
event: pushed(7, 'writer'),
expect: { authorized: 'false', pr_number: '7', head_sha: HEAD, slash_attempt: 'false' },
},
{
// actions/checkout refuses a fork's head under pull_request_target, so
// the run stops here instead of failing in `build`; /preview-docs still works.
name: 'admin pushes to or reopens a fork PR → no automatic preview',
event: pushed(7, 'admin', { fork: 'someone/python-sdk' }),
expect: { authorized: 'false', pr_number: '7', head_sha: HEAD, slash_attempt: 'false' },
permissionLookups: 0,
},
{
name: 'fork PR whose fork has since been deleted → no automatic preview',
event: pushed(7, 'admin', { fork: null }),
expect: { authorized: 'false', pr_number: '7', head_sha: HEAD, slash_attempt: 'false' },
permissionLookups: 0,
},
{
name: 'maintainer comments /preview-docs on a fork PR → previewed like any other',
pr: { fork: 'someone/python-sdk' },
event: slash(7, 'maintainer'),
expect: { authorized: 'true', pr_number: '7', head_sha: HEAD, slash_attempt: 'true' },
},
{
name: 'maintainer comments /preview-docs on an open PR → preview of its current head',
event: slash(7, 'maintainer'),
expect: { authorized: 'true', pr_number: '7', head_sha: HEAD, slash_attempt: 'true' },
},
{
name: 'admin comments /preview-docs → authorized as well',
event: slash(7, 'admin'),
expect: { authorized: 'true', pr_number: '7', head_sha: HEAD, slash_attempt: 'true' },
},
{
name: 'writer without the maintain role comments /preview-docs → refused, recorded as an attempt',
event: slash(7, 'writer'),
expect: { authorized: 'false', pr_number: '7', head_sha: '', slash_attempt: 'true' },
},
{
name: 'outsider comments /preview-docs → refused, recorded as an attempt',
event: slash(7, 'outsider'),
expect: { authorized: 'false', pr_number: '7', head_sha: '', slash_attempt: 'true' },
},
{
name: '/preview-docs on a closed PR → refused even for a maintainer',
pr: { state: 'closed' },
event: slash(7, 'maintainer'),
expect: { authorized: 'false', pr_number: '7', head_sha: '', slash_attempt: 'true' },
},
];
for (const s of authorizeScenarios) {
test(`authorize: ${s.name}`, async () => {
const world = makeWorld({ pr: { number: 7, ...s.pr } });
assert.deepEqual(await runAuthorize(world, s.event), s.expect);
assert.equal(world.writes.length, 0);
if (s.permissionLookups !== undefined) assert.equal(world.permissionLookups, s.permissionLookups);
});
}
test('authorize: a failing permission lookup fails the step instead of deciding either way', async () => {
const world = makeWorld({ pr: { number: 7 } });
world.failPermissionLookup = true;
await assert.rejects(runAuthorize(world, pushed(7, 'admin')), /boom/);
});
// ── comment ────────────────────────────────────────────────────────────────
const DEPLOYED = {
AUTHORIZED: 'true',
PR_NUMBER: '7',
HEAD_SHA: HEAD,
DEPLOY_RESULT: 'success',
DEPLOYMENT_URL: 'https://1a2b3c.mcp-python-sdk-docs.pages.dev',
ALIAS_URL: 'https://pr-7.mcp-python-sdk-docs.pages.dev',
RUN_URL: 'https://github.com/modelcontextprotocol/python-sdk/actions/runs/1',
};
test('comment: a refused /preview-docs gets a plain reply to the commenter, not a preview comment', async () => {
const world = makeWorld({ pr: { number: 7 } });
await runComment(world, { ...DEPLOYED, AUTHORIZED: 'false', HEAD_SHA: '', DEPLOY_RESULT: 'skipped' }, 'outsider');
assert.equal(world.comments.length, 1);
assert.match(world.comments[0].body, /^@outsider — only repository admins or maintainers can run `\/preview-docs`/);
assert.ok(!world.comments[0].body.includes(MARKER));
});
test('comment: first successful deploy posts one preview comment linking the alias URL and the commit', async () => {
const world = makeWorld({ pr: { number: 7 } });
await runComment(world, DEPLOYED, 'admin');
assert.equal(world.comments.length, 1);
const body = world.comments[0].body;
assert.ok(body.startsWith(`${MARKER}\n### 📚 Documentation preview`));
assert.match(body, /\| \*\*Preview\*\* \| https:\/\/pr-7\.mcp-python-sdk-docs\.pages\.dev \|/);
assert.match(body, /\| \*\*Deployment\*\* \| https:\/\/1a2b3c\.mcp-python-sdk-docs\.pages\.dev \|/);
assert.match(body, /\| \*\*Commit\*\* \| `e4dfda7` \|/);
assert.match(body, /\| \*\*Triggered by\*\* \| @admin \|/);
assert.deepEqual(world.writes, ['comment on #7']);
});
test('comment: a later deploy edits the existing preview comment instead of adding another', async () => {
const world = makeWorld({ pr: { number: 7 }, comments: [{ user: 'someone', body: 'LGTM' }, { user: 'github-actions[bot]', body: `${MARKER}\nold table` }] });
await runComment(world, { ...DEPLOYED, HEAD_SHA: 'f'.repeat(40) }, 'admin');
assert.equal(world.comments.length, 2);
assert.match(world.comments[1].body, /\| \*\*Commit\*\* \| `fffffff` \|/);
assert.deepEqual(world.writes, ['edit comment 101']);
});
test("comment: someone else's comment that happens to contain the marker is left alone", async () => {
const world = makeWorld({ pr: { number: 7 }, comments: [{ user: 'someone', body: `quoting ${MARKER} here` }] });
await runComment(world, DEPLOYED, 'admin');
assert.equal(world.comments.length, 2);
assert.equal(world.comments[0].body, `quoting ${MARKER} here`);
assert.deepEqual(world.writes, ['comment on #7']);
});
test('comment: with no alias URL the preview link falls back to the deployment URL', async () => {
const world = makeWorld({ pr: { number: 7 } });
await runComment(world, { ...DEPLOYED, ALIAS_URL: '' }, 'admin');
assert.match(world.comments[0].body, /\| \*\*Preview\*\* \| https:\/\/1a2b3c\.mcp-python-sdk-docs\.pages\.dev \|/);
});
test('comment: a build or deploy that did not succeed is reported with the short SHA and a link to the run', async () => {
const world = makeWorld({ pr: { number: 7 }, comments: [{ user: 'github-actions[bot]', body: `${MARKER}\nold table` }] });
await runComment(world, { ...DEPLOYED, DEPLOY_RESULT: 'skipped', DEPLOYMENT_URL: '', ALIAS_URL: '' }, 'admin');
assert.equal(world.comments.length, 1);
assert.equal(
world.comments[0].body,
`${MARKER}\n### 📚 Documentation preview\n\n❌ Preview build **failed** for \`e4dfda7\` — [workflow logs](${DEPLOYED.RUN_URL}).`
);
});
// ── Harness ────────────────────────────────────────────────────────────────
// `fork`: full name of the fork the head lives on; null for a deleted fork; omitted for a same-repo branch.
function pushed(number, sender, { fork } = {}) {
const repo = fork === undefined ? BASE_REPO : fork === null ? null : { id: 2, full_name: fork };
return { eventName: 'pull_request_target', actor: sender, payload: { action: 'synchronize', repository: BASE_REPO, pull_request: { number, head: { sha: HEAD, repo } }, sender: { login: sender } } };
}
function slash(number, commenter) {
return { eventName: 'issue_comment', actor: commenter, payload: { action: 'created', issue: { number, pull_request: {} }, comment: { body: '/preview-docs', user: { login: commenter } } } };
}
async function runAuthorize(world, event) {
const outputs = {};
const core = { info: () => {}, setOutput: (k, v) => { outputs[k] = v; } };
await authorize({ github: world.github, context: { repo: REPO, ...event }, core });
return outputs;
}
async function runComment(world, env, actor) {
const saved = {};
for (const [k, v] of Object.entries(env)) { saved[k] = process.env[k]; process.env[k] = v; }
try {
await comment({ github: world.github, context: { repo: REPO, actor }, core: {} });
} finally {
for (const [k, v] of Object.entries(saved)) { if (v === undefined) delete process.env[k]; else process.env[k] = v; }
}
}
// ── A tiny in-memory GitHub ────────────────────────────────────────────────
function makeWorld({ pr, comments = [] }) {
const world = { pr: { state: 'open', ...pr }, comments: [], writes: [], failPermissionLookup: false, permissionLookups: 0, nextCommentId: 100 };
for (const c of comments) world.comments.push({ id: world.nextCommentId++, ...c });
const err = (status, message = 'fake error') => Object.assign(new Error(message), { status });
const write = (what) => world.writes.push(what);
const checkPr = (n) => { if (n !== world.pr.number) throw err(404); };
const rest = {
repos: {
getCollaboratorPermissionLevel: async ({ username }) => {
world.permissionLookups++;
if (world.failPermissionLookup) throw err(500, 'boom');
const person = PEOPLE[username];
if (!person) throw err(404, 'not a user');
return { data: { ...person, user: { login: username } } };
},
},
pulls: {
get: async ({ pull_number }) => {
checkPr(pull_number);
const repo = world.pr.fork ? { id: 2, full_name: world.pr.fork } : BASE_REPO;
return { data: { number: pull_number, state: world.pr.state, head: { sha: HEAD, repo } } };
},
},
issues: {
listComments: async ({ issue_number }) => { checkPr(issue_number); return { data: world.comments.map((c) => ({ id: c.id, body: c.body, user: { login: c.user } })) }; },
createComment: async ({ issue_number, body }) => {
checkPr(issue_number);
write(`comment on #${issue_number}`);
world.comments.push({ id: world.nextCommentId++, user: 'github-actions[bot]', body });
},
updateComment: async ({ comment_id, body }) => {
write(`edit comment ${comment_id}`);
const c = world.comments.find((x) => x.id === comment_id);
if (!c) throw err(404);
c.body = body;
},
},
};
world.github = { rest, paginate: async (fn, args) => (await fn(args)).data };
return world;
}

View file

@ -1,312 +0,0 @@
// PR intake gate. The policy lives in CONTRIBUTING.md ("How pull requests get
// in"); .github/workflows/require-linked-issue.yml wires this up to events.
//
// A pull request from someone without triage rights stays open only if its
// description links (Fixes/Closes/Resolves #N) an open issue in this repo that
// is either assigned to the PR author or labeled `help wanted`. Otherwise the
// gate labels it `missing-issue-link`, leaves one comment, and closes it. It
// re-evaluates — and reopens — the PR when the description is edited or the
// author is assigned to the issue. A triage+ user reopening the PR, removing
// the label, or adding `bypass-issue-check` overrides it, and the override
// sticks.
//
// Everything that writes goes through mutate(); when the workflow passes
// ENFORCE=false (its kill switch) the run only logs what it would have done.
'use strict';
const LABEL = 'missing-issue-link'; // marks PRs the gate has closed
const BYPASS_LABEL = 'bypass-issue-check'; // sticky maintainer override
const OPEN_LABEL = 'help wanted'; // issue label that waives assignment
const MARKER = '<!-- require-linked-issue -->';
const BOT_LOGIN = 'github-actions[bot]';
const MAX_ISSUES = 5;
module.exports = async function run({ github, context, core }) {
const { owner, repo } = context.repo;
const enforce = process.env.ENFORCE === 'true';
const contributingUrl = `https://github.com/${owner}/${repo}/blob/main/CONTRIBUTING.md#how-pull-requests-get-in`;
// ── Entry points ─────────────────────────────────────────────────────────
if (context.eventName === 'issues') {
// Someone was assigned an issue: re-evaluate their gate-closed PRs that
// reference it (they may pass now).
const issueNumber = context.payload.issue.number;
const assignee = context.payload.assignee.login;
const closed = await github.paginate(github.rest.issues.listForRepo, {
owner, repo, state: 'closed', creator: assignee, labels: LABEL, per_page: 100,
});
const prs = closed.filter((i) => i.pull_request && closingRefs(i.body).includes(issueNumber));
console.log(`#${issueNumber} assigned to ${assignee}: ${prs.length} gate-closed PR(s) reference it`);
// Evaluate each independently so one transient failure doesn't strand
// the rest (this event won't fire again for the same assignment).
const failures = [];
for (const pr of prs) {
try {
await evaluate(pr.number, 'assigned', context.payload.sender?.login, issueNumber);
} catch (e) {
failures.push(`#${pr.number}: ${e.message}`);
}
}
if (failures.length) throw new Error(`Could not re-evaluate ${failures.join('; ')}`);
return;
}
if (context.eventName === 'workflow_dispatch') {
const n = parseInt(process.env.PR_NUMBER_INPUT, 10);
if (!Number.isInteger(n) || n <= 0) throw new Error(`Bad pr_number input: ${process.env.PR_NUMBER_INPUT}`);
await evaluate(n, 'dispatch', context.payload.sender?.login);
return;
}
await evaluate(context.payload.pull_request.number, context.payload.action, context.payload.sender?.login);
// ── The rules ────────────────────────────────────────────────────────────
async function evaluate(prNumber, action, sender, hintIssue = null) {
// Always read the PR live; the event payload can be stale by the time a
// queued run starts.
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber });
const labels = pr.labels.map((l) => l.name);
// An `unlabeled` run only fires for LABEL (see the workflow `if:`), so the
// event itself proves the label was there a moment ago.
const gated = action === 'unlabeled' || labels.includes(LABEL);
console.log(`PR #${prNumber} by ${pr.user.login} (${pr.state}${pr.draft ? ', draft' : ''}) — ${action} by ${sender ?? '-'}, enforce=${enforce}`);
// 0. Scope: open PRs, plus closed PRs the gate closed itself. Merged PRs
// and PRs someone closed for other reasons are left alone.
if (pr.merged_at) return log('merged — nothing to do');
if (pr.state === 'closed' && !gated) return log('closed by someone else — not ours');
// 1. Exempt authors: bots and anyone with triage or better. Drafts are
// gated like any other PR.
if (pr.user.type === 'Bot') return log('author is a bot — exempt');
if (await isTrusted(pr.user.login)) return pass('author has triage+ on this repo');
// 2. Overrides: a triage+ user reopening the PR or removing the label wants
// it open. Anyone else doing so just triggers a re-check.
if ((action === 'reopened' || action === 'unlabeled') && sender && (await isTrusted(sender))) {
return pass(`${sender} ${action === 'reopened' ? 'reopened it' : 'removed the label'} — override`, { sticky: true });
}
if (labels.includes(BYPASS_LABEL)) return pass(`carries ${BYPASS_LABEL}`);
// 3. The rule: the description links an open issue in this repo that is
// labeled `help wanted` or assigned to the author. Only the first few
// references are fetched; a just-assigned issue is checked first.
const author = pr.user.login.toLowerCase();
const refs = closingRefs(pr.body);
if (hintIssue && refs.includes(hintIssue)) refs.unshift(...refs.splice(refs.indexOf(hintIssue), 1));
const linked = [];
for (const num of refs.slice(0, MAX_ISSUES)) {
const issue = await getIssue(num);
if (!issue) continue; // missing, a PR, closed, or transferred away
linked.push(num);
if (issue.labels.some((l) => l.name.toLowerCase() === OPEN_LABEL)) return pass(`#${num} is labeled "${OPEN_LABEL}"`);
if (issue.assignees.some((a) => a.login.toLowerCase() === author)) return pass(`author is assigned to #${num}`);
}
return fail(linked);
// ── Outcomes ─────────────────────────────────────────────────────────
async function pass(reason, { sticky = false } = {}) {
console.log(`PASS: ${reason}`);
if (sticky) await addLabel(prNumber, BYPASS_LABEL);
if (pr.state === 'closed' && !(await reopen(pr, reason))) return;
if (gated) {
await removeLabel(prNumber, LABEL);
await deleteGateComment(prNumber);
}
}
async function fail(linkedIssues) {
console.log(`FAIL: ${linkedIssues.length ? `not assigned to ${linkedIssues.map((n) => `#${n}`).join(', ')}` : 'no usable issue link'}`);
await addLabel(prNumber, LABEL);
await upsertGateComment(prNumber, closedComment(pr.draft, linkedIssues));
if (pr.state === 'open') {
await mutate(`close PR #${prNumber}`, () => github.rest.pulls.update({ owner, repo, pull_number: prNumber, state: 'closed' }));
}
}
function log(msg) {
console.log(msg);
}
}
// ── Comment text ─────────────────────────────────────────────────────────
function closedComment(draft, linkedIssues) {
const issues = linkedIssues.map((n) => `#${n}`).join(', ');
const opener = draft
? "This PR has been closed automatically. It's still a draft, but we close those early so you don't put in more time only to have it closed the moment you mark it ready.\n\n"
: 'This PR has been closed automatically. ';
const rule = `${opener}This repo only keeps pull requests open when they come from a maintainer, or from a contributor a maintainer has assigned to the linked issue`;
const situation = linkedIssues.length
? [
`${rule}, and you aren't currently assigned to ${issues}.`,
'',
`If a maintainer assigns you to ${issues}, this PR reopens on its own and there's nothing more you need to do here. Assignment is a maintainer call based on capacity; comments that only ask to be assigned don't factor in. What does help is engaging on the issue itself by confirming the repro, explaining why it matters for your use case, or describing the approach you'd take.`,
]
: [
`${rule}, and this PR doesn't link an open issue yet.`,
'',
"- **If you're already assigned to an issue for this**, add `Fixes #<n>` to the description and the PR will reopen on its own.",
`- **If there's no issue yet**, please [open one](https://github.com/${owner}/${repo}/issues/new/choose) instead: what you ran into, why it matters for your use case, and a minimal reproduction. That context is super important to us and is what we use to decide what to prioritise.`,
"- **If there's an issue but you're not assigned**, add `Fixes #<n>` anyway so they're linked, then engage on the issue itself by confirming the repro or describing the approach you'd take. Assignment is a maintainer call based on capacity; comments that only ask to be assigned don't factor in. If you are assigned, this PR reopens automatically.",
];
return [
MARKER,
...situation,
'',
"You're welcome to keep pushing commits here (just avoid force-pushing, since GitHub can't reopen a rewritten branch), but that on its own won't get the PR reviewed or the issue assigned, and realistically most auto-closed PRs stay closed. There's no need to open a new PR either way.",
'',
`[CONTRIBUTING.md](${contributingUrl}) has the full reasoning, but in short:`,
'',
"- We're a small team with very little capacity to review community PRs right now.",
'- Many recent PRs are AI-generated with little human review, and reviewing one carefully still costs a maintainer as much time as it ever did. A well-described issue is usually more useful to us than the code.',
'',
`*Maintainers: reopen, remove \`${LABEL}\`, or add \`${BYPASS_LABEL}\` to override.*`,
].join('\n');
}
function cannotReopenComment(pr, reason) {
return [
MARKER,
`This PR now passes the intake check (${reason}), but GitHub won't let it be reopened — usually because the branch was force-pushed or deleted while the PR was closed, or because another open PR uses the same branch.`,
'',
`If you have another open PR from this branch, please continue there. Otherwise, either push the branch back to \`${pr.head.sha.slice(0, 7)}\` and edit this PR's description to retry, or open a new PR that links the same issue (if a maintainer had waved this one through, mention that so they can do the same there).`,
].join('\n');
}
// ── Helpers ──────────────────────────────────────────────────────────────
async function mutate(description, fn) {
if (!enforce) {
console.log(`[dry-run] would ${description}`);
return undefined;
}
return fn();
}
// Triage-or-better on this repo, from the permission endpoint's capability
// flags (role names can be custom; author_association hides private org
// members). Only a nonexistent user 404s; any other error must throw rather
// than be read as "untrusted", or a maintainer's PR could be closed.
async function isTrusted(username) {
try {
const { data } = await github.rest.repos.getCollaboratorPermissionLevel({ owner, repo, username });
const p = data.user?.permissions;
if (!p) throw new Error(`permission response for ${username} has no capability flags`);
const trusted = Boolean(p.triage || p.push || p.maintain || p.admin);
console.log(` ${username}: ${trusted ? 'trusted' : 'not trusted'} (role ${data.role_name || '-'})`);
return trusted;
} catch (e) {
if (e.status === 404) return false;
throw new Error(`Permission check failed for ${username} (HTTP ${e.status ?? '?'}): ${e.message}`);
}
}
// Issue numbers referenced with a closing keyword, in the forms GitHub itself
// honors: `Fixes #1`, `closes owner/repo#1`, `Resolved https://github.com/owner/repo/issues/1`.
function closingRefs(body) {
const repoRef = `${owner}/${repo}`.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const re = new RegExp(
`\\b(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\\s*:?\\s*(?:${repoRef}#|#|https?://github\\.com/${repoRef}/issues/)(\\d+)`,
'gi',
);
return [...new Set([...(body || '').matchAll(re)].map((m) => parseInt(m[1], 10)))];
}
// The linked issue, or null if it doesn't exist, is actually a PR, isn't
// open, or has been transferred to another repository.
async function getIssue(num) {
let issue;
try {
({ data: issue } = await github.rest.issues.get({ owner, repo, issue_number: num }));
} catch (e) {
if (e.status === 404 || e.status === 410) return null;
throw new Error(`Cannot fetch issue #${num} (HTTP ${e.status ?? '?'}): ${e.message}`);
}
if (issue.pull_request || issue.state !== 'open') return null;
if (!issue.repository_url?.endsWith(`/${owner}/${repo}`)) return null;
return issue;
}
// Reopen a gate-closed PR. GitHub refuses (422) if the branch was rewritten
// or deleted while closed, or another open PR uses it. Explain that in the
// comment and make sure the control label is (still) on, so the PR stays
// gate-managed and a later edit or override retries the reopen.
async function reopen(pr, reason) {
try {
await mutate(`reopen PR #${pr.number}`, () => github.rest.pulls.update({ owner, repo, pull_number: pr.number, state: 'open' }));
return true;
} catch (e) {
if (e.status !== 422) throw e;
core.warning(`GitHub refused to reopen PR #${pr.number}: ${e.message}`);
await addLabel(pr.number, LABEL);
await upsertGateComment(pr.number, cannotReopenComment(pr, reason));
return false;
}
}
async function addLabel(prNumber, name) {
await mutate(`add "${name}" to PR #${prNumber}`, async () => {
await ensureLabelExists(name);
await github.rest.issues.addLabels({ owner, repo, issue_number: prNumber, labels: [name] });
});
}
async function removeLabel(prNumber, name) {
await mutate(`remove "${name}" from PR #${prNumber}`, async () => {
try {
await github.rest.issues.removeLabel({ owner, repo, issue_number: prNumber, name });
} catch (e) {
if (e.status !== 404) throw e;
}
});
}
async function ensureLabelExists(name) {
const meta = {
[LABEL]: ['b76e79', 'Auto-closed: PR needs a linked issue assigned to its author (see CONTRIBUTING.md)'],
[BYPASS_LABEL]: ['0e8a16', 'Maintainer override for the linked-issue intake gate'],
}[name];
try {
await github.rest.issues.getLabel({ owner, repo, name });
} catch (e) {
if (e.status !== 404) throw e;
try {
await github.rest.issues.createLabel({ owner, repo, name, color: meta[0], description: meta[1] });
} catch (createErr) {
if (createErr.status !== 422) throw createErr; // created concurrently
}
}
}
// The gate keeps at most one comment per PR: authored by the Actions bot and
// carrying MARKER. It's created or updated on failure and deleted on pass.
async function findGateComment(prNumber) {
const comments = await github.paginate(github.rest.issues.listComments, { owner, repo, issue_number: prNumber, per_page: 100 });
return comments.find((c) => c.user?.login === BOT_LOGIN && c.body?.includes(MARKER));
}
async function upsertGateComment(prNumber, body) {
const existing = await findGateComment(prNumber);
if (!existing) {
await mutate(`comment on PR #${prNumber}`, () => github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body }));
} else if (existing.body !== body) {
await mutate(`update the gate comment on PR #${prNumber}`, () => github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body }));
}
}
async function deleteGateComment(prNumber) {
const existing = await findGateComment(prNumber);
if (!existing) return;
await mutate(`delete the gate comment on PR #${prNumber}`, async () => {
try {
await github.rest.issues.deleteComment({ owner, repo, comment_id: existing.id });
} catch (e) {
if (e.status !== 404) throw e; // already deleted by a concurrent run
}
});
}
};

View file

@ -1,393 +0,0 @@
// Scenario tests for pr_intake_gate.js. Each case sets up a tiny in-memory
// repository (pull requests, issues, people), fires one event at the gate, and
// checks what the gate left behind: PR state, labels, and its comment.
//
// node --test .github/scripts/pr_intake_gate.test.js
//
// No dependencies; the GitHub client is a small fake defined at the bottom.
// CI runs it in the checks job (.github/workflows/shared.yml).
'use strict';
const test = require('node:test');
const assert = require('node:assert/strict');
const gate = require('./pr_intake_gate.js');
const LABEL = 'missing-issue-link';
const BYPASS = 'bypass-issue-check';
const REPO = { owner: 'modelcontextprotocol', repo: 'python-sdk' };
// People. Only the capability flags matter to the gate.
const PEOPLE = {
maintainer: { type: 'User', perms: { admin: true, maintain: true, push: true, triage: true, pull: true } },
triager: { type: 'User', perms: { triage: true, pull: true } }, // e.g. a trusted-contributors team
outsider: { type: 'User', perms: { pull: true } },
'dependabot[bot]': { type: 'Bot', perms: {} },
'somebot[bot]': { type: 'Bot', perms: {} },
};
// ── Scenarios ──────────────────────────────────────────────────────────────
// `prs` / `issues` describe the world before the event; `expect` describes each
// PR afterwards: state, labels, and comment ('closed' = the "this PR has been
// closed" comment, 'closed-draft' = its draft wording, 'cannot-reopen' = the
// refused-reopen comment, null = none).
// `writes: 0` additionally asserts the gate touched nothing at all.
const scenarios = [
{
name: 'outsider opens a PR with no issue link → labeled, commented, closed',
prs: [pr(3300, 'outsider')],
event: opened(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'closed' } },
},
{
name: 'outsider links an open issue reported by someone else, not assigned → closed',
prs: [pr(3300, 'outsider', { body: 'Fixes #10' })],
issues: [issue(10)],
event: opened(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'closed' } },
},
{
name: 'outsider links an issue they are assigned to → stays open, untouched',
prs: [pr(3300, 'outsider', { body: 'closes #10' })],
issues: [issue(10, { assignees: ['outsider'] })],
event: opened(3300, 'outsider'),
expect: { 3300: { state: 'open', labels: [], comment: null } },
writes: 0,
},
{
name: 'outsider links a `help wanted` issue → stays open without assignment',
prs: [pr(3300, 'outsider', { body: 'Resolves modelcontextprotocol/python-sdk#10' })],
issues: [issue(10, { labels: ['help wanted'] })],
event: opened(3300, 'outsider'),
expect: { 3300: { state: 'open', labels: [], comment: null } },
writes: 0,
},
{
name: 'links only count when they use a closing keyword and point at an open issue here',
prs: [pr(3300, 'outsider', { body: 'Related to #10, fixes owner/other#10, prefixes #10, Fixes #11, Fixes #12' })],
issues: [
issue(10, { assignees: ['outsider'] }), // referenced, but never with a closing keyword for this repo
issue(11, { assignees: ['outsider'], state: 'closed' }), // closed issues don't count
issue(12, { assignees: ['outsider'], repository_url: 'https://api.github.com/repos/modelcontextprotocol/typescript-sdk' }), // transferred away
],
event: opened(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'closed' } },
},
{
name: 'gate-closed PR: author edits the description to link a qualifying issue → reopened, label and comment removed',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #10', gateComment: true })],
issues: [issue(10, { labels: ['help wanted'] })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'open', labels: [], comment: null } },
},
{
name: 'gate-closed PR: maintainer assigns the author on the linked issue → reopened',
prs: [
pr(3300, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #10', gateComment: true }),
pr(3301, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #99', gateComment: true }), // different issue: untouched
],
issues: [issue(10, { assignees: ['outsider'] }), issue(99)],
event: assigned(10, 'outsider', 'maintainer'),
expect: {
3300: { state: 'open', labels: [], comment: null },
3301: { state: 'closed', labels: [LABEL], comment: 'closed' },
},
},
{
name: 'gate-closed PR: maintainer reopens it → stays open with the sticky bypass label',
prs: [pr(3300, 'outsider', { state: 'open', labels: [LABEL], gateComment: true })], // payload arrives post-reopen
event: reopened(3300, 'maintainer'),
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null } },
},
{
name: 'gate-closed PR: triage-role user removes the label → reopened with the sticky bypass label',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [], gateComment: true })], // payload arrives post-unlabel
event: unlabeled(3300, 'triager'),
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null } },
},
{
name: 'gate-closed PR: some other bot strips the label → re-checked, label restored, still closed',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [], gateComment: true })],
event: unlabeled(3300, 'somebot[bot]'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'closed' } },
},
{
name: 'PR carrying the bypass label is never re-closed, even after edits that would fail',
prs: [pr(3300, 'outsider', { labels: [BYPASS], body: 'no link at all' })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null } },
writes: 0,
},
{
name: 'previously passing PR edited to drop its link → closed again',
prs: [pr(3300, 'outsider', { body: 'see #10' })],
issues: [issue(10, { assignees: ['outsider'] })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'closed' } },
},
{
name: "maintainer's own PR is exempt (and reopening it adds no labels)",
prs: [pr(3300, 'maintainer')],
event: reopened(3300, 'maintainer'),
expect: { 3300: { state: 'open', labels: [], comment: null } },
writes: 0,
},
{
name: 'triage-role author is exempt',
prs: [pr(3300, 'triager')],
event: opened(3300, 'triager'),
expect: { 3300: { state: 'open', labels: [], comment: null } },
writes: 0,
},
{
name: 'bot-authored PR (Dependabot) is exempt',
prs: [pr(3300, 'dependabot[bot]')],
event: opened(3300, 'dependabot[bot]'),
expect: { 3300: { state: 'open', labels: [], comment: null } },
writes: 0,
},
{
name: 'draft PR with no issue link is closed like any other, with a note about why drafts are closed early',
prs: [pr(3300, 'outsider', { draft: true })],
event: opened(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'closed-draft' } },
},
{
name: 'pre-existing draft marked ready for review with no link → closed',
prs: [pr(3300, 'outsider')],
event: readyForReview(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'closed' } },
},
{
name: 'PR a maintainer closed by hand (no gate label) is left alone when edited',
prs: [pr(3300, 'outsider', { state: 'closed' })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [], comment: null } },
writes: 0,
},
{
name: 'merged PR still carrying the label is left alone',
prs: [pr(3300, 'outsider', { state: 'closed', merged_at: '2026-08-15T00:00:00Z', labels: [LABEL] })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: null } },
writes: 0,
},
{
name: 'GitHub refuses to reopen (branch rewritten while closed) → stays closed, comment explains',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #10', gateComment: true, refuseReopen: true })],
issues: [issue(10, { assignees: ['outsider'] })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'cannot-reopen' } },
},
{
name: 'gate-closed PR: maintainer adds the bypass label → reopened, and the label sticks',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [LABEL, BYPASS], gateComment: true })], // payload arrives post-label
event: labeled(3300, 'maintainer', BYPASS),
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null } },
},
{
name: 'refused reopen after a label-removal override → both labels on, so the PR stays gate-managed',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [], gateComment: true, refuseReopen: true })],
event: unlabeled(3300, 'triager'),
expect: { 3300: { state: 'closed', labels: [BYPASS, LABEL], comment: 'cannot-reopen' } },
},
{
name: 'after a refused reopen, once the branch is restored an edit retries and reopens',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [BYPASS, LABEL], comments: [{ user: 'github-actions[bot]', body: "<!-- require-linked-issue -->\n…GitHub won't let it be reopened…" }] })],
event: edited(3300, 'outsider'),
expect: { 3300: { state: 'open', labels: [BYPASS], comment: null } },
},
{
name: 'assignment reopens the PR even when the assigned issue is referenced after several others',
prs: [pr(3300, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #1 fixes #2 fixes #3 fixes #4 fixes #5 fixes #6', gateComment: true })],
issues: [issue(1), issue(2), issue(3), issue(4), issue(5), issue(6, { assignees: ['outsider'] })],
event: assigned(6, 'outsider', 'maintainer'),
expect: { 3300: { state: 'open', labels: [], comment: null } },
},
{
name: 'a comment planted by the author with the marker is ignored; the gate posts its own',
prs: [pr(3300, 'outsider', { comments: [{ user: 'outsider', body: '<!-- require-linked-issue -->\nnice try' }] })],
event: opened(3300, 'outsider'),
expect: { 3300: { state: 'closed', labels: [LABEL], comment: 'closed', foreignComments: 1 } },
},
{
name: 'manual dispatch evaluates an old, never-gated PR like a new one',
prs: [pr(3150, 'outsider')],
event: dispatch(3150, 'maintainer'),
expect: { 3150: { state: 'closed', labels: [LABEL], comment: 'closed' } },
},
];
for (const s of scenarios) {
test(s.name, async () => {
const world = makeWorld(s);
await run(world, s.event, { enforce: true });
assert.deepEqual(observe(world, s.expect), s.expect);
if (s.writes !== undefined) assert.equal(world.writes.length, s.writes, `writes: ${world.writes.join(', ')}`);
});
}
test('kill switch (ENFORCE=false) reaches the same verdicts but writes nothing', async () => {
for (const s of scenarios) {
const world = makeWorld(s);
await run(world, s.event, { enforce: false });
assert.equal(world.writes.length, 0, `${s.name}: ${world.writes.join(', ')}`);
}
});
test('on assignment, one PR failing to re-evaluate does not stop the others (run still fails)', async () => {
const world = makeWorld({
prs: [
pr(3300, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #10', gateComment: true }),
pr(3301, 'outsider', { state: 'closed', labels: [LABEL], body: 'Fixes #10', gateComment: true }),
],
issues: [issue(10, { assignees: ['outsider'] })],
});
world.failPullsGet = 3300;
await assert.rejects(run(world, assigned(10, 'outsider', 'maintainer'), { enforce: true }), /Could not re-evaluate #3300/);
assert.equal(world.prs.get(3301).state, 'open');
});
test('an API error while checking permissions fails the run rather than closing the PR', async () => {
const world = makeWorld({ prs: [pr(3300, 'outsider')] });
world.failPermissionLookup = true;
await assert.rejects(run(world, opened(3300, 'outsider'), { enforce: true }), /Permission check failed/);
assert.equal(world.writes.length, 0);
});
// ── Scenario helpers ───────────────────────────────────────────────────────
function pr(number, author, o = {}) {
return { number, author, state: 'open', draft: false, body: '', labels: [], merged_at: null, comments: [], gateComment: false, refuseReopen: false, ...o };
}
function issue(number, o = {}) {
return { number, state: 'open', labels: [], assignees: [], repository_url: `https://api.github.com/repos/${REPO.owner}/${REPO.repo}`, ...o };
}
function prEvent(action, number, sender, extra = {}) {
return { eventName: 'pull_request_target', payload: { action, pull_request: { number }, sender: { login: sender }, ...extra } };
}
function opened(number, sender) { return prEvent('opened', number, sender); }
function edited(number, sender) { return prEvent('edited', number, sender); }
function reopened(number, sender) { return prEvent('reopened', number, sender); }
function readyForReview(number, sender) { return prEvent('ready_for_review', number, sender); }
function unlabeled(number, sender) { return prEvent('unlabeled', number, sender, { label: { name: LABEL } }); }
function labeled(number, sender, name) { return prEvent('labeled', number, sender, { label: { name } }); }
function assigned(issueNumber, assignee, sender) {
return { eventName: 'issues', payload: { action: 'assigned', issue: { number: issueNumber }, assignee: { login: assignee }, sender: { login: sender } } };
}
function dispatch(number, sender) {
return { eventName: 'workflow_dispatch', prNumber: number, payload: { inputs: { pr_number: String(number) }, sender: { login: sender } } };
}
async function run(world, event, { enforce }) {
process.env.ENFORCE = enforce ? 'true' : 'false';
process.env.PR_NUMBER_INPUT = event.prNumber ? String(event.prNumber) : '';
const core = { warning: () => {}, setFailed: (m) => { throw new Error(`setFailed: ${m}`); } };
const quiet = console.log;
console.log = () => {};
try {
await gate({ github: world.github, context: { repo: REPO, ...event }, core });
} finally {
console.log = quiet;
}
}
// What the gate left behind, in the same shape as a scenario's `expect`.
function observe(world, expect) {
const out = {};
for (const num of Object.keys(expect)) {
const p = world.prs.get(Number(num));
const gateComments = p.comments.filter((c) => c.user === 'github-actions[bot]' && c.body.includes('<!-- require-linked-issue -->'));
assert.ok(gateComments.length <= 1, `PR #${num} has ${gateComments.length} gate comments`);
const body = gateComments[0]?.body;
const kind = !body ? null : body.includes("won't let it be reopened") ? 'cannot-reopen' : body.includes('still a draft') ? 'closed-draft' : 'closed';
out[num] = { state: p.state, labels: [...p.labels].sort(), comment: kind };
if ('foreignComments' in expect[num]) out[num].foreignComments = p.comments.length - gateComments.length;
}
return out;
}
// ── A tiny in-memory GitHub ────────────────────────────────────────────────
function makeWorld({ prs = [], issues = [] }) {
const world = { prs: new Map(), issues: new Map(), writes: [], failPermissionLookup: false, failPullsGet: null, nextCommentId: 100 };
for (const p of prs) {
const copy = structuredClone(p);
copy.comments = copy.comments.map((c) => ({ id: world.nextCommentId++, ...c }));
if (copy.gateComment) copy.comments.push({ id: world.nextCommentId++, user: 'github-actions[bot]', body: '<!-- require-linked-issue -->\nThanks for the contribution. …closed for now…' });
world.prs.set(p.number, copy);
}
for (const i of issues) world.issues.set(i.number, structuredClone(i));
const err = (status, message = 'fake error') => Object.assign(new Error(message), { status });
const write = (what) => world.writes.push(what);
const getPr = (n) => { const p = world.prs.get(n); if (!p) throw err(404); return p; };
const apiPr = (p) => ({
number: p.number, state: p.state, draft: p.draft, body: p.body, merged_at: p.merged_at,
labels: p.labels.map((name) => ({ name })),
user: { login: p.author, type: PEOPLE[p.author].type },
head: { sha: 'abc1234def', ref: 'patch-1', repo: { name: REPO.repo, owner: { login: p.author } } },
});
const rest = {
repos: {
getCollaboratorPermissionLevel: async ({ username }) => {
if (world.failPermissionLookup) throw err(500, 'boom');
const person = PEOPLE[username];
if (!person) throw err(404, 'not a user');
return { data: { role_name: 'custom', user: { permissions: { admin: false, maintain: false, push: false, triage: false, pull: false, ...person.perms } } } };
},
},
pulls: {
get: async ({ pull_number }) => { if (world.failPullsGet === pull_number) throw err(502, 'bad gateway'); return { data: apiPr(getPr(pull_number)) }; },
update: async ({ pull_number, state }) => {
const p = getPr(pull_number);
write(`${state === 'open' ? 'reopen' : 'close'} #${pull_number}`);
if (state === 'open' && p.refuseReopen) throw err(422, 'Validation Failed: state cannot be changed');
p.state = state;
return { data: apiPr(p) };
},
},
issues: {
get: async ({ issue_number }) => {
if (world.prs.has(issue_number)) return { data: { ...apiPr(getPr(issue_number)), pull_request: {} } };
const i = world.issues.get(issue_number);
if (!i) throw err(404);
return { data: { ...i, labels: i.labels.map((name) => ({ name })), assignees: i.assignees.map((login) => ({ login })) } };
},
listForRepo: async ({ state, creator, labels }) => ({
data: [...world.prs.values()]
.filter((p) => p.state === state && p.author === creator && p.labels.includes(labels))
.map((p) => ({ number: p.number, body: p.body, pull_request: {} })),
}),
getLabel: async () => ({ data: {} }),
createLabel: async ({ name }) => { write(`create label ${name}`); },
addLabels: async ({ issue_number, labels }) => {
const p = getPr(issue_number);
write(`label #${issue_number} ${labels}`);
for (const l of labels) if (!p.labels.includes(l)) p.labels.push(l);
},
removeLabel: async ({ issue_number, name }) => {
const p = getPr(issue_number);
write(`unlabel #${issue_number} ${name}`);
if (!p.labels.includes(name)) throw err(404);
p.labels = p.labels.filter((l) => l !== name);
},
listComments: async ({ issue_number }) => ({ data: getPr(issue_number).comments.map((c) => ({ id: c.id, body: c.body, user: { login: c.user } })) }),
createComment: async ({ issue_number, body }) => {
write(`comment on #${issue_number}`);
getPr(issue_number).comments.push({ id: world.nextCommentId++, user: 'github-actions[bot]', body });
},
updateComment: async ({ comment_id, body }) => {
write(`edit comment ${comment_id}`);
for (const p of world.prs.values()) for (const c of p.comments) if (c.id === comment_id) c.body = body;
},
deleteComment: async ({ comment_id }) => {
write(`delete comment ${comment_id}`);
for (const p of world.prs.values()) p.comments = p.comments.filter((c) => c.id !== comment_id);
},
},
};
world.github = { rest, paginate: async (fn, args) => (await fn(args)).data };
return world;
}

View file

@ -1,42 +0,0 @@
# Source: https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
name: Claude Code
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
jobs:
claude:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude') && !startsWith(github.event.comment.body, '@claude review')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 1
persist-credentials: false
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@d40ddef4c030e508327d6e35a9c45f3368482c50 # v1.0.195
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} # zizmor: ignore[secrets-outside-env]
use_commit_signing: true
additional_permissions: |
actions: read

View file

@ -2,8 +2,9 @@ name: Conformance Tests
on: on:
push: push:
branches: [main] branches: [v1.x]
pull_request: pull_request:
branches: [v1.x]
workflow_dispatch: workflow_dispatch:
concurrency: concurrency:
@ -13,130 +14,31 @@ concurrency:
permissions: permissions:
contents: read contents: read
env:
# Pinned conformance harness package spec (passed verbatim to `npx --yes`).
# Bump deliberately and reconcile the three
# .github/actions/conformance/expected-failures*.yml files in the same change.
CONFORMANCE_PKG: "@modelcontextprotocol/conformance@0.2.0-alpha.11"
# Each job runs three `--suite all` legs, one per baseline file. The two
# `--spec-version <rev>` legs run every scenario the pinned harness marks
# applicable at that spec revision, at that revision's wire (stateful
# handshake for 2025-11-25, stateless per-request _meta for 2026-07-28). Each
# is a superset of the frozen requirement set that `conformance tier-check`
# scores for that revision (`npx $CONFORMANCE_PKG list --requirements <rev>`
# prints it), so an expected-failures.<rev>.yml entry for a scenario in that
# set means tier-check reports the SDK below 100% for <rev>. The bare leg runs
# every shipped scenario at whichever wire the harness defaults it to; it is
# the only leg that reaches extension-tagged scenarios (tasks-*, auth/dpop*,
# ...), which never match a --spec-version filter. For everything else it
# overlaps one of the pinned legs, so a failure it also sees needs its own
# expected-failures.yml entry besides the pinned file's.
jobs: jobs:
server-conformance: server-conformance:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 15
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with: - uses: astral-sh/setup-uv@803947b9bd8e9f986429fa0c5a41c367cd732b41 # v7.2.1
persist-credentials: false
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with: with:
enable-cache: true enable-cache: true
version: 0.9.5 version: 0.9.5
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 - uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with: with:
node-version: 24 node-version: 24
- run: uv sync --frozen --all-extras --package mcp-everything-server - run: uv sync --frozen --all-extras --package mcp-everything-server
- name: Run server conformance (all suite, 2025-11-25 wire) - run: ./.github/actions/conformance/run-server.sh
run: >-
./.github/actions/conformance/run-server.sh
--suite all
--spec-version 2025-11-25
--expected-failures ./.github/actions/conformance/expected-failures.2025-11-25.yml
--output-dir conformance-results/server-2025-11-25
- name: Run server conformance (all suite, 2026-07-28 wire)
run: >-
./.github/actions/conformance/run-server.sh
--suite all
--spec-version 2026-07-28
--expected-failures ./.github/actions/conformance/expected-failures.2026-07-28.yml
--output-dir conformance-results/server-2026-07-28
- name: Run server conformance (all suite, default wire)
run: >-
./.github/actions/conformance/run-server.sh
--suite all
--expected-failures ./.github/actions/conformance/expected-failures.yml
--output-dir conformance-results/server-all
- name: Upload conformance results
# The log has only summary counts; per-check data is in checks.json.
# always(), not !cancelled(): a timeout-minutes kill is a cancellation,
# and that is when the partial results matter most.
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: server-conformance-results
path: conformance-results/
if-no-files-found: ignore
client-conformance: client-conformance:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 15
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
with: - uses: astral-sh/setup-uv@803947b9bd8e9f986429fa0c5a41c367cd732b41 # v7.2.1
persist-credentials: false
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with: with:
enable-cache: true enable-cache: true
version: 0.9.5 version: 0.9.5
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 - uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
with: with:
node-version: 24 node-version: 24
# --compile-bytecode: without it, ~40 concurrently spawned interpreters - run: uv sync --frozen --all-extras --package mcp
# race to byte-compile site-packages during the timing-sensitive window. - run: npx @modelcontextprotocol/conformance@0.1.13 client --command 'uv run --frozen python .github/actions/conformance/client.py' --suite all --expected-failures .github/actions/conformance/expected-failures.yml
- run: uv sync --frozen --all-extras --package mcp --compile-bytecode
- name: Pre-compile bytecode (editable sources)
run: uv run --frozen python -m compileall -q src .github/actions/conformance
# The harness runs a leg's scenarios via unbounded Promise.all; under
# that concurrency the slowest one (sse-retry, which has a real-time SSE
# reconnect wait) needs more than the 30s default budget.
# `.venv/bin/python` (not `uv run`) avoids lockfile re-checks in ~40
# concurrent spawns; run-client.sh re-runs unexpected failures solo.
- name: Run client conformance (all suite, 2025-11-25 wire)
run: >-
./.github/actions/conformance/run-client.sh
--command '.venv/bin/python .github/actions/conformance/client.py'
--suite all
--timeout 60000
--spec-version 2025-11-25
--expected-failures ./.github/actions/conformance/expected-failures.2025-11-25.yml
--output-dir conformance-results/client-2025-11-25
- name: Run client conformance (all suite, 2026-07-28 wire)
run: >-
./.github/actions/conformance/run-client.sh
--command '.venv/bin/python .github/actions/conformance/client.py'
--suite all
--timeout 60000
--spec-version 2026-07-28
--expected-failures ./.github/actions/conformance/expected-failures.2026-07-28.yml
--output-dir conformance-results/client-2026-07-28
- name: Run client conformance (all suite, default wire)
run: >-
./.github/actions/conformance/run-client.sh
--command '.venv/bin/python .github/actions/conformance/client.py'
--suite all
--timeout 60000
--expected-failures ./.github/actions/conformance/expected-failures.yml
--output-dir conformance-results/client-all
- name: Upload conformance results
# The log has only summary counts; per-check data is in checks.json.
# always(), not !cancelled(): a timeout-minutes kill is a cancellation,
# and that is when the partial results matter most.
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: client-conformance-results
path: conformance-results/
if-no-files-found: ignore

View file

@ -1,68 +0,0 @@
name: Deploy Docs
on:
push:
branches:
# main is the sole deployer of the combined site (v2 at / and /v2/, v1.x
# at /v1/); the v1.x branch has no deploy workflow. A v1.x docs change is
# published by the next main deploy or a manual workflow_dispatch here.
- main
paths:
- docs/**
# docs pages include their code blocks from these files via `--8<--`, so a
# change here changes the rendered site even when no .md file moves.
- docs_src/**
# translated pages and the language registry feed the site/<code>/ sites
- i18n/**
- mkdocs.yml
- src/mcp/**
- src/mcp-types/**
- scripts/build-docs.sh
- scripts/docs/**
- pyproject.toml
- uv.lock
- .github/workflows/deploy-docs.yml
workflow_dispatch:
concurrency:
group: deploy-docs
cancel-in-progress: false
jobs:
deploy-docs:
runs-on: ubuntu-latest
permissions:
contents: read
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: true
version: 0.9.5
- name: Build combined docs (main at / and /v2/, v1.x at /v1/)
run: bash scripts/build-docs.sh site
- name: Configure Pages
uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
- name: Upload Pages artifact
uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
with:
path: site
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0

Some files were not shown because too many files have changed in this diff Show more