Compare commits

..
1705 changed files with 11010 additions and 261713 deletions

View file

@ -1,118 +0,0 @@
Review the pull request: $ARGUMENTS
Follow these steps carefully. Use the `gh` CLI for all GitHub interactions.
## Step 1: Resolve the PR
Parse `$ARGUMENTS` to determine the PR. It can be:
- A full URL like `https://github.com/owner/repo/pull/123`
- A `owner/repo#123` reference
- A bare number like `123` (use the current repo)
- A description — search for it with `gh pr list --search "<description>" --limit 5` and pick the best match
Once resolved, fetch the PR metadata:
```bash
gh pr view <PR> --json number,title,body,author,state,baseRefName,headRefName,url,labels,milestone,additions,deletions,changedFiles,createdAt,updatedAt,mergedAt,reviewDecision,reviews,assignees
```
## Step 2: Gather the diff
Get the full diff of the PR:
```bash
gh pr diff <PR>
```
If the diff is very large (>3000 lines), focus on the most important files first and summarize the rest.
## Step 3: Collect PR discussion context
Fetch all comments and review threads:
```bash
gh api repos/{owner}/{repo}/pulls/{number}/comments --paginate
gh api repos/{owner}/{repo}/issues/{number}/comments --paginate
gh api repos/{owner}/{repo}/pulls/{number}/reviews --paginate
```
Pay attention to:
- Reviewer feedback and requested changes
- Author responses and explanations
- Any unresolved conversations
- Approval or rejection status
## Step 4: Find and read linked issues
Look for issue references in:
- The PR body (patterns like `#123`, `fixes #123`, `closes #123`, `resolves #123`)
- The PR branch name (patterns like `issue-123`, `fix/123`)
- Commit messages
For each linked issue, fetch its content:
```bash
gh issue view <number> --json title,body,comments,labels,state
```
Read through issue comments to understand the original problem, user reports, and any discussed solutions.
## Step 5: Analyze and validate
With all context gathered, analyze the PR critically:
1. **Intent alignment**: Does the code change actually solve the problem described in the PR and/or linked issues?
2. **Completeness**: Are there aspects of the issue or requested feature that the PR doesn't address?
3. **Scope**: Does the PR include changes unrelated to the stated goal? Are there unnecessary modifications?
4. **Correctness**: Based on the diff, are there obvious bugs, edge cases, or logic errors?
5. **Testing**: Does the PR include tests? Are they meaningful and do they cover the important cases?
6. **Breaking changes**: Could this PR break existing functionality or APIs?
7. **Unresolved feedback**: Are there reviewer comments that haven't been addressed?
## Step 6: Produce the review summary
Present the summary in this format:
---
### PR Review: `<title>` (<url>)
**Author:** <author> | **Status:** <state> | **Review decision:** <decision>
**Base:** `<base>` ← `<head>` | **Changed files:** <n> | **+<additions> / -<deletions>**
#### Problem
<1-3 sentences describing what problem this PR is trying to solve, based on the PR description and linked issues>
#### Solution
<1-3 sentences describing the approach taken in the code>
#### Key changes
<Bulleted list of the most important changes, grouped by theme. Include file paths.>
#### Linked issues
<List of linked issues with their title, state, and a one-line summary of the discussion>
#### Discussion highlights
<Summary of important comments from reviewers and the author. Flag any unresolved threads.>
#### Concerns
<List any issues found during validation: bugs, missing tests, scope creep, unaddressed feedback, etc. If none, say "No concerns found.">
#### Verdict
<One of: APPROVE / REQUEST CHANGES / NEEDS DISCUSSION, with a brief justification>
#### Suggested action
<Clear recommendation for the reviewer: what to approve, what to push back on, what to ask about>
---

View file

@ -1,150 +0,0 @@
---
name: test-quality
description: Test quality bar for this repo. Read when writing, reviewing, or designing tests — covers naming, abstraction level, assertions, determinism, and the process for agent-driven test work.
---
# Test & code quality guide
What "best practice for new work" means in this repo. Each rule carries its recorded reasoning
where one exists; a rule with no stated why is a convention — follow it anyway.
## Naming & shape
- **Test names are behaviour sentences** stating the observable outcome, not the feature being
poked: `test_elicit_form_decline_returns_no_content`, never `test_elicit_form_decline`.
- Plain top-level `test_*` functions; no `Test` classes (legacy files have them — don't copy).
- **Docstrings: 1–2 sentences of behaviour, honest about provenance** — spec-mandated,
SDK-defined, or pinning a known gap? Say which: provenance is the triage key when the test
later fails. A pinned-gap assertion breaking usually means a change *fixed* the gap; a
spec-mandated assertion breaking means a regression.
- Define things in dependency order; nothing forward-references. For client↔server tests:
handlers → server construction → client setup → act → assert — the test reads in the order
the conversation happens.
- Inline the server (or equivalent setup) in the test, so the whole observable behaviour fits
on one screen. Lift to a file-level fixture only when several tests in *that file* genuinely
share it; never share across files.
- A big multi-step test is fine when the property is irreducibly multi-step (e.g. resumability).
Split when a failure wouldn't tell you which claim broke — not for shortness. Compensate with
a numbered "Steps:" docstring so a reader sees the choreography before the body.
## Level of abstraction
- **Drive through the highest-level public API that can observe the property.** Hand-built wire
requests are brittle and don't prove the user-facing contract; tests that stay above the
internals keep working when the internals change. Drop to raw HTTP only when the assertion is
about something the high-level API *cannot* observe (status codes, headers, wire framing).
- **Scripting a peer over raw streams is a last resort**, reserved for behaviour the typed API
cannot *produce* (malformed input, an impossible peer response). First ask what it would take
for the public API to express it — often a small helper suffices. Every such test's docstring
states why the public API couldn't do it.
- **In-memory / in-process first.** HTTP-, SSE-, and auth-shaped behaviour can all be driven
through an in-process ASGI transport; threads only when necessary, subprocesses only when the
process boundary is itself the thing under test. In-process isn't just faster — it surfaces
bugs (a real stream leak was found this way) that subprocess indirection masks.
- **Tests read like real user code**: no aliasing shims in conftest, no walls of suppressions,
no private imports unless that is genuinely the documented way to do the thing.
- The assertion must prove the round trip — no side-channel state. What the server saw comes
back through the protocol, or via a closure-captured list asserted after the call. Handlers
assert their dispatch identity first (`assert params.name == "add"`), proving the request
that arrived is the request the test sent.
## Assertions
- **Transformations** (input → output the SDK produced) → full-object `snapshot(...)` equality,
so an added or dropped field fails. Regenerate with `--inline-snapshot=fix` so intentional
changes arrive as a reviewable diff; never hand-edit snapshot literals.
- **Pass-through values** (opaque tokens, `_meta`, cursors) → identity against the same
variable you sent. A snapshot of a pass-through value only "matches" because a human checked
two literals correspond — it proves nothing.
- **Errors** → `pytest.raises` + `.code` against the named constant; snapshot SDK-authored
messages; never `match=` on message text. Third-party text (pydantic, jsonschema) → stable
prefix only — never pin text that changes with a dependency upgrade — with a comment saying so.
## Determinism
- **No sleeps, ever.** A sleep guesses at timing instead of waiting on the condition, so it
either flakes or pads the run — sleeps head the list of the older test code's failure modes.
Coordinate with `anyio.Event` so the wait ends exactly when the condition holds (that
discipline is why 529 e2e tests run in ~10 s). Sole exception: tests *of* time-based
features, with a comment.
- Bound every indefinite wait with `anyio.fail_after(5)`. **5 is the standard** — widening it
needs an articulable reason; "10 to be safe" is covering up a flake, not fixing one, and
unexplained widenings propagate (one agent used 10 and every later one copied it).
- **Never assert wall-clock time, even with huge margins.** `elapsed < 0.9` on a ~0.01 s
operation — a 90× margin — was still rejected in review. `fail_after` bounding a hang is the
only timing primitive allowed.
- **Concurrency tests must prove genuine interleaving.** Without barriers the scheduler is free
to serialize — "a" can start, finish, and return before "b" even begins — so two `start_soon`
calls prove nothing. Gate with events so all parties are mid-flight before any proceeds, emit
interleaved, then assert the demux.
- **Don't over-synchronize either.** When delivery ordering is guaranteed (notifications
emitted during a request you're awaiting, over a single ordered in-memory stream), a plain
collected list asserted after the call is correct; events are for messages not tied to an
awaited operation. Verify the ordering guarantee actually holds for your transport first.
- Async tests use anyio, not asyncio.
## Behaviour philosophy
- **Pin current behaviour; never xfail.** A green suite asserting what actually happens is a
regression bar for any refactor; an xfail proves nothing about it. Where current behaviour
falls short of spec, pin the divergent output and record the gap as data — a tracking issue,
with the docstring naming the known gap (suites with a requirements manifest record it as a
divergence entry). Not hidden, not skipped.
- **Hollow-proof check**: before claiming a test covers a behaviour, re-read the claim and ask
"which assertion proves *this*?" A passing test near a behaviour is not proof of it — a full
review of the e2e suite found two such cases even under this discipline.
## Hygiene
- No new `# pragma`, `# type: ignore`, `# noqa` by default — restructure first; a suppression
usually means a test or a type is missing. The narrow sanctioned escape hatches (and the
audit to run before pushing) are in AGENTS.md. In tests, narrow types with `assert
isinstance`; never `Any`/`object` when a real type exists.
- **Warnings raised during tests are findings, not noise** (the repo runs
`filterwarnings = error`). Fix the cause; if the fix can't land in the same change, scope the
suppression to the one fixture that needs it and track the real fix explicitly.
- Registered-but-never-invoked handler bodies are `raise NotImplementedError`, so they cannot
silently become load-bearing.
- Comments live next to the line they explain, not in docstrings; single backticks for code
refs; match the surrounding comment density (one-liners next to one-liners). No
`from __future__ import annotations` (py310+ repo).
- Test work doesn't change `src/` as a side effect — the one mechanical exception is deleting a
pragma a new test now covers. If a test can't be written without a library change, raise it
as a finding or defer the test; don't quietly edit `src/`.
## Process (for agent-driven work)
- **Small chunks.** ≤10 tests per human-reviewed batch; when fanning out to multiple agents,
≤5 per agent. Review quality scales inversely with batch size: the observed shortcuts
(over-stacked tests, a timing assertion, wrong abstraction level) all surfaced in one
oversized 27-test batch.
- **Design → review → implement, as separate steps.** The design deliverable is not just the
plan — it's the judgement calls (abstraction level, deferrals, contested assertions) stated
explicitly, so the reviewer vetoes them before implementation rather than discovering them
in the diff.
- **High-stakes areas get fresh adversarial reviewers** on both the design and the
implementation — fresh, because whoever designed it (or saw the proposed fix) anchors on the
same layer. SERIOUS findings (wrong assertion, missed MUST/SHOULD, an unrecorded known gap)
re-loop; style doesn't. Reserve the full panel for areas where being wrong is worse than
being slow.
- **Investigations pair a full-context look with a fresh agent given only the problem** —
never the proposed fix — and compare conclusions. The unbiased read regularly catches
anchoring on the wrong layer.
- **When review questions a decision, reconsider genuinely**: re-derive the tradeoff, state the
options, recommend with reasons. Defending the original choice is a valid outcome;
reflexively agreeing with every challenge is as bad as ignoring it.
- **Validate the reference artifact before building on it.** Whatever you treat as ground truth
(a spec import, a baseline, a generated list), check it *first* — discovering it was
incomplete after five batches costs far more than before batch one.
- **Verify as you go**: per-file `uv run --frozen pytest <file> -q` + pyright + ruff while
iterating; full suite, coverage, and `./scripts/test` (when `src/` was touched) at
integration.
- **Every agent writes notes**: what it did, what broke, and — most importantly — what it
couldn't decide, so open questions surface instead of being silently resolved by whoever hit
them.
- **Quality over speed is a stated goal, not a preference** ("rushing something out the door is
not the goal here, explicitly so"). Don't quietly de-scope agreed work mid-stream;
renegotiate scope explicitly.
- **Don't copy patterns from existing test code by default** — much of the repo's older test
code is below the current bar (sleeps, mocks, `Test` classes, raciness). These rules define
the bar for new work.

View file

@ -1,5 +0,0 @@
# Applied 120 line-length rule to all files: https://github.com/modelcontextprotocol/python-sdk/pull/856
543961968c0634e93d919d509cce23a1d6a56c21
# Added 100% code coverage baseline with pragma comments: https://github.com/modelcontextprotocol/python-sdk/pull/1553
89e9c43acf7e23cf766357d776ec1ce63ac2c58e

2
.gitattributes vendored
View file

@ -1,2 +0,0 @@
# Generated
uv.lock linguist-generated=true

View file

@ -1,68 +0,0 @@
name: 🐛 MCP Python SDK Bug
description: Report a bug or unexpected behavior in the MCP Python SDK
labels: ["need confirmation"]
body:
- type: markdown
attributes:
value: Thank you for contributing to the MCP Python SDK! ✊
- type: checkboxes
id: checks
attributes:
label: Initial Checks
description: >
Both the 2.x stable line and the 1.x maintenance line are supported, and only
the newest release of each line receives fixes.
options:
- label: I confirm that I'm using the newest release of my line (the latest 2.x, or the latest 1.x if I'm still on v1)
required: true
- label: I confirm that I searched for my issue in https://github.com/modelcontextprotocol/python-sdk/issues before opening this issue
required: true
- type: dropdown
id: release-line
attributes:
label: Release line
description: Which major version of the SDK are you using?
options:
- 2.x (current stable)
- 1.x (maintenance line, v1.x branch)
validations:
required: true
- type: textarea
id: description
attributes:
label: Description
description: |
Please explain what you're seeing and what you would expect to see.
Please provide as much detail as possible to make understanding and solving your problem as quick as possible. 🙏
validations:
required: true
- type: textarea
id: example
attributes:
label: Example Code
description: >
If applicable, please add a self-contained,
[minimal, reproducible, example](https://stackoverflow.com/help/minimal-reproducible-example)
demonstrating the bug.
placeholder: |
from mcp.server.mcpserver import MCPServer
...
render: Python
- type: textarea
id: version
attributes:
label: Python & MCP Python SDK
description: |
Which version of Python and MCP Python SDK are you using?
render: Text
validations:
required: true

38
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View file

@ -0,0 +1,38 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.

View file

@ -1 +0,0 @@
blank_issues_enabled: false

View file

@ -1,29 +0,0 @@
name: 🚀 MCP Python SDK Feature Request
description: "Suggest a new feature for the MCP Python SDK"
labels: ["feature request"]
body:
- type: markdown
attributes:
value: Thank you for contributing to the MCP Python SDK! ✊
- type: textarea
id: description
attributes:
label: Description
description: |
Please give as much detail as possible about the feature you would like to suggest. 🙏
You might like to add:
* A demo of how code might look when using the feature
* Your use case(s) for the feature
* Reference to other projects that have a similar feature
validations:
required: true
- type: textarea
id: references
attributes:
label: References
description: |
Please add any links or references that might help us understand your feature request better. 📚

View file

@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

View file

@ -1,33 +0,0 @@
name: ❓ MCP Python SDK Question
description: "Ask a question about the MCP Python SDK"
labels: ["question"]
body:
- type: markdown
attributes:
value: Thank you for reaching out to the MCP Python SDK community! We're here to help! 🤝
- type: textarea
id: question
attributes:
label: Question
description: |
Please provide as much detail as possible about your question. 🙏
You might like to include:
* Code snippets showing what you've tried
* Error messages you're encountering (if any)
* Expected vs actual behavior
* Your use case and what you're trying to achieve
validations:
required: true
- type: textarea
id: context
attributes:
label: Additional Context
description: |
Please provide any additional context that might help us better understand your question, such as:
* Your MCP Python SDK version
* Your Python version
* Relevant configuration or environment details 📝

View file

@ -1,59 +0,0 @@
name: v2 feedback
description: Bugs, API friction, or docs gaps in v2 of the SDK
title: "[v2] "
labels: ["v2"]
body:
- type: markdown
attributes:
value: |
Anything in v2 that broke, surprised you, or slowed you down is useful, including API friction and docs gaps.
Docs: https://py.sdk.modelcontextprotocol.io/ · Migration from v1: https://py.sdk.modelcontextprotocol.io/migration/
- type: textarea
id: what
attributes:
label: What happened?
description: What did you do, and what went wrong (or felt wrong)? Paste error output verbatim if there is any.
validations:
required: true
- type: textarea
id: expected
attributes:
label: What did you expect?
validations:
required: false
- type: textarea
id: repro
attributes:
label: Code to reproduce
description: The smallest snippet or repository that shows it. For docs feedback, link the page instead.
render: Python
validations:
required: false
- type: input
id: version
attributes:
label: SDK version
description: The published version (`pip show mcp`) or commit.
validations:
required: false
- type: dropdown
id: area
attributes:
label: Area
options:
- Server
- Client
- Transports
- Auth
- Documentation
- Migration
- Other
validations:
required: false

View file

@ -1,644 +0,0 @@
"""MCP unified conformance test client.
This client is designed to work with the @modelcontextprotocol/conformance npm package.
It handles all conformance test scenarios via environment variables and CLI arguments.
Contract:
- MCP_CONFORMANCE_SCENARIO env var -> scenario name
- MCP_CONFORMANCE_CONTEXT env var -> optional JSON (for client-credentials scenarios)
- MCP_CONFORMANCE_PROTOCOL_VERSION env var -> spec version the harness mock
server is speaking (e.g. "2025-11-25", "2026-07-28"). Always set; when
--spec-version is omitted the harness picks per-scenario (LATEST_SPEC_VERSION
for active scenarios, DRAFT_PROTOCOL_VERSION for draft-only ones).
- Server URL as last CLI argument (sys.argv[1])
- Must exit 0 within the harness --timeout (CI passes 60s; the default is 30s)
Scenarios:
initialize - Connect, initialize, list tools, close
tools_call - Connect, call add_numbers(a=5, b=3), close
sse-retry - Connect, call test_reconnection, close
json-schema-ref-no-deref - Connect, list tools (no $ref deref)
json-schema-2020-12-preservation - List tools, echo the focal inputSchema back verbatim
request-metadata - Connect with all callbacks; client stamps _meta
http-standard-headers - Tool, resource and prompt round-trips (Mcp-* headers checked)
http-invalid-tool-headers - List tools, call every surfaced tool (x-mcp-header filter)
http-custom-headers - Replay the harness's toolCalls (x-mcp-header -> Mcp-Param-*)
elicitation-sep1034-client-defaults - Elicitation with default accept callback
sep-2322-client-request-state - Drive the MRTR auto-loop (SEP-2322)
auth/client-credentials-jwt - Client credentials with private_key_jwt
auth/client-credentials-basic - Client credentials with client_secret_basic
auth/enterprise-managed-authorization - SEP-990 ID-JAG (RFC 8693 + RFC 7523 jwt-bearer)
auth/* - Authorization code flow (default for auth scenarios)
"""
import asyncio
import json
import logging
import os
import sys
from collections.abc import Callable, Coroutine
from typing import Any, cast
from urllib.parse import parse_qs, urlparse
import httpx2
import mcp_types as types
from mcp_types.version import MODERN_PROTOCOL_VERSIONS
from pydantic import AnyUrl
from mcp.client.auth import OAuthClientProvider, TokenStorage
from mcp.client.auth.extensions.client_credentials import (
ClientCredentialsOAuthProvider,
PrivateKeyJWTOAuthProvider,
SignedJWTParameters,
)
from mcp.client.auth.extensions.identity_assertion import IdentityAssertionOAuthProvider
from mcp.client.auth.utils import build_protected_resource_metadata_discovery_urls
from mcp.client.client import Client
from mcp.client.context import ClientRequestContext
from mcp.client.streamable_http import streamable_http_client
from mcp.shared.auth import AuthorizationCodeResult, OAuthClientInformationFull, OAuthClientMetadata, OAuthToken
# Set up logging to stderr (stdout is for conformance test output)
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
stream=sys.stderr,
)
logger = logging.getLogger(__name__)
#: Spec version the harness is running this scenario at (e.g. "2025-11-25",
#: "2026-07-28"). The harness always sets this (when --spec-version is omitted
#: it picks per-scenario: LATEST_SPEC_VERSION for active scenarios,
#: DRAFT_PROTOCOL_VERSION for draft-only ones), so None means we were invoked
#: outside the harness.
PROTOCOL_VERSION: str | None = os.environ.get("MCP_CONFORMANCE_PROTOCOL_VERSION")
def client_mode() -> str:
"""Pick the Client(mode=) for the harness leg.
On a modern leg (2026-07-28+) -> 'auto' so Client.discover() runs and the
_meta envelope + MCP-Protocol-Version header are stamped on every request.
On a handshake-era leg -> 'legacy' so the initialize handshake runs exactly
as before (no server/discover probe is sent against a mock that would 400 it).
Outside the harness -> 'auto' (probe + fallback).
"""
if PROTOCOL_VERSION is None or PROTOCOL_VERSION in MODERN_PROTOCOL_VERSIONS:
return "auto"
return "legacy"
# Type for async scenario handler functions
ScenarioHandler = Callable[[str], Coroutine[Any, None, None]]
# Registry of scenario handlers
HANDLERS: dict[str, ScenarioHandler] = {}
def register(name: str) -> Callable[[ScenarioHandler], ScenarioHandler]:
"""Register a scenario handler."""
def decorator(fn: ScenarioHandler) -> ScenarioHandler:
HANDLERS[name] = fn
return fn
return decorator
def get_conformance_context() -> dict[str, Any]:
"""Load conformance test context from MCP_CONFORMANCE_CONTEXT environment variable."""
context_json = os.environ.get("MCP_CONFORMANCE_CONTEXT")
if not context_json:
raise RuntimeError(
"MCP_CONFORMANCE_CONTEXT environment variable not set. "
"Expected JSON with client_id, client_secret, and/or private_key_pem."
)
try:
return json.loads(context_json)
except json.JSONDecodeError as e:
raise RuntimeError(f"Failed to parse MCP_CONFORMANCE_CONTEXT as JSON: {e}") from e
class InMemoryTokenStorage(TokenStorage):
"""Simple in-memory token storage for conformance testing."""
def __init__(self) -> None:
self._tokens: OAuthToken | None = None
self._client_info: OAuthClientInformationFull | None = None
async def get_tokens(self) -> OAuthToken | None:
return self._tokens
async def set_tokens(self, tokens: OAuthToken) -> None:
self._tokens = tokens
async def get_client_info(self) -> OAuthClientInformationFull | None:
return self._client_info
async def set_client_info(self, client_info: OAuthClientInformationFull) -> None:
self._client_info = client_info
class ConformanceOAuthCallbackHandler:
"""OAuth callback handler that automatically fetches the authorization URL
and extracts the auth code, without requiring user interaction.
"""
def __init__(self) -> None:
self._auth_code: str | None = None
self._state: str | None = None
self._iss: str | None = None
async def handle_redirect(self, authorization_url: str) -> None:
"""Fetch the authorization URL and extract the auth code from the redirect."""
logger.debug(f"Fetching authorization URL: {authorization_url}")
async with httpx2.AsyncClient() as client:
response = await client.get(
authorization_url,
follow_redirects=False,
)
if response.status_code in (301, 302, 303, 307, 308):
location = cast(str, response.headers.get("location"))
if location:
redirect_url = urlparse(location)
query_params: dict[str, list[str]] = parse_qs(redirect_url.query)
if "code" in query_params:
self._auth_code = query_params["code"][0]
state_values = query_params.get("state")
self._state = state_values[0] if state_values else None
iss_values = query_params.get("iss")
self._iss = iss_values[0] if iss_values else None
logger.debug(f"Got auth code from redirect: {self._auth_code[:10]}...")
return
else:
raise RuntimeError(f"No auth code in redirect URL: {location}")
else:
raise RuntimeError(f"No redirect location received from {authorization_url}")
else:
raise RuntimeError(f"Expected redirect response, got {response.status_code} from {authorization_url}")
async def handle_callback(self) -> AuthorizationCodeResult:
"""Return the captured auth code, state, and iss."""
if self._auth_code is None:
raise RuntimeError("No authorization code available - was handle_redirect called?")
result = AuthorizationCodeResult(code=self._auth_code, state=self._state, iss=self._iss)
self._auth_code = None
self._state = None
self._iss = None
return result
# --- Stub callbacks (declare capabilities in _meta without doing real work) ---
async def stub_sampling_callback(
context: ClientRequestContext,
params: types.CreateMessageRequestParams,
) -> types.CreateMessageResult | types.ErrorData:
return types.CreateMessageResult(
role="assistant",
content=types.TextContent(type="text", text=""),
model="conformance-stub",
)
async def stub_list_roots_callback(context: ClientRequestContext) -> types.ListRootsResult | types.ErrorData:
return types.ListRootsResult(roots=[])
async def default_elicitation_callback(
context: ClientRequestContext,
params: types.ElicitRequestParams,
) -> types.ElicitResult | types.ErrorData:
"""Accept elicitation and apply defaults from the schema (SEP-1034)."""
content: dict[str, str | int | float | bool | list[str] | None] = {}
# For form mode, extract defaults from the requested_schema
if isinstance(params, types.ElicitRequestFormParams):
schema = params.requested_schema
logger.debug(f"Elicitation schema: {schema}")
properties = schema.get("properties", {})
for prop_name, prop_schema in properties.items():
if "default" in prop_schema:
content[prop_name] = prop_schema["default"]
logger.debug(f"Applied defaults: {content}")
return types.ElicitResult(action="accept", content=content)
# --- Scenario Handlers ---
@register("initialize")
async def run_initialize(server_url: str) -> None:
"""Connect, initialize, list tools, close."""
async with Client(server_url, mode=client_mode()) as client:
logger.debug("Initialized successfully")
await client.list_tools()
logger.debug("Listed tools successfully")
@register("json-schema-ref-no-deref")
async def run_json_schema_ref_no_deref(server_url: str) -> None:
"""Initialize and list tools; the scenario fails only if the client fetches a network $ref.
The client never walks inputSchema or resolves $refs, so listing is enough (SEP-2106).
Pinned to mode='legacy': the harness reports PROTOCOL_VERSION=2026-07-28 for this
scenario but its mock server only speaks the handshake-era lifecycle and 400s a
modern-stamped tools/list. The check is lifecycle-agnostic so this is harmless.
"""
async with Client(server_url, mode="legacy") as client:
await client.list_tools()
@register("json-schema-2020-12-preservation")
async def run_json_schema_2020_12_preservation(server_url: str) -> None:
"""List tools, then echo the focal tool's inputSchema back verbatim (SEP-1613 / SEP-2106).
The harness diffs what the client round-trips through `json_schema_echo` against its
fixture to detect 2020-12 keywords ($schema, $defs, $anchor, additionalProperties,
allOf/anyOf, if/then/else) being stripped while parsing tools/list. Unlike
json-schema-ref-no-deref, this mock is version-aware, so client_mode() applies.
"""
async with Client(server_url, mode=client_mode()) as client:
listed = await client.list_tools()
focal = next(tool for tool in listed.tools if tool.name == "json_schema_2020_12_tool")
await client.call_tool("json_schema_echo", {"schema": focal.input_schema})
@register("tools_call")
async def run_tools_call(server_url: str) -> None:
"""Connect, list tools, call add_numbers(a=5, b=3), close."""
async with Client(server_url, mode=client_mode()) as client:
await client.list_tools()
result = await client.call_tool("add_numbers", {"a": 5, "b": 3})
logger.debug(f"add_numbers result: {result}")
@register("sse-retry")
async def run_sse_retry(server_url: str) -> None:
"""Connect, list tools, call test_reconnection, close."""
async with Client(server_url, mode=client_mode()) as client:
await client.list_tools()
result = await client.call_tool("test_reconnection", {})
logger.debug(f"test_reconnection result: {result}")
@register("request-metadata")
async def run_request_metadata(server_url: str) -> None:
"""Connect on the modern path with every client capability declared.
The scenario inspects every request's `_meta` envelope (SEP-2575) for
protocolVersion / clientInfo / clientCapabilities, and the matching
MCP-Protocol-Version header. mode='auto' makes the SDK send
server/discover (covering the unsupported-version retry check), then adopt
and stamp the envelope on the follow-up requests.
"""
async with Client(
server_url,
mode=client_mode(),
sampling_callback=stub_sampling_callback,
list_roots_callback=stub_list_roots_callback,
elicitation_callback=default_elicitation_callback,
) as client:
await client.list_tools()
result = await client.call_tool("add_numbers", {"a": 5, "b": 3})
logger.debug(f"add_numbers result: {result}")
@register("http-standard-headers")
async def run_http_standard_headers(server_url: str) -> None:
"""Touch tools, resources and prompts on the modern path so each standard header is checked (SEP-2243).
The scenario inspects Mcp-Method on each request and Mcp-Name on tools/call,
resources/read and prompts/get, and reports methods the client never sent as
SKIPPED rather than failed, so exercise one of each. initialize and
notifications/initialized stay SKIPPED: the modern path discovers via
server/discover and never sends them.
"""
async with Client(server_url, mode=client_mode()) as client:
tools = await client.list_tools()
if tools.tools:
await client.call_tool(tools.tools[0].name, _stub_required_args(tools.tools[0].input_schema))
resources = await client.list_resources()
if resources.resources:
await client.read_resource(resources.resources[0].uri)
prompts = await client.list_prompts()
if prompts.prompts:
await client.get_prompt(prompts.prompts[0].name)
def _stub_required_args(input_schema: dict[str, Any]) -> dict[str, Any]:
"""Minimal arguments satisfying a tool inputSchema's required list."""
by_type: dict[str, Any] = {
"string": "x",
"integer": 0,
"number": 0,
"boolean": False,
"object": {},
"array": [],
"null": None,
}
properties = input_schema.get("properties", {})
return {name: by_type.get(properties.get(name, {}).get("type"), "x") for name in input_schema.get("required", [])}
@register("http-invalid-tool-headers")
async def run_http_invalid_tool_headers(server_url: str) -> None:
"""List tools, then call every tool the SDK surfaces (SEP-2243).
The harness mock advertises one valid tool plus several with malformed
x-mcp-header annotations (empty, non-primitive type, duplicate, invalid
chars). The scenario passes if valid_tool is called and the malformed
ones are not -- so a conforming client filters them out of the list_tools
result and the loop below never sees them. The scenario sets
allowClientError, so a per-call failure is logged and skipped rather
than aborting the whole run.
"""
async with Client(server_url, mode=client_mode()) as client:
listed = await client.list_tools()
logger.debug(f"Surfaced tools: {[t.name for t in listed.tools]}")
for tool in listed.tools:
try:
await client.call_tool(tool.name, _stub_required_args(tool.input_schema))
except Exception:
logger.exception(f"call_tool({tool.name!r}) failed")
@register("http-custom-headers")
async def run_http_custom_headers(server_url: str) -> None:
"""List tools, then replay the harness's `toolCalls` so x-mcp-header args mirror into headers (SEP-2243).
The scenario supplies the exact arguments to send (including the null/edge-case values that
exercise omission and Base64 encoding) via the context `toolCalls`; using them verbatim is
what drives every per-parameter check. `list_tools` first so the SDK caches each tool's
annotations; a tool the SDK dropped (invalid annotations) is skipped. Per-call failures are
logged and skipped rather than aborting the run.
"""
tool_calls: list[dict[str, Any]] = []
if os.environ.get("MCP_CONFORMANCE_CONTEXT"):
tool_calls = get_conformance_context().get("toolCalls", [])
async with Client(server_url, mode=client_mode()) as client:
listed = await client.list_tools()
surfaced = {tool.name for tool in listed.tools}
logger.debug(f"Surfaced tools: {sorted(surfaced)}")
for call in tool_calls:
name = call["name"]
if name not in surfaced:
logger.debug(f"skipping {name!r}: not surfaced by list_tools")
continue
try:
await client.call_tool(name, call.get("arguments") or {})
except Exception:
logger.exception(f"call_tool({name!r}) failed")
@register("elicitation-sep1034-client-defaults")
async def run_elicitation_defaults(server_url: str) -> None:
"""Connect with elicitation callback that applies schema defaults."""
async with Client(server_url, mode=client_mode(), elicitation_callback=default_elicitation_callback) as client:
await client.list_tools()
result = await client.call_tool("test_client_elicitation_defaults", {})
logger.debug(f"test_client_elicitation_defaults result: {result}")
@register("sep-2322-client-request-state")
async def run_mrtr_client(server_url: str) -> None:
"""Drive the SEP-2322 client mock through `Client.call_tool`'s auto-loop.
The mock inspects raw `tools/call` params, so registering an
`elicitation_callback` and letting the driver run is enough to satisfy
all five wire-shape checks: the driver echoes `request_state` byte-exact
and omits it when the server sent none, every retry mints a fresh
JSON-RPC id, the unrelated call between auto-loops carries no MRTR
params, and the no-`resultType` response parses as a terminal
`CallToolResult` so the driver never retries it.
"""
async def confirm(
context: ClientRequestContext, params: types.ElicitRequestParams
) -> types.ElicitResult | types.ErrorData:
return types.ElicitResult(action="accept", content={"confirmed": True})
async with Client(server_url, mode=client_mode(), elicitation_callback=confirm) as client:
await client.list_tools()
await client.call_tool("test_mrtr_echo_state", {})
await client.call_tool("test_mrtr_unrelated", {})
await client.call_tool("test_mrtr_no_state", {})
result = await client.call_tool("test_mrtr_no_result_type", {})
assert isinstance(result, types.CallToolResult)
@register("auth/client-credentials-jwt")
async def run_client_credentials_jwt(server_url: str) -> None:
"""Client credentials flow with private_key_jwt authentication."""
context = get_conformance_context()
client_id = context.get("client_id")
private_key_pem = context.get("private_key_pem")
signing_algorithm = context.get("signing_algorithm", "ES256")
if not client_id:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'client_id'")
if not private_key_pem:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'private_key_pem'")
jwt_params = SignedJWTParameters(
issuer=client_id,
subject=client_id,
signing_algorithm=signing_algorithm,
signing_key=private_key_pem,
)
oauth_auth = PrivateKeyJWTOAuthProvider(
server_url=server_url,
storage=InMemoryTokenStorage(),
client_id=client_id,
assertion_provider=jwt_params.create_assertion_provider(),
)
await _run_auth_session(server_url, oauth_auth)
@register("auth/client-credentials-basic")
async def run_client_credentials_basic(server_url: str) -> None:
"""Client credentials flow with client_secret_basic authentication."""
context = get_conformance_context()
client_id = context.get("client_id")
client_secret = context.get("client_secret")
if not client_id:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'client_id'")
if not client_secret:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'client_secret'")
oauth_auth = ClientCredentialsOAuthProvider(
server_url=server_url,
storage=InMemoryTokenStorage(),
client_id=client_id,
client_secret=client_secret,
token_endpoint_auth_method="client_secret_basic",
)
await _run_auth_session(server_url, oauth_auth)
@register("auth/enterprise-managed-authorization")
async def run_enterprise_managed_authorization(server_url: str) -> None:
"""SEP-990 enterprise-managed authorization: RFC 8693 token-exchange at the
enterprise IdP for an ID-JAG, then RFC 7523 jwt-bearer at the MCP
authorization server."""
context = get_conformance_context()
client_id = context.get("client_id")
client_secret = context.get("client_secret")
idp_client_id = context.get("idp_client_id")
idp_id_token = context.get("idp_id_token")
idp_token_endpoint = context.get("idp_token_endpoint")
if not client_id:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'client_id'")
if not client_secret:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'client_secret'")
if not idp_client_id:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'idp_client_id'")
if not idp_id_token:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'idp_id_token'")
if not idp_token_endpoint:
raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'idp_token_endpoint'")
# IdentityAssertionOAuthProvider takes the AS issuer as configuration (the
# SEP-990 trust model: the resource server is never asked which AS to use).
# The harness does not put the issuer in context, so for conformance we
# learn it from the harness's PRM document (RFC 9728); production
# deployments would supply it as static configuration instead.
prm_url = build_protected_resource_metadata_discovery_urls(None, server_url)[0]
async with httpx2.AsyncClient(timeout=30.0) as http:
prm = (await http.get(prm_url)).raise_for_status().json()
as_issuer = prm["authorization_servers"][0]
async def fetch_id_jag(audience: str, resource: str) -> str:
"""Leg 1 - RFC 8693 token-exchange at the enterprise IdP."""
async with httpx2.AsyncClient(timeout=30.0) as http:
resp = await http.post(
idp_token_endpoint,
data={
"grant_type": "urn:ietf:params:oauth:grant-type:token-exchange",
"requested_token_type": "urn:ietf:params:oauth:token-type:id-jag",
"subject_token": idp_id_token,
"subject_token_type": "urn:ietf:params:oauth:token-type:id_token",
"audience": audience,
"resource": resource,
"client_id": idp_client_id,
},
)
resp.raise_for_status()
return resp.json()["access_token"]
oauth_auth = IdentityAssertionOAuthProvider(
server_url=server_url,
storage=InMemoryTokenStorage(),
client_id=client_id,
client_secret=client_secret,
issuer=as_issuer,
assertion_provider=fetch_id_jag,
token_endpoint_auth_method="client_secret_basic",
)
await _run_auth_session(server_url, oauth_auth)
async def run_auth_code_client(server_url: str) -> None:
"""Authorization code flow (default for auth/* scenarios)."""
callback_handler = ConformanceOAuthCallbackHandler()
storage = InMemoryTokenStorage()
# Check for pre-registered client credentials from context
context_json = os.environ.get("MCP_CONFORMANCE_CONTEXT")
if context_json:
try:
context = json.loads(context_json)
client_id = context.get("client_id")
client_secret = context.get("client_secret")
if client_id:
await storage.set_client_info(
OAuthClientInformationFull(
client_id=client_id,
client_secret=client_secret,
redirect_uris=[AnyUrl("http://localhost:3000/callback")],
token_endpoint_auth_method="client_secret_basic" if client_secret else "none",
)
)
logger.debug(f"Pre-loaded client credentials: client_id={client_id}")
except json.JSONDecodeError:
logger.exception("Failed to parse MCP_CONFORMANCE_CONTEXT")
oauth_auth = OAuthClientProvider(
server_url=server_url,
client_metadata=OAuthClientMetadata(
client_name="conformance-client",
redirect_uris=[AnyUrl("http://localhost:3000/callback")],
grant_types=["authorization_code", "refresh_token"],
response_types=["code"],
),
storage=storage,
redirect_handler=callback_handler.handle_redirect,
callback_handler=callback_handler.handle_callback,
client_metadata_url="https://conformance-test.local/client-metadata.json",
)
await _run_auth_session(server_url, oauth_auth)
async def _run_auth_session(server_url: str, oauth_auth: httpx2.Auth) -> None:
"""Common session logic for all OAuth flows."""
http_client = httpx2.AsyncClient(auth=oauth_auth, timeout=30.0)
transport = streamable_http_client(url=server_url, http_client=http_client)
async with Client(transport, mode=client_mode(), elicitation_callback=default_elicitation_callback) as client:
logger.debug("Initialized successfully")
tools_result = await client.list_tools()
logger.debug(f"Listed tools: {[t.name for t in tools_result.tools]}")
# Call the first available tool (different tests have different tools)
if tools_result.tools:
tool_name = tools_result.tools[0].name
try:
result = await client.call_tool(tool_name, {})
logger.debug(f"Called {tool_name}, result: {result}")
except Exception as e:
logger.debug(f"Tool call result/error: {e}")
logger.debug("Connection closed successfully")
def main() -> None:
"""Main entry point for the conformance client."""
if len(sys.argv) < 2:
print(f"Usage: {sys.argv[0]} <server-url>", file=sys.stderr)
sys.exit(1)
server_url = sys.argv[1]
scenario = os.environ.get("MCP_CONFORMANCE_SCENARIO")
logger.debug(f"Conformance protocol version: {PROTOCOL_VERSION!r} -> mode={client_mode()!r}")
if scenario:
logger.debug(f"Running explicit scenario '{scenario}' against {server_url}")
handler = HANDLERS.get(scenario)
if handler:
asyncio.run(handler(server_url))
elif scenario.startswith("auth/"):
asyncio.run(run_auth_code_client(server_url))
else:
print(f"Unknown scenario: {scenario}", file=sys.stderr)
sys.exit(1)
else:
logger.debug(f"Running default auth flow against {server_url}")
asyncio.run(run_auth_code_client(server_url))
if __name__ == "__main__":
main()

View file

@ -1,26 +0,0 @@
# Expected failures at the 2025-11-25 wire, read by the `--suite all
# --spec-version 2025-11-25` server and client legs: every scenario the pinned
# harness marks applicable at 2025-11-25, run over the stateful handshake.
#
# That selection is a superset of the frozen 2025-11-25 requirement set (`npx
# $CONFORMANCE_PKG list --requirements 2025-11-25`), so an entry here for a
# scenario in that set means `conformance tier-check` scores the SDK below 100%
# for 2025-11-25 even though CI is green; link the tracking issue next to any
# such entry.
#
# Kept separate from expected-failures.yml even though the bare `--suite all`
# legs currently run these same scenarios at 2025-11-25 too: the harness's
# default wire is due to move (modelcontextprotocol/conformance#426) and this
# file must keep meaning 2025-11-25. Until then a failure here also needs an
# entry there. The file is passed even while empty: run-client.sh keys its solo
# re-verification off the harness's baseline output.
#
# Baseline established against the harness pinned via CONFORMANCE_PKG in
# .github/workflows/conformance.yml. New conformance releases are adopted by
# deliberately bumping that pin and reconciling all three expected-failures
# files in the same change. Entries MUST be removed as they start passing (the
# runner fails on stale entries).
client: []
server: []

View file

@ -1,29 +0,0 @@
# Expected failures at the 2026-07-28 wire, read by the `--suite all
# --spec-version 2026-07-28` server and client legs: every scenario the pinned
# harness marks applicable at 2026-07-28, run stateless with per-request _meta.
#
# That selection is a superset of the frozen 2026-07-28 requirement set (`npx
# $CONFORMANCE_PKG list --requirements 2026-07-28`), so an entry here for a
# scenario in that set means `conformance tier-check` scores the SDK below 100%
# for 2026-07-28 even though CI is green; link the tracking issue next to any
# such entry.
#
# This baseline is separate from expected-failures.yml because entries are
# keyed by scenario name only: a scenario that passes at the harness default
# wire in the bare `--suite all` legs but fails when forced to 2026-07-28 (or
# vice versa) cannot be expressed in a shared file. Where the bare legs already
# default a scenario to 2026-07-28, a failure here also needs an entry there.
#
# Baseline established against the harness pinned via CONFORMANCE_PKG in
# .github/workflows/conformance.yml. New conformance releases are adopted by
# deliberately bumping that pin and reconciling all three expected-failures
# files in the same change.
#
# Entries are grouped by what unblocks them. As each gap closes the
# corresponding scenarios start passing and MUST be removed from this list
# (the runner fails on stale entries), so the baseline burns down per
# milestone.
client: []
server: []

View file

@ -1,55 +0,0 @@
# Conformance scenarios not yet passing against the Python SDK on main, read by
# the bare `--suite all` server and client legs (every scenario shipped with the
# pinned harness, each at the harness's default wire for it). CI exits 0 if only
# these fail, exits 1 on unexpected failures or stale entries.
#
# The wire-pinned legs read expected-failures.2025-11-25.yml and
# expected-failures.2026-07-28.yml instead; see the header of
# .github/workflows/conformance.yml for the leg layout.
#
# Baseline established against the harness pinned via CONFORMANCE_PKG in
# .github/workflows/conformance.yml. New conformance releases are adopted by
# deliberately bumping that pin and reconciling all three expected-failures
# files in the same change.
#
# Entries are grouped by SEP. As each SEP lands in the SDK the corresponding
# scenarios start passing and MUST be removed from this list (the runner fails
# on stale entries), so the baseline burns down per milestone.
client:
# SEP-1932 (DPoP): the SDK's OAuth client does not implement DPoP proofs.
# The entries are per-check (conformance #406) because both scenarios
# pass their non-DPoP checks (discovery, token acquisition, request
# flow) live.
- auth/dpop:sep-1932-client-token-request-proof
- auth/dpop:sep-1932-client-dpop-auth-scheme
- auth/dpop:sep-1932-client-fresh-proof
- auth/dpop-nonce:sep-1932-client-token-request-proof
- auth/dpop-nonce:sep-1932-client-dpop-auth-scheme
- auth/dpop-nonce:sep-1932-client-fresh-proof
- auth/dpop-nonce:sep-1932-client-as-nonce
- auth/dpop-nonce:sep-1932-client-rs-nonce
# Workload identity federation: the OAuth client does not implement the
# urn:ietf:params:oauth:grant-type:jwt-bearer grant (it answers with
# authorization_code). Per-check for the same reason.
- auth/wif-jwt-bearer:wif-grant-type
server:
# SEP-2663 (io.modelcontextprotocol/tasks): the SDK does not implement the
# tasks extension yet. The bare `--suite all` leg is the only one that
# reaches these extension-tagged scenarios (they never match a
# --spec-version filter), so these entries live here only.
#
# `tasks-status-notifications` is intentionally NOT listed: the harness
# skips it unconditionally (pending its rewrite against subscriptions/
# listen), and a baseline entry for a scenario with no failing checks is
# flagged stale.
- tasks-lifecycle
- tasks-capability-negotiation
- tasks-wire-fields
- tasks-request-state-removal
- tasks-mrtr-input
- tasks-request-headers
- tasks-dispatch-and-envelope
- tasks-required-task-error
- tasks-mrtr-composition

View file

@ -1,108 +0,0 @@
#!/bin/bash
# Run a client conformance suite, re-verifying unexpected failures solo.
# Concurrent suite runs on a hosted runner can push scenarios with real-time
# waits past tolerance; solo, a real failure fails again while a contention
# artifact passes. Failures that only reproduce under concurrency are excused.
set -uo pipefail
: "${CONFORMANCE_PKG:?set CONFORMANCE_PKG (pinned in .github/workflows/conformance.yml)}"
# One attempt: a solo failure on the quiet runner disproves the contention
# hypothesis; a second try would be the blind retry this script avoids.
SOLO_ATTEMPTS="${CONFORMANCE_SOLO_ATTEMPTS:-1}"
# Relative args resolve from the repo root; same contract as run-server.sh.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR/../../.." || exit 1
log="$(mktemp)"
trap 'rm -f "$log"' EXIT
npx --yes "$CONFORMANCE_PKG" client "$@" 2>&1 | tee "$log"
rc=${PIPESTATUS[0]}
if [ "$rc" -eq 0 ]; then
exit 0
fi
plain="$(sed 's/\x1b\[[0-9;]*m//g' "$log")"
# If the harness's summary wording changes, the list comes up empty and the
# original exit code passes through - never a false green.
mapfile -t scenarios < <(
printf '%s\n' "$plain" |
sed -n '/^Unexpected failures (not in baseline):$/,/^$/p' |
sed -n 's/^ ✗ //p'
)
if [ "${#scenarios[@]}" -eq 0 ]; then
exit "$rc"
fi
for scenario in "${scenarios[@]}"; do
if ! [[ "$scenario" =~ ^[A-Za-z0-9/_-]+$ ]]; then
echo "Extracted unexpected-failure name '${scenario}' does not look like a scenario name; passing the suite failure through." >&2
exit "$rc"
fi
done
# A stale baseline entry is a configuration error a solo rerun cannot excuse.
# Here-string, not a pipe: grep -q quitting early would SIGPIPE printf and,
# under pipefail, skip this guard exactly when the pattern is present.
if grep -q '^Stale baseline entries' <<<"$plain"; then
echo "Suite also reported stale baseline entries; not retrying." >&2
exit "$rc"
fi
# Drop the suite-only flags: --scenario replaces --suite, and solo runs are
# judged directly rather than against the baseline.
rerun_args=()
output_dir=""
skip_next=0
expect_output_dir=0
for arg in "$@"; do
if [ "$skip_next" -eq 1 ]; then
if [ "$expect_output_dir" -eq 1 ]; then
output_dir="$arg"
fi
skip_next=0
expect_output_dir=0
continue
fi
case "$arg" in
--output-dir)
skip_next=1
expect_output_dir=1
;;
--suite | --expected-failures) skip_next=1 ;;
--output-dir=*) output_dir="${arg#--output-dir=}" ;;
--suite=* | --expected-failures=*) ;;
*) rerun_args+=("$arg") ;;
esac
done
if [ -n "$output_dir" ]; then
rerun_args+=(--output-dir "${output_dir}-solo")
fi
for scenario in "${scenarios[@]}"; do
passed=0
for attempt in $(seq 1 "$SOLO_ATTEMPTS"); do
echo ""
echo "Re-running '${scenario}' solo (attempt ${attempt}/${SOLO_ATTEMPTS})..."
if npx --yes "$CONFORMANCE_PKG" client --scenario "$scenario" "${rerun_args[@]}"; then
passed=1
break
fi
done
if [ "$passed" -ne 1 ]; then
echo "'${scenario}' still fails when run alone: real failure, not suite contention." >&2
exit 1
fi
done
if [ -n "$output_dir" ]; then
mkdir -p "$output_dir"
printf '%s\n' "${scenarios[@]}" > "$output_dir/FLAKE_RESCUED"
fi
echo "All ${#scenarios[@]} unexpected failure(s) passed when re-run solo; the suite failures were parallel-run contention."
# Surface the rescue on the run summary; the log line above and the
# FLAKE_RESCUED marker in the uploaded results are otherwise easy to miss on a
# green run.
echo "::warning title=Conformance flake rescued::${scenarios[*]} failed under suite concurrency and passed solo${output_dir:+ (see ${output_dir}-solo and FLAKE_RESCUED in the uploaded results)}."
exit 0

View file

@ -1,51 +0,0 @@
#!/bin/bash
set -e
PORT="${PORT:-3001}"
SERVER_URL="http://localhost:${PORT}/mcp"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR/../../.."
# Refuse to start if something is already listening on the port. The readiness
# check below cannot tell our server apart from a stale one, so a leftover
# listener would mean silently running conformance against old code.
if (: > "/dev/tcp/localhost/${PORT}") 2>/dev/null; then
echo "Error: port ${PORT} is already in use." >&2
echo "Stop the stale process first (lsof -ti:${PORT} -sTCP:LISTEN | xargs kill) or set PORT to a free port." >&2
exit 1
fi
echo "Starting mcp-everything-server on port ${PORT}..."
uv run --frozen mcp-everything-server --port "$PORT" &
SERVER_PID=$!
cleanup() {
echo "Stopping server (PID: ${SERVER_PID})..."
kill $SERVER_PID 2>/dev/null || true
wait $SERVER_PID 2>/dev/null || true
}
trap cleanup EXIT
# Wait for server to be ready. --max-time keeps a hung listener from wedging
# the loop, and a dead server process fails fast instead of retrying.
echo "Waiting for server to be ready..."
MAX_RETRIES=30
RETRY_COUNT=0
while ! curl -s --max-time 2 "$SERVER_URL" > /dev/null 2>&1; do
if ! kill -0 $SERVER_PID 2>/dev/null; then
echo "Server process exited unexpectedly" >&2
exit 1
fi
RETRY_COUNT=$((RETRY_COUNT + 1))
if [ $RETRY_COUNT -ge $MAX_RETRIES ]; then
echo "Server failed to start after ${MAX_RETRIES} retries" >&2
exit 1
fi
sleep 0.5
done
echo "Server ready at $SERVER_URL"
npx --yes "${CONFORMANCE_PKG:?set CONFORMANCE_PKG (pinned in .github/workflows/conformance.yml)}" \
server --url "$SERVER_URL" "$@"

View file

@ -1,22 +0,0 @@
version: 2
updates:
- package-ecosystem: "uv"
directory: "/"
schedule:
interval: monthly
cooldown:
default-days: 14
groups:
python-packages:
patterns:
- "*"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: monthly
cooldown:
default-days: 14
groups:
github-actions:
patterns:
- "*"

View file

@ -1,38 +0,0 @@
<!--
Pull requests from outside the maintainer team need to link an open issue that
a maintainer has assigned to you (or one labeled `help wanted`); others are
closed automatically until that's in place. See CONTRIBUTING.md for details.
-->
Fixes #
<!-- Provide a brief summary of your changes -->
## Motivation and Context
<!-- Why is this change needed? What problem does it solve? -->
## How Has This Been Tested?
<!-- Have you tested this in a real application? Which scenarios were tested? -->
## Breaking Changes
<!-- Will users need to update their code or configurations? -->
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Documentation update
## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes that apply. -->
- [ ] I am assigned to the linked issue (or it is labeled `help wanted`, or I'm a maintainer)
- [ ] I have disclosed any AI assistance and can explain the change in my own words
- [ ] I have read the [MCP Documentation](https://modelcontextprotocol.io)
- [ ] My code follows the repository's style guidelines
- [ ] New and existing tests pass locally
- [ ] I have added appropriate error handling
- [ ] I have added or updated documentation as needed
## Additional context
<!-- Add any other context, implementation notes, or design decisions -->

View file

@ -1,117 +0,0 @@
// Docs preview gate and PR comment. .github/workflows/docs-preview.yml wires
// these up: `authorize` decides whether a run may build and deploy a preview
// (and for which commit), `comment` reports the outcome on the pull request.
// The security model is described in the workflow's header.
'use strict';
const MARKER = '<!-- docs-preview -->';
const BOT_LOGIN = 'github-actions[bot]';
// Sets the job outputs `authorized`, `pr_number`, `head_sha` and
// `slash_attempt` for a pull_request_target or /preview-docs issue_comment run.
async function authorize({ github, context, core }) {
const { owner, repo } = context.repo;
async function permissionFor(username) {
const { data } = await github.rest.repos.getCollaboratorPermissionLevel({ owner, repo, username });
return { level: data.permission, role: data.role_name };
}
let authorized = false;
let prNumber = '';
let headSha = '';
let slashAttempt = false;
if (context.eventName === 'pull_request_target') {
const pr = context.payload.pull_request;
prNumber = String(pr.number);
headSha = pr.head.sha;
// No automatic preview for fork PRs: actions/checkout refuses to fetch a
// fork's head in a pull_request_target run. A maintainer can still request
// one with /preview-docs. (head.repo is null once the fork is deleted.)
const headRepo = pr.head.repo;
if (!headRepo || headRepo.id !== context.payload.repository.id) {
core.info(`PR #${prNumber} head is on ${headRepo ? headRepo.full_name : 'a deleted fork'}; fork PRs are previewed via /preview-docs only.`);
} else {
// Gate on the *sender* (whoever caused this run — on synchronize that
// is the pusher), not the PR author, so a non-admin pushing to an
// admin-opened branch does not get an automatic build.
const actor = context.payload.sender.login;
const perm = await permissionFor(actor);
authorized = perm.level === 'admin';
core.info(`pull_request_target by ${actor} (level=${perm.level}, role=${perm.role}) → authorized=${authorized}`);
}
} else {
// issue_comment: the job-level `if:` already guarantees this is a PR
// comment starting with /preview-docs.
slashAttempt = true;
const actor = context.payload.comment.user.login;
prNumber = String(context.payload.issue.number);
const perm = await permissionFor(actor);
authorized = perm.level === 'admin' || perm.role === 'maintain';
if (authorized) {
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: Number(prNumber) });
if (pr.state !== 'open') {
authorized = false;
core.info(`PR #${prNumber} is ${pr.state}; refusing to preview.`);
} else {
headSha = pr.head.sha;
}
}
core.info(`/preview-docs by ${actor} (level=${perm.level}, role=${perm.role}) → authorized=${authorized}`);
}
core.setOutput('authorized', String(authorized));
core.setOutput('pr_number', prNumber);
core.setOutput('head_sha', headSha);
core.setOutput('slash_attempt', String(slashAttempt));
}
// Posts or updates the preview comment on the PR. Reads the outcome of the
// earlier jobs from the step's env: AUTHORIZED, PR_NUMBER, HEAD_SHA,
// DEPLOY_RESULT, DEPLOYMENT_URL, ALIAS_URL, RUN_URL.
async function comment({ github, context }) {
const { owner, repo } = context.repo;
const env = process.env;
const issue_number = Number(env.PR_NUMBER);
async function upsert(body) {
const comments = await github.paginate(github.rest.issues.listComments, { owner, repo, issue_number, per_page: 100 });
const existing = comments.find((c) => c.user?.login === BOT_LOGIN && c.body?.includes(MARKER));
if (existing) {
await github.rest.issues.updateComment({ owner, repo, comment_id: existing.id, body });
} else {
await github.rest.issues.createComment({ owner, repo, issue_number, body });
}
}
if (env.AUTHORIZED !== 'true') {
await github.rest.issues.createComment({
owner, repo, issue_number,
body: `@${context.actor} — only repository admins or maintainers can run \`/preview-docs\` (and the PR must be open).`,
});
return;
}
if (env.DEPLOY_RESULT !== 'success') {
await upsert(
`${MARKER}\n### 📚 Documentation preview\n\n` +
`❌ Preview build **failed** for \`${env.HEAD_SHA.slice(0, 7)}\` — [workflow logs](${env.RUN_URL}).`
);
return;
}
const previewUrl = env.ALIAS_URL || env.DEPLOYMENT_URL;
const ts = new Date().toISOString().replace('T', ' ').replace(/\.\d+Z$/, ' UTC');
await upsert(
`${MARKER}\n### 📚 Documentation preview\n\n` +
`| | |\n|---|---|\n` +
`| **Preview** | ${previewUrl} |\n` +
`| **Deployment** | ${env.DEPLOYMENT_URL} |\n` +
`| **Commit** | \`${env.HEAD_SHA.slice(0, 7)}\` |\n` +
`| **Triggered by** | @${context.actor} |\n` +
`| **Updated** | ${ts} |\n`
);
}
module.exports = { authorize, comment };

Some files were not shown because too many files have changed in this diff Show more