* feat: add resource parameter validation tests (RFC 8707)
Adds conformance tests for OAuth Resource Indicators (RFC 8707) implementation:
1. Resource parameter checks added to token-endpoint-auth-basic scenario:
- resource-parameter-in-authorization: Verify resource in auth request
- resource-parameter-in-token: Verify resource in token request
- resource-parameter-valid-uri: Verify valid canonical URI
- resource-parameter-consistency: Verify consistency between requests
2. New auth/resource-mismatch scenario:
- Tests that client rejects when PRM resource doesn't match server URL
- Server returns mismatched resource in PRM
- Test passes if client does NOT proceed with authorization
Also adds spec references for RFC 8707 and MCP resource parameter spec.
Closes#33
* fix: make resource consistency check a FAILURE and remove dead code
- Change resource parameter consistency from WARNING to FAILURE
- Remove unreachable protocol check in validateCanonicalUri
(URL constructor already validates scheme presence)
* feat: add PKCE conformance tests
Adds conformance checks for PKCE requirements from the MCP auth spec:
- pkce-code-challenge-sent: Verifies client sends code_challenge in auth request
- pkce-s256-method-used: Verifies client uses S256 code challenge method
- pkce-code-verifier-sent: Verifies client sends code_verifier in token request
- pkce-verifier-matches-challenge: Validates BASE64URL(SHA256(verifier)) = challenge
Also adds auth/pkce-no-s256-support scenario that tests clients correctly
refuse when S256 is not in code_challenge_methods_supported.
All PKCE checks are added to the createAuthServer helper, so they automatically
apply to all existing auth scenarios.
Fixes#77
* feat: add bad client example that skips PKCE
Adds auth-test-no-pkce.ts which implements a custom OAuth flow that
deliberately skips PKCE (no code_challenge in auth request, no
code_verifier in token request).
Also adds negative test to verify the conformance suite correctly
detects this non-compliant behavior.
* chore: skip pkce-no-s256-support scenario for now
* chore: fix formatting
* refactor: remove pkce-no-s256-support scenario and always validate PKCE
- Remove pkce-no-s256-support scenario (was skipped anyway)
- Make pkce-verifier-matches-challenge fail if either code_challenge
or code_verifier is missing, rather than only running when both present
- Update negative test to expect the new failure
* docs: add SDK integration guide and fix spec references
- Add SDK_INTEGRATION.md with guide for SDK maintainers on integrating conformance tests
- Link to guide from README.md
- Update spec references to use 2025-11-25 instead of draft URLs
* fix: prettier formatting and correct action version to v0.1.10
* feat: add pre-registration conformance test
Add conformance test for OAuth pre-registration flow where the server
does NOT support Dynamic Client Registration (DCR) and clients must
use pre-configured static credentials.
This addresses issue #34 (Client Registration Methods) - specifically
the pre-registration section that was previously not covered.
Changes:
- Add disableDynamicRegistration option to createAuthServer helper
- Add auth/pre-registration scenario with pre-registered client creds
- Add MCP_PREREGISTRATION spec reference
- Add context schema for pre-registration credentials
- Add withOAuthRetryWithProvider helper for pre-configured providers
- Add negative test client (auth-test-attempts-dcr.ts) that ignores
pre-registered credentials and fails
The scenario verifies that when a server does not advertise
registration_endpoint in its OAuth metadata, compliant clients
use pre-registered credentials passed via context instead of
attempting DCR.
Closes#34
* fix: add redirect_uris to saveClientInformation call
* chore: remove unused WithOAuthRetryOptions interface
* feat: add core and extensions suites for SDK tiering
Add new suites to support SDK tiering requirements:
Client suites:
- core (19 scenarios): Tier 1 requirements (base + auth)
- extensions (2 scenarios): Optional protocol extensions (client credentials)
- all (21 scenarios): Everything (core + extensions)
- auth (15 scenarios): Auth scenarios only (excludes extensions)
Server suites:
- core: Alias for 'active' (tier 1 requirements)
This allows SDK developers to run --suite core for tier 1 certification
while keeping extension tests available but not blocking.
* feat: promote elicitation tests from pending to active
Move 3 elicitation scenarios to active suite:
- elicitation-sep1330-enums
- tools-call-elicitation
- elicitation-sep1034-defaults
These tests are now passing in the Python SDK and ready for
SDK tiering requirements.
Remaining pending tests (2):
- json-schema-2020-12 (blocked on SDK PR #1135)
- server-sse-polling (blocked on SDK PR #1129)
* feat: add DNS rebinding protection conformance tests
Add a new server conformance scenario to verify that localhost MCP servers
properly validate Host headers to prevent DNS rebinding attacks.
Checks:
- localhost-host-rebinding-rejected: Verifies server returns 403 for
non-localhost Host headers (e.g., evil.example.com)
- localhost-host-valid-accepted: Verifies server accepts requests with
valid localhost Host headers
Also updates the everything-server example to use createMcpExpressApp()
which includes DNS rebinding protection by default.
Closes#103
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: add negative test case for DNS rebinding protection
Add a minimal MCP server that intentionally omits DNS rebinding protection
to serve as a negative test case. This server is expected to FAIL the
dns-rebinding-protection scenario.
Also update README to document the negative test case.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: accept any 4xx status code for DNS rebinding rejection
Update the DNS rebinding protection test to accept any 4xx status code
as a valid rejection response, not just 403. This accommodates different
HTTP semantics:
- 403 Forbidden: Per MCP spec
- 421 Misdirected Request: RFC 7540 (semantically correct for DNS rebinding)
- Other 4xx: Implementation-specific error codes
* refactor: address PR review comments
- Simplify negative test server (remove event store, sessions)
- Update spec reference URL to security_best_practices
- Clarify scope: localhost servers without HTTPS/auth
- DRY up checks.push using spread operator
- Fix valid host check to require 2xx response
* fix: include error details in catch block
* refactor: simplify negative test server to stateless
* refactor: check both Host and Origin headers for DNS rebinding protection
- Send both Host and Origin headers in test requests so servers checking
either header will pass the conformance test
- Add second spec reference for Origin check (transports#security-warning)
- Update descriptions to mention both Host and Origin headers
- Fix protocol version to 2025-11-25
- Remove unnecessary 'as const' type assertions
---------
Co-authored-by: Claude <noreply@anthropic.com>
Add support for SDKs that don't yet pass all conformance tests to still
run conformance in CI without failing, by specifying a YAML baseline file
of known/expected failures.
CLI changes:
- Add --expected-failures <path> flag to both server and client commands
- Exit 0 if only expected failures occur
- Exit 1 if unexpected failures occur (regression)
- Exit 1 if an expected failure now passes (stale baseline, must update)
GitHub Action (action.yml):
- Composite action that builds from source and runs conformance tests
- SDKs reference it as: uses: modelcontextprotocol/conformance@<ref>
- Inputs: mode, url, command, expected-failures, suite, scenario, etc.
New files:
- src/expected-failures.ts: YAML loading, baseline evaluation, reporting
- src/expected-failures.test.ts: 16 unit tests
- action.yml: composite GitHub Action
Closes#99
Add -o, --output-dir option to both client and server commands.
By default, no results directory is created - only console output.
When -o is specified, results are saved to that directory with
timestamped subdirectories.
This eliminates the need to add 'results/' to .gitignore in every
repo where conformance tests are run.
Claude-Generated-By: Claude Code (cli/claude-opus-4-5=100%)
Claude-Steers: 0
Claude-Permission-Prompts: 2
Claude-Escapes: 0
* Add client credentials support to everything-client
- Add ConformanceContextSchema with discriminated union for auth contexts
- Add runClientCredentialsJwt and runClientCredentialsBasic handlers
- Update runner to include scenario name in context for discriminated union parsing
- Update test helpers to set env vars for inline client runners
- Fix JWT audience validation to match SDK behavior (no trailing slash)
- Remove client credentials scenarios from skip list
* Accept both trailing slash forms for JWT audience per RFC 3986
Per RFC 3986, URLs with and without trailing slash are equivalent.
The MCP spec recommends (SHOULD) the form without trailing slash, but
since it's not a MUST, the conformance test should accept both forms
for interoperability with clients like Pydantic that normalize URLs.
* Remove .vscode from tracking and add to gitignore
* Address review feedback
- Remove unused issuerUrl variable and comment
- Simplify audience array to exactly 2 values with clearer naming
- Add comment explaining strip-then-add-back logic
- Rename ConformanceContextSchema to ClientConformanceContextSchema
* Remove dist from tracking and add to gitignore
* Fix .gitignore formatting
* Add getHandler to everything-client for cleaner test imports
Instead of importing individual handlers, tests can now import getHandler
and look up handlers by scenario name.
* Use getHandler exclusively, remove goodClient import
- Add CIMD support to runAuthClient in everything-client
- Add auth fallback to getHandler for unregistered auth/* scenarios
- Remove goodClient import from test file
* Remove runAuthClient fallback, register all auth scenarios explicitly
* [auth] Add metadata suite for running metadata discovery tests
Adds a new 'metadata' suite that runs just the auth/metadata-* scenarios
for faster iteration when testing metadata discovery specifically.
Usage: node dist/index.mjs client --suite metadata --command "..."
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Don't require lefthook to be installed
Set assert_lefthook_installed to false so the hooks gracefully skip
if lefthook is not installed on the system.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add token endpoint auth method conformance tests
Add three new scenarios to test that clients correctly use the
appropriate authentication method based on server metadata:
- auth/token-endpoint-auth-basic: Tests client_secret_basic (HTTP Basic)
- auth/token-endpoint-auth-post: Tests client_secret_post
- auth/token-endpoint-auth-none: Tests public client (no auth)
Each scenario configures the server to only support one auth method
and verifies the client uses the correct method in token requests.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor token endpoint auth tests to use shared createAuthServer
Extend createAuthServer helper with:
- tokenEndpointAuthMethodsSupported option for metadata
- onTokenRequest callback now receives full Request object
- onRegistrationRequest callback for custom client credentials
This eliminates the duplicate auth server implementation in
token-endpoint-auth.ts and reduces code by ~140 lines.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix onTokenRequest callback to use new interface
The createAuthServer onTokenRequest callback now passes an object with
authorizationHeader, body, timestamp etc. instead of (req, timestamp).
Also added missing return value for TokenRequestResult.
* Update @modelcontextprotocol/sdk to 1.25.1
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Olivier Chafik <ochafik@anthropic.com>
* Refactor client testing to pass scenario name and server URL
The CLI now passes both scenario name and server URL as arguments to the
client command (instead of just the server URL). This enables a single
"everything" client that can handle all scenarios based on the scenario
name it receives.
Changes:
- Update executeClient() to pass scenarioName as first argument to client
- Add everything-client.ts that routes to appropriate behavior based on
scenario name
- Update README with new usage examples and documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Use env var for scenario name instead of positional arg
Pass scenario name via MCP_CONFORMANCE_SCENARIO env var instead of as a
positional argument. This is non-breaking for existing clients that just
accept <server-url>.
The env var approach is consistent with how context (including private
keys for JWT auth) is already passed via MCP_CONFORMANCE_CONTEXT.
* Add comment explaining env var design decision
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Olivier Chafik <ochafik@anthropic.com>
- Update client initialization check to accept both 2025-06-18 and 2025-11-25
- Server echoes back client's version if valid (proper version negotiation)
- Fixes compatibility with SDK 1.25.1+ which sends 2025-11-25
🦉 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
spawn('npx', ...) fails on Windows because npx is actually npx.cmd.
Adding shell: true only on Windows fixes this while preserving
Unix signal handling behavior.
🦉 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: validate scope checks ran and report overall test status
- Add FAILURE checks to scope scenarios when authorization flow doesn't
complete (ScopeFromWwwAuthenticateScenario, ScopeFromScopesSupportedScenario,
ScopeOmittedWhenUndefinedScenario)
- Report client timeout and exit errors in test results
- Add OVERALL: PASSED/FAILED summary at end of test output
- Exit with code 1 on any failure (check failures, client timeout, or exit error)
Previously, if the OAuth authorization flow didn't complete, the scope
checks were never triggered and the test would incorrectly report success.
Now these scenarios emit a FAILURE if the expected scope check wasn't recorded.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Count warnings as failures in overall test result
Warnings now contribute to the overall failure status, causing the test
to exit with code 1 when any warnings are present. Also displays warning
checks in the output summary, similar to how failures are displayed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Count warnings as failures in suite runner
Update the suite summary to show ✗ for scenarios with warnings,
and exit with code 1 if any warnings are present across the suite.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add client credentials conformance scenarios with dynamic keypair generation
Implements SEP-1046 client credentials conformance tests:
- auth/client-credentials-jwt: Tests private_key_jwt authentication
- auth/client-credentials-basic: Tests client_secret_basic authentication
Key changes:
- Generate EC P-256 keypair dynamically at test start (no hardcoded keys)
- Pass credentials to client via MCP_CONFORMANCE_CONTEXT environment variable
- Add context field to ScenarioUrls interface for scenario-specific data
- Update client runner to pass context as env var to spawned client process
The MCP_CONFORMANCE_CONTEXT env var contains a JSON object with:
- client_id: The expected client identifier
- private_key_pem: PEM-encoded private key (for JWT scenarios)
- client_secret: Client secret (for basic auth scenarios)
- signing_algorithm: JWT signing algorithm (defaults to ES256)
* Fix client-credentials scenarios for conformance testing
- Generate EC keypair with extractable: true so private key can be
exported and passed to clients via MCP_CONFORMANCE_CONTEXT
- Fix client_secret_basic scenario to use authorizationHeader param
instead of non-existent headers object
* Fix jose type import - use CryptoKey instead of KeyLike
* Add OAuth 2.1 section 4.2 spec reference to client credentials checks
Add OAUTH_2_1_CLIENT_CREDENTIALS spec reference pointing to OAuth 2.1
draft section 4.2 (Client Credentials Grant) and include it in all
client_credentials conformance checks for both JWT and basic auth flows.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add JWT iss claim verification for client credentials
Per RFC 7523, verify that the JWT issuer (iss) claim matches the
expected client_id, in addition to the existing sub claim check.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* lefthook: move deps installs to optional local change
* add test scenario for infinite retry
and negative test
* Add broken client for retry limit negative test and fix lefthook formatting
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix www-authenticate in scope scenario
* add suites and shift pending to reflect current pending
* lint
---------
Co-authored-by: Claude <noreply@anthropic.com>
* add cimd test and negative test
* Improve auth-test-no-cimd.ts documentation for clarity
Address review feedback: expanded comments to better explain what CIMD is,
why this client is non-compliant, and what behavior it's testing.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Remove .js extensions from relative imports
tsx doesn't require .js extensions for TypeScript imports.
Keeps .js for @modelcontextprotocol/sdk imports as those are external packages.
Also improves auth-test-no-cimd.ts documentation for clarity.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Skip auth/basic-cimd test until SDK support lands
Waiting on typescript-sdk PR #1127 for CIMD support.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix trailing whitespace in auth-test-no-cimd.ts
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Add conformance test to validate that MCP servers correctly preserve
JSON Schema 2020-12 keywords ($schema, $defs, additionalProperties)
in tool definitions.
Changes:
- Add JsonSchema2020_12Scenario in src/scenarios/server/json-schema-2020-12.ts
- Add json_schema_2020_12_tool to everything-server with tools/list override
to return raw JSON Schema instead of Zod-converted schema
- Register scenario in pendingClientScenariosList (requires SDK PR #1135)
The test is added to pending because the client SDK currently strips
the JSON Schema 2020-12 keywords. Once PR #1135 is merged into the SDK,
this scenario can be moved to the active list.
* [auth] Add metadata suite for running metadata discovery tests
Adds a new 'metadata' suite that runs just the auth/metadata-* scenarios
for faster iteration when testing metadata discovery specifically.
Usage: node dist/index.mjs client --suite metadata --command "..."
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Don't require lefthook to be installed
Set assert_lefthook_installed to false so the hooks gracefully skip
if lefthook is not installed on the system.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Adds a --suite option to the client command that runs groups of related
tests concurrently and outputs a summary. Currently supports the 'auth' suite.
Usage: client --suite auth --command "tsx examples/clients/typescript/auth-test.ts"
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Claude <noreply@anthropic.com>
* wip scope change
* rm useless bool
* fix-me: package changes for linking
* negative test
* negative test for multiple scopes
* fix displays of warnings
* wip step-up
* negative test for step up auth
* move warning into main request flow
* clean up more checks
* cleanup step-up scenario
* cleanup and simplify
* cleanup middleware
* inline negative tests
* inline the other test
* refactor logging and inlining for standalone execution too
* skip pending tests
* clean up constructor calls for server lifecycle
* add back compat test
* fix dup logging, rm prm
* avoid fallbacks
* better error from example client
* add final endpoint method
* add test for OASM priority order
* s/prepack/prepare/ for npx
* try switching to .js
* make tsdown version happy default
* fix lock
* fix prmPath issue
* back to main package-lock
* fix bin path
* Add comprehensive auth scenario test suite
- Add index.test.ts to test all auth scenarios with auth-test.ts client
- Upgrade @modelcontextprotocol/sdk from 1.20.1 to 1.22.0
- All 5 auth scenarios now pass including auth/basic-metadata-var2
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* consolidate tests into index.test.ts
* rename metadata
* fix comments
* rework auth scenario listing
* add 3rd variation of metadata
* fix some type issues introduced by version bump
* add reusable spec references
* more refs
* fix up spec references
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add SEP-1330 conformance test for elicitation enum schemas
Adds server-side conformance test that validates servers properly request
elicitation with SEP-1330 enum schema improvements:
- Untitled single-select enum (type: string, enum: [...])
- Titled single-select enum (oneOf with const/title)
- Legacy titled enum (enumNames for backward compatibility)
- Untitled multi-select enum (type: array, items.enum)
- Titled multi-select enum (items.anyOf with const/title)
Test expects server to implement `test_elicitation_sep1330_enums` tool.
* Add reference server implementation for SEP-1330 conformance test
Adds test_elicitation_sep1330_enums tool to everything-server that
requests elicitation with all 5 enum schema variants. Also updates
SERVER_REQUIREMENTS.md with full specification.
* Fix prettier formatting
* only test active tests (#24)
* refactor maps to pull names from the test
* bring names back
* separate active from pending
* only test active
---------
Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com>
* feat: add client-side SEP-1034 conformance test for elicitation defaults
Adds a conformance test that validates client SDKs properly apply
default values for omitted fields in elicitation responses per SEP-1034.
The test spins up a server that sends elicitation requests with defaults,
and validates that the client returns content with those defaults applied.
* feat: use elicitInput() for SEP-1034 default validation
Updates client-side test to validate server-side default application
using elicitInput() helper, which applies defaults via validation layer.
This aligns with the TypeScript SDK's SEP-1034 implementation approach.
The test now validates:
- Server uses elicitInput() with schemas containing defaults
- Client returns empty/partial content
- Server automatically applies defaults during validation
- Defaults are correctly merged for all primitive types
* refactor: use server.request() instead of elicitInput() for client-side defaults test
Ensures conformance test validates true client-side default application
rather than server-side validation behavior.
* feat: add applyDefaults capability flag for opt-in default application
Enables conformance test to opt into client-side elicitation defaults
by setting the applyDefaults capability flag.
Add comprehensive conformance test for SEP-1034 which extends MCP
elicitation to support default values across all primitive types.
Changes:
- Add test_elicitation_sep1034_defaults tool to reference server
- Create ElicitationDefaultsScenario with 5 granular checks:
* String schema default value
* Integer schema default value
* Number schema default value
* Enum schema default value (validates it's a valid enum member)
* Boolean schema default value (regression test)
- Register scenario in index.ts
The test validates that servers properly include default values in
elicitation requestedSchema per SEP-1034 specification.
Test results: All 5 checks pass against TypeScript reference server
* readme for auth tests
* wip prm test
* add example helper for local oauth w/o browser
* working through dangling timeout issue
* add better test function
* pretty printing
* add mcp method
* some more logging tweaks
* refactor logger into 1 class
* use common request logger
* fix base scenario
* some renaming
* refactor into helpers
* refactor to support second test
* rm readme
* add comment about www-authenticate
* spec for servers
* initial version of everything server and tests - tets are very simple
* some clean up
* lint etc.
* fix package lock agian
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
