* feat: support "draft" as a first-class spec version target
- Add LATEST_SPEC_VERSION and DATED_SPEC_VERSIONS constants so the next
spec release is a one-line change in types.ts.
- Accept "draft" as a valid protocolVersion in the initialize check and
mock-server response.
- --spec-version draft now selects latest-dated scenarios plus draft-tagged
ones, so SEP authors can run the full suite against an SDK tracking the
in-progress spec without retagging core scenarios.
- Forward --spec-version to the client process via
MCP_CONFORMANCE_SPEC_VERSION so SDK examples can pick the matching
protocolVersion.
Closes#253
* review: collapse SpecVersion to wire strings; dedup NEGOTIABLE_PROTOCOL_VERSIONS; split out 'extension'
Addresses review feedback from @felixweinberger and @mikekistler:
- SpecVersion is now always a wire protocolVersion string
(DatedSpecVersion | typeof DRAFT_PROTOCOL_VERSION). The separate
'draft' tag literal is gone from the type system; 'draft' survives
only as a CLI input alias in resolveSpecVersion. Removes the
tag-vs-wire confusion and the specVersionToProtocolVersion mapping.
- NEGOTIABLE_PROTOCOL_VERSIONS in types.ts is the single source for
what the mock server accepts on initialize (was duplicated in
checks/client.ts and scenarios/client/initialize.ts).
- 'extension' moved out of SpecVersion into a separate ScenarioSpecTag
type. --spec-version extension is no longer valid; extension scenarios
remain reachable via --suite extensions.
- Draft-tagged scenarios now use the DRAFT_PROTOCOL_VERSION constant
so a draft revision bump is a one-line change in types.ts.
Adds src/scenarios/server/negative.test.ts with spawn helpers. Wires
two cases: the previously-orphaned no-dns-rebinding-protection.ts and
a new sep-2164-empty-contents.ts. Strengthens the AGENTS.md guidance
to point at the harness.
* feat: add conformance scenario for SEP-2164 resource-not-found error code
Adds a draft-scoped scenario verifying servers return -32602 (Invalid
Params) for non-existent resources instead of the legacy -32002 or an
empty contents array. Includes a sep-2164.yaml traceability file mapping
each MUST/MUST NOT/SHOULD to its check per the proposed SEP-2484 process.
* fix: align SEP-2164 check severities with spec text (#246)
Splits sep-2164-error-code into:
- sep-2164-no-empty-contents (MUST NOT -> FAILURE)
- sep-2164-error-code (SHOULD -> WARNING)
Updates sep-2164.yaml to quote the spec text verbatim and adds an
excluded row for the client-side -32002 backcompat SHOULD.
See #245 for the SHOULD/WARNING tier-scoring policy discussion.
---------
Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com>
* feat: add specVersions classification to conformance scenarios
Each scenario declares which spec versions it applies to as a list.
Scenarios that carry forward (e.g. initialize) list all applicable versions
['2025-06-18', '2025-11-25']. Scenarios removed from newer specs (e.g.
backcompat auth) only list their original version ['2025-03-26'].
- specVersions list on Scenario and ClientScenario interfaces
- --spec-version CLI filter uses simple .includes()
- Tier-check conformance matrix (Server / Client: Core / Client: Auth)
with per-version columns and unique All* count
- 7 unit tests for specVersions helpers
- Updated tier-audit skill docs with matrix format
* fix: fix console output template for tier audit skill
The template had an orphan table header (Check | Value | T2 | T1)
with no rows above the conformance matrix, causing an empty table
to render. The scorecard rows below the matrix also lacked their
own header.
Fix: two self-contained tables with clear labels — 'Conformance:'
for the per-version matrix, 'Scorecard:' for the check rows.
* fix: align skill console template with tier-check script output
Add asterisk footnote ('unique scenarios — a scenario may apply to
multiple spec versions') and rename 'Scorecard' to 'Repository Health'
to match the labels used by the tier-check CLI output.
* fix: add specVersions to CrossAppAccessCompleteFlowScenario
Added in 83c446d on main after this branch diverged.
* feat: add DNS rebinding protection conformance tests
Add a new server conformance scenario to verify that localhost MCP servers
properly validate Host headers to prevent DNS rebinding attacks.
Checks:
- localhost-host-rebinding-rejected: Verifies server returns 403 for
non-localhost Host headers (e.g., evil.example.com)
- localhost-host-valid-accepted: Verifies server accepts requests with
valid localhost Host headers
Also updates the everything-server example to use createMcpExpressApp()
which includes DNS rebinding protection by default.
Closes#103
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: add negative test case for DNS rebinding protection
Add a minimal MCP server that intentionally omits DNS rebinding protection
to serve as a negative test case. This server is expected to FAIL the
dns-rebinding-protection scenario.
Also update README to document the negative test case.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: accept any 4xx status code for DNS rebinding rejection
Update the DNS rebinding protection test to accept any 4xx status code
as a valid rejection response, not just 403. This accommodates different
HTTP semantics:
- 403 Forbidden: Per MCP spec
- 421 Misdirected Request: RFC 7540 (semantically correct for DNS rebinding)
- Other 4xx: Implementation-specific error codes
* refactor: address PR review comments
- Simplify negative test server (remove event store, sessions)
- Update spec reference URL to security_best_practices
- Clarify scope: localhost servers without HTTPS/auth
- DRY up checks.push using spread operator
- Fix valid host check to require 2xx response
* fix: include error details in catch block
* refactor: simplify negative test server to stateless
* refactor: check both Host and Origin headers for DNS rebinding protection
- Send both Host and Origin headers in test requests so servers checking
either header will pass the conformance test
- Add second spec reference for Origin check (transports#security-warning)
- Update descriptions to mention both Host and Origin headers
- Fix protocol version to 2025-11-25
- Remove unnecessary 'as const' type assertions
---------
Co-authored-by: Claude <noreply@anthropic.com>
spawn('npx', ...) fails on Windows because npx is actually npx.cmd.
Adding shell: true only on Windows fixes this while preserving
Unix signal handling behavior.
🦉 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* add cimd test and negative test
* Improve auth-test-no-cimd.ts documentation for clarity
Address review feedback: expanded comments to better explain what CIMD is,
why this client is non-compliant, and what behavior it's testing.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Remove .js extensions from relative imports
tsx doesn't require .js extensions for TypeScript imports.
Keeps .js for @modelcontextprotocol/sdk imports as those are external packages.
Also improves auth-test-no-cimd.ts documentation for clarity.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Skip auth/basic-cimd test until SDK support lands
Waiting on typescript-sdk PR #1127 for CIMD support.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix trailing whitespace in auth-test-no-cimd.ts
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Add conformance test to validate that MCP servers correctly preserve
JSON Schema 2020-12 keywords ($schema, $defs, additionalProperties)
in tool definitions.
Changes:
- Add JsonSchema2020_12Scenario in src/scenarios/server/json-schema-2020-12.ts
- Add json_schema_2020_12_tool to everything-server with tools/list override
to return raw JSON Schema instead of Zod-converted schema
- Register scenario in pendingClientScenariosList (requires SDK PR #1135)
The test is added to pending because the client SDK currently strips
the JSON Schema 2020-12 keywords. Once PR #1135 is merged into the SDK,
this scenario can be moved to the active list.
* clean up constructor calls for server lifecycle
* add back compat test
* fix dup logging, rm prm
* avoid fallbacks
* better error from example client
* add final endpoint method
* add test for OASM priority order
* s/prepack/prepare/ for npx
* try switching to .js
* make tsdown version happy default
* fix lock
* fix prmPath issue
* back to main package-lock
* fix bin path
* Add comprehensive auth scenario test suite
- Add index.test.ts to test all auth scenarios with auth-test.ts client
- Upgrade @modelcontextprotocol/sdk from 1.20.1 to 1.22.0
- All 5 auth scenarios now pass including auth/basic-metadata-var2
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* consolidate tests into index.test.ts
* rename metadata
* fix comments
* rework auth scenario listing
* add 3rd variation of metadata
* fix some type issues introduced by version bump
* add reusable spec references
* more refs
* fix up spec references
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add SEP-1330 conformance test for elicitation enum schemas
Adds server-side conformance test that validates servers properly request
elicitation with SEP-1330 enum schema improvements:
- Untitled single-select enum (type: string, enum: [...])
- Titled single-select enum (oneOf with const/title)
- Legacy titled enum (enumNames for backward compatibility)
- Untitled multi-select enum (type: array, items.enum)
- Titled multi-select enum (items.anyOf with const/title)
Test expects server to implement `test_elicitation_sep1330_enums` tool.
* Add reference server implementation for SEP-1330 conformance test
Adds test_elicitation_sep1330_enums tool to everything-server that
requests elicitation with all 5 enum schema variants. Also updates
SERVER_REQUIREMENTS.md with full specification.
* Fix prettier formatting
* only test active tests (#24)
* refactor maps to pull names from the test
* bring names back
* separate active from pending
* only test active
---------
Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com>
Add comprehensive conformance test for SEP-1034 which extends MCP
elicitation to support default values across all primitive types.
Changes:
- Add test_elicitation_sep1034_defaults tool to reference server
- Create ElicitationDefaultsScenario with 5 granular checks:
* String schema default value
* Integer schema default value
* Number schema default value
* Enum schema default value (validates it's a valid enum member)
* Boolean schema default value (regression test)
- Register scenario in index.ts
The test validates that servers properly include default values in
elicitation requestedSchema per SEP-1034 specification.
Test results: All 5 checks pass against TypeScript reference server
* spec for servers
* initial version of everything server and tests - tets are very simple
* some clean up
* lint etc.
* fix package lock agian
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
