* docs: add SDK integration guide and fix spec references
- Add SDK_INTEGRATION.md with guide for SDK maintainers on integrating conformance tests
- Link to guide from README.md
- Update spec references to use 2025-11-25 instead of draft URLs
* fix: prettier formatting and correct action version to v0.1.10
Add support for SDKs that don't yet pass all conformance tests to still
run conformance in CI without failing, by specifying a YAML baseline file
of known/expected failures.
CLI changes:
- Add --expected-failures <path> flag to both server and client commands
- Exit 0 if only expected failures occur
- Exit 1 if unexpected failures occur (regression)
- Exit 1 if an expected failure now passes (stale baseline, must update)
GitHub Action (action.yml):
- Composite action that builds from source and runs conformance tests
- SDKs reference it as: uses: modelcontextprotocol/conformance@<ref>
- Inputs: mode, url, command, expected-failures, suite, scenario, etc.
New files:
- src/expected-failures.ts: YAML loading, baseline evaluation, reporting
- src/expected-failures.test.ts: 16 unit tests
- action.yml: composite GitHub Action
Closes#99
* Refactor client testing to pass scenario name and server URL
The CLI now passes both scenario name and server URL as arguments to the
client command (instead of just the server URL). This enables a single
"everything" client that can handle all scenarios based on the scenario
name it receives.
Changes:
- Update executeClient() to pass scenarioName as first argument to client
- Add everything-client.ts that routes to appropriate behavior based on
scenario name
- Update README with new usage examples and documentation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Use env var for scenario name instead of positional arg
Pass scenario name via MCP_CONFORMANCE_SCENARIO env var instead of as a
positional argument. This is non-breaking for existing clients that just
accept <server-url>.
The env var approach is consistent with how context (including private
keys for JWT auth) is already passed via MCP_CONFORMANCE_CONTEXT.
* Add comment explaining env var design decision
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Olivier Chafik <ochafik@anthropic.com>
* spec for servers
* initial version of everything server and tests - tets are very simple
* some clean up
* lint etc.
* fix package lock agian
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
