## Motivation and Context
The three conformance tasks were named `conformance`, `conformance_list`, and `conformance_server`,
spelling out with underscores the grouping that Rake expresses with namespaces.
They are now `conformance:test`, `conformance:list`, and `conformance:server`.
The bare `conformance` task is kept as an alias for `conformance:test`.
## How Has This Been Tested?
By running `rake -T` to confirm the three tasks are grouped under `conformance:`
with the alias listed beside them, `rake --dry-run default` to confirm that the default task
resolves `conformance:test`, and `rake conformance:list` to exercise a renamed task end to end.
## Breaking Changes
None. These are development-only Rake tasks.
## Motivation and Context
The npm `latest` dist-tag of `@modelcontextprotocol/conformance` points at 0.1.16, the old 0.1 line,
while the 2026-07-28 requirement revisions and the frozen-requirements framework live in the 0.2.0 alpha line
(`alpha` currently resolves 0.2.0-alpha.11). The runners invoked the bare package name, so `rake conformance`
and the conformance CI job resolve whatever `latest` points at: a fresh machine silently downgrades to
the old suite while a warm npx cache may keep serving a newer one, and the suite actually in use depends
on the environment. Pinning the `alpha` dist-tag puts every run on the line the frozen conformance requirements
are anchored to (0.2.0-alpha.10, published 2026-07-27).
Running the real alpha suite surfaced the actual client-leg state, so the expected-failures baseline is
updated to match: `initialize` now passes (the harness accepts 2026-07-28, fulfilling the TODO on the old entry),
and the scenarios pending client-side work are recorded with their reasons: the 2026-07-28 stateless lifecycle
and the SEP-2322 MRTR driver, SEP-2243 custom-header mirroring, client-side JSON Schema preservation,
the DPoP and WIF authorization extensions, and one `auth/authorization-server-migration` check added to
the suite after the alpha.10 anchor.
The manual `npx` examples in `conformance/README.md` carry the same pin so local reproduction runs
the same suite as the rake task.
## How Has This Been Tested?
`npx --yes @modelcontextprotocol/conformance@alpha --version` resolves 0.2.0-alpha.11,
and `bundle exec rake conformance` passes the baseline check on both legs: the server leg reports 73 passed with 0 failed
and no expected failures, and the client leg fails exactly the scenarios the baseline records.
## Breaking Changes
None. The change is confined to the conformance tooling; the shipped gem is untouched.
## Motivation and Context
PR #248 introduced server conformance testing using `@modelcontextprotocol/conformance`.
This extends coverage to the client side so that `MCP::Client` behavior can also be validated against
the MCP specification.
### Scope
Client conformance is driven by the conformance runner, which spawns test servers for each scenario
and invokes `conformance/client.rb` against them. The client script delegates the MCP initialize
handshake to `MCP::Client#connect` and executes scenario-specific operations (for example listing
tools or calling a tool).
### Usage
Run all conformance tests (server + client):
```console
bundle exec rake conformance
```
Run a single client scenario:
```console
bundle exec rake conformance SCENARIO=initialize
```
List available scenarios:
```console
bundle exec rake conformance_list
```
## How Has This Been Tested?
Ran `bundle exec rake conformance` locally.
- Server conformance: 40 passed, 0 failed.
- Client conformance: 3 passed, 54 failed (all 54 listed in `expected_failures.yml`).
## Breaking Change
No. This change only adds test infrastructure under `conformance/` and does not modify SDK behavior.
## Motivation
To assess and track the Ruby SDK's compliance with the MCP
specification, integrate `@modelcontextprotocol/conformance` as the
standard evaluation harness. This makes it possible to measure
conformance objectively, catch regressions early, and produce the kind
of evaluation data needed for SDK tier assessments.
## Scope
Only server conformance is included at this point. Client conformance
requires protocol-level features that MCP::Client does not yet
support (initialize handshake, notifications, ping, etc.).
## Usage
Run conformance tests:
```console
bundle exec rake conformance
```
Start the conformance server in the foreground for iterating on a single
scenario without the per-run startup overhead:
```console
bundle exec rake conformance_server
```
## Default Rake Task (`bundle exec rake`)
It is configured to run in order of lower execution cost: `rubocop`, `test`, and then `conformance`.
## CI
`.github/workflows/conformance.yml` file has been added to run the conformance tests in CI.
## SDK Tier Report
The MCP SDK Tier system (https://modelcontextprotocol.io/community/sdk-tiers)
requires SDK maintainers to self-assess and report results to the SDK Working
Group via https://github.com/modelcontextprotocol/modelcontextprotocol/issues.
To generate a full tier assessment report, use the `/mcp-sdk-tier-audit`
slash command from the https://github.com/modelcontextprotocol/conformance repository with
the conformance server running:
```console
# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server
# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp
```
The skill evaluates conformance pass rate, issue label taxonomy, triage
metrics, documentation coverage, and policy compliance, then produces a
markdown report suitable for tier advancement submissions.
