ruby-sdk/conformance
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO e6b5a9b5e3 Add opt-in requestState sealing via RequestStateSecurity
## Motivation and Context

The SEP-2322 `requestState` continuation string leaves the server, sits in the client's hands,
and comes back as client-controlled input. Without protection a client can read the server's continuation state,
tamper with it (for example forging an already-answered elicitation), or replay a state issued for one call against
another tool, other arguments, or another server. The multi-round-trip documentation of the Python SDK is explicit
that the echo must be treated as untrusted input; its high-level server seals the state by default via `RequestStateBoundary`.

New `MCP::Server::RequestStateSecurity` (OpenSSL standard library only) brings that protection to this SDK as an opt-in:

- `seal` encrypts the plaintext state with AES-256-GCM (clients cannot read it, not merely verify it) inside a claims
  envelope binding an expiry window (`ttl:`, default 300 seconds, re-sealed each round), the originating method and target
  (tool/prompt name or resource URI), a digest of the originating arguments (stringified and sorted recursively,
  so symbol/string parses of identical JSON digest identically), and an optional `audience:`. The token format is
  `v1.<base64url(iv || ciphertext || tag)>` with the version prefix bound as GCM associated data,
  following the Python SDK's `AESGCMRequestStateCodec`.
- `unseal` verifies every claim fail-closed and raises `InvalidStateError` on tampering, expiry, or any mismatch.

Passing an instance via `Server.new(request_state_security:)` makes the seal/unseal transparent: issuance seals
the `requestState` of an outgoing `input_required` result, and dispatch unseals the echoed token for `tools/call`,
`prompts/get`, and `resources/read` before any handler runs, so `server_context.request_state` always reads
the plaintext the handler wrote. A tampered, expired, or cross-call echo is rejected as `-32602`
with "Invalid or expired requestState", matching the Python SDK's frozen error.

Without the option the state crosses the wire exactly as the handler wrote it (the Python low-level Server behavior);
the README documents that this is then the handler author's responsibility, and that multi-process deployments
must share the key across workers.

The conformance fixture opts in with a random per-boot key, both to dogfood the feature and because
the tampered-state scenario of the 2026-07-28 conformance requirements retries an MRTR request with
a corrupted `requestState` and requires a JSON-RPC error; without integrity-checked state the fixture
cannot detect the corruption. Every MRTR round trip completes within one server process,
so no key persistence is needed. Sealing is transparent to the fixture tools, whose `requestState` plaintext
round-trips unchanged, and the new `test_input_required_result_tampered_state` tool is therefore
a plain MRTR elicitation flow; the tampered echo is rejected with `-32602` by the server-level unsealing
before dispatch ever reaches it.

Refs #382.

## How Has This Been Tested?

New `test/mcp/server/request_state_security_test.rb` covers constructor validation,
the seal/unseal round trip (opaque `v1.` token, plaintext not visible), expiry via time travel,
fail-closed rejection of method/target/digest/audience mismatches, tampered and malformed tokens,
and tokens sealed under a different key.

New tests in `test/mcp/server_test.rb` drive `Server#handle` with the modern envelope:
transparent sealing on issuance and plaintext restoration on the retry leg, `-32602` for
tampered echoes and for echoes replayed against different arguments, and the pass-through
default when `request_state_security:` is not set.

Against the fixture server at `--spec-version 2026-07-28` (run with `@modelcontextprotocol/conformance@alpha`;
the 2026-07-28 scenarios are not yet in a stable conformance release), `sep-2322-reject-tampered-state` reports SUCCESS,
and the other 13 `input-required-result-*` scenarios stay green with sealing active,
confirming that sealed states round-trip through the suite's retries without false rejections
(the seal binds `method`, target, and an arguments digest, and the suite echoes all three unchanged).
The `--requirements 2025-11-25` server leg still passes.

## Breaking Changes

None. The feature is opt-in via a new keyword argument that defaults to `nil`, in which case behavior is byte-identical to before.
2026-08-08 18:54:14 +09:00
..
client.rb Finalize server/discover and Add Client Modern Lifecycle Support per SEP-2575 2026-08-08 03:45:00 +09:00
client_runner.rb Pin the conformance CLI to the alpha dist-tag 2026-08-08 18:47:24 +09:00
expected_failures.yml Pin the conformance CLI to the alpha dist-tag 2026-08-08 18:47:24 +09:00
README.md Pin the conformance CLI to the alpha dist-tag 2026-08-08 18:47:24 +09:00
server.rb Add opt-in requestState sealing via RequestStateSecurity 2026-08-08 18:54:14 +09:00
server_runner.rb Pin the conformance CLI to the alpha dist-tag 2026-08-08 18:47:24 +09:00

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Runs both server and client conformance tests in sequence. Server conformance starts the conformance server and tests it. Client conformance spawns test servers for each scenario and invokes conformance/client.rb against them. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port (server conformance only) 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off)
# Run a single server scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance@alpha server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance@alpha server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  server_runner.rb       # Starts the server, runs npx conformance server, exits with result code
  client.rb              # Conformance client (invoked by npx conformance client)
  client_runner.rb       # Runs npx conformance client, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios (server and client)
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.