ruby-sdk/conformance
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO 6e2b7a1971 Let Handlers Return Multi Round-Trip input_required Results per SEP-2322
## Motivation and Context

SEP-2322 (modelcontextprotocol/modelcontextprotocol#2322, merged for the 2026-07-28 MCP spec release)
replaces server-initiated JSON-RPC requests, which the stateless modern lifecycle (SEP-2575) forbids,
with multi round-trip results: a `tools/call`, `prompts/get`, or `resources/read` handler returns
`resultType: "input_required"` with `inputRequests` (server-assigned keys mapping to `elicitation/create`,
`sampling/createMessage`, or `roots/list` request shapes) and/or an opaque `requestState`,
and the client retries the ORIGINAL request with `inputResponses` under the same keys plus the echoed state.
The client-side recognition landed earlier (`MCP::ResultType`, `Client::InputRequiredError`);
this adds the server side, mirroring the Python SDK's low-level Server (python-sdk#2967/#2986)
and the TypeScript SDK's umbrella 2026-07-28 work.

- New `MCP::Server::InputRequiredResult` value object: construction validation
  (at least one of the two fields, embedded methods restricted to the three shapes), `to_h` wire serialization,
  and the capability mapping shared with the TypeScript SDK's `requiredClientCapabilitiesForInputRequest`
  (url-mode elicitation requires `elicitation.url`, form otherwise; sampling with `tools`/`toolChoice` requires
  `sampling.tools`; `roots/list` requires `roots`), including the 2025 back-compat rule that a bare `elicitation: {}`
  declaration implies form support.
- Handlers return the object through the existing paths: both branches of `call_tool_with_args` and
  `call_prompt_template_with_args` pass it through instead of calling `.to_h`, `call_tool` skips
  output schema validation and the structured-content fallback for it (output schema validation would otherwise
  run against a `nil` `structuredContent`), and the `resources/read` dispatch skips the `contents` wrapping and
  SEP-2549 cache-hint stamping.
- One central hook in the dispatch lambda gates and serializes the result, after the cancellation check
  so cancelled requests stay suppressed: a legacy request (no SEP-2575 envelope) gets an internal error,
  because pre-2026 clients treat an unknown `resultType` as a final result; embedded requests exceeding
  the request's declared client capabilities get `-32021` with the full merged `requiredCapabilities` set.
- `MCP::ServerContext` gains `input_responses`, `request_state`, and the key-tolerant `input_response(key)` reader.
  The retry fields are params-top-level siblings of `name`/`arguments`/`uri` (not `_meta`),
  so only handlers that opt in to `server_context:` can participate, matching the envelope readers' access model.
  The server holds no memory between rounds: handlers re-run from the start on every retry
  (deterministic replay, as in the Python SDK).

The 2026-07-28 conformance requirements score 14 `input-required-result-*` scenarios and the capability
checks of `server-stateless`, and each drives a diagnostic fixture tool the server under test must define.
The fixture defines the tools those scenarios call, mirroring the suite's TypeScript reference fixture
(`examples/servers/typescript/everything-server.ts`) in tool names and `inputRequests` keys:

- `test_input_required_result_elicitation` / `_sampling` / `_list_roots`: one embedded request each,
  answered on the retry round (also exercised by the missing-response, result-type, ignore-extra-params,
  and validate-input scenarios)
- `test_input_required_result_request_state`: stamps an opaque `requestState` and completes only after
  it round-trips
- `test_input_required_result_multiple_inputs`: all three embeddable methods in one result
- `test_input_required_result_multi_round`: two elicitation rounds with per-round `requestState`
- `test_input_required_result_capabilities`: builds `inputRequests` from the declared client capabilities,
  so a sampling-only client is never asked to elicit
- prompt `test_input_required_result_prompt`: the non-tool MRTR flow over `prompts/get`
- `test_missing_capability`: guards on `require_client_capability!(:sampling)`, giving
  the `server-stateless` capability checks a target that answers -32021 with HTTP 400

`test_input_required_result_tampered_state` is deliberately absent: rejecting a tampered `requestState` needs
the sealing machinery of the request-state security change, and its fixture tool lands there.
Sealing of the echoed `requestState` (it arrives as client-controlled input) and the client-side auto-fulfillment loop
follow in the next changes.

Refs #382.

## How Has This Been Tested?

New `test/mcp/server/input_required_result_test.rb` covers construction validation, key normalization and freezing,
the wire shape of `to_h`, the full capability mapping, missing-capability subtraction with symbol/string declarations,
and the implied-form back-compat rule.

New tests in `test/mcp/server_test.rb` drive `Server#handle` with the modern envelope: issuance wire shape for all three methods,
the retry leg exposing `input_responses`/`request_state`/ `input_response(key)` to the handler, the legacy-request internal error,
`-32021` with the merged `requiredCapabilities` data, output-schema validation bypass under `validate_tool_call_results: true`,
and `resources/read` results staying unwrapped without `ttlMs`/`cacheScope` stamping even when the server configures cache hints.

Against the fixture server at `--spec-version 2026-07-28`, all SEP-2322 checks of the 13 covered scenarios pass,
and `server-stateless` reports `sep-2575-server-rejects-undeclared-capability` and
`sep-2575-missing-capability-http-400` as SUCCESS. The `wire-schema-valid` failures on rounds that reach
a complete result come from the missing `resultType` stamp, which is the separate stamping change's job.
The `--requirements 2025-11-25` server leg still passes 78/78, and `bundle exec rake` is green.

## Breaking Changes

None. The serialization seams only branch on a return type that previously could not occur, all new keyword arguments default to `nil`,
and requests that carry no `inputResponses`/`requestState` behave exactly as before.
2026-08-08 15:07:16 +09:00
..
client.rb Finalize server/discover and Add Client Modern Lifecycle Support per SEP-2575 2026-08-08 03:45:00 +09:00
client_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
expected_failures.yml Support 2026-07-28 as the Latest Protocol Version 2026-07-31 12:55:21 +09:00
README.md Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
server.rb Let Handlers Return Multi Round-Trip input_required Results per SEP-2322 2026-08-08 15:07:16 +09:00
server_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Runs both server and client conformance tests in sequence. Server conformance starts the conformance server and tests it. Client conformance spawns test servers for each scenario and invokes conformance/client.rb against them. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port (server conformance only) 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off)
# Run a single server scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  server_runner.rb       # Starts the server, runs npx conformance server, exits with result code
  client.rb              # Conformance client (invoked by npx conformance client)
  client_runner.rb       # Runs npx conformance client, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios (server and client)
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.