| Filename | Latest commit message | Latest commit date |
|---|---|---|
da2a7f80d7
## Motivation and Context The modern (stateless) lifecycle of MCP 2026-07-28 serves each request as a self-contained exchange, and two delivery requirements of that model were unimplemented: - Notifications a handler emits while serving a modern request (progress, log messages) had nowhere to go: the modern path answered with a single JSON object, and the ephemeral per-request session degraded every notification to non-delivery. The conformance requirement expects them as frames on the request's own response stream (`tools-call-with-progress` fails with "No progress notifications received" at 2026-07-28). - The envelope's `logLevel` member, which replaces the removed `logging/setLevel` RPC, was parsed but never applied: log delivery fell back to the server-wide level, violating the requirement that no `notifications/message` is sent for requests that did not set `_meta` `logLevel`. The changes wire both up: - `StreamableHTTPTransport#handle_modern` registers a per-request notification sink for its ephemeral session; `send_notification` routes into the sink ahead of the stateless guard. A request whose handler emitted notifications answers as an SSE stream carrying them ahead of the final response; a request that emitted none keeps the single JSON exchange and its HTTP status ladder. - `Server#handle_request` applies the lifted envelope's `logLevel` to the session via the existing `configure_logging`, and on modern-era sessions `ServerSession#notify_log_message` no longer falls back to the server-wide level: without a per-request `logLevel` it stays silent, on every transport. An unrecognized level reads as absent, the safe direction, matching the Python SDK's `allowed_log_levels`. The `server-stateless` scenario of the 2026-07-28 conformance requirements probes two stream-discipline rules through diagnostic fixture tools the Ruby fixture did not define, so both checks failed as not testable: `sep-2575-http-server-no-independent-requests-on-stream` calls `test_streaming_elicitation` and requires that every frame on the request's response stream is a notification or the final response, and `sep-2575-server-no-log-without-loglevel` calls `test_logging_tool` without an envelope `logLevel` and requires that no `notifications/message` reaches the stream. The fixture now defines both tools. `test_streaming_elicitation` reports progress and completes; `test_logging_tool` calls `notify_log_message` unconditionally and relies on the SDK's per-request gate, which drops the notification for modern requests that did not authorize logging through `io.modelcontextprotocol/logLevel`. Both lean on the notification streaming this change adds: with a `progressToken` or an authorizing `logLevel`, the notification is delivered as an SSE frame ahead of the final response rather than being dropped. One known limitation relative to the TypeScript and Python SDKs, which stream notifications live while the handler runs: delivery here is buffered, flushing the frames after the handler returns. Ordering is preserved and the conformance checks are satisfied, but long-running handlers get no real-time interleaving. The sink is bounded (`MAX_MODERN_REQUEST_NOTIFICATIONS`, 1000): a handler emitting notifications in proportion to client-supplied input cannot grow one request's memory without bound, and past the cap the notify helpers report `false`, the same non-delivery degradation they have on every other undeliverable path. Live streaming can follow without changing the wire shape. ## How Has This Been Tested? `bundle exec rake test` passes with zero failures and RuboCop reports no offenses. New tests cover: a modern `tools/call` streaming `notifications/progress` frames ahead of the result, the single JSON exchange staying in place without notifications, log suppression without the envelope `logLevel`, log delivery with it, the modern-era session ignoring the server-wide level while honoring a per-request one, and the notification cap bounding one request's buffered frames while the notify helpers report `false` past it. Against the conformance fixture server, the `tools-call-with-progress` progress check passes at `--spec-version 2026-07-28` (its remaining wire-schema check belongs to the separate `resultType` change), the stream-discipline checks `sep-2575-http-server-no-independent-requests-on-stream` and `sep-2575-server-no-log-without-loglevel` report SUCCESS, and manual probes confirm the positive directions the suite leaves untested: with `io.modelcontextprotocol/logLevel` in `_meta`, `test_logging_tool` streams its `notifications/message` frame ahead of the result, and with a `progressToken`, `test_streaming_elicitation` streams its `notifications/progress` frame. The `--requirements 2025-11-25` server leg passes 78/78, unchanged. ## Breaking Changes None for stable protocol versions: legacy requests and their notification delivery are unchanged. Within the modern lifecycle, notifications become deliverable where they were previously dropped, and log messages without a per-request `logLevel` stop leaking through the server-wide level, as the specification requires. |
||
| .. | ||
| client.rb | ||
| client_runner.rb | ||
| expected_failures.yml | ||
| README.md | ||
| server.rb | ||
| server_runner.rb | ||
MCP Conformance Tests
Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.
Prerequisites
- Node.js (for
npx) bundle installcompleted
Running the Tests
Run all scenarios
bundle exec rake conformance
Runs both server and client conformance tests in sequence. Server conformance starts the
conformance server and tests it. Client conformance spawns test servers for each scenario
and invokes conformance/client.rb against them. Scenarios listed in expected_failures.yml
are allowed to fail without affecting the exit code.
Environment variables
| Variable | Description | Default |
|---|---|---|
PORT |
Server port (server conformance only) | 9292 |
SCENARIO |
Run a single scenario by name | (all) |
SPEC_VERSION |
Filter scenarios by spec version | (all) |
VERBOSE |
Show raw JSON output when set | (off) |
# Run a single server scenario
bundle exec rake conformance SCENARIO=ping
# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1
# Start the server on a specific port
bundle exec rake conformance_server PORT=3000
Start the server and test separately
# Terminal 1: start the server
bundle exec rake conformance_server
# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp
# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping
Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.
List available scenarios
bundle exec rake conformance_list
Prints all scenario names that can be passed to SCENARIO.
SDK Tier Report
The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.
To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from
the modelcontextprotocol/conformance
repository with the conformance server running:
# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server
# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp
The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.
File Structure
conformance/
server.rb # Conformance server (Rack + Puma, default port 9292)
server_runner.rb # Starts the server, runs npx conformance server, exits with result code
client.rb # Conformance client (invoked by npx conformance client)
client_runner.rb # Runs npx conformance client, exits with result code
expected_failures.yml # Baseline of known-failing scenarios (server and client)
README.md # This file
Known Limitations
Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to
fail without affecting the exit code and are tracked to catch regressions.
These are shown in the output of bundle exec rake conformance.