modelcontextprotocol/ruby-sdk
Watch
1
0
Fork
You've already forked ruby-sdk
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ruby-sdk/conformance
History
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO da5f9225ac Support sampling/createMessage per MCP specification 
## Motivation and Context

The MCP specification defines `sampling/createMessage` to request LLM completions from the client.
The Ruby SDK did not yet support server-to-client requests,
so the server had no way to ask the client to sample an LLM.
This caused the `tools-call-sampling` conformance test to fail.

This adds `Server#create_sampling_message` to send a `sampling/createMessage` request
to the client via the transport layer. The method validates that the client declared
the `sampling` capability during initialization, and optionally checks the
`sampling.tools` capability when tools or tool_choice parameters are provided.

On the transport side, `Transport#send_request` is introduced as an abstract method,
with implementations in both `StdioTransport` and `StreamableHTTPTransport`.
Each transport assigns a unique request ID, sends the JSON-RPC request to the client,
and blocks until the client responds.

Client capabilities are stored per session via `ServerSession`, so validation is scoped
to the originating client. `ServerSession#create_sampling_message` uses
`build_sampling_params` for capability validation and sends the request directly through
the transport with the session ID, following the same pattern as `notify_progress`.

`Server#create_sampling_message` is the public API for single-client transports
(e.g., StdioTransport).

For multi-client transports (e.g., StreamableHTTPTransport),
`ServerSession#create_sampling_message` must be used to route requests to the correct client.
Tools using `server_context.create_sampling_message` automatically route to the correct session
regardless of transport.

`StreamableHTTPTransport#send_request` requires `session_id` to prevent broadcasting sampling requests
to all connected clients. Missing or invalid session IDs raise explicit errors instead of silently failing.

The `include_context` parameter is soft-deprecated in the MCP specification but is included
for compatibility with existing clients that declare the `sampling.context` capability.
Python and TypeScript SDKs also retain this parameter.

Ref: https://modelcontextprotocol.io/specification/latest/server/utilities/sampling

## How Has This Been Tested?

Added comprehensive tests in `test/mcp/server_sampling_test.rb` covering required and optional parameters,
capability validation, error handling, nil-param omission, per-session capability isolation,
and HTTP init capability scoping.

Added transport tests in `test/mcp/server/transports/stdio_transport_test.rb` and
`test/mcp/server/transports/streamable_http_transport_test.rb` for `send_request` round-trip behavior,
error responses, stateless mode rejection, missing session_id, invalid session_id, and no-active-stream error.

Conformance: `tools-call-sampling` is removed from expected failures. The conformance server's `TestSampling`
tool now calls `server_context.create_sampling_message` instead of returning a stub error.

This aligns the Ruby SDK with the MCP specification and other SDK implementations.

## Breaking Changes

None for end users.
`Transport#send_request` is a new abstract method. `ServerSession#create_sampling_message` is a new method.
2026-04-04 05:55:27 +09:00
..
expected_failures.yml Support sampling/createMessage per MCP specification 2026-04-04 05:55:27 +09:00
README.md Support running MCP conformance tests against the Ruby SDK 2026-03-02 14:50:03 +09:00
runner.rb Support running MCP conformance tests against the Ruby SDK 2026-03-02 14:50:03 +09:00
server.rb Support sampling/createMessage per MCP specification 2026-04-04 05:55:27 +09:00

README.md

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Starts the conformance server, runs all active scenarios against it, prints a pass/fail summary for each scenario, and exits with a non-zero status code if any unexpected failures are detected. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off) 
# Run a single scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  runner.rb              # Starts the server, runs npx conformance, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.