Watch

History

Repository files (latest commit first)
Filename	Latest commit message	Latest commit date
Koichi ITO bbe637612b Support progress notifications per MCP specification ## Motivation and Context The MCP specification defines `notifications/progress` for reporting incremental progress during long-running tool operations. The Ruby SDK did not yet support this, so tools had no way to send progress updates to clients. This caused the `tools-call-with-progress` conformance test to fail. This adds an `MCP::ServerContext` wrapper that provides a `report_progress` method on `server_context`. Tools call `server_context.report_progress(value, total:, message:)` to send notifications back to the client. The method is a no-op when no `progressToken` was provided, so tools can call it unconditionally without checking for a token. This design is consistent with the TypeScript and Python SDKs, which provide progress reporting through a context object rather than a separate parameter. On the client side, `Client#call_tool` now accepts a `progress_token:` parameter that sets `_meta.progressToken` in the request. Ref: https://modelcontextprotocol.io/specification/latest/server/utilities/progress ## How Has This Been Tested? Added comprehensive tests in `test/mcp/server_progress_test.rb` covering `notify_progress`, tool integration with progress, block-defined tools, `**kwargs` tools, and no-op behavior without a token. Added client tests for `progress_token:` parameter inclusion and omission. Added `test/mcp/server_context_test.rb` test for non-Hash `server_context` preservation when `_meta` is present. Conformance: `tools-call-with-progress` removed from expected failures. ## Breaking Change `server_context` is now always an `MCP::ServerContext` instance when passed to tools, even if the server has no context configured. Code that checks `server_context ?` to detect a nil context will need to be updated, as `ServerContext` is always truthy. `ServerContext` delegates method calls to the underlying context object via `method_missing`, so `server_context[:key]` and other accessor patterns continue to work. In practice, this is unlikely to affect users. A tool that declares `server_context:` is opting in to use the context; if the server has no context configured, that is a server configuration issue rather than something the tool should handle with a nil check.		2026-03-10 01:49:09 +09:00
..
expected_failures.yml	Support progress notifications per MCP specification	2026-03-10 01:49:09 +09:00
README.md	Support running MCP conformance tests against the Ruby SDK	2026-03-02 14:50:03 +09:00
runner.rb	Support running MCP conformance tests against the Ruby SDK	2026-03-02 14:50:03 +09:00
server.rb	Support progress notifications per MCP specification	2026-03-10 01:49:09 +09:00

README.md

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

Node.js (for npx)
bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Starts the conformance server, runs all active scenarios against it, prints a pass/fail summary for each scenario, and exits with a non-zero status code if any unexpected failures are detected. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable	Description	Default
`PORT`	Server port	`9292`
`SCENARIO`	Run a single scenario by name	(all)
`SPEC_VERSION`	Filter scenarios by spec version	(all)
`VERBOSE`	Show raw JSON output when set	(off)

# Run a single scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  runner.rb              # Starts the server, runs npx conformance, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.