modelcontextprotocol/ruby-sdk
Watch
1
0
Fork
You've already forked ruby-sdk
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ruby-sdk/conformance
History
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO d80fba4fbc Support resource subscriptions per MCP specification 
## Motivation and Context

The MCP specification defines `resources/subscribe`, `resources/unsubscribe`,
and `notifications/resources/updated` for clients to monitor resource changes:
https://modelcontextprotocol.io/specification/2025-03-26/server/resources#subscriptions

The Ruby SDK had stub (no-op) handlers but provided no way for server developers to
customize subscription behavior or send update notifications.

Following the Python SDK approach, the SDK does not track subscription state internally.
Server developers register handler blocks and manage their own subscription state,
allowing flexibility for different subscription semantics
(per-session tracking, persistence, debouncing, etc.).

Three methods are added:

- `Server#resources_subscribe_handler`: registers a handler for
  `resources/subscribe` requests
- `Server#resources_unsubscribe_handler`: registers a handler for
  `resources/unsubscribe` requests
- `ServerSession#notify_resources_updated`: sends a
  `notifications/resources/updated` notification to the subscribing client

`ServerContext#notify_resources_updated` is also added so that tool handlers
can send the notification scoped to the originating session.

## How Has This Been Tested?

All tests pass (`rake test`), RuboCop is clean. New tests cover custom handler registration
for `resources/subscribe` and `resources/unsubscribe`, session-scoped `notify_resources_updated` notifications,
error handling, and `ServerContext` delegation.

## Breaking Changes

None.
2026-04-19 03:48:46 +09:00
..
expected_failures.yml Support elicitation/create per MCP specification 2026-04-13 15:21:45 +09:00
README.md Support running MCP conformance tests against the Ruby SDK 2026-03-02 14:50:03 +09:00
runner.rb Support running MCP conformance tests against the Ruby SDK 2026-03-02 14:50:03 +09:00
server.rb Support resource subscriptions per MCP specification 2026-04-19 03:48:46 +09:00

README.md

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Starts the conformance server, runs all active scenarios against it, prints a pass/fail summary for each scenario, and exits with a non-zero status code if any unexpected failures are detected. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off) 
# Run a single scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  runner.rb              # Starts the server, runs npx conformance, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.