modelcontextprotocol/ruby-sdk
Watch
1
0
Fork
You've already forked ruby-sdk
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ruby-sdk/conformance
History
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO 985fd03ecb Support elicitation/create per MCP specification 
## Motivation and Context

The MCP specification (2025-11-25) defines elicitation as a server-to-client
request mechanism that allows servers to request user input during tool execution:
https://modelcontextprotocol.io/specification/2025-11-25/client/elicitation

This aligns with the Python SDK, which also exposes separate methods for form and URL mode elicitation.
Three methods are added following the Python SDK's API design:

- `create_form_elicitation`: collects structured data via JSON Schema
- `create_url_elicitation`: directs users to external URLs for sensitive
  interactions (OAuth, credentials, etc.)
- `notify_elicitation_complete`: notifies the client that a URL mode
  interaction has completed

`URLElicitationRequiredError` (error code `-32042`) is defined as a subclass of `RequestHandlerError`
with its own `error_code` and `error_data` attributes. `JsonRpcHandler#handle_request_error` is
extended to check for these attributes so that MCP-specific error codes can be propagated without
embedding MCP knowledge in the generic JSON-RPC layer.

## How Has This Been Tested?

- Unit tests for `ServerSession` form/URL mode, capability checks, nil
  capabilities, notification, and `URLElicitationRequiredError`
- `ServerContext` delegation tests for all three elicitation methods
- `JsonRpcHandler` tests for custom error code and fallback behavior
- `Methods` capability test for `NOTIFICATIONS_ELICITATION_COMPLETE`
- Conformance tests pass (40/40), including `tools-call-elicitation`,
  `elicitation-sep1034-defaults`, and `elicitation-sep1330-enums`
- All tests pass (`rake test`), RuboCop is clean
- Verified form mode elicitation end-to-end with a sample MCP server
  running in Claude Code

## Breaking Changes

None. This is a new feature addition.
2026-04-13 15:21:45 +09:00
..
expected_failures.yml Support elicitation/create per MCP specification 2026-04-13 15:21:45 +09:00
README.md Support running MCP conformance tests against the Ruby SDK 2026-03-02 14:50:03 +09:00
runner.rb Support running MCP conformance tests against the Ruby SDK 2026-03-02 14:50:03 +09:00
server.rb Support elicitation/create per MCP specification 2026-04-13 15:21:45 +09:00

README.md

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Starts the conformance server, runs all active scenarios against it, prints a pass/fail summary for each scenario, and exits with a non-zero status code if any unexpected failures are detected. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off) 
# Run a single scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  runner.rb              # Starts the server, runs npx conformance, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.