modelcontextprotocol/ruby-sdk
Watch
1
0
Fork
You've already forked ruby-sdk
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ruby-sdk/conformance
History
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO b54eebac75 Add MCP::Elicitation::EnumSchema builders for SEP-1330 
## Motivation and Context

MCP 2025-11-25 (SEP-1330) replaces the ad-hoc `enumNames` convention with four standards-based enum schema variants
for elicitation requests: titled/untitled single-select and titled/untitled multi-select. The TypeScript SDK ships
typed Zod definitions for each shape (`packages/core/src/types/schemas.ts`), and the Python SDK relies on hand-built
schemas with the same shapes in its conformance fixtures.

Adds builder class methods on `MCP::Elicitation::EnumSchema` that produce the canonical JSON-Schema shapes for each variant,
plus a `legacy_titled` builder retained for backward compatibility with clients that have not yet migrated.
Each constructor returns an `EnumSchema` instance whose `to_h` produces the property-value Hash to pass through
`requested_schema:` on `ServerSession#create_form_elicitation`.

Each builder maps one-to-one to the spec's schema type names in `schema/2025-11-25/schema.ts`,
which the TypeScript and Python SDKs also carry verbatim in their type layers:

| Spec type (schema.ts)                             | Builder                                         |
|---------------------------------------------------|-------------------------------------------------|
| `UntitledSingleSelectEnumSchema`                  | `untitled_single_select`                        |
| `TitledSingleSelectEnumSchema`                    | `titled_single_select`                          |
| `UntitledMultiSelectEnumSchema`                   | `untitled_multi_select`                         |
| `TitledMultiSelectEnumSchema`                     | `titled_multi_select`                           |
| `LegacyTitledEnumSchema` (deprecated in the spec) | `legacy_titled`                                 |
| `EnumSchema` (the union of all five)              | the `MCP::Elicitation::EnumSchema` class itself |

These shapes stay relevant on the stateless 2026-07-28 lifecycle: elicitation is not deprecated by SEP-2577
and reaches modern clients as `elicitation/create` requests embedded in SEP-2322 `input_required` results,
whose `requestedSchema` uses the same variants. The README documents that pairing; the example references
`MCP::Server::InputRequiredResult`, so this change should merge after the SEP-2322 server-side PR.

The conformance server now builds the enum schemas of its `test_elicitation_sep1330_enums` tool
(and the `status` enum of the SEP-1034 defaults tool) through these builders instead of hand-written hashes,
so the conformance run exercises the exact schemas the SDK produces for users.

## How Has This Been Tested?

`test/mcp/elicitation/enum_schema_test.rb` covers each of the five shapes with happy-path emission,
`default` propagation, `title` / `description` propagation, and the four ArgumentError cases
(empty values, missing `:value`, missing `:title`, mismatched `value_titles` length). Two additional regression tests
assert that empty-but-non-nil defaults survive `to_h`:

- `EnumSchema.untitled_single_select(default: "")` keeps the empty string in the emitted Hash
- `EnumSchema.untitled_multi_select(default: [])` keeps the empty array in the emitted Hash

`bundle exec rake` (tests, RuboCop, and conformance baseline) passes; the `elicitation-sep1330-enums`
and `elicitation-sep1034-defaults` conformance scenarios pass with the builder-produced schemas,
confirming they emit the same wire shapes as the previous hand-written hashes.

## Breaking Changes

None. The change is purely additive: new module `MCP::Elicitation::EnumSchema` with new class methods.
Existing callers of `ServerSession#create_form_elicitation` that pass raw hashes continue to work unchanged.
2026-08-05 12:26:13 +09:00
..
client.rb [Conformance] Report expected client terminal failures quietly 2026-07-19 10:53:31 +09:00
client_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
expected_failures.yml Support 2026-07-28 as the Latest Protocol Version 2026-07-31 12:55:21 +09:00
README.md Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
server.rb Add MCP::Elicitation::EnumSchema builders for SEP-1330 2026-08-05 12:26:13 +09:00
server_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00

README.md

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Runs both server and client conformance tests in sequence. Server conformance starts the conformance server and tests it. Client conformance spawns test servers for each scenario and invokes conformance/client.rb against them. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port (server conformance only) 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off) 
# Run a single server scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  server_runner.rb       # Starts the server, runs npx conformance server, exits with result code
  client.rb              # Conformance client (invoked by npx conformance client)
  client_runner.rb       # Runs npx conformance client, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios (server and client)
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.