ruby-sdk/conformance
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO 14b4f7c8c7 Validate the RFC 9207 iss Authorization Response Parameter per SEP-2468
## Motivation and Context

SEP-2468 (modelcontextprotocol/modelcontextprotocol#2468, merged for the 2026-07-28 spec release) recommends
that authorization servers include the RFC 9207 `iss` parameter in authorization responses and requires MCP clients
to validate a present `iss` against the expected issuer before redeeming the authorization code.
This mitigates mix-up attacks in multi-IdP setups, where an attacker-controlled authorization server could otherwise capture
a code minted by an honest server.

This mirrors the TypeScript SDK's approach (typescript-sdk#2272, with typescript-sdk#1957 as the reference implementation),
adapted to the Ruby provider contract:

- `Provider`'s `callback_handler` may now return `[code, state, iss]` in addition to the existing `[code, state]`.
  The 3-element form opts into validation; returning `iss = nil` asserts the caller inspected the authorization response
  and found no `iss`.
- `Flow#run!` validates the issuer after the `state` check and before `exchange_authorization_code`,
  implementing the RFC 9207 Section 2.4 rules: a present `iss` must equal the AS metadata `issuer` exactly
  (the value `ensure_issuer_matches!` already pinned to the discovery URL); a nil `iss` from a 3-element callback is rejected
  when the AS metadata advertises `authorization_response_iss_parameter_supported: true`; a legacy 2-element callback skips the check,
  so every existing integration keeps working. On any violation the flow aborts with `AuthorizationError` before
  the code reaches a token endpoint.
- The conformance client now returns the 3-element form, plumbing `iss` through from the redirect URL.

Resolves #386.

## How Has This Been Tested?

New tests in `test/mcp/client/oauth/flow_test.rb`:

- a matching `iss` completes the flow and persists tokens
- a mismatched `iss` raises `AuthorizationError` and never contacts the token endpoint (`assert_not_requested` on the token POST)
- a nil `iss` with `authorization_response_iss_parameter_supported: true` raises without contacting the token endpoint (fail closed)
- a legacy 2-element callback completes even when the AS advertises `iss` support (backward compatibility)
- a nil `iss` without the advertisement completes
- a mismatched `iss` is rejected even without the advertisement (validate-when-present is unconditional)

`bundle exec rake` (tests, RuboCop, and conformance baseline with the 3-element conformance callback) passes.

## Breaking Changes

None. The 2-element `callback_handler` contract continues to work unchanged; issuer validation activates only
when a callback opts in by returning the 3-element form.
2026-06-28 20:40:29 +09:00
..
client.rb Validate the RFC 9207 iss Authorization Response Parameter per SEP-2468 2026-06-28 20:40:29 +09:00
client_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
expected_failures.yml Fall Back to Legacy 2025-03-26 OAuth Discovery for Servers Without PRM 2026-06-16 17:48:18 +09:00
README.md Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
server.rb Support resource subscriptions per MCP specification 2026-04-19 03:48:46 +09:00
server_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Runs both server and client conformance tests in sequence. Server conformance starts the conformance server and tests it. Client conformance spawns test servers for each scenario and invokes conformance/client.rb against them. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port (server conformance only) 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off)
# Run a single server scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  server_runner.rb       # Starts the server, runs npx conformance server, exits with result code
  client.rb              # Conformance client (invoked by npx conformance client)
  client_runner.rb       # Runs npx conformance client, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios (server and client)
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.