ruby-sdk/conformance
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO 5766069923 Add OAuth 2.1 client support for MCP authorization flow
## Motivation and Context

The MCP Authorization spec (2025-11-25) requires clients to support
Protected Resource Metadata discovery (RFC 9728), Authorization
Server Metadata discovery (RFC 8414), Dynamic Client Registration,
and OAuth 2.1 Authorization Code grant with PKCE (S256). The Ruby SDK
previously had no client-side OAuth support: the HTTP transport raised
`RequestHandlerError` on a 401 with no recovery, and 21 `auth/*` scenarios
were listed as known failures in `conformance/expected_failures.yml`.

This aligns the Ruby SDK with the Python and TypeScript SDKs,
both of which expose a pluggable OAuth client provider that drives
the full discovery + DCR + authorization + token exchange flow.
`MCP::Client::OAuth::Provider` mirrors that surface: callers supply
`client_metadata`, a `redirect_uri`, and pluggable `redirect_handler`
/ `callback_handler` hooks plus a token/client information storage.
`MCP::Client::OAuth::InMemoryStorage` is provided as a default.

`MCP::Client::HTTP` accepts a new optional `oauth:` keyword. When present,
the transport attaches `Authorization: Bearer` to every request,
and on a 401 parses `WWW-Authenticate` for the `resource_metadata` URL
and `scope` challenge, drives the discovery + DCR + authorization code flow
through the provider, and retries the failed request once with the acquired token.

Spec compliance details:

- The `WWW-Authenticate` parser handles multiple challenges
  (e.g. `Basic ..., Bearer ...` or `Bearer ..., DPoP ...`), extracting only
  the parameters that belong to the Bearer challenge.
- The PRM `resource` value is required to cover the MCP server URL
  (same scheme/host/port, with the PRM path as a prefix of the server path)
  before the flow continues, preventing redirection of issued tokens to a different audience.
- Both authorization and token requests always include a `resource`
  parameter (RFC 8707), defaulting to a canonicalized form of
  the MCP server URL when PRM does not advertise one.
- `client_secret_basic`, `client_secret_post`, and `none` token endpoint
  authentication methods are supported. When pre-registered client information
  specifies no `token_endpoint_auth_method`, the transport defaults to
  `client_secret_basic` per RFC 6749 §2.3.1 an dthe TS/Python SDK convention.
  Pre-registered client credentials supplied via `Storage#save_client_information`
  with either string or symbol keys are honored, skipping DCR.
- Scope resolution follows the spec order: the `WWW-Authenticate` challenge scope wins first,
  then PRM `scopes_supported`, and `Provider#scope` is the last-resort fallback only
  if neither is supplied (the provider-supplied scope must not pre-empt server-advertised scope).
- Token endpoint error responses (RFC 6749 §5.2) are parsed: a 4x with
  `error: "invalid_grant"` raises `Flow::InvalidGrantError`,
  so `MCP::Client::HTTP` can discard the stored refresh token only when
  the AS truly says it is dead. Generic `AuthorizationError`
  (5xx, network failures, other error codes) leaves the refresh token
  intact, so a transient AS outage does not force interactive reauth
  on the next attempt.
- The OAuth `state` parameter is compared via
  `OpenSSL.fixed_length_secure_compare` (with a `bytesize` guard) to
  prevent timing-based discovery of the expected value.
- The 401 retry guard is a request-scoped local variable, so an MCP server
  that keeps returning 401 is surfaced as an error after one
  retry rather than triggering an infinite re-authorization loop.

## How Has This Been Tested?

Unit tests cover PKCE generation, multi-challenge `WWW-Authenticate` parsing,
PRM and Authorization Server metadata URL builders, the full Flow orchestrator
(success path, state mismatch, missing PKCE support, skipping DCR for both string
and symbol-keyed preregistered client information, `client_secret_basic`
token endpoint authentication, PRM `resource` mismatch rejection, fallback to
the server URL when PRM omits `resource`, scope resolution preferring PRM
`scopes_supported` over `Provider#scope`, default `client_secret_basic` for
confidential clients without an explicit `token_endpoint_auth_method`,
and the new `InvalidGrantError` vs generic `AuthorizationError` split for
token endpoint rejections), and the HTTP transport behavior
(401 recovery + retry, no-provider fallthrough, no infinite loop on repeated 401,
no infinite loop on flow failure, refresh-token retention on transient 5xx from
the token endpoint).

Conformance: `bundle exec rake conformance` newly passes 16 `auth/*`
scenarios beyond the existing baseline, including
`auth/metadata-default`, `metadata-var{1,2,3}`,
`scope-from-www-authenticate`, `scope-from-scopes-supported`,
`scope-omitted-when-undefined`, `scope-retry-limit`,
`token-endpoint-auth-{basic,post,none}`, `pre-registration`,
`resource-mismatch`, and `offline-access-not-supported`. Remaining
`auth/*` scenarios stay in `expected_failures.yml`.
`bundle exec rake test rubocop` is clean.

## Breaking Changes

None. `MCP::Client::HTTP#initialize` accepts a new optional `oauth:`
keyword that defaults to `nil`, preserving existing behavior. When unset,
the transport raises `MCP::Client::RequestHandlerError` on a 401 exactly as before.
2026-05-17 00:48:41 +09:00
..
client.rb Add OAuth 2.1 client support for MCP authorization flow 2026-05-17 00:48:41 +09:00
client_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
expected_failures.yml Add OAuth 2.1 client support for MCP authorization flow 2026-05-17 00:48:41 +09:00
README.md Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
server.rb Support resource subscriptions per MCP specification 2026-04-19 03:48:46 +09:00
server_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Runs both server and client conformance tests in sequence. Server conformance starts the conformance server and tests it. Client conformance spawns test servers for each scenario and invokes conformance/client.rb against them. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port (server conformance only) 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off)
# Run a single server scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  server_runner.rb       # Starts the server, runs npx conformance server, exits with result code
  client.rb              # Conformance client (invoked by npx conformance client)
  client_runner.rb       # Runs npx conformance client, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios (server and client)
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.