modelcontextprotocol/ruby-sdk
Watch
1
0
Fork
You've already forked ruby-sdk
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ruby-sdk/conformance
History
Exact
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Koichi ITO 2c223dd2f0 Fall Back to Legacy 2025-03-26 OAuth Discovery for Servers Without PRM 
## Motivation and Context

The MCP 2025-03-26 authorization spec predates Protected Resource Metadata:
the MCP server itself acted as the authorization base URL, its RFC 8414 metadata was fetched from
the server origin, and the spec's "Fallbacks for Servers without Metadata Discovery" section
required clients to use the default endpoints `/authorize`, `/token`, and `/register` relative to
the authorization base URL when no metadata was published. Both the TypeScript SDK
(`discoverOAuthServerInfo` falls back to the server base URL on PRM failure; `startAuthorization` /
`executeTokenRequest` / `registerClient` default the endpoint paths) and the Python SDK
(`build_oauth_authorization_server_metadata_discovery_urls(None, ...)` plus the same endpoint defaults)
keep this as client-side backwards compatibility, and the `auth/2025-03-26-oauth-metadata-backcompat` and
`auth/2025-03-26-oauth-endpoint-fallback` conformance scenarios exercise it.

The Ruby flow previously raised as soon as PRM discovery failed, so both scenarios were
expected conformance failures. This change adds the same fallback:

- `Flow#locate_authorization_server` first attempts PRM discovery; on any discovery failure
  (404s, network errors, malformed documents, matching the TypeScript and Python SDKs' broad fallback)
  the MCP server's origin becomes the legacy authorization base URL. The Communication Security check
  (HTTPS or loopback) still applies to that origin, and PRM documents that parse correctly keep
  their strict shape validation.
- `Flow#authorization_server_metadata` fetches RFC 8414 metadata from the base URL.
  On the legacy path the `issuer` byte-match is skipped: the 2025-03-26 spec predates that requirement,
  and a pre-PRM server may host its OAuth endpoints under a path prefix whose `issuer` legitimately differs
  from the discovery origin (neither reference SDK validates the issuer on this path). On the modern path
  the byte-match is unchanged.
- When even the metadata document is absent, synthetic metadata carrying the legacy spec's default endpoints
  is used, with PKCE S256 assumed (the legacy spec mandates PKCE; the TypeScript and Python SDKs hardcode S256
  on this path too). Endpoint HTTPS checks still apply.
- `run!` and `refresh!` share the new discovery helpers, and both 2025-03-26 scenarios are removed from
  `conformance/expected_failures.yml`.

## How Has This Been Tested?

New tests in `test/mcp/client/oauth/flow_test.rb`:

- the metadata-backcompat shape: no PRM, AS metadata at the server origin with `/oauth`-prefixed endpoints
  and a mismatched `issuer` completes the flow against those endpoints (no issuer error)
- the endpoint-fallback shape: no metadata at all registers, authorizes, and exchanges the code at `/register`,
  `/authorize`, and `/token` on the origin, still sending `code_challenge` (S256) and `code_verifier`
- a remote plain-http origin is rejected as the legacy authorization base
- strict-mode regression: with PRM present, a mismatched `issuer` still aborts the flow
- a malformed PRM (top-level JSON array) now selects the legacy path instead of leaking a `TypeError`,
  and surfaces a domain error when that path also dead-ends

Two existing tests that asserted PRM failure was terminal were updated to dead-end the legacy path explicitly,
preserving their original intent (single flow attempt, domain error type).

## Breaking Changes

None for spec-compliant 2025-06-18+ servers, whose PRM-based discovery and issuer validation are unchanged.
Servers that previously failed hard during PRM discovery now get one legacy discovery attempt before the flow errors,
which can only turn previously failing flows into working ones.
2026-06-16 17:48:18 +09:00
..
client.rb Merge pull request #399 from koic/client_credentials_basic 2026-06-13 20:06:29 +09:00
client_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
expected_failures.yml Fall Back to Legacy 2025-03-26 OAuth Discovery for Servers Without PRM 2026-06-16 17:48:18 +09:00
README.md Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00
server.rb Support resource subscriptions per MCP specification 2026-04-19 03:48:46 +09:00
server_runner.rb Support running MCP client conformance tests against the Ruby SDK 2026-05-08 15:11:28 +09:00

README.md

MCP Conformance Tests

Validates the Ruby SDK's conformance to the MCP specification using @modelcontextprotocol/conformance.

Prerequisites

  • Node.js (for npx)
  • bundle install completed

Running the Tests

Run all scenarios

bundle exec rake conformance

Runs both server and client conformance tests in sequence. Server conformance starts the conformance server and tests it. Client conformance spawns test servers for each scenario and invokes conformance/client.rb against them. Scenarios listed in expected_failures.yml are allowed to fail without affecting the exit code.

Environment variables

Variable Description Default
PORT Server port (server conformance only) 9292
SCENARIO Run a single scenario by name (all)
SPEC_VERSION Filter scenarios by spec version (all)
VERBOSE Show raw JSON output when set (off) 
# Run a single server scenario
bundle exec rake conformance SCENARIO=ping

# Use a different port with verbose output
bundle exec rake conformance PORT=3000 VERBOSE=1

# Start the server on a specific port
bundle exec rake conformance_server PORT=3000

Start the server and test separately

# Terminal 1: start the server
bundle exec rake conformance_server

# Terminal 2: run all scenarios
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp

# Terminal 2: run a single scenario
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping

Keeps the server alive between test runs, which avoids the startup overhead when iterating on a single scenario. Stop the server with Ctrl+C when done.

List available scenarios

bundle exec rake conformance_list

Prints all scenario names that can be passed to SCENARIO.

SDK Tier Report

The MCP SDK Tier system requires SDK maintainers to self-assess and report results to the SDK Working Group via modelcontextprotocol/modelcontextprotocol issues.

To generate a full tier assessment report, use the /mcp-sdk-tier-audit slash command from the modelcontextprotocol/conformance repository with the conformance server running:

# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp

The skill evaluates conformance pass rate, issue label taxonomy, triage metrics, documentation coverage, and policy compliance, then produces a markdown report suitable for tier advancement submissions.

File Structure

conformance/
  server.rb              # Conformance server (Rack + Puma, default port 9292)
  server_runner.rb       # Starts the server, runs npx conformance server, exits with result code
  client.rb              # Conformance client (invoked by npx conformance client)
  client_runner.rb       # Runs npx conformance client, exits with result code
  expected_failures.yml  # Baseline of known-failing scenarios (server and client)
  README.md              # This file

Known Limitations

Known-failing scenarios are registered in conformance/expected_failures.yml. They are allowed to fail without affecting the exit code and are tracked to catch regressions. These are shown in the output of bundle exec rake conformance.