## Motivation and Context
The MCP 2025-03-26 authorization spec predates Protected Resource Metadata:
the MCP server itself acted as the authorization base URL, its RFC 8414 metadata was fetched from
the server origin, and the spec's "Fallbacks for Servers without Metadata Discovery" section
required clients to use the default endpoints `/authorize`, `/token`, and `/register` relative to
the authorization base URL when no metadata was published. Both the TypeScript SDK
(`discoverOAuthServerInfo` falls back to the server base URL on PRM failure; `startAuthorization` /
`executeTokenRequest` / `registerClient` default the endpoint paths) and the Python SDK
(`build_oauth_authorization_server_metadata_discovery_urls(None, ...)` plus the same endpoint defaults)
keep this as client-side backwards compatibility, and the `auth/2025-03-26-oauth-metadata-backcompat` and
`auth/2025-03-26-oauth-endpoint-fallback` conformance scenarios exercise it.
The Ruby flow previously raised as soon as PRM discovery failed, so both scenarios were
expected conformance failures. This change adds the same fallback:
- `Flow#locate_authorization_server` first attempts PRM discovery; on any discovery failure
(404s, network errors, malformed documents, matching the TypeScript and Python SDKs' broad fallback)
the MCP server's origin becomes the legacy authorization base URL. The Communication Security check
(HTTPS or loopback) still applies to that origin, and PRM documents that parse correctly keep
their strict shape validation.
- `Flow#authorization_server_metadata` fetches RFC 8414 metadata from the base URL.
On the legacy path the `issuer` byte-match is skipped: the 2025-03-26 spec predates that requirement,
and a pre-PRM server may host its OAuth endpoints under a path prefix whose `issuer` legitimately differs
from the discovery origin (neither reference SDK validates the issuer on this path). On the modern path
the byte-match is unchanged.
- When even the metadata document is absent, synthetic metadata carrying the legacy spec's default endpoints
is used, with PKCE S256 assumed (the legacy spec mandates PKCE; the TypeScript and Python SDKs hardcode S256
on this path too). Endpoint HTTPS checks still apply.
- `run!` and `refresh!` share the new discovery helpers, and both 2025-03-26 scenarios are removed from
`conformance/expected_failures.yml`.
## How Has This Been Tested?
New tests in `test/mcp/client/oauth/flow_test.rb`:
- the metadata-backcompat shape: no PRM, AS metadata at the server origin with `/oauth`-prefixed endpoints
and a mismatched `issuer` completes the flow against those endpoints (no issuer error)
- the endpoint-fallback shape: no metadata at all registers, authorizes, and exchanges the code at `/register`,
`/authorize`, and `/token` on the origin, still sending `code_challenge` (S256) and `code_verifier`
- a remote plain-http origin is rejected as the legacy authorization base
- strict-mode regression: with PRM present, a mismatched `issuer` still aborts the flow
- a malformed PRM (top-level JSON array) now selects the legacy path instead of leaking a `TypeError`,
and surfaces a domain error when that path also dead-ends
Two existing tests that asserted PRM failure was terminal were updated to dead-end the legacy path explicitly,
preserving their original intent (single flow attempt, domain error type).
## Breaking Changes
None for spec-compliant 2025-06-18+ servers, whose PRM-based discovery and issuer validation are unchanged.
Servers that previously failed hard during PRM discovery now get one legacy discovery attempt before the flow errors,
which can only turn previously failing flows into working ones.
## Motivation and Context
The MCP authorization specification (2025-11-25 scope-selection-strategy)
and RFC 6750 Section 3.1 describe OAuth 2.0 step-up authentication:
the client already holds a valid access token, but the server returns a 403
with `WWW-Authenticate: Bearer error="insufficient_scope", scope="..."`
to demand a higher-privileged scope for a specific operation.
The client is expected to run a fresh authorization request with
the escalated scope and retry.
Until now the Ruby SDK only ran the OAuth flow on 401 responses.
A 403 surfaced unchanged as a `RequestHandlerError` and the operation
that needed the escalated scope failed permanently, even with a valid token
in storage. This lined up with the `auth/scope-step-up` conformance
scenario being listed in `conformance/expected_failures.yml`.
The change adds:
- `MCP::Client::HTTP#send_request` recognises a 403 carrying
an `insufficient_scope` Bearer challenge as a step-up signal,
re-runs a full authorization request with the scope from the challenge,
and retries the original request. Retries are capped at
one per `send_request` call so a server that repeatedly demands more
scope cannot loop the transport.
- `run_step_up_flow!` deliberately bypasses the refresh-token path that
`run_oauth_flow!` uses on 401. Refreshing would re-issue the same
scope set the existing token already has and the server already
rejected. Only a fresh `authorization_code` grant with the escalated
scope can succeed.
- A plain 403 with no `insufficient_scope` challenge continues to
surface as `RequestHandlerError` with `error_type: :forbidden`.
- `conformance/client.rb` invokes a tool for the `auth/scope-step-up`
scenario; the scenario's authorization middleware only requires
the escalated scope on `tools/call`, so the previous `tools/list`-only
client could never trigger the second authorization.
- `auth/scope-step-up` is removed from `conformance/expected_failures.yml`.
Resolves#383.
## How Has This Been Tested?
New `HTTPOAuthTest` cases cover: the 403 insufficient_scope path
re-authorizes with the escalated scope from the challenge and retries
the original request with the new bearer token; a plain 403 without
the `insufficient_scope` challenge is surfaced as
`error_type: :forbidden` with no OAuth side effects; repeated
`insufficient_scope` responses retry at most once and then surface the 403;
a provider with a saved `refresh_token` still goes through
the full authorization request rather than exchanging the refresh token,
because the existing scope set is precisely what was rejected.
Conformance: `auth/scope-step-up` now passes 22/22 with no warnings.
`bundle exec rake test`, `bundle exec rake rubocop`, and
`bundle exec rake conformance` are all green.
## Breaking Changes
None. Behaviour on 401 is unchanged. A 403 without the `insufficient_scope`
Bearer challenge still raises the same `RequestHandlerError` as before.
The new retry only fires when both the WWW-Authenticate header signals step-up
and the transport was constructed with an OAuth provider.
## Motivation and Context
The MCP Authorization spec (2025-11-25) requires clients to support
Protected Resource Metadata discovery (RFC 9728), Authorization
Server Metadata discovery (RFC 8414), Dynamic Client Registration,
and OAuth 2.1 Authorization Code grant with PKCE (S256). The Ruby SDK
previously had no client-side OAuth support: the HTTP transport raised
`RequestHandlerError` on a 401 with no recovery, and 21 `auth/*` scenarios
were listed as known failures in `conformance/expected_failures.yml`.
This aligns the Ruby SDK with the Python and TypeScript SDKs,
both of which expose a pluggable OAuth client provider that drives
the full discovery + DCR + authorization + token exchange flow.
`MCP::Client::OAuth::Provider` mirrors that surface: callers supply
`client_metadata`, a `redirect_uri`, and pluggable `redirect_handler`
/ `callback_handler` hooks plus a token/client information storage.
`MCP::Client::OAuth::InMemoryStorage` is provided as a default.
`MCP::Client::HTTP` accepts a new optional `oauth:` keyword. When present,
the transport attaches `Authorization: Bearer` to every request,
and on a 401 parses `WWW-Authenticate` for the `resource_metadata` URL
and `scope` challenge, drives the discovery + DCR + authorization code flow
through the provider, and retries the failed request once with the acquired token.
Spec compliance details:
- The `WWW-Authenticate` parser handles multiple challenges
(e.g. `Basic ..., Bearer ...` or `Bearer ..., DPoP ...`), extracting only
the parameters that belong to the Bearer challenge.
- The PRM `resource` value is required to cover the MCP server URL
(same scheme/host/port, with the PRM path as a prefix of the server path)
before the flow continues, preventing redirection of issued tokens to a different audience.
- Both authorization and token requests always include a `resource`
parameter (RFC 8707), defaulting to a canonicalized form of
the MCP server URL when PRM does not advertise one.
- `client_secret_basic`, `client_secret_post`, and `none` token endpoint
authentication methods are supported. When pre-registered client information
specifies no `token_endpoint_auth_method`, the transport defaults to
`client_secret_basic` per RFC 6749 §2.3.1 an dthe TS/Python SDK convention.
Pre-registered client credentials supplied via `Storage#save_client_information`
with either string or symbol keys are honored, skipping DCR.
- Scope resolution follows the spec order: the `WWW-Authenticate` challenge scope wins first,
then PRM `scopes_supported`, and `Provider#scope` is the last-resort fallback only
if neither is supplied (the provider-supplied scope must not pre-empt server-advertised scope).
- Token endpoint error responses (RFC 6749 §5.2) are parsed: a 4x with
`error: "invalid_grant"` raises `Flow::InvalidGrantError`,
so `MCP::Client::HTTP` can discard the stored refresh token only when
the AS truly says it is dead. Generic `AuthorizationError`
(5xx, network failures, other error codes) leaves the refresh token
intact, so a transient AS outage does not force interactive reauth
on the next attempt.
- The OAuth `state` parameter is compared via
`OpenSSL.fixed_length_secure_compare` (with a `bytesize` guard) to
prevent timing-based discovery of the expected value.
- The 401 retry guard is a request-scoped local variable, so an MCP server
that keeps returning 401 is surfaced as an error after one
retry rather than triggering an infinite re-authorization loop.
## How Has This Been Tested?
Unit tests cover PKCE generation, multi-challenge `WWW-Authenticate` parsing,
PRM and Authorization Server metadata URL builders, the full Flow orchestrator
(success path, state mismatch, missing PKCE support, skipping DCR for both string
and symbol-keyed preregistered client information, `client_secret_basic`
token endpoint authentication, PRM `resource` mismatch rejection, fallback to
the server URL when PRM omits `resource`, scope resolution preferring PRM
`scopes_supported` over `Provider#scope`, default `client_secret_basic` for
confidential clients without an explicit `token_endpoint_auth_method`,
and the new `InvalidGrantError` vs generic `AuthorizationError` split for
token endpoint rejections), and the HTTP transport behavior
(401 recovery + retry, no-provider fallthrough, no infinite loop on repeated 401,
no infinite loop on flow failure, refresh-token retention on transient 5xx from
the token endpoint).
Conformance: `bundle exec rake conformance` newly passes 16 `auth/*`
scenarios beyond the existing baseline, including
`auth/metadata-default`, `metadata-var{1,2,3}`,
`scope-from-www-authenticate`, `scope-from-scopes-supported`,
`scope-omitted-when-undefined`, `scope-retry-limit`,
`token-endpoint-auth-{basic,post,none}`, `pre-registration`,
`resource-mismatch`, and `offline-access-not-supported`. Remaining
`auth/*` scenarios stay in `expected_failures.yml`.
`bundle exec rake test rubocop` is clean.
## Breaking Changes
None. `MCP::Client::HTTP#initialize` accepts a new optional `oauth:`
keyword that defaults to `nil`, preserving existing behavior. When unset,
the transport raises `MCP::Client::RequestHandlerError` on a 401 exactly as before.
Koichi ITO