Re-run OAuth flow on 403 insufficient_scope (step-up)

## Motivation and Context

The MCP authorization specification (2025-11-25 scope-selection-strategy)
and RFC 6750 Section 3.1 describe OAuth 2.0 step-up authentication:
the client already holds a valid access token, but the server returns a 403
with `WWW-Authenticate: Bearer error="insufficient_scope", scope="..."`
to demand a higher-privileged scope for a specific operation.
The client is expected to run a fresh authorization request with
the escalated scope and retry.

Until now the Ruby SDK only ran the OAuth flow on 401 responses.
A 403 surfaced unchanged as a `RequestHandlerError` and the operation
that needed the escalated scope failed permanently, even with a valid token
in storage. This lined up with the `auth/scope-step-up` conformance
scenario being listed in `conformance/expected_failures.yml`.

The change adds:

- `MCP::Client::HTTP#send_request` recognises a 403 carrying
  an `insufficient_scope` Bearer challenge as a step-up signal,
  re-runs a full authorization request with the scope from the challenge,
  and retries the original request. Retries are capped at
  one per `send_request` call so a server that repeatedly demands more
  scope cannot loop the transport.
- `run_step_up_flow!` deliberately bypasses the refresh-token path that
  `run_oauth_flow!` uses on 401. Refreshing would re-issue the same
  scope set the existing token already has and the server already
  rejected. Only a fresh `authorization_code` grant with the escalated
  scope can succeed.
- A plain 403 with no `insufficient_scope` challenge continues to
  surface as `RequestHandlerError` with `error_type: :forbidden`.
- `conformance/client.rb` invokes a tool for the `auth/scope-step-up`
  scenario; the scenario's authorization middleware only requires
  the escalated scope on `tools/call`, so the previous `tools/list`-only
  client could never trigger the second authorization.
- `auth/scope-step-up` is removed from `conformance/expected_failures.yml`.

Resolves #383.

## How Has This Been Tested?

New `HTTPOAuthTest` cases cover: the 403 insufficient_scope path
re-authorizes with the escalated scope from the challenge and retries
the original request with the new bearer token; a plain 403 without
the `insufficient_scope` challenge is surfaced as
`error_type: :forbidden` with no OAuth side effects; repeated
`insufficient_scope` responses retry at most once and then surface the 403;
a provider with a saved `refresh_token` still goes through
the full authorization request rather than exchanging the refresh token,
because the existing scope set is precisely what was rejected.

Conformance: `auth/scope-step-up` now passes 22/22 with no warnings.
`bundle exec rake test`, `bundle exec rake rubocop`, and
`bundle exec rake conformance` are all green.

## Breaking Changes

None. Behaviour on 401 is unchanged. A 403 without the `insufficient_scope`
Bearer challenge still raises the same `RequestHandlerError` as before.
The new retry only fires when both the WWW-Authenticate header signals step-up
and the transport was constructed with an OAuth provider.
This commit is contained in:
Koichi ITO 2026-05-28 13:06:32 +09:00
commit a407cea086

View file

@ -1913,6 +1913,9 @@ pass an `MCP::Client::OAuth::Provider` to the transport instead of a static `Aut
- On a `401 Unauthorized`, parse the `WWW-Authenticate` header, discover the authorization server (Protected Resource Metadata + RFC 8414 Authorization Server Metadata),
perform Dynamic Client Registration if needed, run the OAuth 2.1 Authorization Code flow with PKCE (S256), and retry the failed request with the acquired token.
- On subsequent 401s with a saved `refresh_token`, exchange it at the token endpoint before falling back to the full interactive flow (RFC 6749 Section 6).
- On a `403 Forbidden` whose `WWW-Authenticate` header carries `error="insufficient_scope"` (OAuth 2.0 step-up, RFC 6750 Section 3.1 and the MCP scope-selection-strategy),
run a fresh authorization request for the union of the currently granted scope and the scope named in the challenge, then retry the failed request once.
The refresh path is bypassed because refreshing would re-issue the same scope set the server just rejected. A `403` without that challenge is surfaced unchanged.
- Request the `offline_access` scope when `client_metadata[:grant_types]` includes `refresh_token` and the authorization server advertises `offline_access` in its metadata
`scopes_supported` (SEP-2207). This is what lets the server issue the `refresh_token` used above. As an SDK-level safeguard, when the authorization server does not advertise
`offline_access` the scope is also stripped from any other source (challenge, PRM, or provider-supplied scope) so a server that does not support it never receives it.