ruby-sdk/conformance/expected_failures.yml
Koichi ITO 9603b2fe93 Support running MCP client conformance tests against the Ruby SDK
## Motivation and Context

PR #248 introduced server conformance testing using `@modelcontextprotocol/conformance`.
This extends coverage to the client side so that `MCP::Client` behavior can also be validated against
the MCP specification.

### Scope

Client conformance is driven by the conformance runner, which spawns test servers for each scenario
and invokes `conformance/client.rb` against them. The client script delegates the MCP initialize
handshake to `MCP::Client#connect` and executes scenario-specific operations (for example listing
tools or calling a tool).

### Usage

Run all conformance tests (server + client):

```console
bundle exec rake conformance
```

Run a single client scenario:

```console
bundle exec rake conformance SCENARIO=initialize
```

List available scenarios:

```console
bundle exec rake conformance_list
```

## How Has This Been Tested?

Ran `bundle exec rake conformance` locally.

- Server conformance: 40 passed, 0 failed.
- Client conformance: 3 passed, 54 failed (all 54 listed in `expected_failures.yml`).

## Breaking Change

No. This change only adds test infrastructure under `conformance/` and does not modify SDK behavior.
2026-05-08 15:11:28 +09:00

28 lines
909 B
YAML

server: []
client:
# TODO: SSE reconnection not implemented in Ruby client.
- sse-retry
# TODO: Elicitation not implemented in Ruby client.
- elicitation-sep1034-client-defaults
# TODO: OAuth/auth not implemented in Ruby client.
- auth/metadata-default
- auth/metadata-var1
- auth/metadata-var2
- auth/metadata-var3
- auth/basic-cimd
- auth/scope-from-www-authenticate
- auth/scope-from-scopes-supported
- auth/scope-omitted-when-undefined
- auth/scope-step-up
- auth/scope-retry-limit
- auth/token-endpoint-auth-basic
- auth/token-endpoint-auth-post
- auth/token-endpoint-auth-none
- auth/pre-registration
- auth/2025-03-26-oauth-metadata-backcompat
- auth/2025-03-26-oauth-endpoint-fallback
- auth/client-credentials-jwt
- auth/client-credentials-basic
- auth/cross-app-access-complete-flow
- auth/offline-access-scope
- auth/offline-access-not-supported