Support running MCP client conformance tests against the Ruby SDK

## Motivation and Context

PR #248 introduced server conformance testing using `@modelcontextprotocol/conformance`.
This extends coverage to the client side so that `MCP::Client` behavior can also be validated against
the MCP specification.

### Scope

Client conformance is driven by the conformance runner, which spawns test servers for each scenario
and invokes `conformance/client.rb` against them. The client script delegates the MCP initialize
handshake to `MCP::Client#connect` and executes scenario-specific operations (for example listing
tools or calling a tool).

### Usage

Run all conformance tests (server + client):

```console
bundle exec rake conformance
```

Run a single client scenario:

```console
bundle exec rake conformance SCENARIO=initialize
```

List available scenarios:

```console
bundle exec rake conformance_list
```

## How Has This Been Tested?

Ran `bundle exec rake conformance` locally.

- Server conformance: 40 passed, 0 failed.
- Client conformance: 3 passed, 54 failed (all 54 listed in `expected_failures.yml`).

## Breaking Change

No. This change only adds test infrastructure under `conformance/` and does not modify SDK behavior.
This commit is contained in:
Koichi ITO 2026-03-13 03:33:13 +09:00
commit 9603b2fe93

View file

@ -20,7 +20,8 @@ desc "Run MCP conformance tests (PORT, SCENARIO, SPEC_VERSION, VERBOSE)"
task :conformance do |t|
next unless npx_available?(t.name)
require_relative "conformance/runner"
require_relative "conformance/server_runner"
require_relative "conformance/client_runner"
options = {}
options[:port] = Integer(ENV["PORT"]) if ENV["PORT"]
@ -28,7 +29,8 @@ task :conformance do |t|
options[:spec_version] = ENV["SPEC_VERSION"] if ENV["SPEC_VERSION"]
options[:verbose] = true if ENV["VERBOSE"]
Conformance::Runner.new(**options).run
Conformance::ServerRunner.new(**options).run
Conformance::ClientRunner.new(**options.except(:port)).run
end
desc "List available conformance scenarios"
@ -36,6 +38,7 @@ task :conformance_list do |t|
next unless npx_available?(t.name)
system("npx", "--yes", "@modelcontextprotocol/conformance", "list", "--server")
system("npx", "--yes", "@modelcontextprotocol/conformance", "list", "--client")
end
desc "Start the conformance server (PORT)"