Support running MCP client conformance tests against the Ruby SDK
## Motivation and Context PR #248 introduced server conformance testing using `@modelcontextprotocol/conformance`. This extends coverage to the client side so that `MCP::Client` behavior can also be validated against the MCP specification. ### Scope Client conformance is driven by the conformance runner, which spawns test servers for each scenario and invokes `conformance/client.rb` against them. The client script delegates the MCP initialize handshake to `MCP::Client#connect` and executes scenario-specific operations (for example listing tools or calling a tool). ### Usage Run all conformance tests (server + client): ```console bundle exec rake conformance ``` Run a single client scenario: ```console bundle exec rake conformance SCENARIO=initialize ``` List available scenarios: ```console bundle exec rake conformance_list ``` ## How Has This Been Tested? Ran `bundle exec rake conformance` locally. - Server conformance: 40 passed, 0 failed. - Client conformance: 3 passed, 54 failed (all 54 listed in `expected_failures.yml`). ## Breaking Change No. This change only adds test infrastructure under `conformance/` and does not modify SDK behavior.
This commit is contained in:
parent
d18698890a
commit
9603b2fe93
1 changed files with 131 additions and 17 deletions
7
Rakefile
7
Rakefile
|
|
@ -20,7 +20,8 @@ desc "Run MCP conformance tests (PORT, SCENARIO, SPEC_VERSION, VERBOSE)"
|
|||
task :conformance do |t|
|
||||
next unless npx_available?(t.name)
|
||||
|
||||
require_relative "conformance/runner"
|
||||
require_relative "conformance/server_runner"
|
||||
require_relative "conformance/client_runner"
|
||||
|
||||
options = {}
|
||||
options[:port] = Integer(ENV["PORT"]) if ENV["PORT"]
|
||||
|
|
@ -28,7 +29,8 @@ task :conformance do |t|
|
|||
options[:spec_version] = ENV["SPEC_VERSION"] if ENV["SPEC_VERSION"]
|
||||
options[:verbose] = true if ENV["VERBOSE"]
|
||||
|
||||
Conformance::Runner.new(**options).run
|
||||
Conformance::ServerRunner.new(**options).run
|
||||
Conformance::ClientRunner.new(**options.except(:port)).run
|
||||
end
|
||||
|
||||
desc "List available conformance scenarios"
|
||||
|
|
@ -36,6 +38,7 @@ task :conformance_list do |t|
|
|||
next unless npx_available?(t.name)
|
||||
|
||||
system("npx", "--yes", "@modelcontextprotocol/conformance", "list", "--server")
|
||||
system("npx", "--yes", "@modelcontextprotocol/conformance", "list", "--client")
|
||||
end
|
||||
|
||||
desc "Start the conformance server (PORT)"
|
||||
|
|
|
|||
Loading…
Reference in a new issue