5932521552
## Motivation and Context The npm `latest` dist-tag of `@modelcontextprotocol/conformance` points at 0.1.16, the old 0.1 line, while the 2026-07-28 requirement revisions and the frozen-requirements framework live in the 0.2.0 alpha line (`alpha` currently resolves 0.2.0-alpha.11). The runners invoked the bare package name, so `rake conformance` and the conformance CI job resolve whatever `latest` points at: a fresh machine silently downgrades to the old suite while a warm npx cache may keep serving a newer one, and the suite actually in use depends on the environment. Pinning the `alpha` dist-tag puts every run on the line the frozen conformance requirements are anchored to (0.2.0-alpha.10, published 2026-07-27). Running the real alpha suite surfaced the actual client-leg state, so the expected-failures baseline is updated to match: `initialize` now passes (the harness accepts 2026-07-28, fulfilling the TODO on the old entry), and the scenarios pending client-side work are recorded with their reasons: the 2026-07-28 stateless lifecycle and the SEP-2322 MRTR driver, SEP-2243 custom-header mirroring, client-side JSON Schema preservation, the DPoP and WIF authorization extensions, and one `auth/authorization-server-migration` check added to the suite after the alpha.10 anchor. The manual `npx` examples in `conformance/README.md` carry the same pin so local reproduction runs the same suite as the rake task. ## How Has This Been Tested? `npx --yes @modelcontextprotocol/conformance@alpha --version` resolves 0.2.0-alpha.11, and `bundle exec rake conformance` passes the baseline check on both legs: the server leg reports 73 passed with 0 failed and no expected failures, and the client leg fails exactly the scenarios the baseline records. ## Breaking Changes None. The change is confined to the conformance tooling; the shipped gem is untouched.
48 lines
1.3 KiB
Ruby
48 lines
1.3 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
# Runs `npx @modelcontextprotocol/conformance@alpha client` against the conformance client script.
|
|
require "English"
|
|
|
|
module Conformance
|
|
class ClientRunner
|
|
def initialize(scenario: nil, spec_version: nil, verbose: false)
|
|
@scenario = scenario
|
|
@spec_version = spec_version
|
|
@verbose = verbose
|
|
end
|
|
|
|
def run
|
|
command = build_command
|
|
puts "Command: #{command.join(" ")}\n\n"
|
|
|
|
system(*command)
|
|
conformance_exit_code = $CHILD_STATUS.exitstatus
|
|
exit(conformance_exit_code || 1) unless conformance_exit_code == 0
|
|
end
|
|
|
|
private
|
|
|
|
def build_command
|
|
expected_failures_yml = File.expand_path("expected_failures.yml", __dir__)
|
|
client_script = File.expand_path("client.rb", __dir__)
|
|
|
|
npx_command = [
|
|
"npx",
|
|
"--yes",
|
|
"@modelcontextprotocol/conformance@alpha",
|
|
"client",
|
|
"--command",
|
|
"bundle exec ruby #{client_script}",
|
|
]
|
|
npx_command += if @scenario
|
|
["--scenario", @scenario]
|
|
else
|
|
["--suite", "all"]
|
|
end
|
|
npx_command += ["--spec-version", @spec_version] if @spec_version
|
|
npx_command += ["--verbose"] if @verbose
|
|
npx_command += ["--expected-failures", expected_failures_yml]
|
|
npx_command
|
|
end
|
|
end
|
|
end
|