Pin the conformance CLI to the alpha dist-tag
## Motivation and Context The npm `latest` dist-tag of `@modelcontextprotocol/conformance` points at 0.1.16, the old 0.1 line, while the 2026-07-28 requirement revisions and the frozen-requirements framework live in the 0.2.0 alpha line (`alpha` currently resolves 0.2.0-alpha.11). The runners invoked the bare package name, so `rake conformance` and the conformance CI job resolve whatever `latest` points at: a fresh machine silently downgrades to the old suite while a warm npx cache may keep serving a newer one, and the suite actually in use depends on the environment. Pinning the `alpha` dist-tag puts every run on the line the frozen conformance requirements are anchored to (0.2.0-alpha.10, published 2026-07-27). Running the real alpha suite surfaced the actual client-leg state, so the expected-failures baseline is updated to match: `initialize` now passes (the harness accepts 2026-07-28, fulfilling the TODO on the old entry), and the scenarios pending client-side work are recorded with their reasons: the 2026-07-28 stateless lifecycle and the SEP-2322 MRTR driver, SEP-2243 custom-header mirroring, client-side JSON Schema preservation, the DPoP and WIF authorization extensions, and one `auth/authorization-server-migration` check added to the suite after the alpha.10 anchor. The manual `npx` examples in `conformance/README.md` carry the same pin so local reproduction runs the same suite as the rake task. ## How Has This Been Tested? `npx --yes @modelcontextprotocol/conformance@alpha --version` resolves 0.2.0-alpha.11, and `bundle exec rake conformance` passes the baseline check on both legs: the server leg reports 73 passed with 0 failed and no expected failures, and the client leg fails exactly the scenarios the baseline records. ## Breaking Changes None. The change is confined to the conformance tooling; the shipped gem is untouched.
This commit is contained in:
parent
05c3585bba
commit
5932521552
4 changed files with 23 additions and 10 deletions
|
|
@ -47,10 +47,10 @@ bundle exec rake conformance_server PORT=3000
|
|||
bundle exec rake conformance_server
|
||||
|
||||
# Terminal 2: run all scenarios
|
||||
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp
|
||||
npx @modelcontextprotocol/conformance@alpha server --url http://localhost:9292/mcp
|
||||
|
||||
# Terminal 2: run a single scenario
|
||||
npx @modelcontextprotocol/conformance server --url http://localhost:9292/mcp --scenario ping
|
||||
npx @modelcontextprotocol/conformance@alpha server --url http://localhost:9292/mcp --scenario ping
|
||||
```
|
||||
|
||||
Keeps the server alive between test runs, which avoids the startup overhead when iterating
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# frozen_string_literal: true
|
||||
|
||||
# Runs `npx @modelcontextprotocol/conformance client` against the conformance client script.
|
||||
# Runs `npx @modelcontextprotocol/conformance@alpha client` against the conformance client script.
|
||||
require "English"
|
||||
|
||||
module Conformance
|
||||
|
|
@ -29,7 +29,7 @@ module Conformance
|
|||
npx_command = [
|
||||
"npx",
|
||||
"--yes",
|
||||
"@modelcontextprotocol/conformance",
|
||||
"@modelcontextprotocol/conformance@alpha",
|
||||
"client",
|
||||
"--command",
|
||||
"bundle exec ruby #{client_script}",
|
||||
|
|
|
|||
|
|
@ -1,6 +1,19 @@
|
|||
server: []
|
||||
client:
|
||||
# TODO: The conformance framework's client-initialization check pins the expected requested version to 2025-11-25,
|
||||
# while this SDK's latest protocol version is 2026-07-28. Remove once the @modelcontextprotocol/conformance package
|
||||
# accepts 2026-07-28.
|
||||
- initialize
|
||||
# Pending client-side support for the 2026-07-28 stateless lifecycle: the conformance client
|
||||
# still connects through the legacy handshake and has no SEP-2322 MRTR driver yet.
|
||||
- request-metadata
|
||||
- sep-2322-client-request-state
|
||||
# SEP-2243 client-side custom-header mirroring and invalid-tool header gating
|
||||
# are not implemented in `MCP::Client::HTTP` yet.
|
||||
- http-custom-headers
|
||||
- http-invalid-tool-headers
|
||||
# Client-side JSON Schema preservation gaps: `$ref` values must reach the caller unresolved.
|
||||
- json-schema-ref-no-deref
|
||||
- json-schema-2020-12-preservation
|
||||
# Unimplemented authorization extensions.
|
||||
- auth/dpop
|
||||
- auth/dpop-nonce
|
||||
- auth/wif-jwt-bearer
|
||||
# Fails one check added to the suite after the 0.2.0-alpha.10 requirements anchor.
|
||||
- auth/authorization-server-migration
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# frozen_string_literal: true
|
||||
|
||||
# Starts the conformance server and runs `npx @modelcontextprotocol/conformance` against it.
|
||||
# Starts the conformance server and runs `npx @modelcontextprotocol/conformance@alpha` against it.
|
||||
require "English"
|
||||
require "net/http"
|
||||
require_relative "server"
|
||||
|
|
@ -31,7 +31,7 @@ module Conformance
|
|||
def build_command
|
||||
expected_failures_yml = File.expand_path("expected_failures.yml", __dir__)
|
||||
|
||||
npx_command = ["npx", "--yes", "@modelcontextprotocol/conformance", "server", "--url", "http://localhost:#{@port}/mcp"]
|
||||
npx_command = ["npx", "--yes", "@modelcontextprotocol/conformance@alpha", "server", "--url", "http://localhost:#{@port}/mcp"]
|
||||
npx_command += ["--scenario", @scenario] if @scenario
|
||||
npx_command += ["--spec-version", @spec_version] if @spec_version
|
||||
npx_command += ["--verbose"] if @verbose
|
||||
|
|
|
|||
Loading…
Reference in a new issue