Feat/tasks mrtr extension (#262)
* feat(tasks): SEP-2663 lifecycle scenario (8 checks)
Adds the first scenario for the SEP-2663 io.modelcontextprotocol/tasks
extension — a single TasksLifecycleScenario covering sync vs async
dispatch, DetailedTask shape on tasks/get, tool errors vs protocol
errors, and cancellation semantics. 8 ConformanceCheck records, all
passing against a SEP-2663-conformant Go fixture.
Why "tasks" (not "tasks-v2"): SEP-2663 IS the tasks surface once it
lands; the v2 suffix is only meaningful in implementations that
maintain a v1 surface alongside, which the conformance suite does not.
Layout:
- src/scenarios/server/tasks/lifecycle.ts — scenario class
- src/scenarios/server/tasks/helpers.ts — raw-fetch escape hatch
(the SDK's typed schemas strip resultType/inputRequests/...)
- src/scenarios/server/tasks/lifecycle.test.ts — fork-local vitest
runner. Two modes: spawn a fixture binary via MCPKIT_TASKS_BINARY,
or point at an already-running server via MCPKIT_TASKS_SERVER_URL.
Skips when neither is set so it doesn't break upstream CI runs that
go through everything-server (which doesn't yet implement
io.modelcontextprotocol/tasks).
Scenario is registered in pendingClientScenariosList so
all-scenarios.test.ts skips it; promote to active once the upstream
fixture grows extension support.
Tagged ['extension', DRAFT_PROTOCOL_VERSION] — selectable via
--suite extensions and --spec-version draft.
* style(tasks): apply prettier formatting
* feat(tasks,mrtr): port full SEP-2663 + SEP-2322 scenario suite
Builds out the rest of the tasks scenarios (atop the lifecycle canary)
and adds the SEP-2322 ephemeral MRTR scenario in a sibling folder.
Both target their own fixtures; both runners are brand-neutral and
language-agnostic (TASKS_SERVER_URL / TASKS_SERVER_CMD,
MRTR_SERVER_URL / MRTR_SERVER_CMD; readiness via TCP polling).
Tasks ClientScenario classes:
- TasksLifecycleScenario (8 checks; v2-01..v2-08)
- TasksCapabilityNegotiationScenario (4 checks; v2-11/22/23/25, SEP-2575)
- TasksWireFieldsScenario (3 checks; v2-12/13/21)
- TasksRequestStateScenario (3 checks; v2-14/15/28)
- TasksMRTRInputScenario (3 checks; v2-16/17/29 partial fulfillment)
- TasksRequestHeadersScenario (3 checks; SEP-2243 request-header tolerance)
- TasksDispatchScenario (8 checks; v2-09/10/19/20/26/27/30/31)
- TasksStatusNotificationsScenario (1 check; SEP-2663 §notifications, optional)
MRTR ClientScenario class:
- MrtrEphemeralFlowScenario (7 checks + 1 SKIPPED; mrtr-01..07,
mrtr-08 deferred for spec terminology +
reference-impl reasons)
Both runners spawn the fixture via a shell command and detect readiness
by TCP-polling the URL's host/port — no log-line scanning, no
language-specific assumptions. The same env vars work for any server
implementation.
Scenarios are tagged ['extension', DRAFT_PROTOCOL_VERSION] and registered
in pendingClientScenariosList so all-scenarios.test.ts (which targets
the upstream everything-server) skips them until the fixture grows
SEP-2322 / SEP-2663 support.
* docs(tasks,mrtr): scenario READMEs for upstream porting
Restructured around ClientScenario classes (one row per class with
check-list under it) rather than per-numbered-test slugs. Documents
fixture requirements, env vars, open spec questions, and the
wire-format diff for each suite.
Per AGENTS.md, severity follows spec keyword (MUST/MUST NOT → FAILURE,
SHOULD/SHOULD NOT → WARNING). The READMEs explain why some checks emit
INFO rather than FAILURE (optional emission paths per SEP-2322).
* tasks: assert createdAt + lastUpdatedAt; factor _shared/ helpers
Two reviewer-driven additions:
1. SEP-2663 createdAt / lastUpdatedAt ISO-8601 assertion in
`tasks-server-task-creation` (per Luca's PR #262 review feedback).
The check now flags servers that emit non-ISO timestamps (epoch
seconds, RFC-2822, etc.) on TaskInfoV2 envelopes.
2. Factor cross-cutting test-harness helpers into _shared/:
- `_shared/test-runner.ts` — `waitForServerReady` (renamed from
`waitForTcpReady`; the call site cares about server readiness,
not the TCP-poll mechanism). Imported by tasks/ and mrtr/
all-scenarios.test.ts; replaces ~30 LOC of inline duplication
in each.
- `_shared/wire-format.ts` — `ISO_8601_PATTERN` constant +
`isIso8601(s)` predicate. Documented rationale for choosing a
regex over `Date.parse` (too permissive),
`new Date(s).toISOString()` (too strict), or
`Temporal.Instant.from` (Node 24+ experimental). Future
wire-shape predicates (data URI, percent-encoded filename,
etc.) can land here.
Cherry-pick footprint when graduating to upstream PR is the SEP
folder + the imported `_shared/` files. First PR through carries
them upstream; subsequent feat branches inherit via standard
upstream-sync flow.
All 9 scenario tests still pass against the Go reference fixtures.
* refactor(tasks,mrtr): use SDK Client + AnyResult instead of raw-fetch helpers
Drops initRawSession/rawRequest/rawRequestFull from tasks/helpers.ts in
favor of the SDK's Client + StreamableHTTPClientTransport, paired with
a Zod passthrough schema (AnyResult) that preserves SEP-2663 / SEP-2322
draft fields the SDK's typed schemas would strip.
headers.ts and notifications.ts keep a small inline fetch where the SDK
can't reach: per-request HTTP headers (SEP-2243) and SSE notification
observation. Both reuse the SDK session via transport.sessionId.
All SEP-2663 + MRTR ephemeral-flow scenarios pass against the Go fixture.
* style: prettier formatting on tasks/mrtr scenarios
* tasks: align ttlMs / pollIntervalMs assertions per 2026-05-07 SEP
PR 2663 commit 62758914 standardised every duration field on the Ms
suffix, integer milliseconds. wire-fields.ts now asserts ttlMs and
pollIntervalMs are present on CreateTaskResult, the legacy v1 ttl
and pollInterval keys are absent (already covered), and the interim
ttlSeconds / pollIntervalMilliseconds keys are also absent on a
post-2026-05-07 server. lifecycle.ts and the scenario README pick up
matching prose updates.
Verified by make testconf-tasks-v2 (8/8) against a renamed mcpkit
fixture, and make testconf-mrtr (7/7 + 1 SKIPPED) against the paired
MRTR surface.
* style: prettier column alignment on tasks README
* mrtr: flip MRTR_INPUT_REQUIRED_RESULT_TYPE per merged SEP-2322
SEP-2322 merged on 2026-05-06 with the variant renamed from
IncompleteResult to InputRequiredResult and the resultType discriminator
from "incomplete" to "input_required" (commit de6d76fb, per dsp-ant
request). The MRTR_INCOMPLETE_RESULT_TYPE constant was specifically
designed as a one-line flip point for this scenario.
Renames
- MRTR_INCOMPLETE_RESULT_TYPE = "incomplete" ->
MRTR_INPUT_REQUIRED_RESULT_TYPE = "input_required"
- isIncompleteResult -> isInputRequiredResult
- All "IncompleteResult" -> "InputRequiredResult" in scenario prose
and check descriptions (ephemeral-flow.ts, README.md)
SEP-2663 had not yet flipped its discriminator literal as of PR head
82fb2c4d (5/7 21:52 UTC). Caitie's 5/15 RC commitment (issue comment
4384052694 on PR 2322) tracks the alignment to "input_required" both
sides. The constant remains the one-line flip point in case the 2663
follow-up surprises us.
Tested via mcpkit's make testconf-mrtr (7/7 + 1 SKIPPED green against
a renamed mcpkit fixture) and make testconf-tasks-v2 (8/8 still
green, no regressions on the paired surface).
* mrtr: prettier formatting + fix README rename typo
Lefthook prettier reformatted column alignment on first push attempt;
README also had a stale "renamed from InputRequiredResult" — should
read "renamed from IncompleteResult". Fix both.
* mrtr-08: refresh SKIPPED prose post SEP-2322 merge
Two stale references in the mrtr-tasks-composition SKIPPED check:
- Comment block + errorMessage framed blocker (a) as "spec authors
disagree" / "input_required vs incomplete". SEP-2322 merged
2026-05-06 with "input_required" (commit de6d76fb). The blocker
now reads as "SEP-2663 has not yet aligned to the merged 2322
literal" — Caitie's 5/15 RC commitment (PR 2322 issue comment
4384052694) tracks the alignment.
- errorMessage referenced "IsIncomplete signal" — that field was
renamed to IsInputRequired on the mcpkit side in lockstep with
the SEP-2322 wire-variant rename. Updated to match.
Status stays SKIPPED because blocker (b) — the mcpkit middleware
refactor (issue 347) — is still open.
* mrtr-08: drop the resolved-discriminator framing from the SKIPPED message
After SEP-2322 merged with "input_required", the only blocker that
actually keeps mrtr-08 SKIPPED is the eager-task-creation pattern in
reference-server middleware (panyam/mcpkit issue 347). The earlier
two-blocker framing read as if the test were waiting on both, but
blocker (a) is effectively resolved for any server that emits the
merged-2322 literal — leaving (b) as the sole gate.
Tighten the comment block + description + errorMessage to lead with
the middleware refactor and demote the discriminator history to a
parenthetical aside.
* style: prettier line-break on mrtr-08 description string
* tasks-wire-fields: address PR 262 review feedback
Two requested changes from the SEP-2663 author's review pass on
modelcontextprotocol/conformance PR 262
(pullrequestreview-4254601106).
(1) Drop the interim-key absence checks. The transition window between
"ttlSeconds / pollIntervalMilliseconds" and the final
"ttlMs / pollIntervalMs" wording is over now that the merged
spec settled on the Ms suffix. Useful while the spec was in
flight, noise once it stabilized. Removes the absence checks
plus the surrounding comment + description + details fields.
(2) Add Number.isInteger() to ttlMs and pollIntervalMs validation.
Spec says integer milliseconds; the previous typeof + range check
would have allowed fractional values. Now both fields fail if
they're not integers.
README scenario table tightened: "ttlMs + pollIntervalMs present and
integer-valued; legacy ttl / pollInterval keys absent".
* style: prettier column alignment on wire-fields README row
* feat(tasks): align scenarios with the Final-merged SEP-2663 spec
SEP-2663 merged Final on 2026-05-15. Four normative wire changes baked in at merge time. This commit updates the tasks server-conformance scenarios so the suite validates each one as a MUST rather than tolerating divergence as INFO/MAY.
1. notifications/tasks/status renamed to notifications/tasks. The status-notifications scenario now FAILs on observing the legacy method on the v2 surface, and validates the new name's payload shape when emitted.
2. notifications/progress and notifications/message MUST NOT be sent for tasks. Two new absence-asserts on the status-notifications scenario, reusing the existing SSE-observation harness.
3. requestState removed from the tasks-v2 wire. The previous request-state scenario tested the deleted "Request State Management" section; replaced with absence-asserts on CreateTaskResult, DetailedTask, and notifications/tasks payloads. SEP-2322's InputRequiredResult on the MRTR surface still carries requestState and is unchanged.
4. -32003 (Missing Required Client Capability) for required-task tools when the client did not negotiate the extension. New scenario TasksRequiredTaskErrorScenario initializes without declaring the extension, calls a TaskSupport=required tool, and asserts the rejection code plus the structured requiredCapabilities payload. Documents `failing_job` as a fixture requirement.
Verified locally against panyam/mcpkit examples/tasks-v2: 8/9 scenarios pass; the request-state-removal scenario intentionally FAILs against the current mcpkit main because the implementation still emits requestState on DetailedTask. That gap is the next item on the mcpkit catch-up plan.
* chore(tasks): address PR 262 review feedback
- notifications.ts: skip the entire scenario; SEP-2663 routes
notifications/tasks over the SEP-2575 subscriptions/listen stream,
not the tools/call POST SSE the prior harness observed. Body
preserved in git blame for the rewrite follow-up.
- request-state.ts: keep the absence-asserts (per the asymmetric
recommendation in review) and reframe the doc block around the
SEP-2322 lexical-adjacency motivation rather than pre-merge SEP
history.
- required-task-error.ts: cite SEP-2575 §"Missing Required
Capabilities" as the canonical home of -32003; add SEP_2575_REF to
specReferences on the -32003 checks.
- Sweep decontextualized "merged spec dropped X" / "pre-merge SEP-2663
said Y" framing from tasks/* doc blocks and inline check
descriptions.
* chore(tasks): prettier — strip trailing blank line in notifications.ts
* chore(tasks): drop `requestState` from `tasks/update` params
`taskId` is the session-continuation handle on the tasks-v2 MRTR-resume
path; `requestState` lives only on the SEP-2322 ephemeral
InputRequiredResult surface, not on the tasks-v2 wire.
* chore(tasks): loosen strict ack equality on tasks/cancel + tasks/update
The cancel-empty-ack and tasks/update-ack checks were comparing the
whole response against the literal `{resultType:"complete"}` via
`JSON.stringify` equality. That rejected acks carrying `_meta` or any
other future result-shape metadata even though the spec permits them.
Now structural: assert `resultType === "complete"` plus absence of the
task-envelope fields (`taskId`, `status`, `result`, `error`,
`inputRequests`). `_meta` and other metadata pass through.
* chore(tasks): poll for terminal status after tasks/cancel
The cancel-empty-ack check observed status via an immediate tasks/get
after the cancel ack. Cancellation is eventually-consistent, so the
task could still be `working` (or transitioning) at the moment of
observation, making the assertion flaky against any reasonable server.
Now polls via `waitForTerminal()` and asserts the terminal status is
`cancelled`. README's `slow_compute` fixture row documents the
matching tool contract ("MUST settle to `cancelled` when
`tasks/cancel` arrives while running") so the test has a deterministic
anchor.
* chore(tasks): tasks/cancel on a terminal task is idempotent
The previous check asserted `-32602` when cancelling an already-terminal
task. That forces every client to handle the race between observing a
task's terminal status and racing in a cancel request — the task can
terminate between the two and the cancel then errors instead of acking.
Check 8 renamed to `tasks-cancel-terminal-idempotent-ack` and now
asserts the same empty-ack shape (`resultType:"complete"` + no task-
envelope fields) as on an active task. README's lifecycle table row
updated to match.
* docs(tasks): drop `requestState` from SEP-2663 wire-field listings
`requestState` belongs to the SEP-2322 ephemeral MRTR result shape
(`InputRequiredResult`), not to the tasks-v2 wire. The README listed
it under SEP-2663 wire fields in four spots — top blurb, the SEP-2663
row of the Specs table, the v1↔v2 wire-format diff, and the raw-fetch
design note. All four now describe only the actual SEP-2663 shapes;
the raw-fetch note still calls out the SEP-2322 ephemeral
`requestState` separately so the rationale for stripping it via raw
fetch stays intact. The SEP-2322 row of the Specs table is unchanged
since `requestState` is correctly listed there as an MRTR base type.
* docs(tasks): rewrite stale request-state table to match absence-asserts
The README's `tasks-request-state` table still described the prior
scenario shape (shape / echo / stale-tolerance checks), which had
assumed `requestState` was on the tasks-v2 wire. The scenario itself
was rewritten earlier into the two-check absence assertion
(`tasks-create-result-no-request-state` /
`tasks-get-detailed-no-request-state`); the README now matches.
A motivational paragraph above the table explains why a negative
test exists for a field the spec never defines (the SEP-2322
lexical-adjacency confusion vector).
* chore(tasks): assert -32003 (not -32601) for gated tasks methods
When a client calls tasks/get / tasks/update / tasks/cancel without
having negotiated the io.modelcontextprotocol/tasks extension, the
suite now asserts -32003 (Missing Required Client Capability,
SEP-2575 §"Missing Required Capabilities") instead of -32601.
The spec doesn't currently mandate this code for the gated-method
path, so the assertion is forward-looking: it follows the SEP-2575
pattern that already governs `required` tools (and that
subscriptions/listen for tasks is expected to use). README's
open-questions list calls this out so readers know it's a
spec-clarification ahead-of-the-curve.
Drops the now-obsolete "Invalid requestState — silent ack vs -32602"
open-questions item: the previous request-state scenario covered
that, but the rewritten absence-asserts scenario no longer exercises
that path.
* chore(tasks): prettier — realign README table pipes after row edits
* feat(tasks/mrtr): initRawSession helper for draft-version handshake
The SDK's Client.connect() pins protocolVersion to LATEST_PROTOCOL_VERSION
in the initialize body. Scenarios tagged DRAFT_PROTOCOL_VERSION would
still negotiate the previous stable on the wire, which means a strict
draft-only server rejects the handshake with unsupported_protocol_version.
initRawSession sidesteps the SDK: a raw fetch initialize that carries the
draft version, captures the session ID, sends notifications/initialized,
and exposes the small surface scenarios actually use - request,
requestFull, notification, close. Errors are thrown as McpError so
existing instanceof / .code checks keep working. SSE response parsing
folds in the per-scenario raw helper from headers.ts.
All 8 SEP-2663 tasks scenarios + the MRTR ephemeral-flow scenario migrate
to the raw session. tsc clean.
* feat(tasks/mrtr): wire-mode matrix; SEP-2575 _meta + SEP-2243 routing
headers
initRawSession grows a `stateless` option that picks between the
legacy session wire (initialize handshake + Mcp-Session-Id) and the
SEP-2575 stateless wire (no initialize; server/discover up front;
per-request `_meta.io.modelcontextprotocol/{protocolVersion,
clientInfo, clientCapabilities}` envelope; `MCP-Protocol-Version`
header on every call).
Tasks behavior is wire-independent in spec, so the harness can run
each scenario against both wires. To keep the legacy run green
through this PR's refactor (and to surface stateless-mode gaps
deliberately rather than as a regression), the default is legacy-
only — set TASKS_WIRE_MODES=legacy,stateless (or just stateless) to
opt the matrix on. Same toggle for MRTR via MRTR_WIRE_MODES.
Two follow-on cleanups in the same commit so the legacy run actually
passes against an SEP-2243-enforcing server:
- All request paths now auto-emit Mcp-Method (and Mcp-Name for
tools/call.params.name / resources/read.params.uri) on protocol
versions in `SEP_2243_ENFORCED_VERSIONS` (today DRAFT-2026-v1).
Scenarios that probe the mismatch path can still override via
`extraHeaders`. Empty record on older versions so the helper stays
silent when the spec doesn't require it.
- `tasks-headers-body-method-authoritative` was written under an
older SEP-2243 reading where Mcp-Method was tolerant. The current
spec (and mcpkit's PR 477 implementation, plus the upstream
http-header-validation scenario) says the server MUST reject
mismatches with -32001 HeaderMismatch. Renamed to
`tasks-headers-reject-mismatched-method` and the assertion flipped
to expect the rejection.
Verification: testconf-tasks-v2 + testconf-mrtr both green on
legacy wire against mcpkit's tasks-v2 fixture (9/9 + 1/1).
* rename TASKS_WIRE_MODES / MRTR_WIRE_MODES → MCP_WIRE_MODES
Wire-mode choice is a cross-cutting conformance concern, not a
per-suite knob. Operators wouldn't realistically toggle tasks and
mrtr to different wires (both fixtures speak the same MCP server,
either it supports both wires or one). Single env var read by both
harnesses; default unchanged at ['legacy'].
* default MCP_WIRE_MODES to legacy+stateless
The conformance suite is brand-neutral — every SDK consuming it
should validate both MCP wires for SEP-2663 / SEP-2322 behavior by
default. Pinning the default to legacy-only let mcpkit's CI stay
green while gaps in its stateless dispatcher pended, but quietly
under-tested any other SDK that adopts the suite.
Default now runs every tasks-v2 scenario × 2 wires and the mrtr
ephemeral-flow scenario × 2 wires. SDKs with only one wire
implemented pin via MCP_WIRE_MODES=legacy or =stateless. mcpkit's
own audit will need that pin (or to close panyam/mcpkit issue 480)
until the stateless dispatcher consumes per-request
_meta.clientCapabilities for the tasks extension declaration.
* helpers: send MCP-Protocol-Version on every post-initialize POST
Per 2025-11-25 transport spec §Protocol-Version Header, clients MUST
include MCP-Protocol-Version on every subsequent HTTP request after
initialize. Universal post-initialize requirement; applies to both
the legacy session wire and the SEP-2575 stateless wire.
The helper was dropping the header on legacy follow-up calls as a
workaround for our server's detectWireKind treating the header as a
stateless-wire signal. That workaround was non-spec-compliant; the
right fix is for our server to drop the header from its wire-detection
precedence list and rely on body-level _meta.protocolVersion as the
SEP-2575 signal. Filed separately.
Comment thread: PR 5 review (Luca, 2026-05-27).
* types/helpers: parameterize wire mode via ScenarioRunOptions
Drop the module-level defaultStateless flag + setDefaultWireStateless
shim that the harness was toggling in beforeEach. Wire mode now flows
as an explicit parameter through ClientScenario.run, so tests can be
parallelized safely (no shared mutable state between concurrent runs).
ClientScenario.run gains an optional ScenarioRunOptions second arg.
Existing scenarios that ignore the arg satisfy the interface as-is
(TypeScript permits an implementation to declare fewer parameters than
the interface). The 10 task/mrtr scenarios that consume wire mode add
one line each to thread `opts?.stateless` into initRawSession.
Harness loops call `scenario.run(SERVER_URL, { stateless: wire === 'stateless' })`
directly; no more beforeEach setter.
Comment thread: PR 5 review (Luca, 2026-05-27, comments 2 and 7 on parallelism).
* harness: use describe.each + it.each for the (wire x scenario) matrix
Replaces the nested for-loops in both the tasks and mrtr harnesses
with vitest's table-driven describe.each + it.each idioms. Same N x M
cardinality of tests, but vitest sees the parameter table directly:
- Reporter labels render the wire row inline:
`... > stateless wire > 'tasks-lifecycle' - all checks ...`
- Each (wire, scenario) becomes a leaf test vitest can schedule
independently, opening the door to test.concurrent / pool config
later without restructuring.
WIRE_TABLE pairs each WireMode with the boolean flag the harness
needs to pass via ScenarioRunOptions, so the `it.each` callback
receives the prepared `stateless` directly rather than deriving it
inside every test body.
No on-wire change. Behavior identical against the reference fixture
(9 task scenarios + 1 mrtr scenario, each per wire).
Comment thread: PR 5 review (Luca, 2026-05-27, comments 5, 6, 9, 10).
* helpers: extend Mcp-Name to prompts/get + tasks/* (SEP-2243 + SEP-2663)
routingHeaders now auto-emits Mcp-Name for the full standard-header
surface the spec defines:
- prompts/get -> params.name (SEP-2243 §Standard Headers)
- tasks/get -> params.taskId (SEP-2663 §Streamable HTTP routing headers)
- tasks/update -> params.taskId (SEP-2663)
- tasks/cancel -> params.taskId (SEP-2663)
tools/call (params.name) and resources/read (params.uri) were already
covered. Switch from if/else chain to a method-name switch since the
surface is now wider; cleaner to extend further.
Helper doc updated with the full surface list and a TODO for SEP-2243
§Custom Headers from Tool Parameters (Mcp-Param-* mirroring from
x-mcp-header annotations), which requires caching the tools/list
schema. Tracked as a separate follow-up; no current tasks/mrtr
scenario exercises a tool with x-mcp-header annotated inputs.
Comment thread: PR 5 review (Luca, 2026-05-27, comment 4).
* _shared/wire-mode: hoist parseWireModes out of the per-suite harnesses
Tasks and mrtr harnesses had byte-identical copies of WireMode type,
VALID_MODES, DEFAULT_WIRE_MODES, parseWireModes, and WIRE_MODES.
Hoisted to src/scenarios/server/_shared/wire-mode.ts so the single
parser drives both suites. Each harness now does:
import { parseWireModes, type WireMode } from '../_shared/wire-mode';
const WIRE_MODES: WireMode[] = parseWireModes();
No behavior change. MCP_WIRE_MODES env var semantics identical;
default still legacy+stateless.
Companion TODO landed in tasks/helpers.ts: most of that module
(initRawSession, RawSession, routingHeaders, SEP_2243_ENFORCED_VERSIONS,
readJsonRpcResponse, AnyResult, the failureCheck/skipCheck/errMsg test
helpers) isn't actually tasks-specific and belongs in _shared too.
Holding that file reshuffle for a separate follow-up PR so the move
gets its own focused review thread without crowding the SEP work.
Comment thread: PR 5 review (Luca, 2026-05-27, comment 8).
* refactor: hoist raw-session + sep-refs + checks helpers into _shared/
Picks up the TODO left in tasks/helpers.ts on PR 5: the raw fetch
session machinery, the SEP-2243 routing-header builder, the SEP_*
reference constants, and the failureCheck / skipCheck / errMsg /
AnyResult test scaffolding were never tasks-specific. They lived in
tasks/helpers.ts only because tasks/ was the first server suite to
need them. mrtr/helpers.ts re-declared SEP_2322_REF + errMsg +
failureCheck for the same reason.
Move them to src/scenarios/server/_shared/:
- sep-refs.ts: SEP_2243_REF, SEP_2322_REF, SEP_2575_REF, SEP_2663_REF
- checks.ts: errMsg, failureCheck, skipCheck, AnyResult
- raw-session.ts: JsonRpcResponse, RawSession, InitRawOptions,
initRawSession (legacy + stateless paths),
routingHeaders, SEP_2243_ENFORCED_VERSIONS,
readJsonRpcResponse, nextRawId
tasks/helpers.ts shrinks to the tasks-shaped pieces it actually owns:
TASKS_EXTENSION_ID and the waitForTerminal / waitForStatus polling
loops. mrtr/helpers.ts drops the duplicate SEP ref + the duplicate
test-scaffolding wrappers and re-imports from _shared/.
skipCheck and failureCheck in _shared/checks.ts make specReferences
required (a shared scaffold can't pick a sensible default), so all
call sites now pass the SEP they're grading against explicitly.
Pure file move + import-path churn. No behavioral change. Typecheck
+ lint clean.
* feat: emit SEP-2243 Mcp-Param-* headers from tool inputSchema annotations
Closes the TODO in routingHeaders. SEP-2243 §"Custom Headers from
Tool Parameters" requires the client to mirror primitive-typed
tools/call arguments as `Mcp-Param-<Suffix>` HTTP headers when the
tool's inputSchema annotates the property with
`x-mcp-header: "<Suffix>"`. Servers behind L7 routers / WAFs use
these headers to make routing or policy decisions without parsing
the JSON body.
Implementation:
- RawSession gains a `toolSchemas: Map<string, unknown>` cache.
- `initRawSession` issues a best-effort `tools/list` after the
handshake (after `notifications/initialized` for legacy; after
`server/discover` for SEP-2575 stateless) and populates the
cache from the response. Failures are swallowed — empty cache is
the safe default.
- `routingHeaders` takes an optional `toolSchemas` and walks the
cached schema for tools/call: for each primitive-typed property
(string / number / integer / boolean) with `x-mcp-header`, it
encodes the argument value per SEP-2243 §value-encoding (plain
printable ASCII verbatim; everything else wrapped as
`=?base64?{b64-utf8}?=`) and emits `Mcp-Param-<Suffix>`.
- Both wire flavours (legacy + SEP-2575) call routingHeaders the
same way, so the same Mcp-Param-* headers ship regardless of
transport mode.
Behavior wash for tools without `x-mcp-header` annotations — they
emit only Mcp-Method + Mcp-Name as before. Behavior wash for
non-tools/call methods. No new dependencies (uses
`globalThis.btoa` + `TextEncoder`, both standard in Node 20+).
Typecheck + lint + the existing vitest suite stay green.
* chore(_shared/raw-session): drop reference-impl naming from doc comments
Tightens three doc-comment passages that named or hinted at a
specific server-side implementation as the canonical reference. The
conformance suite is positioned brand-neutral; the doc comments now
describe the wire behavior in spec-anchored language only.
No behavioral change.
* sep-refs: point at rendered SEP pages on modelcontextprotocol.io
PR URLs freeze at merge commit; the rendered SEP pages on the spec
website absorb post-merge amendments. Switch all four refs:
SEP-2243 → /seps/2243-http-standardization
SEP-2322 → /seps/2322-MRTR
SEP-2575 → /seps/2575-stateless-mcp
SEP-2663 → /seps/2663-tasks-extension
* raw-session: use eventsource-parser for SSE parsing
Delegate the SSE branch of readJsonRpcResponse to eventsource-parser
(the same parser the MCP TS SDK uses). The hand-rolled splitter
worked for the single-frame JSON-RPC response we need but didn't
cover the full WHATWG SSE semantics — multi-line `data:`
continuation, CR/CRLF line endings, comment lines, the
`event:`/`id:`/`retry:` fields. The library covers all of it.
Already a direct dependency (3.x), so no new transitive footprint.
* test: pin readJsonRpcResponse contract for SSE + JSON branches
12 vitest cases against `_shared/raw-session.ts`'s content-type
dispatcher: application/json (id-match, mismatch, error frames) and
text/event-stream (single-frame extraction, id-filter across multi-
event streams, multi-line `data:` continuation, CRLF tolerance,
comment/retry/id field handling, error-frame return, empty body,
non-JSON data interleaving).
Builds synthetic `Response` objects locally; no fixture server
required, runs in every CI. Tests pin the contract the
eventsource-parser swap inherits — guards against future parser-
version regressions and against any subsequent rewrite that subtly
changes the SSE branch.
* tasks: isolate negative-path dispatch checks (conformance#262)
The removed-method (-32601) and unknown-taskId (-32602) checks
in tasks-dispatch-and-envelope previously sent requests missing
Mcp-Name, so strict servers validating routing headers per
SEP-2243 §Server Behavior could reject before reaching method
dispatch — making the asserted error code unreachable.
- routingHeaders(): emit Mcp-Name on tasks/result so the
removed-method check fails specifically on -32601, not
-32001 HeaderMismatch.
- validTasksParams() helper: baseline params per
tasks-namespace method; negative checks override one field
to express isolation intent declaratively.
- ISOLATION_HINT on wrong-code failures points at the most
likely cause (routing headers / _meta / params shape
short-circuit before dispatch).
Forward-safe under either reading of the open SEP-2663
client-MUST → server-MUST-reject elevation question: the added
Mcp-Name matches the body, so SEP-2243's universal mismatch
rule can't fire either way.
Verified: 18/18 tasks scenarios pass.
* tasks: rename scenario check IDs to sep-2663-* slugs
Aligns 13 tasks scenario check IDs with the SEP-2484 traceability
convention used by sep-2322.yaml and others: the slug emitted at
runtime matches the check: row in the SEP's requirement-traceability
yaml, so plan.modelcontextprotocol.io can link the SEP requirement
to its implementing scenario.
Rename map:
tasks-tools-call-without-extension-sync
→ sep-2663-server-rejects-undeclared-client
tasks-methods-gated-without-extension
→ sep-2663-tasks-methods-non-declaring-32003
tasks-server-task-creation
→ sep-2663-result-type-task-on-create
tasks-get-during-working
→ sep-2663-tasks-get-status-working
tasks-get-terminal-inlined-result
→ sep-2663-tasks-get-status-completed
tasks-tool-error-completed-iserror
→ sep-2663-tool-error-uses-completed-status
tasks-protocol-error-failed-shape
→ sep-2663-tasks-get-status-failed
tasks-cancel-empty-ack
→ sep-2663-cancel-ack-empty-result
tasks-removed-tasks-result
→ sep-2663-tasks-result-removed-method-not-found
tasks-legacy-task-param-ignored
→ sep-2663-legacy-task-param-ignored
tasks-strong-consistency-immediate-get
→ sep-2663-durable-create-strong-consistency
tasks-get-unknown-task-id-rejected
→ sep-2663-tasks-get-invalid-task-id-32602
tasks-mrtr-input-requests-on-tasks-get
→ sep-2663-tasks-get-status-input-required
Wire behavior unchanged — only the reported slugs change. Verified
18/18 tasks scenarios still pass.
Untouched (intentionally): scenario-flow gates like
tasks-extension-advertised, tasks-sync-tool-call, tasks-wire-field-renames,
tasks-immediate-result-shortcut etc., which test end-to-end flows
rather than a specific RFC-2119 sentence and surface in the
traceability manifest as untracked (same pattern as sep-2322.yaml).
* seps: add sep-2663.yaml requirement-traceability declaration
Declares the 37 conformance checks (and 12 excluded requirements)
the SEP-2663 Tasks Extension surface should produce, mapping each
normative MUST/SHOULD sentence on the SEP page to either a check ID
emitted by an existing scenario or a documented reason for exclusion.
Exclusions are all client-internal, host-internal, or architectural
requirements that aren't observable at the protocol level — confirmed
during the new-sep skill's exclusion round.
The 13 scenario checks renamed in the previous commit now match the
check: slugs declared here; the remaining ~12 yaml rows describe
requirements without a scenario emitter yet — those are gaps to fill
in follow-up work (notifications via subscriptions/listen, key
uniqueness across MRTR rounds, legacy-capability migration probes).
spec_url points at the SEP page itself (status: Final) — SEP-2663's
normative content lives on /seps/2663-tasks-extension and is mirrored
in modelcontextprotocol/experimental-ext-tasks for source-control
purposes. Every quoted text: field was verified copy-paste findable
against the rendered SEP page after stripping markdown formatting.
* tasks: enforce SEP-2663 routing-header validation on tasks surface
Two related fixes:
1. tasks/headers.ts description previously said servers "MUST NOT
require" the Mcp-Method / Mcp-Name routing headers on the tasks
surface and "MUST NOT change dispatch behavior based on them —
including when the headers disagree with the body." Both claims
are spec-incorrect. SEP-2663 author confirmed in the conformance
review thread that the client-MUST in §"Streamable HTTP: Routing
Headers" elevates Mcp-Name to "required standard header" status
for tasks/get / tasks/update / tasks/cancel, so SEP-2243's
§"Server Behavior" mismatch + missing-header rejection applies.
Description rewritten to match the spec: routing headers are
REQUIRED, server MUST reject missing or mismatched with -32001.
2. Add a negative-path check covering the most common failure mode
on the tasks surface: tasks/get with mismatched Mcp-Name MUST
return -32001 HeaderMismatch. Probes tasks/get specifically;
tasks/update / tasks/cancel follow by extension. Missing-header
case is deferred (requires raw-session enhancement to suppress
a header entirely; current extraHeaders mechanism only overrides).
3. New yaml row pinning the rule to the SEP-2243 §"Server Behavior"
sentence via row-level url override (the rule itself is SEP-2243;
the scope of application on the tasks surface is what's
distinctively SEP-2663).
Known impact: surfaces a real gap in SDKs that derived their
routing-header validation from the SEP-2243 base table only (which
lists tasks/* as NOT requiring Mcp-Name). Those SDKs will see this
check go FAILURE until they extend their validation to cover
tasks-namespace methods per SEP-2663 §540.
CI against everything-server is unaffected (skips the tasks suite
entirely; everything-server doesn't implement the extension).
* mrtr: activate mrtr-08 composition check + fold into sep-2663.yaml
Folds PR 7 (panyam/mcpconformance) into PR 262:
- Cherry-picks the mrtr-tasks-composition activation from
feat/sep-2663-mrtr-08-composition (ad4f267 upstream): real end-to-end
check driving the SEP-2663 commit 451f5e1 promotion flow
(InputRequiredResult rounds gather user_name via elicitation/create,
then CreateTaskResult on the final round escalates to async; the
final tasks/get result reflects the answer gathered during MRTR).
- Addresses LucaButBoring's PR 7 nit: replaces the inline SEP-2663
reference literal with the shared SEP_2663_REF constant (now also
pointing at the rendered SEP page).
- Renames the scenario's emitted check ID from `mrtr-tasks-composition`
to `sep-2663-mrtr-synchronous-before-task-creation` to match the
corresponding row in sep-2663.yaml — closes the gap between
declared requirement and implementing check.
- Updates mrtr/README.md: refreshes the Specs-covered table and the
per-check description table for the new slug, and drops the "Open
issues" section (both blockers are resolved — discriminator value
aligned upstream; mcpkit middleware refactored per panyam/mcpkit#347).
- Updates the sep-2663.yaml flow-gate comment block to note that
the ephemeral mrtr-* flow gates also fall into the untracked
category for the same reason as the tasks-* flow gates.
One fewer skipped check on PR 262. Verified 123/123 server-suite
tests pass against mcpkit's tasks-v2 + mrtr fixtures (post
panyam/mcpkit#499 + #501).
Closes panyam/mcpconformance#7.
* tasks/mrtr: drop legacy wire when targeting DRAFT-2026-v1
SEP-2575 (Accepted) removes the initialize handshake on DRAFT-2026-v1
and replaces it with self-contained per-request _meta. Driving the
legacy initialize+Mcp-Session-Id wire against a draft fixture is
therefore a logical contradiction: the harness asserts
`protocolVersion: "DRAFT-2026-v1"` while doing the handshake the
version removed. Servers that strictly enforce SEP-2575 correctly
reject these requests with -32602 (see Randgalt's report on
conformance PR 262, 2026-05-31); lenient servers silently accept
them and mask the spec gap.
Both effects are wrong for the harness — we should only emit
spec-permitted traffic.
Adds `effectiveWireModes(protocolVersion)` in _shared/wire-mode.ts.
It wraps `parseWireModes()` and filters `legacy` out when the target
version is in POST_SEP_2575_VERSIONS (currently just DRAFT-2026-v1;
widen when a future dated release picks SEP-2575 up). If the user
explicitly pinned MCP_WIRE_MODES=legacy against a post-SEP-2575
version, the helper warns and falls back to ['stateless'] so the
suite still runs.
Both draft-only suites (tasks, mrtr) switch to
`effectiveWireModes(DRAFT_PROTOCOL_VERSION)`. The MCP_WIRE_MODES
override remains honored within the spec-permitted set.
Verified end-to-end with a strict mcpkit fixture (panyam/mcpkit
fix/sep-2575-enforce-meta-on-draft):
- pre-fix harness → 7/18 fail with the spec-required error
- post-fix harness → 9/9 pass (stateless wire only)
* tasks/mrtr: derive wire from spec version, drop RunContext.wire
`RunContext.wire` was a parallel knob to `specVersion` that the tasks
and MRTR harnesses used to loop scenarios across both the legacy
session wire and the SEP-2575 stateless wire. After SEP-2575 collapsed
the legacy wire on DRAFT-2026-v1 (a6f7c27), the matrix shrank to a
single column on draft — and an explicit `wire` field was both
redundant and unsafe: it let callers go out of sync with the
`specVersion` they declared on the wire.
The CLI's `runServerConformanceTest` never set the field, so every
`npm start -- server --scenario tasks-* --url ...` invocation silently
emitted the legacy `initialize` handshake against a draft server,
producing requests with no `_meta.io.modelcontextprotocol/*` envelope.
Strict SEP-2575 servers correctly rejected the traffic; lenient ones
accepted it and masked the gap.
Changes:
- Drop `wire?` from `RunContext`.
- Add `isStateless(ctx)` in `_shared/wire-mode.ts`; scenarios derive
`stateless` from `POST_SEP_2575_VERSIONS.has(ctx.specVersion)`.
- Extend the CLI's spec-version inference to default extension
scenarios to draft (today every extension scenario in the repo lives
there). Without this, the inference stayed on `LATEST_SPEC_VERSION`
for tasks scenarios and kept emitting legacy traffic.
- Flatten the now-1-element wire loop in the tasks + mrtr harnesses;
remove the parseWireModes/effectiveWireModes/MCP_WIRE_MODES infra
that drove it (no remaining consumers).
Adds a vitest regression in `runner/server.test.ts` that spins up an
in-process recording HTTP server, drives `runServerConformanceTest`
at `tasks-lifecycle`, and asserts the first outgoing call is
`server/discover` carrying `_meta.io.modelcontextprotocol/*` — the
SEP-2575 fingerprint — so the CLI wire selection can't silently
regress.
* scenarios + connection: address conformance#262 layout review
Folds the parallel mini-framework the tasks/MRTR suites had grown into
the existing harness shape pcarleton called out in conformance PR 262.
No new spec coverage; existing scenarios still pass 241/241 unit tests
and the regression added for the wire-derivation fix still pins the CLI
fingerprint.
Layout
scenarios/server/_shared/ is gone. The helper bundle moves to
scenarios/server/tasks-mrtr-helpers.ts (matching the existing
input-required-result-helpers.ts precedent); raw-session.ts and its
test fold into connection/stateless.ts.
Per-suite runners + READMEs (tasks/all-scenarios.test.ts,
mrtr/all-scenarios.test.ts, _shared/test-runner.ts, both READMEs) are
removed. The scenarios are already registered in
scenarios/index.ts pendingClientScenariosList, so the CLI runner
(`npm start -- server --scenario tasks-lifecycle --url ...` or
`--suite pending --url ...`) is the supported entry point. Per
AGENTS.md: don't reimplement the runner.
Connection extension
Connection grows the three things tasks/MRTR scenarios actually need
beyond the existing surface:
- `request(method, params, extraHeaders?)` for the SEP-2243
header-mismatch tests.
- `discover(): Promise<Record<string, unknown>>` for
capability-negotiation scenarios. Resolved eagerly from the SDK
accessors on the stateful wire; issued lazily as `server/discover`
on the stateless wire and memoized.
- `ConnectOptions { capabilities?, clientInfo? }` on
RunContext.connect() so scenarios can negotiate extensions
(`extensions: { 'io.modelcontextprotocol/tasks': {} }`) at
bootstrap on either wire.
Crucially the stateless wire does NOT eagerly emit `server/discover`
when capabilities are declared. SEP-2575 says per-request
`_meta.io.modelcontextprotocol/clientCapabilities` is authoritative;
a server that gates extensions on a prior `server/discover` is
non-conformant, and the conformance suite's job is to surface that,
not paper over it.
Scenario migration
Every tasks/MRTR scenario switches from `initRawSession(...)` to
`await ctx.connect({ capabilities })`, and the capability scenario
switches from `session.initializeResult.capabilities` to
`await withExt.discover()`. The wire choice now flows entirely from
`specVersion` — `isStateless(ctx)` (moved to connection/select.ts
next to `connectFor`) is the single source of truth.
Description sweep
Each scenario's `description` string now spells out the required
server fixtures (tool names + behavior) directly. Previously that
content lived only in the file-header comment, so an implementer
reading the runner's emitted description was missing it.
Reviewer notes
- 23 files changed, +455 / -291 (net + because the Connection
abstraction now carries surface that used to live in raw-session;
the raw-session deletion was 692 lines on its own).
- Unit tests: 241 pass (the 12 deleted tests were redundant with
connection-level coverage; the genuinely-unique SSE multi-line
`data:` test pinned behavior the connection parser doesn't have).
- Reference fixture runs: tasks scenarios surface a fixture-side
spec gap (per-request `_meta` not honored without a prior
`server/discover`). That's the suite working; the gap will be
filed upstream against the fixture, not papered over here.
* tasks: dedupe MRTR ephemeral-flow against #188, fold composition into tasks/
Of the 8 checks in mrtr/ephemeral-flow.ts, 7 substantially overlapped
the SEP-2322 InputRequiredResult* scenarios that #188 already shipped
(basic-elicitation, sampling, list-roots, request-state, multiple-input-
requests, multi-round, wrong-input-key). Only sep-2663-mrtr-synchronous-
before-task-creation — the SEP-2663 commit 451f5e1 composition path
where the MRTR loop's final round returns a CreateTaskResult — is
genuinely new versus #188's coverage, and it's SEP-2663-specific, not
SEP-2322-specific.
Moves that one check to its own tasks scenario in tasks/composition.ts
(extensionId = io.modelcontextprotocol/tasks, registered alongside the
other tasks-* scenarios in scenarios/index.ts) and deletes the entire
mrtr/ directory.
Reuses the existing #188 helpers (isInputRequiredResult, mockElicitResponse
from input-required-result-helpers.ts) rather than reintroducing the
parallel mrtr/helpers.ts ones.
* address review on conformance#262 follow-up
- connectStateless.discover() now reuses the request() pipeline (send +
drainEvents + unwrap) instead of inlining a duplicate copy. The local
memoization stays via discoverPromise ??=.
- runner/server.test.ts mock server no longer silently swallows
unparseable bodies. They land in a parseFailures sidecar that the
test asserts is empty — a non-JSON body to this mock is exactly the
kind of regression the test exists to catch.
- Drop the tautological connectFor identity tests. They asserted
`typeof === 'function'` and `not.toBe(connectStateful)`, both of which
hold trivially once connectFor wraps in a closure. Wire-shape behavior
is already covered by the connectStateless mock-fetch suite and by
runner/server.test.ts.
* tasks/mrtr: address PR 262 audit before review
Pre-emptive cleanup against the upstream patterns established by recent
merges (#318 Connection abstraction, #319 draft-spec consistency, #303
SEP-2243 check IDs, #321 MockServer surface). No behavior change for
scenarios — every check is still emitted; only the shape of how they're
emitted moves toward upstream's house style.
Surface
- Revert withRequestMeta / sendStatelessRequest signatures to match
upstream byte-for-byte. The need that pushed me to widen those
(per-request _meta.clientCapabilities for extension declaration on
the SEP-2575 stateless wire) is now handled inside connectStateless:
ConnectOptions.capabilities / clientInfo are folded into params._meta
via the trailing _meta spread that withRequestMeta already supports.
This keeps the new "extension declaration" concept on the Connection
layer only — the lower-level helpers stay shared with every other
scenario in the suite.
- Document that Connection.request's extraHeaders throws on the
stateful wire instead of "ignored with a warning" — the SDK transport
manages headers internally, so silent drop in a conformance harness
would mask test correctness.
Layout
- src/scenarios/server/tasks-mrtr-helpers.ts moves under tasks/ as
tasks/mrtr-helpers.ts. All 10 importers live in tasks/; the file's
prior placement next to input-required-result-helpers.ts no longer
applies once those scenarios moved into tasks/.
- Drop the unused AnyResult export + zod import. The Connection wrapper
doesn't take a zod schema, so the helper was dead on arrival.
- SEP-2322 spec reference now re-exports MRTR_SPEC_REFERENCES[0] from
input-required-result-helpers so the codebase holds one URL per SEP
instead of two for SEP-2322.
Naming
- All 10 tasks/*.ts switch to `readonly source = { ... } as const` to
match the convention used by caching.ts, resources.ts,
http-standard-headers.ts. Drops the ScenarioSource import where now
unused.
- Scenario-scope each scenario's connect-failure check ID
(tasks-lifecycle-bootstrap, tasks-headers-bootstrap, …) instead of
sharing tasks-session-bootstrap. Matches the upstream caching.ts
pattern (sep-2549-caching-connection) where each scenario owns its
own bootstrap-failure ID.
Traceability
- sep-2663.yaml gains the positive-path counterpart to the existing
rejection requirement under Streamable HTTP: Routing Headers, so
every sep-2663-* check ID emitted by tasks/*.ts is declared in the
manifest.
Verification
- Conformance vitest: 270/270.
- Build: clean.
- mcpkit testconf-tasks-v2 (47 fork scenarios + 1 sentinel): pass.
- mcpkit testconf-mrtr (3 fork scenarios + 1 sentinel): pass.
* tasks/required-task-error: anchor -32003 checks to sep-2663.yaml rows
The two checks in required-task-error.ts each verified a distinct
normative requirement that sep-2663.yaml already declared, but the
check IDs (tasks-required-task-error-code, tasks-required-task-error-data-shape)
used the tasks-* flow-gate prefix and went untracked by the traceability
manifest.
Promote both check IDs to sep-2663-*:
- tasks-required-task-error-code → sep-2663-server-returns-32003-when-required
- tasks-required-task-error-data-shape → sep-2663-server-returns-32003-data-shape
Split the existing YAML entry to mirror the split. The spec sentence
glues two observable requirements with a trailing colon (the JSON-RPC
code, and the data-shape payload that names the missing extension);
each gets its own row so each is verifiable independently.
Remove the now-stale tasks-required-task-error-code reference from the
sep-2663.yaml flow-gate comment block — that check is no longer a
flow-gate.
No behavior change. 270/270 vitest still pass.
* runner: forward ConnectOptions from RunContext.connect
server.ts built the RunContext with `connect: () => connectFor(...)(serverUrl)`,
dropping the opts argument scenarios pass to ctx.connect(). TypeScript
doesn't catch this because `() => T` is assignable to `(opts?) => T`.
Result: scenarios declaring capabilities sent DEFAULT_CLIENT_CAPABILITIES
on the wire in CLI runs while testContext() forwarded opts correctly,
so unit tests passed but `npm start -- server --scenario tasks-*` against
a strict-draft server failed with missing-capability errors.
Tightens the regression test to assert scenario-passed capabilities reach
_meta.clientCapabilities.extensions, not just that defaults are present.
* tasks: align scenario descriptions and cancel-ack check with SEP-2663
capability.ts: description said non-declaring tasks/* requests get
-32601 MethodNotFound; the assertion checks MissingRequiredClientCapability
per SEP-2663 §Error Handling. Reworded to the named error type and
per-request _meta/server-discover framing (SEP-2575).
lifecycle.ts: description claimed cancel-on-terminal MUST return
-32602; Check 8 in the same file asserts an idempotent ack and the
SEP has no -32602 carve-out for terminal tasks. Removed the bullet.
lifecycle.ts Check 7: SEP-2663 §Task Cancellation says transition to
`cancelled` is not guaranteed; stop failing the ack-shape check on
the settled status and record it in details instead.
* connection: fix stale docstring and drop redundant eta-expansion
stateless.ts docstring referenced `connection.initializeResult`; the
implementation exposes `discover()`.
select.ts wrapped connectStateful in an identity arrow; pass it
directly now that signatures match.
---------
Co-authored-by: Paul Carleton <paulc@anthropic.com>
This commit is contained in:
Sri Panyam
GitHub
parent
fe3a3103bc
commit
d70d7ad475
25 changed files with 3647 additions and 65 deletions
|
|
@ -1,30 +1,9 @@
|
|||
import { describe, it, expect, vi, afterEach } from 'vitest';
|
||||
import {
|
||||
connectFor,
|
||||
isStatefulVersion,
|
||||
STATELESS_SPEC_VERSIONS
|
||||
} from './select';
|
||||
import { connectStateful } from './stateful';
|
||||
import { isStatefulVersion, STATELESS_SPEC_VERSIONS } from './select';
|
||||
import { connectStateless } from './stateless';
|
||||
import { JsonRpcError } from './index';
|
||||
import { DRAFT_PROTOCOL_VERSION } from '../types';
|
||||
|
||||
describe('connectFor', () => {
|
||||
it('returns stateful for dated 2025-x versions', () => {
|
||||
expect(connectFor('2025-03-26')).toBe(connectStateful);
|
||||
expect(connectFor('2025-06-18')).toBe(connectStateful);
|
||||
expect(connectFor('2025-11-25')).toBe(connectStateful);
|
||||
});
|
||||
it('returns stateless for the draft version', () => {
|
||||
// connectFor wraps connectStateless in a closure (to pass the spec
|
||||
// version through), so identity with connectStateless no longer holds;
|
||||
// assert it did not select the stateful implementation. The wire-level
|
||||
// behaviour of the wrapper is covered in stateless.test.ts.
|
||||
expect(connectFor('2026-07-28')).not.toBe(connectStateful);
|
||||
expect(connectFor('2026-07-28')).not.toBe(connectStateless);
|
||||
});
|
||||
});
|
||||
|
||||
describe('STATELESS_SPEC_VERSIONS', () => {
|
||||
it('contains exactly the versions isStatefulVersion rejects', () => {
|
||||
expect(STATELESS_SPEC_VERSIONS.length).toBeGreaterThan(0);
|
||||
|
|
|
|||
|
|
@ -15,14 +15,40 @@
|
|||
import type { SpecVersion } from '../types';
|
||||
import type { JSONRPCNotification } from '../spec-types/2025-11-25';
|
||||
|
||||
/**
|
||||
* Options accepted at session bootstrap. On the stateful (2025-x) wire
|
||||
* these flow into the `initialize` request params; on the stateless
|
||||
* (2026-x) wire they live in `_meta.io.modelcontextprotocol/*` on the
|
||||
* `server/discover` request.
|
||||
*/
|
||||
export interface ConnectOptions {
|
||||
/**
|
||||
* Capabilities declared during session bootstrap (e.g.
|
||||
* `{ extensions: { 'io.modelcontextprotocol/tasks': {} }, elicitation: {} }`).
|
||||
*/
|
||||
capabilities?: Record<string, unknown>;
|
||||
/** Client info advertised at bootstrap; defaults to the harness's own info. */
|
||||
clientInfo?: { name: string; version: string };
|
||||
}
|
||||
|
||||
export interface Connection {
|
||||
/**
|
||||
* Send a JSON-RPC request and return its result.
|
||||
* Throws `JsonRpcError` on JSON-RPC error responses.
|
||||
*
|
||||
* `extraHeaders` extend or override the standard headers
|
||||
* (Content-Type, Accept, MCP-Protocol-Version, Mcp-Method, Mcp-Name)
|
||||
* for this call only, used by SEP-2243 routing-header tests that
|
||||
* inject a mismatch. Honored on the stateless wire; throws on the
|
||||
* stateful wire (the SDK transport manages headers internally, so
|
||||
* a per-call override would require dropping to raw fetch — silently
|
||||
* dropping the header in a conformance harness would mask test
|
||||
* correctness).
|
||||
*/
|
||||
request<R = unknown>(
|
||||
method: string,
|
||||
params?: Record<string, unknown>
|
||||
params?: Record<string, unknown>,
|
||||
extraHeaders?: Record<string, string>
|
||||
): Promise<R>;
|
||||
|
||||
/**
|
||||
|
|
@ -33,6 +59,20 @@ export interface Connection {
|
|||
*/
|
||||
readonly notifications: JSONRPCNotification[];
|
||||
|
||||
/**
|
||||
* Return the server's advertised capabilities, serverInfo, and
|
||||
* instructions. On the stateful wire this is synthesized from the
|
||||
* SDK Client's post-`initialize` accessors and resolves immediately;
|
||||
* on the stateless wire this issues `server/discover` (SEP-2575's
|
||||
* equivalent of the missing handshake) on first call and memoizes
|
||||
* the result.
|
||||
*
|
||||
* Scenarios that don't inspect server-side state never call this —
|
||||
* SEP-2575 has no required handshake, so paying for the extra request
|
||||
* is opt-in.
|
||||
*/
|
||||
discover(): Promise<Record<string, unknown>>;
|
||||
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
|
|
@ -48,7 +88,7 @@ export interface RunContext {
|
|||
* Scenarios that test the connection mechanics themselves (initialize,
|
||||
* GET-SSE, DNS rebinding) bypass this and use raw fetch.
|
||||
*/
|
||||
connect(): Promise<Connection>;
|
||||
connect(opts?: ConnectOptions): Promise<Connection>;
|
||||
}
|
||||
|
||||
export class JsonRpcError extends Error {
|
||||
|
|
@ -75,4 +115,4 @@ export {
|
|||
type JsonRpcResponse,
|
||||
type StatelessResponse
|
||||
} from './stateless';
|
||||
export { connectFor } from './select';
|
||||
export { connectFor, isStateless } from './select';
|
||||
|
|
|
|||
|
|
@ -9,30 +9,36 @@ import {
|
|||
ProgressNotificationSchema
|
||||
} from '@modelcontextprotocol/sdk/types.js';
|
||||
|
||||
import type { ConnectOptions } from './index';
|
||||
|
||||
const DEFAULT_CLIENT_INFO = {
|
||||
name: 'conformance-test-client',
|
||||
version: '1.0.0'
|
||||
} as const;
|
||||
|
||||
const DEFAULT_CAPABILITIES = {
|
||||
sampling: {},
|
||||
elicitation: {}
|
||||
} as const;
|
||||
|
||||
export interface MCPClientConnection {
|
||||
client: Client;
|
||||
close: () => Promise<void>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create and connect an MCP client to a server
|
||||
* Create and connect an MCP client to a server. `opts.capabilities` and
|
||||
* `opts.clientInfo` override the harness defaults — scenarios that
|
||||
* negotiate extensions (tasks, EMA, ...) pass them through to drive a
|
||||
* conformant `initialize`.
|
||||
*/
|
||||
export async function connectToServer(
|
||||
serverUrl: string
|
||||
serverUrl: string,
|
||||
opts: ConnectOptions = {}
|
||||
): Promise<MCPClientConnection> {
|
||||
const client = new Client(
|
||||
{
|
||||
name: 'conformance-test-client',
|
||||
version: '1.0.0'
|
||||
},
|
||||
{
|
||||
capabilities: {
|
||||
// Client capabilities
|
||||
sampling: {},
|
||||
elicitation: {}
|
||||
}
|
||||
}
|
||||
);
|
||||
const client = new Client(opts.clientInfo ?? DEFAULT_CLIENT_INFO, {
|
||||
capabilities: opts.capabilities ?? DEFAULT_CAPABILITIES
|
||||
});
|
||||
|
||||
const transport = new StreamableHTTPClientTransport(new URL(serverUrl));
|
||||
|
||||
|
|
|
|||
|
|
@ -3,13 +3,15 @@ import {
|
|||
DRAFT_PROTOCOL_VERSION,
|
||||
type SpecVersion
|
||||
} from '../types';
|
||||
import type { Connection } from './index';
|
||||
import type { Connection, ConnectOptions, RunContext } from './index';
|
||||
import { connectStateful } from './stateful';
|
||||
import { connectStateless } from './stateless';
|
||||
|
||||
/**
|
||||
* Spec versions that use the stateful lifecycle (initialize handshake,
|
||||
* Mcp-Session-Id). Anything not in this list uses the stateless lifecycle.
|
||||
* Mcp-Session-Id). Anything not in this list uses the stateless lifecycle
|
||||
* — SEP-2575 (Accepted) removed the initialize handshake on 2026-07-28
|
||||
* and later.
|
||||
*/
|
||||
const STATEFUL_VERSIONS: ReadonlySet<string> = new Set([
|
||||
'2024-11-05',
|
||||
|
|
@ -40,10 +42,22 @@ export const STATELESS_SPEC_VERSIONS: readonly SpecVersion[] =
|
|||
|
||||
export function connectFor(
|
||||
specVersion: SpecVersion
|
||||
): (serverUrl: string) => Promise<Connection> {
|
||||
): (serverUrl: string, opts?: ConnectOptions) => Promise<Connection> {
|
||||
return isStatefulVersion(specVersion)
|
||||
? connectStateful
|
||||
: // Pass the version through so stateless requests declare the spec
|
||||
// version the run was invoked with (matters under --force).
|
||||
(serverUrl) => connectStateless(serverUrl, specVersion);
|
||||
(serverUrl, opts) => connectStateless(serverUrl, specVersion, opts);
|
||||
}
|
||||
|
||||
/**
|
||||
* True when the spec version on the context requires the SEP-2575
|
||||
* stateless wire (no initialize handshake; per-request `_meta` envelope).
|
||||
*
|
||||
* Mirrors `connectFor` so scenarios that drive the wire directly (not via
|
||||
* the SDK-wrapped Connection) pick the wire the same way the connection
|
||||
* factory does.
|
||||
*/
|
||||
export function isStateless(ctx: Pick<RunContext, 'specVersion'>): boolean {
|
||||
return !isStatefulVersion(ctx.specVersion);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -15,10 +15,13 @@ import {
|
|||
} from '@modelcontextprotocol/sdk/types.js';
|
||||
import { connectToServer } from './sdk-client';
|
||||
import type { JSONRPCNotification } from '../spec-types/2025-11-25';
|
||||
import { JsonRpcError, type Connection } from './index';
|
||||
import { JsonRpcError, type Connection, type ConnectOptions } from './index';
|
||||
|
||||
export async function connectStateful(serverUrl: string): Promise<Connection> {
|
||||
const { client, close } = await connectToServer(serverUrl);
|
||||
export async function connectStateful(
|
||||
serverUrl: string,
|
||||
opts: ConnectOptions = {}
|
||||
): Promise<Connection> {
|
||||
const { client, close } = await connectToServer(serverUrl, opts);
|
||||
|
||||
const notifications: JSONRPCNotification[] = [];
|
||||
const collect = (n: unknown) => {
|
||||
|
|
@ -45,10 +48,30 @@ export async function connectStateful(serverUrl: string): Promise<Connection> {
|
|||
return {
|
||||
notifications,
|
||||
|
||||
// Synthesize the discover-shape from the SDK Client's post-`initialize`
|
||||
// accessors so the stateful Connection exposes the same surface the
|
||||
// stateless wire's `server/discover` produces.
|
||||
async discover(): Promise<Record<string, unknown>> {
|
||||
return {
|
||||
capabilities: client.getServerCapabilities() ?? {},
|
||||
serverInfo: client.getServerVersion() ?? {},
|
||||
instructions: client.getInstructions()
|
||||
};
|
||||
},
|
||||
|
||||
async request<R>(
|
||||
method: string,
|
||||
params: Record<string, unknown> = {}
|
||||
params: Record<string, unknown> = {},
|
||||
extraHeaders?: Record<string, string>
|
||||
): Promise<R> {
|
||||
if (extraHeaders && Object.keys(extraHeaders).length > 0) {
|
||||
// The SDK Client transport manages headers internally; per-call
|
||||
// override would require dropping to raw fetch. No 2025-x
|
||||
// scenario needs this today; flag loudly if one shows up.
|
||||
throw new Error(
|
||||
'connectStateful.request: extraHeaders is unsupported on the stateful wire (per-call header overrides require raw fetch on the stateless wire only)'
|
||||
);
|
||||
}
|
||||
try {
|
||||
return (await client.request({ method, params }, ResultSchema)) as R;
|
||||
} catch (e) {
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@
|
|||
|
||||
import { DRAFT_PROTOCOL_VERSION, type SpecVersion } from '../types';
|
||||
import type { JSONRPCNotification } from '../spec-types/2025-11-25';
|
||||
import { JsonRpcError, type Connection } from './index';
|
||||
import { JsonRpcError, type Connection, type ConnectOptions } from './index';
|
||||
|
||||
export interface JsonRpcResponse {
|
||||
jsonrpc: '2.0';
|
||||
|
|
@ -58,7 +58,9 @@ let nextRequestId = 1;
|
|||
|
||||
/**
|
||||
* The `Mcp-Name` source field per SEP-2243: `params.name` for tools/call and
|
||||
* prompts/get, `params.uri` for resources/read; absent otherwise.
|
||||
* prompts/get, `params.uri` for resources/read, `params.taskId` for the
|
||||
* SEP-2663 tasks methods (`tasks/get`, `tasks/update`, `tasks/cancel`).
|
||||
* Absent otherwise.
|
||||
*/
|
||||
export function mcpNameForRequest(
|
||||
method: string,
|
||||
|
|
@ -70,6 +72,13 @@ export function mcpNameForRequest(
|
|||
if (method === 'resources/read') {
|
||||
return typeof params?.uri === 'string' ? params.uri : undefined;
|
||||
}
|
||||
if (
|
||||
method === 'tasks/get' ||
|
||||
method === 'tasks/update' ||
|
||||
method === 'tasks/cancel'
|
||||
) {
|
||||
return typeof params?.taskId === 'string' ? params.taskId : undefined;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
|
|
@ -298,21 +307,52 @@ export async function sendStatelessRequest(
|
|||
* `sendStatelessRequest()`: classifies SSE-stream events into the notification
|
||||
* sink, surfaces server→client *requests* on the response stream as a spec
|
||||
* violation, and throws `JsonRpcError` on error responses.
|
||||
*
|
||||
* Session bootstrap on the stateless wire is `server/discover` (SEP-2575's
|
||||
* replacement for `initialize`). The result is exposed via
|
||||
* `connection.discover()` so scenarios can inspect server capabilities,
|
||||
* serverInfo, and supported protocol versions.
|
||||
*/
|
||||
export async function connectStateless(
|
||||
serverUrl: string,
|
||||
specVersion: SpecVersion = DRAFT_PROTOCOL_VERSION
|
||||
specVersion: SpecVersion = DRAFT_PROTOCOL_VERSION,
|
||||
opts: ConnectOptions = {}
|
||||
): Promise<Connection> {
|
||||
const notifications: JSONRPCNotification[] = [];
|
||||
const capabilities = opts.capabilities ?? DEFAULT_CLIENT_CAPABILITIES;
|
||||
const clientInfo = opts.clientInfo ?? CONFORMANCE_CLIENT_INFO;
|
||||
|
||||
async function request<R>(
|
||||
method: string,
|
||||
// The Connection layer is the single place that knows about
|
||||
// connect-time capabilities / clientInfo. We fold them into the
|
||||
// request's `_meta` here (the trailing `params._meta` spread in
|
||||
// `withRequestMeta` lets us override its defaults) so that
|
||||
// `sendStatelessRequest` and `withRequestMeta` keep their upstream
|
||||
// signatures untouched.
|
||||
function withConnectMeta(
|
||||
params?: Record<string, unknown>
|
||||
): Promise<R> {
|
||||
const response = await sendStatelessRequest(serverUrl, method, params, {
|
||||
specVersion
|
||||
});
|
||||
): Record<string, unknown> {
|
||||
return {
|
||||
...params,
|
||||
_meta: {
|
||||
'io.modelcontextprotocol/clientCapabilities': capabilities,
|
||||
'io.modelcontextprotocol/clientInfo': clientInfo,
|
||||
...(params?._meta as Record<string, unknown> | undefined)
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
async function send(
|
||||
method: string,
|
||||
params?: Record<string, unknown>,
|
||||
extraHeaders?: Record<string, string>
|
||||
): Promise<StatelessResponse> {
|
||||
return sendStatelessRequest(serverUrl, method, withConnectMeta(params), {
|
||||
specVersion,
|
||||
headers: extraHeaders
|
||||
});
|
||||
}
|
||||
|
||||
function drainEvents(response: StatelessResponse): void {
|
||||
for (const event of response.events ?? []) {
|
||||
if (typeof event !== 'object' || event === null) continue;
|
||||
if ('method' in event && !('id' in event)) {
|
||||
|
|
@ -324,7 +364,9 @@ export async function connectStateless(
|
|||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function unwrap<R>(method: string, response: StatelessResponse): R {
|
||||
const rpcError = response.body?.error;
|
||||
// Only a properly-shaped JSON-RPC error becomes a JsonRpcError; anything
|
||||
// else (e.g. a proxy's `{"error": "upstream timeout"}`) falls through so
|
||||
|
|
@ -350,8 +392,31 @@ export async function connectStateless(
|
|||
return response.body.result as R;
|
||||
}
|
||||
|
||||
// SEP-2575 has no required handshake. `_meta.clientCapabilities` on
|
||||
// every request is authoritative per spec, so `server/discover` is
|
||||
// strictly a client-side query ("what does the server advertise?").
|
||||
// We deliberately do NOT run it eagerly — a server that ignores
|
||||
// per-request `_meta` until a prior `server/discover` has registered
|
||||
// the session is non-conformant, and the conformance suite's job is
|
||||
// to surface that, not paper over it.
|
||||
let discoverPromise: Promise<Record<string, unknown>> | undefined;
|
||||
|
||||
async function request<R>(
|
||||
method: string,
|
||||
params?: Record<string, unknown>,
|
||||
extraHeaders?: Record<string, string>
|
||||
): Promise<R> {
|
||||
const response = await send(method, params, extraHeaders);
|
||||
drainEvents(response);
|
||||
return unwrap<R>(method, response);
|
||||
}
|
||||
|
||||
return {
|
||||
notifications,
|
||||
discover(): Promise<Record<string, unknown>> {
|
||||
return (discoverPromise ??=
|
||||
request<Record<string, unknown>>('server/discover'));
|
||||
},
|
||||
request,
|
||||
close: async () => {}
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
import { LATEST_SPEC_VERSION, type SpecVersion } from '../types';
|
||||
import { connectFor } from './select';
|
||||
import type { RunContext } from './index';
|
||||
import type { ConnectOptions, RunContext } from './index';
|
||||
|
||||
/**
|
||||
* Build a RunContext for unit tests that drive a scenario directly.
|
||||
|
|
@ -14,6 +14,6 @@ export function testContext(
|
|||
return {
|
||||
serverUrl,
|
||||
specVersion,
|
||||
connect: () => connectFor(specVersion)(serverUrl)
|
||||
connect: (opts?: ConnectOptions) => connectFor(specVersion)(serverUrl, opts)
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,7 +3,9 @@
|
|||
* skipping (an explicitly-requested version outside a scenario's window
|
||||
* skips rather than silently testing something else; --force overrides).
|
||||
*/
|
||||
import { describe, test, expect } from 'vitest';
|
||||
import http from 'http';
|
||||
import type { AddressInfo } from 'net';
|
||||
import { afterEach, beforeEach, describe, test, expect } from 'vitest';
|
||||
import { runServerConformanceTest } from './server';
|
||||
import { DRAFT_PROTOCOL_VERSION, LATEST_SPEC_VERSION } from '../types';
|
||||
|
||||
|
|
@ -50,3 +52,92 @@ describe('runServerConformanceTest spec-version applicability', () => {
|
|||
expect(result.checks.length).toBeGreaterThan(0);
|
||||
}, 60000);
|
||||
});
|
||||
|
||||
describe('runServerConformanceTest wire selection for draft-only scenarios', () => {
|
||||
// Regression: the CLI used to silently emit the legacy initialize+session
|
||||
// wire when running a draft-only scenario, producing requests with no
|
||||
// `_meta.io.modelcontextprotocol/*` envelope (and `initialize` rather
|
||||
// than `server/discover`). Deriving wire from spec version on the
|
||||
// RunContext makes the CLI emit SEP-2575 stateless traffic on draft.
|
||||
let server: http.Server;
|
||||
let url: string;
|
||||
const captured: Array<{ method?: string; params?: Record<string, unknown> }> =
|
||||
[];
|
||||
// Bodies the mock couldn't parse as JSON. We surface these via an
|
||||
// explicit assertion (rather than silently dropping) because in this
|
||||
// test's scope, every body MUST be a JSON-RPC request — anything else
|
||||
// is the kind of malformed-wire regression this test exists to catch.
|
||||
const parseFailures: string[] = [];
|
||||
|
||||
beforeEach(async () => {
|
||||
captured.length = 0;
|
||||
parseFailures.length = 0;
|
||||
server = http.createServer((req, res) => {
|
||||
let buf = '';
|
||||
req.on('data', (chunk) => {
|
||||
buf += chunk;
|
||||
});
|
||||
req.on('end', () => {
|
||||
let id: unknown = null;
|
||||
try {
|
||||
const body = JSON.parse(buf);
|
||||
captured.push({ method: body.method, params: body.params });
|
||||
id = body.id ?? null;
|
||||
} catch {
|
||||
parseFailures.push(buf);
|
||||
}
|
||||
res.statusCode = 200;
|
||||
res.setHeader('Content-Type', 'application/json');
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
jsonrpc: '2.0',
|
||||
id,
|
||||
error: { code: -32603, message: 'mock server: scenario aborted' }
|
||||
})
|
||||
);
|
||||
});
|
||||
});
|
||||
await new Promise<void>((resolve) =>
|
||||
server.listen(0, '127.0.0.1', resolve)
|
||||
);
|
||||
const port = (server.address() as AddressInfo).port;
|
||||
url = `http://127.0.0.1:${port}/mcp`;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
});
|
||||
|
||||
test('emits SEP-2575 stateless wire (no initialize, _meta envelope) on tasks-lifecycle', async () => {
|
||||
await runServerConformanceTest(url, 'tasks-lifecycle');
|
||||
|
||||
// Every outgoing body MUST be JSON-RPC. Hitting this assertion would
|
||||
// mean the harness emitted something the mock couldn't parse — a
|
||||
// significant regression in its own right, surface it loudly.
|
||||
expect(parseFailures).toEqual([]);
|
||||
|
||||
expect(captured.length).toBeGreaterThan(0);
|
||||
|
||||
// The legacy wire opens with an `initialize` handshake; SEP-2575
|
||||
// removes it. The scenario MUST NOT have sent one.
|
||||
expect(captured.some((c) => c.method === 'initialize')).toBe(false);
|
||||
|
||||
// Every body MUST carry the SEP-2575 `_meta` envelope.
|
||||
const first = captured[0];
|
||||
const meta = first.params?._meta as Record<string, unknown> | undefined;
|
||||
expect(meta).toBeDefined();
|
||||
expect(meta?.['io.modelcontextprotocol/protocolVersion']).toBe(
|
||||
DRAFT_PROTOCOL_VERSION
|
||||
);
|
||||
expect(meta?.['io.modelcontextprotocol/clientInfo']).toBeDefined();
|
||||
expect(meta?.['io.modelcontextprotocol/clientCapabilities']).toBeDefined();
|
||||
|
||||
// Scenario-passed capabilities (not just defaults) must reach the wire.
|
||||
const caps = meta?.['io.modelcontextprotocol/clientCapabilities'] as
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
expect(caps?.extensions).toMatchObject({
|
||||
'io.modelcontextprotocol/tasks': {}
|
||||
});
|
||||
}, 30000);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -74,10 +74,12 @@ export async function runServerConformanceTest(
|
|||
|
||||
// When --spec-version is omitted, infer the version from the scenario's
|
||||
// declared source so draft-only scenarios get the draft (stateless)
|
||||
// connection rather than the stateful latest-spec default.
|
||||
// connection rather than the stateful latest-spec default. Extension
|
||||
// scenarios are off-timeline; today every extension in this repo lives
|
||||
// on draft, so they fall under the same inference.
|
||||
const resolvedSpecVersion =
|
||||
specVersion ??
|
||||
('introducedIn' in scenario.source &&
|
||||
('extensionId' in scenario.source ||
|
||||
scenario.source.introducedIn === DRAFT_PROTOCOL_VERSION
|
||||
? DRAFT_PROTOCOL_VERSION
|
||||
: LATEST_SPEC_VERSION);
|
||||
|
|
@ -89,7 +91,7 @@ export async function runServerConformanceTest(
|
|||
const ctx: RunContext = {
|
||||
serverUrl,
|
||||
specVersion: resolvedSpecVersion,
|
||||
connect: () => connectFor(resolvedSpecVersion)(serverUrl)
|
||||
connect: (opts) => connectFor(resolvedSpecVersion)(serverUrl, opts)
|
||||
};
|
||||
const checks = await scenario.run(ctx);
|
||||
|
||||
|
|
|
|||
|
|
@ -86,6 +86,17 @@ import {
|
|||
InputRequiredResultValidateInputScenario
|
||||
} from './server/input-required-result';
|
||||
|
||||
import { TasksLifecycleScenario } from './server/tasks/lifecycle';
|
||||
import { TasksCapabilityNegotiationScenario } from './server/tasks/capability';
|
||||
import { TasksWireFieldsScenario } from './server/tasks/wire-fields';
|
||||
import { TasksRequestStateRemovalScenario } from './server/tasks/request-state';
|
||||
import { TasksMRTRInputScenario } from './server/tasks/mrtr-input';
|
||||
import { TasksRequestHeadersScenario } from './server/tasks/headers';
|
||||
import { TasksDispatchScenario } from './server/tasks/dispatch';
|
||||
import { TasksStatusNotificationsScenario } from './server/tasks/notifications';
|
||||
import { TasksRequiredTaskErrorScenario } from './server/tasks/required-task-error';
|
||||
import { TasksMrtrCompositionScenario } from './server/tasks/composition';
|
||||
|
||||
import {
|
||||
HttpHeaderValidationScenario,
|
||||
HttpCustomHeaderServerValidationScenario
|
||||
|
|
@ -122,7 +133,22 @@ const pendingClientScenariosList: ClientScenario[] = [
|
|||
// Pending until the everything-server fully implements SEP-2243
|
||||
// header validation (case-insensitive names, whitespace trimming, -32020 error code)
|
||||
new HttpHeaderValidationScenario(),
|
||||
new HttpCustomHeaderServerValidationScenario()
|
||||
new HttpCustomHeaderServerValidationScenario(),
|
||||
|
||||
// SEP-2663 Tasks extension. Pending because the everything-server
|
||||
// does not implement io.modelcontextprotocol/tasks; targeted runs
|
||||
// point at a SEP-2663-conformant fixture via
|
||||
// `npm start -- server --scenario tasks-* --url <fixture>`.
|
||||
new TasksLifecycleScenario(),
|
||||
new TasksCapabilityNegotiationScenario(),
|
||||
new TasksWireFieldsScenario(),
|
||||
new TasksRequestStateRemovalScenario(),
|
||||
new TasksMRTRInputScenario(),
|
||||
new TasksRequestHeadersScenario(),
|
||||
new TasksDispatchScenario(),
|
||||
new TasksStatusNotificationsScenario(),
|
||||
new TasksRequiredTaskErrorScenario(),
|
||||
new TasksMrtrCompositionScenario()
|
||||
];
|
||||
|
||||
// All client scenarios
|
||||
|
|
@ -189,6 +215,19 @@ const allClientScenariosList: ClientScenario[] = [
|
|||
new HttpHeaderValidationScenario(),
|
||||
new HttpCustomHeaderServerValidationScenario(),
|
||||
|
||||
// SEP-2663 Tasks extension. Pending against the everything-server;
|
||||
// targeted runs point at a SEP-2663-conformant fixture.
|
||||
new TasksLifecycleScenario(),
|
||||
new TasksCapabilityNegotiationScenario(),
|
||||
new TasksWireFieldsScenario(),
|
||||
new TasksRequestStateRemovalScenario(),
|
||||
new TasksMRTRInputScenario(),
|
||||
new TasksRequestHeadersScenario(),
|
||||
new TasksDispatchScenario(),
|
||||
new TasksStatusNotificationsScenario(),
|
||||
new TasksRequiredTaskErrorScenario(),
|
||||
new TasksMrtrCompositionScenario(),
|
||||
|
||||
// InputRequiredResult scenarios (SEP-2322)
|
||||
new InputRequiredResultBasicElicitationScenario(),
|
||||
new InputRequiredResultBasicSamplingScenario(),
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import { testContext } from '../../connection/testing';
|
|||
/**
|
||||
* SEP-2322 MRTR negative tests.
|
||||
*
|
||||
* Positive tests run via all-scenarios.test.ts against the everything-server
|
||||
* Positive tests run via the CLI runner against the everything-server
|
||||
* (which implements MRTR in its stateless path). These negative tests run
|
||||
* against a deliberately broken server to verify checks emit FAILURE.
|
||||
*/
|
||||
|
|
|
|||
275
src/scenarios/server/tasks/capability.ts
Normal file
275
src/scenarios/server/tasks/capability.ts
Normal file
|
|
@ -0,0 +1,275 @@
|
|||
/**
|
||||
* SEP-2663 Tasks Extension — capability negotiation conformance.
|
||||
*
|
||||
* Tests that the server advertises the io.modelcontextprotocol/tasks
|
||||
* extension correctly, gates the v2 task surface on negotiation, and
|
||||
* supports SEP-2575 per-request capability overrides.
|
||||
*
|
||||
* Required server fixtures:
|
||||
* - greet — sync-only, returns "Hello, {name}!"
|
||||
* - slow_compute — task-supporting, sleeps N seconds
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../../types';
|
||||
import type { Connection, RunContext } from '../../../connection';
|
||||
import { SEP_2575_REF, SEP_2663_REF } from './mrtr-helpers';
|
||||
import { errMsg, failureCheck } from './mrtr-helpers';
|
||||
import { TASKS_EXTENSION_ID } from './helpers';
|
||||
|
||||
export class TasksCapabilityNegotiationScenario implements ClientScenario {
|
||||
name = 'tasks-capability-negotiation';
|
||||
readonly source = { extensionId: 'io.modelcontextprotocol/tasks' } as const;
|
||||
description = `Test SEP-2663 capability negotiation for the tasks extension.
|
||||
|
||||
**Server Implementation Requirements:**
|
||||
|
||||
**Capability advertisement (SEP-2663):**
|
||||
- The server MUST advertise \`io.modelcontextprotocol/tasks\` under
|
||||
\`capabilities.extensions\` in its \`server/discover\` response.
|
||||
- It MUST NOT use a v1-style \`capabilities.tasks\` slot (the v1 surface
|
||||
is replaced by the extension).
|
||||
|
||||
**Gating without negotiation (SEP-2663):**
|
||||
- For requests that do NOT carry the \`io.modelcontextprotocol/tasks\`
|
||||
extension in \`_meta.clientCapabilities\`, the server MUST reject
|
||||
\`tasks/get\`, \`tasks/update\`, and \`tasks/cancel\` with a
|
||||
\`MissingRequiredClientCapability\` error.
|
||||
- A \`tools/call\` from such a session MUST NOT return
|
||||
\`CreateTaskResult\`. Task-supporting tools fall through to synchronous
|
||||
execution and return a plain \`ToolResult\` with
|
||||
\`resultType:"complete"\`.
|
||||
|
||||
**Per-request opt-in (SEP-2575):**
|
||||
- A session that did not declare the extension at session level can
|
||||
opt into task creation for a single \`tools/call\` by including the
|
||||
extension under \`_meta.io.modelcontextprotocol/clientCapabilities.extensions\`.
|
||||
The server MUST honor the per-request opt-in and produce a
|
||||
\`CreateTaskResult\` for that call.
|
||||
|
||||
**Required server fixtures (\`tools/list\` MUST include all):**
|
||||
- \`greet\` — sync-only, returns \`Hello, {name}!\`.
|
||||
- \`slow_compute\` — task-supporting, \`seconds\`-second sleep then a
|
||||
result.`;
|
||||
|
||||
async run(ctx: RunContext): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
// Two parallel sessions: one declares the extension, one does NOT.
|
||||
let withExt: Connection;
|
||||
let withoutExt: Connection;
|
||||
try {
|
||||
withExt = await ctx.connect({
|
||||
capabilities: {
|
||||
elicitation: {},
|
||||
sampling: {},
|
||||
extensions: { [TASKS_EXTENSION_ID]: {} }
|
||||
}
|
||||
});
|
||||
withoutExt = await ctx.connect({
|
||||
capabilities: {}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tasks-capability-bootstrap',
|
||||
name: 'TasksCapabilityBootstrap',
|
||||
description: 'Initialize handshakes (with + without extension) succeed',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed to initialize: ${errMsg(error)}`,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
return checks;
|
||||
}
|
||||
|
||||
// Check 1: server advertises extension under capabilities.extensions.
|
||||
{
|
||||
const id = 'tasks-extension-advertised';
|
||||
const name = 'TasksExtensionAdvertised';
|
||||
const description = `Server advertises ${TASKS_EXTENSION_ID} under capabilities.extensions (and not capabilities.tasks)`;
|
||||
const discovered = await withExt.discover();
|
||||
const caps: any = (discovered.capabilities as any) ?? {};
|
||||
const errs: string[] = [];
|
||||
if (caps.tasks) {
|
||||
errs.push(
|
||||
'v1-style capabilities.tasks slot MUST NOT be used; tasks lives under capabilities.extensions'
|
||||
);
|
||||
}
|
||||
if (!caps.extensions) {
|
||||
errs.push('capabilities.extensions MUST be advertised');
|
||||
} else if (!caps.extensions[TASKS_EXTENSION_ID]) {
|
||||
errs.push(
|
||||
`capabilities.extensions["${TASKS_EXTENSION_ID}"] MUST be present`
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF],
|
||||
details: {
|
||||
hasExtensions: Boolean(caps.extensions),
|
||||
hasTasksExtension: Boolean(caps.extensions?.[TASKS_EXTENSION_ID]),
|
||||
hasV1TasksSlot: Boolean(caps.tasks)
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Check 2: tasks/* methods rejected with -32003 (Missing Required
|
||||
// Client Capability) when the client did not negotiate the tasks
|
||||
// extension. Follows the SEP-2575 §"Missing Required Capabilities"
|
||||
// pattern — same code path as the required-task-error scenario and
|
||||
// (when implemented) subscriptions/listen for tasks.
|
||||
{
|
||||
const id = 'sep-2663-tasks-methods-non-declaring-32003';
|
||||
const name = 'TasksMethodsGatedWithoutExtension';
|
||||
const description =
|
||||
'tasks/get, tasks/update, tasks/cancel return -32003 when the client did not negotiate the tasks extension (SEP-2575 Missing Required Capabilities)';
|
||||
const cases: Array<{ method: string; params: any }> = [
|
||||
{ method: 'tasks/get', params: { taskId: 'gate-test' } },
|
||||
{
|
||||
method: 'tasks/update',
|
||||
params: { taskId: 'gate-test', inputResponses: {} }
|
||||
},
|
||||
{ method: 'tasks/cancel', params: { taskId: 'gate-test' } }
|
||||
];
|
||||
const errs: string[] = [];
|
||||
for (const tc of cases) {
|
||||
try {
|
||||
await withoutExt.request(tc.method, tc.params);
|
||||
errs.push(`${tc.method} MUST reject (it returned a result)`);
|
||||
} catch (e: any) {
|
||||
if (e.code !== -32003) {
|
||||
errs.push(
|
||||
`${tc.method} MUST return -32003; got ${e.code ?? '<missing>'}`
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2575_REF, SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
|
||||
// Check 3: tools/call without extension returns sync ToolResult, not task.
|
||||
{
|
||||
const id = 'sep-2663-server-rejects-undeclared-client';
|
||||
const name = 'TasksToolsCallWithoutExtensionSync';
|
||||
const description =
|
||||
'tools/call from a session without the extension MUST fall through to sync (no CreateTaskResult, even for task-supporting tools)';
|
||||
try {
|
||||
const result = (await withoutExt.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 0, label: 'capability-no-ext' }
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (result.resultType === 'task') {
|
||||
errs.push(
|
||||
'tools/call without extension MUST NOT return resultType:"task"'
|
||||
);
|
||||
}
|
||||
if (result.taskId) {
|
||||
errs.push(
|
||||
`tools/call without extension MUST NOT carry top-level taskId; got ${result.taskId}`
|
||||
);
|
||||
}
|
||||
if (result.task) {
|
||||
errs.push(
|
||||
'tools/call without extension MUST NOT carry the v1-style nested `task` envelope'
|
||||
);
|
||||
}
|
||||
if (!result.content) {
|
||||
errs.push(
|
||||
'tools/call without extension MUST return sync ToolResult with content[]'
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF],
|
||||
details: {
|
||||
resultType: result.resultType,
|
||||
hasTaskId: Boolean(result.taskId)
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 4: SEP-2575 per-request _meta opt-in produces CreateTaskResult.
|
||||
{
|
||||
const id = 'tasks-per-request-meta-opt-in';
|
||||
const name = 'TasksPerRequestMetaOptIn';
|
||||
const description =
|
||||
'tools/call with extension declared in _meta.io.modelcontextprotocol/clientCapabilities produces a CreateTaskResult even when the session did not negotiate the extension';
|
||||
try {
|
||||
const result = (await withoutExt.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 1, label: 'capability-meta-opt' },
|
||||
_meta: {
|
||||
'io.modelcontextprotocol/clientCapabilities': {
|
||||
extensions: { [TASKS_EXTENSION_ID]: {} }
|
||||
}
|
||||
}
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (result.resultType !== 'task') {
|
||||
errs.push(
|
||||
`expected resultType:"task" via per-request opt-in; got ${JSON.stringify(result.resultType)}`
|
||||
);
|
||||
}
|
||||
if (!result.taskId) {
|
||||
errs.push(
|
||||
'per-request opt-in MUST produce a CreateTaskResult with top-level taskId'
|
||||
);
|
||||
}
|
||||
if (result.task) {
|
||||
errs.push(
|
||||
'CreateTaskResult MUST be flat (no nested `task` wrapper) even on per-request opt-in path'
|
||||
);
|
||||
}
|
||||
// Best-effort cleanup: cancel the task so we don't leak a 1s
|
||||
// background goroutine on the server.
|
||||
if (result.taskId) {
|
||||
try {
|
||||
await withExt.request('tasks/cancel', { taskId: result.taskId });
|
||||
} catch {
|
||||
/* swallow — cleanup best-effort */
|
||||
}
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2575_REF, SEP_2663_REF],
|
||||
details: {
|
||||
resultType: result.resultType,
|
||||
taskId: result.taskId
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2575_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
await withExt.close().catch(() => {});
|
||||
await withoutExt.close().catch(() => {});
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
179
src/scenarios/server/tasks/composition.ts
Normal file
179
src/scenarios/server/tasks/composition.ts
Normal file
|
|
@ -0,0 +1,179 @@
|
|||
/**
|
||||
* SEP-2663 + SEP-2322 — MRTR → Tasks composition.
|
||||
*
|
||||
* Tests the composition path made normative by SEP-2663 commit 451f5e1:
|
||||
* a tool gathers input via the SEP-2322 MRTR loop on `tools/call`, then
|
||||
* the handler escalates to async on the final round and the server
|
||||
* returns a `CreateTaskResult`. The inlined result of the eventual task
|
||||
* MUST reflect the answer gathered during the MRTR phase, so an
|
||||
* implementation that wires MRTR and tasks as independent surfaces
|
||||
* fails this end-to-end check.
|
||||
*
|
||||
* The SEP-2322 ephemeral-MRTR contract (round 1 InputRequiredResult →
|
||||
* round 2 ToolResult) is covered exhaustively by the `input-required-*`
|
||||
* scenarios; only the composition with `CreateTaskResult` on the final
|
||||
* round is SEP-2663-specific, which is why this is the only check in
|
||||
* the tasks suite that drives the MRTR loop end-to-end.
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../../types';
|
||||
import type { Connection, RunContext } from '../../../connection';
|
||||
import { SEP_2322_REF, SEP_2663_REF, failureCheck } from './mrtr-helpers';
|
||||
import {
|
||||
isInputRequiredResult,
|
||||
mockElicitResponse
|
||||
} from '../input-required-result-helpers';
|
||||
import { TASKS_EXTENSION_ID, waitForTerminal } from './helpers';
|
||||
|
||||
const MRTR_INPUT_REQUIRED_RESULT_TYPE = 'input_required';
|
||||
|
||||
export class TasksMrtrCompositionScenario implements ClientScenario {
|
||||
name = 'tasks-mrtr-composition';
|
||||
readonly source = { extensionId: 'io.modelcontextprotocol/tasks' } as const;
|
||||
description = `Test SEP-2663 MRTR → Tasks composition (SEP-2663 commit 451f5e1).
|
||||
|
||||
**Server Implementation Requirements:**
|
||||
|
||||
A tool that gathers input via the SEP-2322 MRTR loop and then escalates
|
||||
to async on the final round MUST return a \`CreateTaskResult\` on that
|
||||
final round, NOT a sync \`ToolResult\`. The composition is what makes
|
||||
the two surfaces interoperate — clients should not need to choose one
|
||||
or the other up front.
|
||||
|
||||
**Spec separation that MUST stay observable:**
|
||||
|
||||
1. Round 1 (MRTR) carries \`inputRequests\` + (optionally) \`requestState\`
|
||||
and MUST NOT carry \`taskId\`.
|
||||
2. Round 2 (CreateTaskResult) carries \`taskId\` + \`status\` and MUST
|
||||
NOT carry \`requestState\` — SEP-2663 removed it from the v2 wire
|
||||
shape, so the MRTR phase's \`requestState\` does not leak into the
|
||||
task envelope and clients don't have to deduplicate across flows.
|
||||
3. The final task \`result\` (inlined on \`tasks/get\` once terminal)
|
||||
MUST reflect the answer gathered during the MRTR phase, end-to-end.
|
||||
|
||||
**Required server fixtures (\`tools/list\` MUST include all):**
|
||||
- \`test_tool_with_task\` — registered with \`taskSupport=required\`.
|
||||
Round 1: returns \`InputRequiredResult\` asking for \`user_name\`.
|
||||
Round 2: with the elicit response echoed back, escalates to async
|
||||
and returns \`CreateTaskResult\`. The task's eventual result text MUST
|
||||
contain the gathered \`user_name\` so the round-trip is observable.`;
|
||||
|
||||
async run(ctx: RunContext): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
const id = 'sep-2663-mrtr-synchronous-before-task-creation';
|
||||
const name = 'Sep2663MrtrSynchronousBeforeTaskCreation';
|
||||
const description =
|
||||
'MRTR loop gathers input then escalates to a task on the final round (SEP-2322 + SEP-2663 451f5e1).';
|
||||
const specRefs = [SEP_2322_REF, SEP_2663_REF];
|
||||
|
||||
let conn: Connection | undefined;
|
||||
try {
|
||||
// Round 2's CreateTaskResult requires the client to declare
|
||||
// io.modelcontextprotocol/tasks (SEP-2663) at session bootstrap;
|
||||
// elicitation is needed for the MRTR round.
|
||||
conn = await ctx.connect({
|
||||
capabilities: {
|
||||
elicitation: {},
|
||||
extensions: { [TASKS_EXTENSION_ID]: {} }
|
||||
}
|
||||
});
|
||||
|
||||
// Round 1: MRTR — no inputResponses. Expect InputRequiredResult.
|
||||
const r1 = (await conn.request('tools/call', {
|
||||
name: 'test_tool_with_task',
|
||||
arguments: {}
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (!isInputRequiredResult(r1)) {
|
||||
errs.push(
|
||||
`round 1 MUST be InputRequiredResult; got ${JSON.stringify(r1)}`
|
||||
);
|
||||
}
|
||||
if (r1?.resultType !== MRTR_INPUT_REQUIRED_RESULT_TYPE) {
|
||||
errs.push(
|
||||
`round 1 resultType MUST be "${MRTR_INPUT_REQUIRED_RESULT_TYPE}"; got ${JSON.stringify(r1?.resultType)}`
|
||||
);
|
||||
}
|
||||
if (r1 && 'taskId' in r1) {
|
||||
errs.push(
|
||||
'round 1 (MRTR) MUST NOT carry taskId — task is only minted on the final round'
|
||||
);
|
||||
}
|
||||
const key = Object.keys(r1?.inputRequests ?? {})[0];
|
||||
if (!key) {
|
||||
errs.push('round 1 missing inputRequests; cannot drive round 2');
|
||||
}
|
||||
|
||||
// Round 2: MRTR retry with the elicit response. The handler now
|
||||
// returns GoAsync, so the server MUST return CreateTaskResult.
|
||||
let taskId: string | undefined;
|
||||
let r2: any;
|
||||
if (key) {
|
||||
r2 = (await conn.request('tools/call', {
|
||||
name: 'test_tool_with_task',
|
||||
arguments: {},
|
||||
inputResponses: {
|
||||
[key]: mockElicitResponse({ name: 'Alice' })
|
||||
},
|
||||
...(r1.requestState !== undefined
|
||||
? { requestState: r1.requestState }
|
||||
: {})
|
||||
})) as any;
|
||||
if (r2?.resultType !== 'task') {
|
||||
errs.push(
|
||||
`round 2 MUST be CreateTaskResult (resultType:"task"); got ${JSON.stringify(r2?.resultType)}`
|
||||
);
|
||||
}
|
||||
if (!r2?.taskId) {
|
||||
errs.push('round 2 CreateTaskResult MUST carry top-level taskId');
|
||||
} else {
|
||||
taskId = r2.taskId;
|
||||
}
|
||||
if (r2 && 'requestState' in r2) {
|
||||
errs.push(
|
||||
'round 2 CreateTaskResult MUST NOT carry requestState (SEP-2663 spec separation — MRTR requestState does not leak into the task envelope)'
|
||||
);
|
||||
}
|
||||
if (r2 && 'inputRequests' in r2) {
|
||||
errs.push(
|
||||
'round 2 CreateTaskResult MUST NOT carry inputRequests (those belong on DetailedTask returned by tasks/get)'
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Round 3: poll tasks/get until terminal; assert the inlined result
|
||||
// reflects the answer gathered in the MRTR phase.
|
||||
if (taskId) {
|
||||
const terminal = await waitForTerminal(conn, taskId);
|
||||
if (terminal.status !== 'completed') {
|
||||
errs.push(
|
||||
`final task status MUST be "completed"; got ${JSON.stringify(terminal.status)}`
|
||||
);
|
||||
}
|
||||
const text = terminal?.result?.content?.[0]?.text ?? '';
|
||||
if (!/Alice/.test(text)) {
|
||||
errs.push(
|
||||
`final task result MUST reflect the user_name gathered during the MRTR phase (expected "Alice"); got ${JSON.stringify(text)}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: specRefs
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, specRefs));
|
||||
} finally {
|
||||
await conn?.close().catch(() => {});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
529
src/scenarios/server/tasks/dispatch.ts
Normal file
529
src/scenarios/server/tasks/dispatch.ts
Normal file
|
|
@ -0,0 +1,529 @@
|
|||
/**
|
||||
* SEP-2663 Tasks Extension — dispatch + envelope conformance.
|
||||
*
|
||||
* Bundles a number of small, related checks under one scenario:
|
||||
* - Removed v1 methods (tasks/result, tasks/list) reject as -32601.
|
||||
* - Server-directed task creation works without a client `task` hint
|
||||
* param; legacy v1 `task` param on tools/call is tolerated and
|
||||
* ignored on sync tools.
|
||||
* - Immediate-result shortcut: a fast operation MAY skip task creation
|
||||
* and return a sync ToolResult.
|
||||
* - SEP-2322 resultType:"complete" discriminator on every non-task
|
||||
* response (sync tools/call, tasks/get, tasks/update, tasks/cancel).
|
||||
* - Strong consistency: tasks/get immediately after CreateTaskResult
|
||||
* MUST resolve.
|
||||
* - tasks/get with an unknown taskId MUST return -32602.
|
||||
*
|
||||
* Required server fixtures:
|
||||
* - greet — sync-only
|
||||
* - slow_compute — task-supporting (seconds:0 = instant)
|
||||
* - confirm_delete — task-supporting, parks for elicitation
|
||||
* - failing_job — task-supporting, returns tool error
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../../types';
|
||||
import type { Connection, RunContext } from '../../../connection';
|
||||
import { SEP_2322_REF, SEP_2663_REF } from './mrtr-helpers';
|
||||
import { errMsg, failureCheck } from './mrtr-helpers';
|
||||
import {
|
||||
TASKS_EXTENSION_ID,
|
||||
validTasksParams,
|
||||
waitForStatus,
|
||||
waitForTerminal
|
||||
} from './helpers';
|
||||
|
||||
/**
|
||||
* Diagnostic suffix for negative-path checks. When a server returns
|
||||
* the wrong JSON-RPC error code, the most common cause (per the
|
||||
* SEP-2663 review on conformance#262) is that the server validated
|
||||
* some other dimension of the request first — routing headers,
|
||||
* `_meta`, params shape — and short-circuited before reaching the
|
||||
* code path the check is actually probing. The check fixture
|
||||
* already sends an otherwise-valid envelope via `validTasksParams`
|
||||
* + raw-session auto-headers, so a wrong code here usually means
|
||||
* either the server has a different validation order than expected
|
||||
* or the fixture envelope is still missing something this server
|
||||
* requires. Mention both possibilities so the failure is debuggable.
|
||||
*/
|
||||
const ISOLATION_HINT =
|
||||
'(if the server is otherwise compliant, verify it does not validate other dimensions — routing headers, _meta, params shape — before method dispatch)';
|
||||
|
||||
export class TasksDispatchScenario implements ClientScenario {
|
||||
name = 'tasks-dispatch-and-envelope';
|
||||
readonly source = { extensionId: 'io.modelcontextprotocol/tasks' } as const;
|
||||
description = `Test SEP-2663 dispatch / envelope rules across the tasks surface.
|
||||
|
||||
**Server Implementation Requirements:**
|
||||
|
||||
**Removed v1 methods (SEP-2663):**
|
||||
- \`tasks/result\` is removed in v2 — the result is inlined on
|
||||
\`tasks/get\`. Servers MUST reject the method with JSON-RPC \`-32601\`.
|
||||
- \`tasks/list\` is removed in v2. Servers MUST reject it with
|
||||
\`-32601\`.
|
||||
|
||||
**Server-directed task creation (SEP-2663):**
|
||||
- The client does NOT send a \`task\` hint param. The server alone
|
||||
decides whether to create a task. A \`tools/call\` against a
|
||||
task-supporting tool MUST produce \`CreateTaskResult\` even with no
|
||||
client hint.
|
||||
|
||||
**Legacy \`task\` param tolerated (SEP-2663):**
|
||||
- A v1 client may still send \`task: { ttl, pollInterval }\` on
|
||||
\`tools/call\`. The server MUST tolerate it (no error) AND MUST NOT
|
||||
promote a sync-only tool to a task on its presence. The body
|
||||
arguments + tool registration are authoritative.
|
||||
|
||||
**Immediate-result shortcut (SEP-2663):**
|
||||
- A server MAY return a sync \`ToolResult\` for task-supporting tools
|
||||
when the operation completes fast enough. Either return a
|
||||
\`CreateTaskResult\` (with \`resultType:"task"\`) or a sync
|
||||
\`ToolResult\` (with \`resultType:"complete"\`); both are valid.
|
||||
|
||||
**resultType:"complete" on non-task responses (SEP-2322):**
|
||||
- Every JSON-RPC response on the tools+tasks surface other than a
|
||||
CreateTaskResult MUST carry \`resultType:"complete"\`. This applies
|
||||
to: sync \`tools/call\`, \`tasks/get\`, \`tasks/update\` ack,
|
||||
\`tasks/cancel\` ack.
|
||||
|
||||
**Strong consistency / durable create (SEP-2663):**
|
||||
- A server MUST NOT return \`CreateTaskResult\` until the task is
|
||||
durably created — that is, until a \`tasks/get\` for the returned
|
||||
\`taskId\` would resolve. Issuing \`tasks/get\` immediately after the
|
||||
CreateTaskResult arrives MUST succeed, not -32602.
|
||||
|
||||
**Unknown taskId on tasks/get (SEP-2663):**
|
||||
- \`tasks/get\` for a taskId the server doesn't recognize MUST return
|
||||
JSON-RPC \`-32602\` (InvalidParams). Mirrors the same rule for
|
||||
\`tasks/cancel\` (clarified upstream in spec commit d963ad0).
|
||||
|
||||
**Required server fixtures (\`tools/list\` MUST include all):**
|
||||
- \`greet\` — sync-only.
|
||||
- \`slow_compute\` — task-supporting (\`seconds: 0\` for the immediate
|
||||
shortcut path).
|
||||
- \`confirm_delete\` — task-supporting, parks for elicitation.
|
||||
- \`failing_job\` — task-supporting, returns a tool execution error.`;
|
||||
|
||||
async run(ctx: RunContext): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
let conn: Connection;
|
||||
try {
|
||||
conn = await ctx.connect({
|
||||
capabilities: {
|
||||
elicitation: {},
|
||||
sampling: {},
|
||||
extensions: { [TASKS_EXTENSION_ID]: {} }
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tasks-dispatch-bootstrap',
|
||||
name: 'TasksDispatchBootstrap',
|
||||
description:
|
||||
'Initialize handshake declaring io.modelcontextprotocol/tasks extension succeeds',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed to initialize: ${errMsg(error)}`,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
return checks;
|
||||
}
|
||||
|
||||
// Check 1: tasks/result removed.
|
||||
//
|
||||
// Isolation: the request body uses a v1-shaped { taskId } via
|
||||
// validTasksParams, and the raw-session emits Mcp-Method +
|
||||
// Mcp-Name routing headers and the standard _meta envelope. The
|
||||
// ONLY thing the server should object to is the method name.
|
||||
{
|
||||
const id = 'sep-2663-tasks-result-removed-method-not-found';
|
||||
const name = 'TasksRemovedTasksResult';
|
||||
const description =
|
||||
'tasks/result is removed in v2 and MUST reject with -32601';
|
||||
try {
|
||||
await conn.request('tasks/result', validTasksParams('tasks/result'));
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'tasks/result returned a result instead of -32601',
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} catch (e: any) {
|
||||
const errs: string[] = [];
|
||||
if (e.code !== -32601) {
|
||||
errs.push(
|
||||
`expected -32601; got ${e.code ?? '<missing>'} ${ISOLATION_HINT}`
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Check 2: tasks/list removed.
|
||||
//
|
||||
// Isolation: v1 tasks/list took no required params; the empty
|
||||
// body is fully valid. Routing headers + _meta come from raw-
|
||||
// conn. Only the method name itself is wrong.
|
||||
{
|
||||
const id = 'tasks-removed-tasks-list';
|
||||
const name = 'TasksRemovedTasksList';
|
||||
const description =
|
||||
'tasks/list is removed in v2 and MUST reject with -32601';
|
||||
try {
|
||||
await conn.request('tasks/list', validTasksParams('tasks/list'));
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'tasks/list returned a result instead of -32601',
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} catch (e: any) {
|
||||
const errs: string[] = [];
|
||||
if (e.code !== -32601) {
|
||||
errs.push(
|
||||
`expected -32601; got ${e.code ?? '<missing>'} ${ISOLATION_HINT}`
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Check 3: server-directed task creation without client hint.
|
||||
{
|
||||
const id = 'tasks-server-directed-creation-no-hint';
|
||||
const name = 'TasksServerDirectedCreationNoHint';
|
||||
const description =
|
||||
'tools/call with no client `task` hint param MUST still produce CreateTaskResult for task-supporting tools';
|
||||
try {
|
||||
const result = (await conn.request('tools/call', {
|
||||
name: 'failing_job',
|
||||
arguments: {}
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (result.resultType !== 'task' || !result.taskId) {
|
||||
errs.push(
|
||||
`expected CreateTaskResult; got resultType=${JSON.stringify(result.resultType)}, taskId=${JSON.stringify(result.taskId)}`
|
||||
);
|
||||
}
|
||||
// Best-effort wait so we don't leak.
|
||||
if (result.taskId) {
|
||||
try {
|
||||
await waitForTerminal(conn, result.taskId);
|
||||
} catch {
|
||||
/* swallow */
|
||||
}
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 4: legacy `task` param tolerated + ignored on sync tool.
|
||||
{
|
||||
const id = 'sep-2663-legacy-task-param-ignored';
|
||||
const name = 'TasksLegacyTaskParamIgnored';
|
||||
const description =
|
||||
'tools/call with legacy `task` param against a sync tool MUST NOT error and MUST NOT be promoted to a task';
|
||||
try {
|
||||
const result = (await conn.request('tools/call', {
|
||||
name: 'greet',
|
||||
arguments: { name: 'legacy-hint' },
|
||||
// Legacy v1 hint that the server MUST ignore.
|
||||
task: { ttl: 60_000, pollInterval: 100 }
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (result.resultType === 'task') {
|
||||
errs.push(
|
||||
'legacy `task` param MUST NOT promote a sync tool to a task'
|
||||
);
|
||||
}
|
||||
if (result.taskId) {
|
||||
errs.push(
|
||||
`sync tool with legacy hint MUST NOT carry top-level taskId; got ${result.taskId}`
|
||||
);
|
||||
}
|
||||
if (!Array.isArray(result.content) || result.content.length === 0) {
|
||||
errs.push('sync tool MUST still return content[]');
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 5: immediate-result shortcut. Either CreateTaskResult OR
|
||||
// sync ToolResult is acceptable for an instant operation.
|
||||
{
|
||||
const id = 'tasks-immediate-result-shortcut';
|
||||
const name = 'TasksImmediateResultShortcut';
|
||||
const description =
|
||||
'For a fast operation, a task-supporting tool MAY skip task creation and return a sync ToolResult; either path is valid';
|
||||
try {
|
||||
const result = (await conn.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 0, label: 'instant' }
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (result.resultType === 'task') {
|
||||
if (!result.taskId) {
|
||||
errs.push(
|
||||
'task-path response MUST carry top-level taskId on CreateTaskResult'
|
||||
);
|
||||
}
|
||||
} else {
|
||||
// Sync path
|
||||
if (!Array.isArray(result.content)) {
|
||||
errs.push(
|
||||
'sync-path response MUST carry content[] for the immediate ToolResult'
|
||||
);
|
||||
}
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF],
|
||||
details: { resultType: result.resultType }
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 6: resultType:"complete" on every non-task response.
|
||||
{
|
||||
const id = 'tasks-result-type-complete-on-non-task-responses';
|
||||
const name = 'TasksResultTypeCompleteOnNonTaskResponses';
|
||||
const description =
|
||||
'Sync tools/call, tasks/get, tasks/update ack, and tasks/cancel ack MUST all carry resultType:"complete"';
|
||||
const errs: string[] = [];
|
||||
try {
|
||||
// Sync tools/call.
|
||||
const sync = (await conn.request('tools/call', {
|
||||
name: 'greet',
|
||||
arguments: { name: 'rt' }
|
||||
})) as any;
|
||||
if (sync.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`sync tools/call resultType = ${JSON.stringify(sync.resultType)}, want "complete"`
|
||||
);
|
||||
}
|
||||
|
||||
// tasks/get against a fresh task.
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 0, label: 'rt-get' }
|
||||
})) as any;
|
||||
const taskIdForGet = created.taskId;
|
||||
if (taskIdForGet) {
|
||||
await waitForTerminal(conn, taskIdForGet);
|
||||
const got = (await conn.request('tasks/get', {
|
||||
taskId: taskIdForGet
|
||||
})) as any;
|
||||
if (got.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`tasks/get resultType = ${JSON.stringify(got.resultType)}, want "complete"`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// tasks/cancel ack on a fresh long-running task.
|
||||
const longLived = (await conn.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 60, label: 'rt-cancel' }
|
||||
})) as any;
|
||||
if (longLived.taskId) {
|
||||
const cancelAck = (await conn.request('tasks/cancel', {
|
||||
taskId: longLived.taskId
|
||||
})) as any;
|
||||
if (cancelAck.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`tasks/cancel ack resultType = ${JSON.stringify(cancelAck.resultType)}, want "complete"`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// tasks/update ack on a parked elicitation task.
|
||||
const elicit = (await conn.request('tools/call', {
|
||||
name: 'confirm_delete',
|
||||
arguments: { filename: 'rt.txt' }
|
||||
})) as any;
|
||||
const elicitTaskId = elicit.taskId;
|
||||
if (elicitTaskId) {
|
||||
await waitForStatus(conn, elicitTaskId, 'input_required', 5_000);
|
||||
const updateAck = (await conn.request('tasks/update', {
|
||||
taskId: elicitTaskId,
|
||||
inputResponses: { 'unknown-key': { ignored: true } }
|
||||
})) as any;
|
||||
if (updateAck.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`tasks/update ack resultType = ${JSON.stringify(updateAck.resultType)}, want "complete"`
|
||||
);
|
||||
}
|
||||
try {
|
||||
await conn.request('tasks/cancel', { taskId: elicitTaskId });
|
||||
} catch {
|
||||
/* swallow */
|
||||
}
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2322_REF, SEP_2663_REF]
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2322_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 7: strong consistency — immediate tasks/get after CreateTaskResult.
|
||||
{
|
||||
const id = 'sep-2663-durable-create-strong-consistency';
|
||||
const name = 'TasksStrongConsistencyImmediateGet';
|
||||
const description =
|
||||
'tasks/get issued immediately after CreateTaskResult arrives MUST resolve (server MUST NOT return CreateTaskResult before the task is durably created)';
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 60, label: 'consistency' }
|
||||
})) as any;
|
||||
const taskId = created.taskId;
|
||||
if (!taskId) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'slow_compute did not create a task',
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} else {
|
||||
// No await/sleep between create and get — codifies the
|
||||
// strong-consistency ordering.
|
||||
const got = (await conn.request('tasks/get', { taskId })) as any;
|
||||
const errs: string[] = [];
|
||||
if (got.taskId !== taskId) {
|
||||
errs.push(
|
||||
`immediate tasks/get MUST resolve the same taskId; got ${got.taskId}`
|
||||
);
|
||||
}
|
||||
// Cleanup.
|
||||
try {
|
||||
await conn.request('tasks/cancel', { taskId });
|
||||
} catch {
|
||||
/* swallow */
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 8: tasks/get with unknown taskId returns -32602.
|
||||
//
|
||||
// Isolation: tasks/get is a supported method; the taskId is the
|
||||
// single property under test. The unknown taskId appears in both
|
||||
// body and Mcp-Name header (raw-session mirrors it) so a server
|
||||
// that validates routing headers first will route to tasks/get
|
||||
// and reject for "unknown taskId" specifically — not for header
|
||||
// mismatch.
|
||||
{
|
||||
const id = 'sep-2663-tasks-get-invalid-task-id-32602';
|
||||
const name = 'TasksGetUnknownTaskIdRejected';
|
||||
const description =
|
||||
'tasks/get for a taskId the server does not recognize MUST return -32602';
|
||||
try {
|
||||
await conn.request(
|
||||
'tasks/get',
|
||||
validTasksParams('tasks/get', {
|
||||
taskId: 'tasks-conformance-nonexistent-12345'
|
||||
})
|
||||
);
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'tasks/get with unknown taskId returned a result',
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} catch (e: any) {
|
||||
const errs: string[] = [];
|
||||
if (e.code !== -32602) {
|
||||
errs.push(
|
||||
`expected -32602; got ${e.code ?? '<missing>'} ${ISOLATION_HINT}`
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
await conn.close().catch(() => {});
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
316
src/scenarios/server/tasks/headers.ts
Normal file
316
src/scenarios/server/tasks/headers.ts
Normal file
|
|
@ -0,0 +1,316 @@
|
|||
/**
|
||||
* SEP-2243 Mcp-Method / Mcp-Name request-header validation, tasks
|
||||
* surface.
|
||||
*
|
||||
* SEP-2243 defines Mcp-Method and Mcp-Name as REQUEST headers (client →
|
||||
* server) used by HTTP infrastructure (proxies, gateways, observability)
|
||||
* to route or shape JSON-RPC traffic without parsing the body. The
|
||||
* server MUST reject requests where the routing headers disagree with
|
||||
* (or are missing for a name-carrying body) the JSON-RPC envelope, with
|
||||
* HTTP 400 + JSON-RPC `-32001 HeaderMismatch`.
|
||||
*
|
||||
* This scenario exercises the validation on the tasks surface
|
||||
* specifically — the upstream `http-header-validation` scenario covers
|
||||
* the general case; here we verify mcpkit's tasks/* methods route
|
||||
* through the same validator (matched headers → success; mismatched
|
||||
* header → -32001).
|
||||
*
|
||||
* Required server fixtures:
|
||||
* - greet — sync-only, returns "Hello, {name}!"
|
||||
* - slow_compute — task-supporting, sleeps N seconds
|
||||
*/
|
||||
|
||||
import { McpError } from '@modelcontextprotocol/sdk/types.js';
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../../types';
|
||||
import type { Connection, RunContext } from '../../../connection';
|
||||
import { SEP_2243_REF, SEP_2663_REF } from './mrtr-helpers';
|
||||
import { errMsg, failureCheck } from './mrtr-helpers';
|
||||
import { TASKS_EXTENSION_ID } from './helpers';
|
||||
|
||||
const HEADER_MISMATCH_ERROR_CODE = -32001;
|
||||
|
||||
export class TasksRequestHeadersScenario implements ClientScenario {
|
||||
name = 'tasks-request-headers';
|
||||
readonly source = { extensionId: 'io.modelcontextprotocol/tasks' } as const;
|
||||
description = `Test SEP-2243 Mcp-Method / Mcp-Name request-header validation, tasks surface.
|
||||
|
||||
**Server Implementation Requirements:**
|
||||
|
||||
SEP-2243 defines two required request headers that mirror body fields
|
||||
into the HTTP layer for routing intermediaries:
|
||||
|
||||
- \`Mcp-Method: <jsonrpc-method>\` — REQUIRED on every JSON-RPC request,
|
||||
matching the body \`method\`.
|
||||
- \`Mcp-Name: <name-shaped-identifier>\` — REQUIRED on requests with a
|
||||
name-shaped body field. Per SEP-2243 §"Standard Headers" this covers
|
||||
\`tools/call\` (\`params.name\`), \`resources/read\` (\`params.uri\`), and
|
||||
\`prompts/get\` (\`params.name\`). SEP-2663 §"Streamable HTTP: Routing
|
||||
Headers" extends the requirement to tasks-namespace methods:
|
||||
\`tasks/get\`, \`tasks/update\`, and \`tasks/cancel\` MUST carry
|
||||
\`Mcp-Name: <taskId>\` matching \`params.taskId\`.
|
||||
|
||||
Per SEP-2243 §"Server Behavior", servers that process the request body
|
||||
MUST validate that header values match the body. Per its "Validation
|
||||
Failure Conditions", both missing required headers and mismatched
|
||||
values trigger rejection with JSON-RPC error code \`-32001\`
|
||||
(HeaderMismatch) and HTTP 400.
|
||||
|
||||
**Required server fixtures (\`tools/list\` MUST include all):**
|
||||
- \`greet\` — sync-only, returns \`Hello, {name}!\`.
|
||||
- \`slow_compute\` — task-supporting, \`seconds\`-second sleep then a
|
||||
result.`;
|
||||
|
||||
async run(ctx: RunContext): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
let conn: Connection;
|
||||
try {
|
||||
conn = await ctx.connect({
|
||||
capabilities: { extensions: { [TASKS_EXTENSION_ID]: {} } }
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tasks-headers-bootstrap',
|
||||
name: 'TasksHeadersBootstrap',
|
||||
description:
|
||||
'Initialize handshake declaring io.modelcontextprotocol/tasks extension succeeds',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed to initialize: ${errMsg(error)}`,
|
||||
specReferences: [SEP_2243_REF]
|
||||
});
|
||||
return checks;
|
||||
}
|
||||
|
||||
// Check 1: Mcp-Method on tools/call against a sync tool.
|
||||
{
|
||||
const id = 'tasks-headers-tolerate-mcp-method-on-tools-call';
|
||||
const name = 'TasksHeadersTolerateMcpMethodOnToolsCall';
|
||||
const description =
|
||||
'Server tolerates Mcp-Method request header on tools/call (sync tool dispatch unaffected)';
|
||||
try {
|
||||
const result = (await conn.request(
|
||||
'tools/call',
|
||||
{ name: 'greet', arguments: { name: 'sep-2243' } },
|
||||
{ 'Mcp-Method': 'tools/call' }
|
||||
)) as any;
|
||||
const errs: string[] = [];
|
||||
if (result.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`sync ToolResult.resultType MUST be "complete" regardless of routing header; got ${JSON.stringify(result.resultType)}`
|
||||
);
|
||||
}
|
||||
if (
|
||||
!Array.isArray(result.content) ||
|
||||
result.content.length === 0 ||
|
||||
result.content[0]?.text !== 'Hello, sep-2243!'
|
||||
) {
|
||||
errs.push(
|
||||
'tool result content MUST be unaffected by the Mcp-Method header'
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2243_REF]
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2243_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 2: Mcp-Method + Mcp-Name on tasks/get (drive a task first
|
||||
// so we have a real taskId to route on). With matched routing
|
||||
// headers the server MUST dispatch normally; this is the positive
|
||||
// case complementing the negative-path mismatch checks below.
|
||||
let routingTaskId: string | undefined;
|
||||
{
|
||||
const id = 'sep-2663-routing-headers-accepted-on-tasks-get';
|
||||
const name = 'Sep2663RoutingHeadersAcceptedOnTasksGet';
|
||||
const description =
|
||||
'Server accepts matched Mcp-Method + Mcp-Name request headers on tasks/get and dispatches normally';
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 60, label: 'headers-tasks-get' }
|
||||
})) as any;
|
||||
routingTaskId = created.taskId;
|
||||
if (!routingTaskId) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'slow_compute did not create a task',
|
||||
specReferences: [SEP_2243_REF]
|
||||
});
|
||||
} else {
|
||||
const got = (await conn.request(
|
||||
'tasks/get',
|
||||
{ taskId: routingTaskId },
|
||||
{
|
||||
'Mcp-Method': 'tasks/get',
|
||||
'Mcp-Name': routingTaskId
|
||||
}
|
||||
)) as any;
|
||||
const errs: string[] = [];
|
||||
if (got.taskId !== routingTaskId) {
|
||||
errs.push(
|
||||
`tasks/get MUST resolve body taskId regardless of routing headers; got ${got.taskId}`
|
||||
);
|
||||
}
|
||||
if (!got.status) {
|
||||
errs.push(
|
||||
'tasks/get MUST still return status when routing headers are set'
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2243_REF]
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2243_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 3: Mismatched Mcp-Method header is rejected on the tasks
|
||||
// surface (SEP-2243 §Server Validation, tasks dispatch path).
|
||||
{
|
||||
const id = 'tasks-headers-reject-mismatched-method';
|
||||
const name = 'TasksHeadersRejectMismatchedMethod';
|
||||
const description =
|
||||
'When Mcp-Method header disagrees with body on a tools/call, server MUST reject with -32001 HeaderMismatch (SEP-2243 §Server Validation)';
|
||||
try {
|
||||
await conn.request(
|
||||
'tools/call',
|
||||
{ name: 'greet', arguments: { name: 'header-mismatch' } },
|
||||
{ 'Mcp-Method': 'tasks/get' }
|
||||
);
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage:
|
||||
'tools/call with Mcp-Method: tasks/get returned a successful response; SEP-2243 requires -32001 rejection',
|
||||
specReferences: [SEP_2243_REF]
|
||||
});
|
||||
} catch (error) {
|
||||
const code =
|
||||
error instanceof McpError ? error.code : (error as any)?.code;
|
||||
if (code === HEADER_MISMATCH_ERROR_CODE) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'SUCCESS',
|
||||
timestamp: new Date().toISOString(),
|
||||
specReferences: [SEP_2243_REF]
|
||||
});
|
||||
} else {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `expected -32001 HeaderMismatch; got ${code ?? errMsg(error)}`,
|
||||
specReferences: [SEP_2243_REF]
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check 4: Mismatched Mcp-Name header on tasks/get is rejected.
|
||||
// Per SEP-2663 §"Streamable HTTP: Routing Headers" the client MUST
|
||||
// set Mcp-Name to params.taskId on tasks/get / tasks/update /
|
||||
// tasks/cancel; per SEP-2243 §"Server Behavior" the server MUST
|
||||
// reject any header that disagrees with the body. tasks/get is the
|
||||
// representative probe — the same rule applies to update/cancel by
|
||||
// extension. Confirmed by SEP-2663 author on conformance#262.
|
||||
{
|
||||
const id = 'sep-2663-server-rejects-mismatched-mcp-name-on-tasks-get';
|
||||
const name = 'Sep2663ServerRejectsMismatchedMcpNameOnTasksGet';
|
||||
const description =
|
||||
'When Mcp-Name header disagrees with params.taskId on tasks/get, server MUST reject with -32001 HeaderMismatch';
|
||||
if (!routingTaskId) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage:
|
||||
'routing-task fixture from Check 2 unavailable; cannot drive negative-path probe',
|
||||
specReferences: [SEP_2243_REF, SEP_2663_REF]
|
||||
});
|
||||
} else {
|
||||
try {
|
||||
await conn.request(
|
||||
'tasks/get',
|
||||
{ taskId: routingTaskId },
|
||||
{ 'Mcp-Name': 'sep-2663-wrong-task-id-for-routing-header-mismatch' }
|
||||
);
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage:
|
||||
'tasks/get with mismatched Mcp-Name returned a result; SEP-2243 requires -32001 rejection',
|
||||
specReferences: [SEP_2243_REF, SEP_2663_REF]
|
||||
});
|
||||
} catch (error) {
|
||||
const code =
|
||||
error instanceof McpError ? error.code : (error as any)?.code;
|
||||
if (code === HEADER_MISMATCH_ERROR_CODE) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'SUCCESS',
|
||||
timestamp: new Date().toISOString(),
|
||||
specReferences: [SEP_2243_REF, SEP_2663_REF]
|
||||
});
|
||||
} else {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `expected -32001 HeaderMismatch; got ${code ?? errMsg(error)}`,
|
||||
specReferences: [SEP_2243_REF, SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Cleanup the long-lived task.
|
||||
if (routingTaskId) {
|
||||
try {
|
||||
await conn.request('tasks/cancel', { taskId: routingTaskId });
|
||||
} catch {
|
||||
/* swallow */
|
||||
}
|
||||
}
|
||||
|
||||
await conn.close().catch(() => {});
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
110
src/scenarios/server/tasks/helpers.ts
Normal file
110
src/scenarios/server/tasks/helpers.ts
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
/**
|
||||
* Tasks-specific helpers for SEP-2663 server-conformance scenarios.
|
||||
*
|
||||
* - `TASKS_EXTENSION_ID`: the extension identifier scenarios assert
|
||||
* when checking server capabilities.
|
||||
* - `validTasksParams`: well-formed body shapes for negative-path
|
||||
* checks that need to isolate a single failure dimension.
|
||||
* - `waitForTerminal` / `waitForStatus`: tasks/get polling loops.
|
||||
*/
|
||||
|
||||
import type { Connection } from '../../../connection';
|
||||
|
||||
export const TASKS_EXTENSION_ID = 'io.modelcontextprotocol/tasks';
|
||||
|
||||
/**
|
||||
* Baseline "otherwise well-formed" params for a tasks-namespace
|
||||
* method, used by negative-path checks that need to isolate a single
|
||||
* failure dimension.
|
||||
*
|
||||
* When a check asserts a specific JSON-RPC error code (e.g. -32601
|
||||
* for a removed method, -32602 for an unknown taskId), the rest of
|
||||
* the request MUST be valid — otherwise a spec-compliant server can
|
||||
* reject for a different reason (missing routing headers, missing
|
||||
* `_meta`, malformed params) and the assertion fires on the wrong
|
||||
* dimension, producing misleading conformance results.
|
||||
*
|
||||
* The Connection already auto-populates SEP-2243 routing headers
|
||||
* and the SEP-2322 `_meta` envelope based on the method and params,
|
||||
* so this helper's job is to supply a sensible *body* — a params
|
||||
* object whose shape would be acceptable on the v1 surface — and
|
||||
* let the caller mutate exactly the field under test.
|
||||
*
|
||||
* Example:
|
||||
*
|
||||
* // tasks/result is removed in v2; everything else is valid, so
|
||||
* // -32601 is the only error that can fire on a compliant server.
|
||||
* await conn.request('tasks/result', validTasksParams('tasks/result'));
|
||||
*
|
||||
* // tasks/get with an unknown taskId — the taskId is the property
|
||||
* // under test; the rest of the envelope is valid.
|
||||
* await conn.request(
|
||||
* 'tasks/get',
|
||||
* validTasksParams('tasks/get', { taskId: 'nonexistent-12345' })
|
||||
* );
|
||||
*/
|
||||
export function validTasksParams(
|
||||
method: string,
|
||||
overrides?: Record<string, unknown>
|
||||
): Record<string, unknown> {
|
||||
const probeTaskId = 'tasks-conformance-isolation-probe';
|
||||
let base: Record<string, unknown>;
|
||||
switch (method) {
|
||||
case 'tasks/result': // v1 shape: { taskId }
|
||||
case 'tasks/get':
|
||||
case 'tasks/cancel':
|
||||
base = { taskId: probeTaskId };
|
||||
break;
|
||||
case 'tasks/update':
|
||||
base = { taskId: probeTaskId, inputResponses: {} };
|
||||
break;
|
||||
case 'tasks/list': // v1 shape: optional { cursor? }
|
||||
base = {};
|
||||
break;
|
||||
default:
|
||||
base = {};
|
||||
}
|
||||
return { ...base, ...(overrides ?? {}) };
|
||||
}
|
||||
|
||||
/** Poll tasks/get until the task reaches a terminal state. */
|
||||
export async function waitForTerminal(
|
||||
conn: Connection,
|
||||
taskId: string,
|
||||
timeoutMs = 10_000
|
||||
): Promise<any> {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
const task = (await conn.request('tasks/get', { taskId })) as any;
|
||||
if (['completed', 'failed', 'cancelled'].includes(task.status)) {
|
||||
return task;
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 200));
|
||||
}
|
||||
throw new Error(
|
||||
`Task ${taskId} did not reach terminal state within ${timeoutMs}ms`
|
||||
);
|
||||
}
|
||||
|
||||
/** Poll tasks/get until a specific status (or any terminal state). */
|
||||
export async function waitForStatus(
|
||||
conn: Connection,
|
||||
taskId: string,
|
||||
status: string,
|
||||
timeoutMs = 10_000
|
||||
): Promise<any> {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
const task = (await conn.request('tasks/get', { taskId })) as any;
|
||||
if (
|
||||
task.status === status ||
|
||||
['completed', 'failed', 'cancelled'].includes(task.status)
|
||||
) {
|
||||
return task;
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 200));
|
||||
}
|
||||
throw new Error(
|
||||
`Task ${taskId} did not reach status ${status} within ${timeoutMs}ms`
|
||||
);
|
||||
}
|
||||
534
src/scenarios/server/tasks/lifecycle.ts
Normal file
534
src/scenarios/server/tasks/lifecycle.ts
Normal file
|
|
@ -0,0 +1,534 @@
|
|||
/**
|
||||
* SEP-2663 Tasks Extension — server lifecycle conformance.
|
||||
*
|
||||
* Tests a server that implements the io.modelcontextprotocol/tasks
|
||||
* extension end-to-end: sync vs async dispatch, DetailedTask shape on
|
||||
* tasks/get, tool errors vs protocol errors, and cancellation
|
||||
* semantics.
|
||||
*
|
||||
* Required server fixtures (tools/list output must include all):
|
||||
* - greet — sync-only, returns "Hello, {name}!"
|
||||
* - slow_compute — task-supporting, sleeps N seconds
|
||||
* - failing_job — task-supporting, returns a tool error
|
||||
* - protocol_error_job — task-supporting, panics into a protocol error
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../../types';
|
||||
import type { Connection, RunContext } from '../../../connection';
|
||||
import { SEP_2322_REF, SEP_2663_REF } from './mrtr-helpers';
|
||||
import { errMsg, failureCheck, skipCheck } from './mrtr-helpers';
|
||||
import { TASKS_EXTENSION_ID, waitForTerminal } from './helpers';
|
||||
import { isIso8601 } from './mrtr-helpers';
|
||||
|
||||
export class TasksLifecycleScenario implements ClientScenario {
|
||||
name = 'tasks-lifecycle';
|
||||
readonly source = { extensionId: 'io.modelcontextprotocol/tasks' } as const;
|
||||
description = `Test SEP-2663 Tasks extension lifecycle on the server.
|
||||
|
||||
**Server Implementation Requirements (SEP-2663):**
|
||||
|
||||
The server MUST advertise \`io.modelcontextprotocol/tasks\` under
|
||||
\`capabilities.extensions\` and gate the task surface on negotiation.
|
||||
|
||||
**Sync dispatch (no task created):**
|
||||
- A \`tools/call\` against a sync-only tool MUST return a flat
|
||||
\`ToolResult\` with \`resultType:"complete"\` and a \`content[]\` array.
|
||||
- It MUST NOT carry \`taskId\` at the top level (that would imply a
|
||||
CreateTaskResult).
|
||||
|
||||
**Server-directed task creation:**
|
||||
- For task-supporting tools, the server decides whether to create a task —
|
||||
the client MUST NOT need to opt in via a request param.
|
||||
- The response MUST be a \`CreateTaskResult\` — a flat \`Result & Task\`
|
||||
intersection: \`resultType:"task"\`, plus \`taskId\` / \`status\` /
|
||||
\`createdAt\` / \`lastUpdatedAt\` / \`ttlMs\` at the top level.
|
||||
There MUST NOT be a nested \`task\` wrapper key.
|
||||
|
||||
**tasks/get DetailedTask:**
|
||||
- Working tasks return \`status\` and basic metadata; result/error are
|
||||
absent.
|
||||
- Completed tasks MUST inline the original tool result under \`result\`
|
||||
with \`content[]\`. There is no separate \`tasks/result\` method.
|
||||
|
||||
**Tool errors vs protocol errors (SEP-2663 §error-semantics):**
|
||||
- A tool that ran but reported an error MUST surface as
|
||||
\`status:"completed"\` with \`result.isError:true\`. The status
|
||||
\`"failed"\` is reserved for protocol-level errors.
|
||||
- A protocol-level error (server crash, internal failure) MUST surface
|
||||
as \`status:"failed"\` with an inlined \`error\` object (JSON-RPC
|
||||
error shape: code/message/data) and MUST NOT carry \`result\`.
|
||||
|
||||
**Cancellation:**
|
||||
- \`tasks/cancel\` MUST return an empty
|
||||
\`{resultType:"complete"}\` ack — no task envelope (SEP-2322
|
||||
discriminator). The cancelled status is observed via the next
|
||||
\`tasks/get\`.
|
||||
- \`tasks/cancel\` against a terminal task returns the same empty ack
|
||||
(idempotent) — the spec reserves \`-32602\` for unknown taskIds only.
|
||||
|
||||
**Required server fixtures (\`tools/list\` MUST include all):**
|
||||
- \`greet\` — sync-only, returns \`Hello, {name}!\`.
|
||||
- \`slow_compute\` — task-supporting, \`seconds\`-second sleep then a
|
||||
result. \`seconds: 0\` for the immediate path. MUST settle to
|
||||
\`cancelled\` (not \`completed\`/\`failed\`) when \`tasks/cancel\`
|
||||
arrives while running, so the lifecycle cancel check has a
|
||||
deterministic terminal status.
|
||||
- \`failing_job\` — task-supporting, always returns a tool execution
|
||||
error after ~1s.
|
||||
- \`protocol_error_job\` — task-supporting, panics into a protocol
|
||||
error.`;
|
||||
|
||||
async run(ctx: RunContext): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
let conn: Connection;
|
||||
try {
|
||||
conn = await ctx.connect({
|
||||
capabilities: {
|
||||
elicitation: {},
|
||||
sampling: {},
|
||||
extensions: { [TASKS_EXTENSION_ID]: {} }
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tasks-lifecycle-bootstrap',
|
||||
name: 'TasksLifecycleBootstrap',
|
||||
description:
|
||||
'Initialize handshake declaring io.modelcontextprotocol/tasks extension succeeds',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed to initialize: ${errMsg(error)}`,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
return checks;
|
||||
}
|
||||
|
||||
// Check 1: sync tool call returns ToolResult, no task creation.
|
||||
{
|
||||
const id = 'tasks-sync-tool-call';
|
||||
const name = 'TasksSyncToolCall';
|
||||
const description =
|
||||
'Sync tool returns ToolResult (resultType:"complete"), no taskId at top level';
|
||||
try {
|
||||
const result = (await conn.request('tools/call', {
|
||||
name: 'greet',
|
||||
arguments: { name: 'World' }
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (result.resultType === 'task') {
|
||||
errs.push('sync tool result MUST NOT carry resultType:"task"');
|
||||
}
|
||||
if (result.taskId) {
|
||||
errs.push(
|
||||
`sync tool result MUST NOT carry top-level taskId; got ${result.taskId}`
|
||||
);
|
||||
}
|
||||
if (!Array.isArray(result.content) || result.content.length === 0) {
|
||||
errs.push('sync tool result MUST carry a non-empty content[] array');
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF, SEP_2322_REF],
|
||||
details: {
|
||||
resultType: result.resultType,
|
||||
hasTaskId: Boolean(result.taskId),
|
||||
contentLength: result.content?.length
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 2: server-directed task creation produces flat CreateTaskResult.
|
||||
let workingTaskId: string | undefined;
|
||||
{
|
||||
const id = 'sep-2663-result-type-task-on-create';
|
||||
const name = 'TasksServerTaskCreation';
|
||||
const description =
|
||||
'Task-supporting tool returns flat CreateTaskResult (no nested `task` wrapper)';
|
||||
try {
|
||||
const result = (await conn.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 2, label: 'lifecycle-create' }
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (result.resultType !== 'task') {
|
||||
errs.push(
|
||||
`expected resultType:"task"; got ${JSON.stringify(result.resultType)}`
|
||||
);
|
||||
}
|
||||
if (result.task) {
|
||||
errs.push(
|
||||
'CreateTaskResult MUST be flat (Result & Task); there must be no nested `task` wrapper key'
|
||||
);
|
||||
}
|
||||
if (!result.taskId) {
|
||||
errs.push('CreateTaskResult MUST carry top-level taskId');
|
||||
}
|
||||
if (!result.status) {
|
||||
errs.push('CreateTaskResult MUST carry top-level status');
|
||||
}
|
||||
if ('result' in result) {
|
||||
errs.push(
|
||||
'CreateTaskResult MUST NOT carry `result` (lives on tasks/get DetailedTask)'
|
||||
);
|
||||
}
|
||||
if ('error' in result) {
|
||||
errs.push(
|
||||
'CreateTaskResult MUST NOT carry `error` (lives on tasks/get DetailedTask)'
|
||||
);
|
||||
}
|
||||
if ('inputRequests' in result) {
|
||||
errs.push(
|
||||
'CreateTaskResult MUST NOT carry `inputRequests` (lives on tasks/get DetailedTask)'
|
||||
);
|
||||
}
|
||||
// Timestamps — both keys present, both ISO-8601 formatted. Per
|
||||
// SEP-2663 these are required on every TaskInfoV2. See
|
||||
// `mrtr-helpers.ts` for the regex rationale.
|
||||
if (!isIso8601(result.createdAt)) {
|
||||
errs.push(
|
||||
`createdAt MUST be an ISO-8601 string; got ${JSON.stringify(result.createdAt)}`
|
||||
);
|
||||
}
|
||||
if (!isIso8601(result.lastUpdatedAt)) {
|
||||
errs.push(
|
||||
`lastUpdatedAt MUST be an ISO-8601 string; got ${JSON.stringify(result.lastUpdatedAt)}`
|
||||
);
|
||||
}
|
||||
if (result.taskId) workingTaskId = result.taskId;
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF],
|
||||
details: {
|
||||
resultType: result.resultType,
|
||||
taskId: result.taskId,
|
||||
status: result.status
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 3: tasks/get during working state returns status + metadata.
|
||||
{
|
||||
const id = 'sep-2663-tasks-get-status-working';
|
||||
const name = 'TasksGetDuringWorking';
|
||||
const description =
|
||||
'tasks/get returns status + metadata for an active task';
|
||||
if (!workingTaskId) {
|
||||
checks.push(
|
||||
skipCheck(id, name, description, 'no task created', [SEP_2663_REF])
|
||||
);
|
||||
} else {
|
||||
try {
|
||||
const task = (await conn.request('tasks/get', {
|
||||
taskId: workingTaskId
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (task.taskId !== workingTaskId) {
|
||||
errs.push(
|
||||
`taskId mismatch: expected ${workingTaskId}, got ${task.taskId}`
|
||||
);
|
||||
}
|
||||
if (!task.status) errs.push('tasks/get response MUST carry status');
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF],
|
||||
details: { status: task.status }
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(
|
||||
failureCheck(id, name, description, error, [SEP_2663_REF])
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check 4: terminal tasks/get inlines result with content[].
|
||||
{
|
||||
const id = 'sep-2663-tasks-get-status-completed';
|
||||
const name = 'TasksGetTerminalInlinedResult';
|
||||
const description =
|
||||
'Completed task tasks/get inlines result with content[] (no separate tasks/result method)';
|
||||
if (!workingTaskId) {
|
||||
checks.push(
|
||||
skipCheck(id, name, description, 'no task created', [SEP_2663_REF])
|
||||
);
|
||||
} else {
|
||||
try {
|
||||
const terminal = await waitForTerminal(conn, workingTaskId);
|
||||
const errs: string[] = [];
|
||||
if (terminal.status !== 'completed') {
|
||||
errs.push(
|
||||
`expected status:"completed"; got ${JSON.stringify(terminal.status)}`
|
||||
);
|
||||
}
|
||||
if (!terminal.result) {
|
||||
errs.push('completed task MUST inline `result`');
|
||||
} else if (
|
||||
!Array.isArray(terminal.result.content) ||
|
||||
terminal.result.content.length === 0
|
||||
) {
|
||||
errs.push(
|
||||
'completed task `result.content[]` MUST be a non-empty array'
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF],
|
||||
details: {
|
||||
status: terminal.status,
|
||||
hasResult: Boolean(terminal.result),
|
||||
contentLength: terminal.result?.content?.length
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(
|
||||
failureCheck(id, name, description, error, [SEP_2663_REF])
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check 5: tool execution error → completed with isError:true.
|
||||
{
|
||||
const id = 'sep-2663-tool-error-uses-completed-status';
|
||||
const name = 'TasksToolErrorCompletedIsError';
|
||||
const description =
|
||||
'Tool execution error reports as completed + result.isError (NOT failed)';
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'failing_job',
|
||||
arguments: {}
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (!created.taskId) {
|
||||
errs.push('failing_job MUST create a task');
|
||||
} else {
|
||||
const terminal = await waitForTerminal(conn, created.taskId);
|
||||
if (terminal.status !== 'completed') {
|
||||
errs.push(
|
||||
`tool error MUST surface as completed (not "${terminal.status}")`
|
||||
);
|
||||
}
|
||||
if (!terminal.result) {
|
||||
errs.push('completed task with tool error MUST carry `result`');
|
||||
} else if (terminal.result.isError !== true) {
|
||||
errs.push('result.isError MUST be true for tool execution errors');
|
||||
}
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 6: protocol-level error → failed with inlined error, no result.
|
||||
{
|
||||
const id = 'sep-2663-tasks-get-status-failed';
|
||||
const name = 'TasksProtocolErrorFailedShape';
|
||||
const description =
|
||||
'Protocol-level error reports as failed + inlined error{code,message}, no result';
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'protocol_error_job',
|
||||
arguments: {}
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (!created.taskId) {
|
||||
errs.push('protocol_error_job MUST create a task');
|
||||
} else {
|
||||
const terminal = await waitForTerminal(conn, created.taskId);
|
||||
if (terminal.status !== 'failed') {
|
||||
errs.push(
|
||||
`protocol error MUST surface as failed (not "${terminal.status}")`
|
||||
);
|
||||
}
|
||||
if (!terminal.error) {
|
||||
errs.push('failed task MUST carry inlined `error`');
|
||||
} else {
|
||||
if (typeof terminal.error.code !== 'number') {
|
||||
errs.push('failed task error MUST carry numeric `code`');
|
||||
}
|
||||
if (typeof terminal.error.message !== 'string') {
|
||||
errs.push('failed task error MUST carry string `message`');
|
||||
}
|
||||
}
|
||||
if (terminal.result) {
|
||||
errs.push('failed task MUST NOT carry `result`');
|
||||
}
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 7: tasks/cancel returns empty {resultType:"complete"} ack;
|
||||
// status settles to cancelled.
|
||||
{
|
||||
const id = 'sep-2663-cancel-ack-empty-result';
|
||||
const name = 'TasksCancelEmptyAck';
|
||||
const description =
|
||||
'tasks/cancel returns {resultType:"complete"} ack; status settles to cancelled';
|
||||
let cancelTaskId: string | undefined;
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 60, label: 'lifecycle-cancel' }
|
||||
})) as any;
|
||||
cancelTaskId = created.taskId;
|
||||
if (!cancelTaskId) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'slow_compute did not create a task',
|
||||
specReferences: [SEP_2663_REF, SEP_2322_REF]
|
||||
});
|
||||
} else {
|
||||
const ack = (await conn.request('tasks/cancel', {
|
||||
taskId: cancelTaskId
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (ack?.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`cancel ack MUST carry resultType:"complete"; got resultType=${ack?.resultType}`
|
||||
);
|
||||
}
|
||||
// Task-envelope fields MUST NOT appear on the ack; _meta and
|
||||
// other result-shape metadata are permitted.
|
||||
const ackOffenders = (
|
||||
['taskId', 'status', 'result', 'error', 'inputRequests'] as const
|
||||
).filter((f) => f in ack);
|
||||
if (ackOffenders.length > 0) {
|
||||
errs.push(
|
||||
`cancel ack MUST NOT carry task-envelope fields; got: ${ackOffenders.join(', ')}`
|
||||
);
|
||||
}
|
||||
// SEP-2663 §Task Cancellation: transition to `cancelled` is not
|
||||
// guaranteed; record the settled status as diagnostic detail only.
|
||||
const after = await waitForTerminal(conn, cancelTaskId);
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF, SEP_2322_REF],
|
||||
details: { cancelAck: ack, statusAfterCancel: after.status }
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 8: tasks/cancel on a terminal task is idempotent — returns
|
||||
// the same {resultType:"complete"} empty-ack as on an active task,
|
||||
// rather than -32602. Required so clients don't have to handle the
|
||||
// race where a task terminates between observation and the cancel
|
||||
// request.
|
||||
{
|
||||
const id = 'tasks-cancel-terminal-idempotent-ack';
|
||||
const name = 'TasksCancelTerminalIdempotentAck';
|
||||
const description =
|
||||
'tasks/cancel on a terminal task returns the same empty-ack as on an active task (idempotent)';
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'slow_compute',
|
||||
arguments: { seconds: 1, label: 'lifecycle-cancel-terminal' }
|
||||
})) as any;
|
||||
const completedTaskId = created.taskId;
|
||||
if (!completedTaskId) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'slow_compute did not create a task',
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} else {
|
||||
await waitForTerminal(conn, completedTaskId);
|
||||
const ack = (await conn.request('tasks/cancel', {
|
||||
taskId: completedTaskId
|
||||
})) as any;
|
||||
const errs: string[] = [];
|
||||
if (ack?.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`idempotent cancel ack MUST carry resultType:"complete"; got resultType=${ack?.resultType}`
|
||||
);
|
||||
}
|
||||
const ackOffenders = (
|
||||
['taskId', 'status', 'result', 'error', 'inputRequests'] as const
|
||||
).filter((f) => f in ack);
|
||||
if (ackOffenders.length > 0) {
|
||||
errs.push(
|
||||
`idempotent cancel ack MUST NOT carry task-envelope fields; got: ${ackOffenders.join(', ')}`
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2663_REF],
|
||||
details: { cancelAck: ack }
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2663_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
await conn.close();
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
107
src/scenarios/server/tasks/mrtr-helpers.ts
Normal file
107
src/scenarios/server/tasks/mrtr-helpers.ts
Normal file
|
|
@ -0,0 +1,107 @@
|
|||
/**
|
||||
* Generic conformance-check helpers shared by the SEP-2663 `tasks/*`
|
||||
* scenarios. Pure: no I/O, no scenario-specific state.
|
||||
*
|
||||
* Split from `helpers.ts` (which holds tasks-specific helpers like
|
||||
* `validTasksParams` and the polling loops) so that the SEP references
|
||||
* and the FAILURE/SKIPPED check builders can be reused by any future
|
||||
* extension scenario without dragging tasks-domain types along.
|
||||
*/
|
||||
|
||||
import type { ConformanceCheck, SpecReference } from '../../../types';
|
||||
import { MRTR_SPEC_REFERENCES } from '../input-required-result-helpers';
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// Check builders
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
|
||||
export function errMsg(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
|
||||
/** Build a FAILURE check from a thrown error, preserving id/name/description. */
|
||||
export function failureCheck(
|
||||
id: string,
|
||||
name: string,
|
||||
description: string,
|
||||
error: unknown,
|
||||
specReferences: SpecReference[]
|
||||
): ConformanceCheck {
|
||||
return {
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errMsg(error),
|
||||
specReferences
|
||||
};
|
||||
}
|
||||
|
||||
/** Build a SKIPPED check (preserves id stability so Ctrl+F still finds it). */
|
||||
export function skipCheck(
|
||||
id: string,
|
||||
name: string,
|
||||
description: string,
|
||||
reason: string,
|
||||
specReferences: SpecReference[]
|
||||
): ConformanceCheck {
|
||||
return {
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'SKIPPED',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Skipped: ${reason}`,
|
||||
specReferences
|
||||
};
|
||||
}
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// SEP references
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
|
||||
export const SEP_2243_REF: SpecReference = {
|
||||
id: 'SEP-2243',
|
||||
url: 'https://modelcontextprotocol.io/seps/2243-http-standardization'
|
||||
};
|
||||
|
||||
/**
|
||||
* SEP-2322 (MRTR). Imported from `input-required-result-helpers` so the
|
||||
* codebase holds a single URL per SEP — the rendered spec page absorbs
|
||||
* post-merge amendments and `MRTR_SPEC_REFERENCES` is already the
|
||||
* traceability anchor for SEP-2322 elsewhere in `server/`.
|
||||
*/
|
||||
export const SEP_2322_REF: SpecReference = MRTR_SPEC_REFERENCES[0];
|
||||
|
||||
export const SEP_2575_REF: SpecReference = {
|
||||
id: 'SEP-2575',
|
||||
url: 'https://modelcontextprotocol.io/seps/2575-stateless-mcp'
|
||||
};
|
||||
|
||||
export const SEP_2663_REF: SpecReference = {
|
||||
id: 'SEP-2663',
|
||||
url: 'https://modelcontextprotocol.io/seps/2663-tasks-extension'
|
||||
};
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// Wire-format predicates
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* ISO-8601 timestamp prefix (YYYY-MM-DDThh:mm:ss). Tolerant about the
|
||||
* timezone tail (`Z`, `+00:00`, `+0000`) and sub-second precision —
|
||||
* matches what real servers emit (Go `time.RFC3339Nano`, Python
|
||||
* `datetime.isoformat()`, JavaScript `toISOString()`).
|
||||
*
|
||||
* A regex over `Date.parse` (too permissive, accepts RFC-2822, "May 4
|
||||
* 2026") / `new Date(s).toISOString() === s` (too strict, rejects
|
||||
* valid `+00:00` offsets that don't survive the canonical `Z`
|
||||
* round-trip) / `Temporal.Instant.from` (Node 24+ experimental).
|
||||
*/
|
||||
export const ISO_8601_PATTERN = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}/;
|
||||
|
||||
/** Returns true when the input is a string matching ISO-8601 prefix. */
|
||||
export function isIso8601(s: unknown): boolean {
|
||||
return typeof s === 'string' && ISO_8601_PATTERN.test(s);
|
||||
}
|
||||
372
src/scenarios/server/tasks/mrtr-input.ts
Normal file
372
src/scenarios/server/tasks/mrtr-input.ts
Normal file
|
|
@ -0,0 +1,372 @@
|
|||
/**
|
||||
* SEP-2322 / SEP-2663 — MRTR input flow on the tasks surface.
|
||||
*
|
||||
* Tests the input_required → tasks/update → resume loop, including
|
||||
* partial inputResponses fulfillment when a tool fans out multiple
|
||||
* simultaneous input requests.
|
||||
*
|
||||
* Required server fixtures:
|
||||
* - confirm_delete — task-supporting, calls TaskElicit once
|
||||
* - multi_input — task-supporting, fans out two TaskElicits in
|
||||
* parallel so two keys are pending at once
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../../types';
|
||||
import type { Connection, RunContext } from '../../../connection';
|
||||
import { SEP_2322_REF, SEP_2663_REF } from './mrtr-helpers';
|
||||
import { errMsg, failureCheck } from './mrtr-helpers';
|
||||
import { TASKS_EXTENSION_ID, waitForStatus, waitForTerminal } from './helpers';
|
||||
|
||||
export class TasksMRTRInputScenario implements ClientScenario {
|
||||
name = 'tasks-mrtr-input';
|
||||
readonly source = { extensionId: 'io.modelcontextprotocol/tasks' } as const;
|
||||
description = `Test SEP-2322 MRTR input flow on the tasks surface.
|
||||
|
||||
**Server Implementation Requirements:**
|
||||
|
||||
**Surfacing inputRequests (SEP-2322):**
|
||||
- A task waiting on client input MUST report \`status:"input_required"\`
|
||||
on tasks/get and surface a non-empty \`inputRequests\` map keyed by
|
||||
server-minted opaque ids. Each entry carries the underlying request
|
||||
(\`elicitation/create\`, \`sampling/createMessage\`, etc.).
|
||||
|
||||
**Resuming via tasks/update (SEP-2663):**
|
||||
- The client delivers responses through \`tasks/update\` with
|
||||
\`inputResponses\` keyed to match the server-emitted ids. The server
|
||||
MUST return an empty \`{resultType:"complete"}\` ack on the
|
||||
tasks/update response — the resulting task state is observed via the
|
||||
next tasks/get.
|
||||
- After the response is delivered, the task MUST resume execution and
|
||||
proceed to a terminal state (or back to input_required for another
|
||||
round).
|
||||
|
||||
**Partial fulfillment (SEP-2663):**
|
||||
- A tool that emits multiple simultaneous input requests parks the task
|
||||
with multiple keys in \`inputRequests\`. A client MAY answer them one
|
||||
at a time:
|
||||
- tasks/update with a subset of keys MUST be acked.
|
||||
- The task MUST stay in \`input_required\` until every pending request
|
||||
has been answered.
|
||||
- tasks/get after a partial update MUST surface only the still-pending
|
||||
keys; the answered key MUST be removed.
|
||||
|
||||
**Required server fixtures (\`tools/list\` MUST include all):**
|
||||
- \`confirm_delete\` — task-supporting, emits a single
|
||||
\`elicitation/create\` inputRequest then completes when the response
|
||||
arrives.
|
||||
- \`multi_input\` — task-supporting, fans out two \`elicitation/create\`
|
||||
inputRequests in parallel so two keys are pending at once (used by
|
||||
the partial-fulfillment check).`;
|
||||
|
||||
async run(ctx: RunContext): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
let conn: Connection;
|
||||
try {
|
||||
conn = await ctx.connect({
|
||||
capabilities: {
|
||||
elicitation: {},
|
||||
sampling: {},
|
||||
extensions: { [TASKS_EXTENSION_ID]: {} }
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tasks-mrtr-input-bootstrap',
|
||||
name: 'TasksMrtrInputBootstrap',
|
||||
description:
|
||||
'Initialize handshake declaring io.modelcontextprotocol/tasks extension succeeds',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed to initialize: ${errMsg(error)}`,
|
||||
specReferences: [SEP_2322_REF]
|
||||
});
|
||||
return checks;
|
||||
}
|
||||
|
||||
// Check 1: tasks/get surfaces inputRequests when status=input_required.
|
||||
{
|
||||
const id = 'sep-2663-tasks-get-status-input-required';
|
||||
const name = 'TasksMRTRInputRequestsOnTasksGet';
|
||||
const description =
|
||||
'tasks/get on an input_required task MUST surface a non-empty inputRequests map';
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'confirm_delete',
|
||||
arguments: { filename: 'mrtr-input.txt' }
|
||||
})) as any;
|
||||
const taskId = created.taskId;
|
||||
if (!taskId) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'confirm_delete did not create a task',
|
||||
specReferences: [SEP_2322_REF]
|
||||
});
|
||||
} else {
|
||||
const task = await waitForStatus(
|
||||
conn,
|
||||
taskId,
|
||||
'input_required',
|
||||
5_000
|
||||
);
|
||||
const errs: string[] = [];
|
||||
if (task.status !== 'input_required') {
|
||||
errs.push(
|
||||
`expected status:"input_required"; got ${JSON.stringify(task.status)}`
|
||||
);
|
||||
}
|
||||
if (
|
||||
!task.inputRequests ||
|
||||
typeof task.inputRequests !== 'object' ||
|
||||
Array.isArray(task.inputRequests)
|
||||
) {
|
||||
errs.push('inputRequests MUST be a non-null object (map)');
|
||||
} else {
|
||||
const keys = Object.keys(task.inputRequests);
|
||||
if (keys.length === 0) {
|
||||
errs.push('inputRequests MUST have at least one entry');
|
||||
} else {
|
||||
const firstReq = task.inputRequests[keys[0]];
|
||||
if (!firstReq?.method) {
|
||||
errs.push(
|
||||
'each inputRequest MUST carry a `method` (e.g., elicitation/create)'
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Cancel so we don't leave the task parked.
|
||||
try {
|
||||
await conn.request('tasks/cancel', { taskId });
|
||||
} catch {
|
||||
/* swallow */
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2322_REF, SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
checks.push(failureCheck(id, name, description, error, [SEP_2322_REF]));
|
||||
}
|
||||
}
|
||||
|
||||
// Check 2: tasks/update delivers inputResponses + resumes the task.
|
||||
{
|
||||
const id = 'tasks-mrtr-tasks-update-resumes';
|
||||
const name = 'TasksMRTRTasksUpdateResumes';
|
||||
const description =
|
||||
'tasks/update with matching inputResponses MUST be acked with {resultType:"complete"} and resume the task to a terminal state';
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'confirm_delete',
|
||||
arguments: { filename: 'mrtr-resume.txt' }
|
||||
})) as any;
|
||||
const taskId = created.taskId;
|
||||
if (!taskId) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'confirm_delete did not create a task',
|
||||
specReferences: [SEP_2322_REF, SEP_2663_REF]
|
||||
});
|
||||
} else {
|
||||
const inputTask = await waitForStatus(
|
||||
conn,
|
||||
taskId,
|
||||
'input_required',
|
||||
5_000
|
||||
);
|
||||
const errs: string[] = [];
|
||||
const responses: Record<string, any> = {};
|
||||
for (const key of Object.keys(inputTask.inputRequests ?? {})) {
|
||||
responses[key] = {
|
||||
action: 'accept',
|
||||
content: { confirm: true }
|
||||
};
|
||||
}
|
||||
const ack = (await conn.request('tasks/update', {
|
||||
taskId,
|
||||
inputResponses: responses
|
||||
})) as any;
|
||||
if (ack?.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`tasks/update ack MUST carry resultType:"complete"; got resultType=${ack?.resultType}`
|
||||
);
|
||||
}
|
||||
const updateAckOffenders = (
|
||||
['taskId', 'status', 'result', 'error', 'inputRequests'] as const
|
||||
).filter((f) => f in ack);
|
||||
if (updateAckOffenders.length > 0) {
|
||||
errs.push(
|
||||
`tasks/update ack MUST NOT carry task-envelope fields; got: ${updateAckOffenders.join(', ')}`
|
||||
);
|
||||
}
|
||||
const terminal = await waitForTerminal(conn, taskId);
|
||||
if (terminal.status !== 'completed') {
|
||||
errs.push(
|
||||
`task MUST resume to completed after tasks/update; got status ${JSON.stringify(terminal.status)}`
|
||||
);
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2322_REF, SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
checks.push(
|
||||
failureCheck(id, name, description, error, [
|
||||
SEP_2322_REF,
|
||||
SEP_2663_REF
|
||||
])
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Check 3: partial inputResponses fulfillment leaves the rest pending.
|
||||
{
|
||||
const id = 'tasks-mrtr-partial-fulfillment';
|
||||
const name = 'TasksMRTRPartialFulfillment';
|
||||
const description =
|
||||
'tasks/update with a subset of keys MUST keep the task in input_required with only the unanswered key remaining';
|
||||
try {
|
||||
const created = (await conn.request('tools/call', {
|
||||
name: 'multi_input',
|
||||
arguments: {}
|
||||
})) as any;
|
||||
const taskId = created.taskId;
|
||||
if (!taskId) {
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: 'multi_input did not create a task',
|
||||
specReferences: [SEP_2663_REF]
|
||||
});
|
||||
} else {
|
||||
// Wait until two keys are pending (the fan-out tool races two
|
||||
// TaskElicits, so we may briefly see one before the second).
|
||||
let inputTask: any;
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < 5_000) {
|
||||
inputTask = (await conn.request('tasks/get', { taskId })) as any;
|
||||
if (
|
||||
inputTask.status === 'input_required' &&
|
||||
inputTask.inputRequests &&
|
||||
Object.keys(inputTask.inputRequests).length >= 2
|
||||
) {
|
||||
break;
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 100));
|
||||
}
|
||||
const errs: string[] = [];
|
||||
if (inputTask.status !== 'input_required') {
|
||||
errs.push(
|
||||
`task with two parallel elicits MUST be input_required; got ${JSON.stringify(inputTask.status)}`
|
||||
);
|
||||
}
|
||||
const keys = Object.keys(inputTask.inputRequests ?? {});
|
||||
if (keys.length < 2) {
|
||||
errs.push(
|
||||
`multi_input MUST surface 2 inputRequests; got ${keys.length}`
|
||||
);
|
||||
} else {
|
||||
const [firstKey, secondKey] = keys;
|
||||
|
||||
// Answer first key only.
|
||||
const firstAck = (await conn.request('tasks/update', {
|
||||
taskId,
|
||||
inputResponses: {
|
||||
[firstKey]: {
|
||||
action: 'accept',
|
||||
content: { name: 'partial-1', confirm: true }
|
||||
}
|
||||
}
|
||||
})) as any;
|
||||
if (firstAck.resultType !== 'complete') {
|
||||
errs.push(
|
||||
`partial tasks/update ack MUST carry resultType:"complete"; got ${JSON.stringify(firstAck)}`
|
||||
);
|
||||
}
|
||||
|
||||
// Status MUST still be input_required with only the second
|
||||
// key remaining.
|
||||
const afterFirst = (await conn.request('tasks/get', {
|
||||
taskId
|
||||
})) as any;
|
||||
if (afterFirst.status !== 'input_required') {
|
||||
errs.push(
|
||||
`task MUST stay input_required while another input is still pending; got ${JSON.stringify(afterFirst.status)}`
|
||||
);
|
||||
}
|
||||
const remaining = Object.keys(afterFirst.inputRequests ?? {});
|
||||
if (!remaining.includes(secondKey)) {
|
||||
errs.push(
|
||||
`unanswered key MUST remain in inputRequests; got ${JSON.stringify(remaining)}`
|
||||
);
|
||||
}
|
||||
if (remaining.includes(firstKey)) {
|
||||
errs.push(
|
||||
`answered key MUST be removed from inputRequests; still saw ${firstKey}`
|
||||
);
|
||||
}
|
||||
|
||||
// Answer second key — task resumes and finishes.
|
||||
await conn.request('tasks/update', {
|
||||
taskId,
|
||||
inputResponses: {
|
||||
[secondKey]: {
|
||||
action: 'accept',
|
||||
content: { name: 'partial-2', confirm: true }
|
||||
}
|
||||
}
|
||||
});
|
||||
const terminal = await waitForTerminal(conn, taskId);
|
||||
if (terminal.status !== 'completed') {
|
||||
errs.push(
|
||||
`task MUST complete after both inputs are satisfied; got ${JSON.stringify(terminal.status)}`
|
||||
);
|
||||
}
|
||||
}
|
||||
checks.push({
|
||||
id,
|
||||
name,
|
||||
description,
|
||||
status: errs.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errs.length > 0 ? errs.join('; ') : undefined,
|
||||
specReferences: [SEP_2322_REF, SEP_2663_REF]
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
checks.push(
|
||||
failureCheck(id, name, description, error, [
|
||||
SEP_2322_REF,
|
||||
SEP_2663_REF
|
||||
])
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
await conn.close().catch(() => {});
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
56
src/scenarios/server/tasks/notifications.ts
Normal file
56
src/scenarios/server/tasks/notifications.ts
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
/**
|
||||
* SEP-2663 Tasks Extension — status notifications conformance.
|
||||
*
|
||||
* **SKIPPED — pending subscriptions/listen rewrite.** SEP-2663 delivers
|
||||
* `notifications/tasks` over the `subscriptions/listen` stream defined
|
||||
* by SEP-2575, not over the POST SSE response of `tools/call`. This
|
||||
* scenario's harness reads SSE frames off the `tools/call` POST
|
||||
* response, so its observation point misses notifications on servers
|
||||
* that follow the spec — they emit on a stream this harness never
|
||||
* opens.
|
||||
*
|
||||
* The scenario is preserved here as a reference point; rewriting the
|
||||
* harness against subscriptions/listen is tracked as a follow-up.
|
||||
*
|
||||
* Required server fixtures (for the future re-enabled scenario):
|
||||
* - slow_compute — task-supporting, sleeps N seconds.
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../../types';
|
||||
import type { RunContext } from '../../../connection';
|
||||
import { SEP_2663_REF } from './mrtr-helpers';
|
||||
import { skipCheck } from './mrtr-helpers';
|
||||
|
||||
export class TasksStatusNotificationsScenario implements ClientScenario {
|
||||
name = 'tasks-status-notifications';
|
||||
readonly source = { extensionId: 'io.modelcontextprotocol/tasks' } as const;
|
||||
description = `Test SEP-2663 status notifications (optional).
|
||||
|
||||
**Server Implementation Requirements:**
|
||||
|
||||
Servers MAY emit \`notifications/tasks\` on the subscriptions/listen
|
||||
stream to inform clients of task state changes without polling.
|
||||
Notifications are optional — a server is conformant whether it sends
|
||||
them or not. When sent, the notification params MUST carry:
|
||||
|
||||
- \`taskId\`: the task the notification refers to.
|
||||
- \`status\`: the new task status.
|
||||
- For terminal statuses (\`completed\`/\`failed\`/\`cancelled\`),
|
||||
notifications MAY inline the corresponding \`result\` or \`error\`
|
||||
per the SEP-2663 DetailedTask shape.`;
|
||||
|
||||
async run(_ctx: RunContext): Promise<ConformanceCheck[]> {
|
||||
// Prior implementation (POST-SSE observation harness on tools/call)
|
||||
// was removed in this commit; see git blame for the working code to
|
||||
// adapt when rewriting against subscriptions/listen.
|
||||
return [
|
||||
skipCheck(
|
||||
'tasks-status-notifications',
|
||||
'TasksStatusNotificationsScenario',
|
||||
'Status notifications conformance, pending subscriptions/listen rewrite.',
|
||||
'SEP-2663 delivers notifications/tasks via the SEP-2575 subscriptions/listen stream, not via the tools/call POST SSE response that the prior harness observed. The harness needs a rewrite against the new channel; tracked as a follow-up.',
|
||||
[SEP_2663_REF]
|
||||
)
|
||||
];
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue