inspector/.claude/skills/test-servers/SKILL.md
Cliff Hall ab80221ce6
chore: land the #2247 round-9 review findings on v2/main (#2322)
* chore: land the #2247 round-9 review findings on v2/main

Copilot filed a ninth review round on #2247 after it had merged, so none of
it reached v2/main. All five findings are addressed here.

1. The testing -> test-servers hand-off delivered the wrong half of the
   procedure. test-servers documented only the two-process manual path
   (server-composable.js --config <name>.json plus a second Inspector
   process), but an integration test builds the server in-process from the
   @modelcontextprotocol/inspector-test-server API and owns its lifecycle.
   Adds a "Two ways to use a fixture — pick the right one first" section
   with the in-process recipe, the four mechanics that bite (one barrel,
   start/stop in afterEach, read started.url, era is a constructor option),
   and a pointer to the reference test; renames "Run one" to "Run one by
   hand (two processes)" so the manual path names itself. testing now says
   in the connecting bullet that an automated test does NOT spawn.

2. testing's heading asked "does the test need a real server?", which its
   own hand-rolled-server exception contradicts. It now asks whether the
   test uses a test-servers/ fixture.

3. "imports" was too broad a dependency test — test-server-scope.test.ts
   imports the package to unit-test scope middleware as a pure function and
   never starts a server. The condition is now "starts, spawns, or
   configures", with that test named as the counterexample.

4. docs/skill-authoring.md attributed the 33% -> 100% rise to imperative
   wording and placement independently. The #2247 edit changed both at
   once, so only the combination is established; the split into two levers
   is now labelled the plausible mechanism, not a measured result.

5. The CHAIN_THRESHOLD comment generalised "a well-shaped pointer clears
   0.8" from one two-case run of that same multi-variable edit. It now
   reads as an existence proof about the ceiling, not a property of
   pointers in general.

Also records the headline lesson where it can be found again: a green
hand-off number proves skill B loaded, never that B answered the question —
in AGENTS.md's chain-case rules and in the skill-authoring checklist.

Docs, skill bodies and one comment only; no runtime code changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zwYWanPhvgu6wiKoeGg4N
Signed-off-by: cliffhall <cliff@futurescale.com>

* fix: address Copilot review round 1 on #2264

The new section claimed two shapes and there are three — the automated
stdio path was missing, and it is the repo's established pattern
(getTestMcpServerCommand() into a stdio transport, used by five web
integration files and the whole CLI suite). Telling callers an automated
test "does not spawn" told them not to use it.

- test-servers: "Two ways" -> "Three ways", with a table and a subsection
  for the spawned-stdio shape — the transport owns the child (disconnect,
  not stop()), no config can be selected because that entry point runs the
  stdio default, and it is still the build. Names a reference test for it.
- test-servers: the build is no longer scoped to the manual path. All three
  shapes resolve test-servers/build/ — two through the barrel alias, one by
  running the emitted .js — so the staleness hazard is called out as
  applying to every shape, and `Build first` no longer opens by claiming
  the only reason is subprocess spawning.
- testing: the "does not spawn" instruction is replaced by the two
  automated shapes and when to pick each. What they share, and what the
  hand-off actually needed to say, is that neither picks a showcase config.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zwYWanPhvgu6wiKoeGg4N
Signed-off-by: cliffhall <cliff@futurescale.com>

* fix: address Copilot review round 2 on #2264

Round 1 fixed the missing stdio shape but kept the wrong axis. Cutting
the taxonomy by "automated vs by hand" excluded the config-driven smokes
— smoke-web-elicitation.mjs spawns server-composable.js --config
directly, and lib/mcp-app-flow.mjs does it for smoke:web:app,
smoke:web:tabs and pack:verify. Those are automated AND need every bit
of the config and protocol-era guidance a person at two terminals does,
so the skill was routing their authors away from it.

- test-servers: the cut is now how the SERVER IS STOOD UP, stated as
  such. Third shape is "spawned composable HTTP", whose consumers are
  the smokes, pack:verify, and a person by hand — the config/era
  guidance belongs to the shape, whoever starts it. Table gains a
  Consumers column and loses the Who column that caused this.
- test-servers: the stale "an automated test builds the server
  in-process instead" line under `Run one by hand` is gone (suppressed
  comment); that heading is now the same shape driven by a person.
- testing: routes by what is driving — integration/CLI test to
  in-process HTTP, stdio-is-the-point to spawned stdio, and a smoke to
  the composable-config path with the era guidance it actually needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zwYWanPhvgu6wiKoeGg4N
Signed-off-by: cliffhall <cliff@futurescale.com>

* fix: address Copilot review round 3 on #2264

Round 2 replaced one wrong axis with a narrower one: it routed "a smoke"
wholesale to the config-driven composable path, and smoke:cli does not
use it. smoke-cli.mjs starts an in-process createTestServerHttp for the
header round-trip (so it can read back what the CLI sent) and writes the
built stdio entry into a --catalog for the connect checks; smoke:tui uses
only the latter. Neither picks a config.

- test-servers: consumers filled in per row — in-process HTTP gains CLI
  tests and smoke:cli, spawned stdio gains smoke:cli and smoke:tui, and
  the composable row is narrowed to the WEB smokes. A warning states the
  point directly: "a smoke" is not a shape, pick by the row.
- test-servers: the stdio shape now covers naming the built entry in a
  catalog without connecting, which is what those two smokes do.
- testing: same narrowing, plus the reason spelled out so the word
  "smoke" stops being a routing key.
- testing: the dependency criterion said "starts, spawns, or configures",
  which excluded the build-only smoke:tui case documented directly above
  it — the skill both required and rejected the hand-off for it
  (suppressed comment). It now reads "starts, spawns, configures, or
  hands a built entry to the subject under test".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018zwYWanPhvgu6wiKoeGg4N
Signed-off-by: cliffhall <cliff@futurescale.com>

---------

Signed-off-by: cliffhall <cliff@futurescale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 08:22:16 -04:00

263 lines
12 KiB
Markdown

---
name: test-servers
description: Run a composable MCP test server by hand — pick the showcase config for a feature or bug, build it, and connect with the right protocol era. Use when a change, a PR or a smoke test needs a real server to exercise it; when reproducing a reported bug by hand; when choosing which fixture or protocol era to run; when a fixture keeps serving stale code after an edit; or when the config or preset you need does not exist yet.
disable-model-invocation: false
---
# Running a test server
`test-servers/` provides **composable MCP servers** so tests and manual checks
exercise a real server over a real transport instead of mocks. A server is
assembled from **presets** (fixture factories in
`test-servers/src/preset-registry.ts`) and configured declaratively with a JSON
file under `test-servers/configs/`.
The full catalogue of showcase configs — one per feature, each with what to click
and what the broken build did — is
[`docs/test-servers.md`](../../../docs/test-servers.md). This skill is how to
run one.
## Three ways to use a fixture — pick the right one first
A fixture is stood up in **one of three shapes**, and most of what follows is
about the third. Establish which one you are in before reading further, because
the showcase **config file and the protocol-era table** belong to that one alone.
⚠️ **The cut is how the server is stood up, not who is driving.** Automated and
by-hand is the wrong axis: the composable-config shape has *both* kinds of
consumer, and a smoke that spawns it needs every bit of the config and era
guidance a person at two terminals does.
| Shape | Server runs | Config file | Consumers |
| --- | --- | --- | --- |
| **In-process HTTP** | inside the test process, built from the API | none — options are constructor args | integration tests, CLI tests, **`smoke:cli`** |
| **Spawned stdio** | a child process the transport (or the binary under test) starts | none — the stdio fixture runs its default config | integration tests, the CLI suites, **`smoke:cli`**, **`smoke:tui`** |
| **Spawned composable HTTP** | a child process started with `--config` | **yes** — a showcase `--config <name>.json` | the **web** smokes, `pack:verify`, **and** a person by hand |
⚠️ **"A smoke" is not a shape** — `smoke:cli` uses the first two and `smoke:tui`
the second, while only the config-driven web smokes use the third. Pick by the
row, never by the caller's category.
- **In-process HTTP — `createTestServerHttp`.** The caller *constructs* the
server and owns its lifecycle. No subprocess, no JSON config, no showcase
config to pick. This is the shape for anything needing HTTP/SSE, a specific
tool set, or the modern handler — and it is not test-only: `scripts/smoke-cli.mjs`
starts one in the smoke process so it can read back the headers the CLI sent.
- **Spawned stdio — `getTestMcpServerCommand()`.** The test hands the built
fixture's `{ command, args }` to a stdio transport (or to the built CLI), and
the transport spawns it. A subprocess *is* started, but **still no config
file**: that entry point runs the stdio server's default config, so there is
nothing to pick. Reach for it when stdio is the point (`InspectorClient` over
stdio, the CLI's out-of-process E2E suite) and the default tool set is enough.
A caller may also just *name* the built entry rather than connect to it —
`smoke:cli` and `smoke:tui` write it into a `--catalog` as
`{ type: "stdio", command: node, args: [<built entry>] }` — which is still
this shape, and still the build.
- **Spawned composable HTTP — `server-composable.js --config <name>.json`.**
Picking the showcase config and the protocol era applies **to this shape**,
whoever starts it. Two consumers, and they differ only in who runs the second
process:
- **A script.** `scripts/smoke-web-elicitation.mjs` spawns it directly;
`scripts/lib/mcp-app-flow.mjs` (`startMcpAppServer`) does it for
`smoke:web:app`, `smoke:web:tabs` and `pack:verify`. These are automated and
config-driven, and the whole of this skill applies to them. ⚠️ **Not every
smoke is here** — `smoke:cli` and `smoke:tui` use the two shapes above and
pick no config at all.
- **You, in a terminal**, with the Inspector in another — `Run one by hand`
below.
⚠️ **The build applies to all three.** Every shape resolves
`test-servers/build/` — the two API-driven ones through the
`@modelcontextprotocol/inspector-test-server` alias, the composable one by
running the emitted `.js` directly — so `Build first` and its stale-build hazard
are **not** guidance for one path. Read that section whichever shape you are in.
### Automated, in-process HTTP: build the server from the API
```ts
import {
createTestServerHttp,
type TestServerHttp,
createTestServerInfo,
createEchoTool,
} from "@modelcontextprotocol/inspector-test-server";
let server: TestServerHttp | null = null;
afterEach(async () => {
// Stop it even when the assertion threw, or the port leaks into the next test.
if (server) {
try {
await server.stop();
} catch {
// ignore
}
server = null;
}
});
it("…", async () => {
const started = createTestServerHttp({
serverInfo: createTestServerInfo("excluded-tools-test", "1.0.0"),
tools: [createEchoTool()],
// `modern: {}` opts the fixture into the 2026-07-28 handler; omit it for legacy.
});
await started.start();
server = started;
// `started.url` is the bound URL — read it, never reconstruct it from a port.
// …connect an InspectorClient to it and assert.
});
```
The reference test is
[`clients/web/src/test/integration/mcp/inspectorClient-excluded-tools.test.ts`](../../../clients/web/src/test/integration/mcp/inspectorClient-excluded-tools.test.ts)
— read it before writing a new one; it is the shape fixture-backed integration
tests follow when the server is built in-process. (Stdio-backed ones follow the
next subsection instead.)
Four mechanics of this path:
- **The factories come from one barrel.** `createTestServerHttp` /
`createTestServerStdio` build the server; the `create*Tool`,
`create*Resource` and `create*Prompt` fixtures in
`test-servers/src/test-server-fixtures.ts` populate it;
`createTestServerInfo` fills in `serverInfo`. Prefer an existing fixture
factory to hand-writing a `ToolDefinition` — that is what makes the fixture a
shared one.
- **`start()` then `stop()`, and `stop()` in an `afterEach`.** The server binds a
real port, so a test that throws before stopping leaks it into the rest of the
file.
- **Read `started.url`.** `createTestServerHttp` resolves through
`findAvailablePort()`, which walks upward when the port is taken, so an
assumed port is the same bug the two-process path has.
- **Era is a constructor option, not a config file.** `modern: {}` on the config
object selects the modern handler; the client side picks its own negotiation
(`eraToVersionNegotiation`). The showcase-config era table below does not
apply.
⚠️ **The barrel is an alias to the BUILD, not to the source** —
`vitest.shared.mts` maps `@modelcontextprotocol/inspector-test-server` to
`test-servers/build/index.js`. So `Build first` applies to this path in full,
stale-build hazard included: an edit to `test-servers/src` that is not rebuilt
is invisible to an in-process test exactly as it is to a spawned one.
### Automated, spawned stdio: hand over the command
```ts
import { getTestMcpServerCommand } from "@modelcontextprotocol/inspector-test-server";
const { command, args } = getTestMcpServerCommand();
const client = new InspectorClient(
{ type: "stdio", command, args },
{ environment: { transport: createTransportNode } },
);
await client.connect();
// … afterEach → client.disconnect(), which is what stops the child.
```
`getTestMcpServerCommand()` returns `node <test-servers/build/test-server-stdio.js>`.
Three consequences:
- **You do not own the process, the transport does.** There is no `start()` /
`stop()` pair — disconnecting the client is what reaps the child, so the
`afterEach` that matters is `client.disconnect()`.
- **No config is selected and none can be.** That entry point starts the stdio
server on its **default** config, so the showcase-config table and the
protocol-era guidance below do not apply. If the case needs a specific tool
set or the modern handler, it is an in-process HTTP test, not this.
- **It is still the build.** The path comes from the module's own resolved
location under the alias, so it is `test-servers/build/`, with the same
staleness hazard.
The same command feeds the CLI's out-of-process E2E suite
(`clients/cli/__tests__/e2e.test.ts`), which spawns the built CLI *and* lets it
spawn the fixture. Reference tests for this shape:
`clients/web/src/test/integration/mcp/inspectorClient-response-rejected.test.ts`
and `clients/cli/__tests__/methods.test.ts`.
## Build first
Every shape above resolves generated output — the in-process one imports the
barrel, which is **aliased to `test-servers/build/index.js`**, and the stdio and
composable-config ones run emitted `.js` as real subprocesses. So the build must
exist whichever one you are in:
```sh
cd clients/web && npm run test-servers:build # tsc -p test-servers → test-servers/build/
```
Scripts reach this through `scripts/ensure-test-servers.mjs`, which builds
**unconditionally** (once per process per repo root).
⚠️ **Unconditional emit is not a clean.** A **deleted** source file leaves its
stale `.js` behind, existence checks pass against it, and anything still
importing that module silently runs the old code — reported not as staleness but
as a product failure in whatever was being tested. So after deleting or renaming
a source file:
```sh
rm -rf test-servers/build
```
The `.tsbuildinfo` is pinned inside `build/` so that clean actually invalidates
the cache.
## Run one by hand (two processes)
This is the **spawned composable HTTP** shape from the section above, driven by
you rather than by a smoke script — the config and era guidance is the same
either way. Two processes: the test server, then the Inspector.
```sh
# 1. The server, from the repo root, with the config you picked:
node test-servers/build/server-composable.js --config test-servers/configs/<name>.json
```
```sh
# 2. The Inspector, in another terminal (needs a built launcher — `npm run build`):
node clients/launcher/build/index.js --web
```
Then add the server in the Inspector using the URL the first process announced.
Two mechanics that bite:
- **The server announces its URL on _stderr_**, not stdout (`console.error` in
`server-composable.ts`). Watching stdout alone looks like a server that never
started.
- **The bound port is not necessarily the config's.** `createTestServerHttp`
resolves through `findAvailablePort()`, which walks upward when the configured
port is taken — so read the announced URL rather than assuming.
## Pick the right protocol era
Each config in [`docs/test-servers.md`](../../../docs/test-servers.md) says
which era to connect with. The default is **legacy**; configs setting
`transport.modern` need **Protocol Era = Modern**. Connecting with the wrong era
usually looks like a missing capability rather than an error.
## Common starting points
| Want to see | Config |
| --- | --- |
| An MCP App in the Apps tab | `mcp-app-http.json` (legacy) |
| An App-rendered elicitation | `app-elicitation-http.json` (legacy) |
| `Mcp-*` headers + the modern error taxonomy | `modern-network-http.json` |
| A tool result's `structuredContent` section | `structured-output-http.json` (legacy) |
| RFC 6570 resource-template expansion | `rfc6570-templates-http.json` |
| OAuth token revocation on clear | `oauth-revocation-http.json` (legacy) |
| A token endpoint the SDK refuses | `oauth-insecure-token-endpoint-http.json` (legacy) |
| Cancelling a call mid-flight | `cancellation-modern-http.json` (modern) |
## Adding a config or preset
- Presets live in `test-servers/src/preset-registry.ts`; configs in
`test-servers/configs/*.json`.
- A new showcase config gets a row in `docs/test-servers.md` saying what to do
and what the broken build did — the "what it looked like broken" half is what
makes the fixture reproducible later.
- ⚠️ **An `outputSchema` override must ride a tool that returns structured
content.** A conforming client validates the result against the advertised
schema, so an override on a preset returning none makes every call fail with
"declares an output schema but returned no structured content".