inspector/.claude/skills/test-servers/SKILL.md
cliffhall a51e62dfa1 chore: address Copilot review round 1
- skill-manifest: the closing `---` fence must occupy a whole line. `---oops`
  was accepted as a terminator, silently truncating the frontmatter and passing
  validation — the exact failure the guard exists to catch. Regression cases
  added (prefix, trailing whitespace, EOF, CRLF).
- skill-eval: reject a nonzero `claude -p` exit instead of resolving it as an
  empty observation. An auth error or rate limit was scoring as "no skill
  invoked", which passes every negative case and reads as a trigger miss on
  every positive one.
- skill-eval: extract `collectSkillInvocations` / `sampleHit` behind a `main()`
  guard and cover them in `skill-eval.test.mjs` — the stream parsing and the
  process-outcome handling are unreachable from an eval run's happy path.
- docs/test-servers: annotate the era on every catalogue row. Six legacy rows
  carried no annotation under a "unless noted, Modern" default, so the doc sent
  readers to the wrong era for the feature they were trying to see.
- docs/quality-gate: `validate` runs four guards, not three; add the
  `verify:skills` row and the two new `test:scripts` suites; fix the `#publishing`
  fragment left dangling by the split.
- issue-create / pr-flow: delegate the card steps to `/board-ops` rather than
  restating its option IDs, which are explicitly unstable.
- issue-triage: the audit counted presence, not cardinality — an issue carrying
  both `v1` and `v2` passed a check that claims "exactly one". Same for the five
  type labels. Also scope the two milestone checks to board #28.
- AGENTS.md: codify the v1 milestone exception. Every milestone is a v2 release
  bucket, so a v1 issue has none to take and the Incoming-milestone invariant
  cannot apply to board #11.
- clients/launcher/README: its pointer promised an explanation of why the cli/tui
  manifests ship, which the extracted guide does not make.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 00:52:45 -04:00

98 lines
3.9 KiB
Markdown

---
name: test-servers
description: Run a composable MCP test server by hand — pick the showcase config that reproduces a given feature or bug, build test-servers, and connect with the right protocol era. Covers the stale-build hazard.
disable-model-invocation: true
---
# Running a test server
`test-servers/` provides **composable MCP servers** so tests and manual checks
exercise a real server over a real transport instead of mocks. A server is
assembled from **presets** (fixture factories in
`test-servers/src/preset-registry.ts`) and configured declaratively with a JSON
file under `test-servers/configs/`.
The full catalogue of showcase configs — one per feature, each with what to click
and what the broken build did — is
[`docs/test-servers.md`](../../../docs/test-servers.md). This skill is how to
run one.
## Build first
The servers are spawned as real subprocesses, so the build output must exist:
```sh
cd clients/web && npm run test-servers:build # tsc -p test-servers → test-servers/build/
```
Scripts reach this through `scripts/ensure-test-servers.mjs`, which builds
**unconditionally** (once per process per repo root).
⚠️ **Unconditional emit is not a clean.** A **deleted** source file leaves its
stale `.js` behind, existence checks pass against it, and anything still
importing that module silently runs the old code — reported not as staleness but
as a product failure in whatever was being tested. So after deleting or renaming
a source file:
```sh
rm -rf test-servers/build
```
The `.tsbuildinfo` is pinned inside `build/` so that clean actually invalidates
the cache.
## Run one
Two processes: the test server, then the Inspector.
```sh
# 1. The server, from the repo root, with the config you picked:
node test-servers/build/server-composable.js --config test-servers/configs/<name>.json
```
```sh
# 2. The Inspector, in another terminal (needs a built launcher — `npm run build`):
node clients/launcher/build/index.js --web
```
Then add the server in the Inspector using the URL the first process announced.
Two mechanics that bite:
- **The server announces its URL on _stderr_**, not stdout (`console.error` in
`server-composable.ts`). Watching stdout alone looks like a server that never
started.
- **The bound port is not necessarily the config's.** `createTestServerHttp`
resolves through `findAvailablePort()`, which walks upward when the configured
port is taken — so read the announced URL rather than assuming.
## Pick the right protocol era
Each config in [`docs/test-servers.md`](../../../docs/test-servers.md) says
which era to connect with. The default is **legacy**; configs setting
`transport.modern` need **Protocol Era = Modern**. Connecting with the wrong era
usually looks like a missing capability rather than an error.
## Common starting points
| Want to see | Config |
| --- | --- |
| An MCP App in the Apps tab | `mcp-app-http.json` (legacy) |
| An App-rendered elicitation | `app-elicitation-http.json` (legacy) |
| `Mcp-*` headers + the modern error taxonomy | `modern-network-http.json` |
| A tool result's `structuredContent` section | `structured-output-http.json` (legacy) |
| RFC 6570 resource-template expansion | `rfc6570-templates-http.json` |
| OAuth token revocation on clear | `oauth-revocation-http.json` (legacy) |
| Cancelling a call mid-flight | `cancellation-modern-http.json` (modern) |
## Adding a config or preset
- Presets live in `test-servers/src/preset-registry.ts`; configs in
`test-servers/configs/*.json`.
- A new showcase config gets a row in `docs/test-servers.md` saying what to do
and what the broken build did — the "what it looked like broken" half is what
makes the fixture reproducible later.
- ⚠️ **An `outputSchema` override must ride a tool that returns structured
content.** A conforming client validates the result against the advertised
schema, so an override on a preset returning none makes every call fail with
"declares an output schema but returned no structured content".