modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

Review round 1: gate the hand-off on connecting, not on tier depth

Browse source 
Copilot, both findings accepted.

1. The trigger was over-broad. "a smoke" and "everything from web
   integration rightwards" routed work to `test-servers` that needs no
   fixture at all: `smoke:launcher` only checks `--help`, `smoke:web`
   asserts the SPA is served, `smoke:web:browser` stops at first paint,
   and Storybook play functions render from fixture props. The condition
   is now "does this exercise MCP behaviour over a transport?", stated
   as such, with both sides named — the smokes that connect
   (`smoke:cli`, `smoke:web:app`, `smoke:web:elicit`, `smoke:web:tabs`)
   and `pack:verify` on one side, the boot-only ones and Storybook on
   the other. Depth in the tier list is explicitly not the boundary.

2. The threshold rationale contradicted the new result. It said an 0.8
   bar would mark a hand-off red "regardless of how strongly the first
   skill points at the second" — which the 100% measurement refutes.
   Both copies (docs and `skill-eval.mjs`) now say what 0.5 actually
   buys: a floor on useful reliability for a noisier second-hop load, so
   the column still separates an adequate pointer from a broken one. The
   value is unchanged.

Re-measured after the narrowing, since a tighter trigger could have
weakened the pointer: 100% / 100% on the hand-off and 63/63 first-move
at 100%, full suite at RUNS=5. Recorded figure updated from the
intermediate 100% / 80%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-05 16:17:31 -04:00
commit 02f094f001
1 changed files with 54 additions and 34 deletions
Download patch file Download diff file Expand all files Collapse all files

45
.claude/skills/testing/SKILL.md
View file

@ -14,15 +14,22 @@ test goes, how to run it, and how to clear the gate.
## Before you write it: does the test need a real server?
**If it does, load the `test-servers` skill now — that is step one, before
choosing a location or writing a line.** It does, whenever the task is: an
integration test; an end-to-end test; a smoke; a coverage gap that has to be
exercised over a transport; or reproducing a reported bug by hand.
choosing a location or writing a line.**
Every one of those drives a **real server over a real transport, never a mock**,
and picking the fixture, building it, and connecting with the right protocol era
is a procedure this skill does not carry. Writing one without `test-servers`
means hand-rolling a fixture that already exists, or mocking the thing the tier
exists to avoid mocking.
The condition is **"does this exercise MCP behaviour over a transport?"**, not
which tier the test lands in. It does whenever the task is: an integration test;
an end-to-end test that connects; a smoke that drives a **connected** flow; a
coverage gap only reachable over a real connection; or reproducing a reported
bug against a server. It does **not** for a test that renders a component from
fixture props, or for a boot-only smoke that never connects — `smoke:launcher`
checks `--help` and `smoke:web` / `smoke:web:browser` only assert the SPA is
served and paints.
When it does apply, the test drives a **real server over a real transport, never
a mock**, and picking the fixture, building it, and connecting with the right
protocol era is a procedure this skill does not carry. Writing one without
`test-servers` means hand-rolling a fixture that already exists, or mocking the
thing the tier exists to avoid mocking.
## Where the test file goes
@ -91,9 +98,14 @@ spawns the built binary) → smokes through the built launcher (`npm run smoke`)
Storybook play functions (`test:storybook`) → the published-tarball check
(`npm run pack:verify`, local/release only — needs network).
Everything from **web integration** rightwards needs a fixture from
`test-servers/` — load the `test-servers` skill as soon as a task puts you at
that tier or deeper.
⚠️ **Depth in that list is not the fixture boundary — connecting is.** Web
integration, the out-of-process CLI tests, the smokes that actually connect
(`smoke:cli`, `smoke:web:app`, `smoke:web:elicit`, `smoke:web:tabs`) and
`pack:verify` all need a fixture from `test-servers/`; `smoke:launcher`,
`smoke:web`, `smoke:web:browser` and every Storybook play function do not — the
first three stop at boot, and play functions render from fixture props. **Load
the `test-servers` skill as soon as a task puts you on the connecting side of
that line.**
`validate` runs the per-client `test` scripts — so web **unit** plus cli's
out-of-process `e2e.test.ts`, but **not** web's integration project, which runs
@ -172,8 +184,9 @@ shared helper that wraps one.
## Test servers, not mocks
Integration and smoke tests drive a real server over a real transport, never a
mock. **Load the `test-servers` skill to pick, build and run the fixture** —
which showcase config covers the feature, which protocol era to connect with,
how to add a combination that does not exist yet, and why a fixture can keep
serving stale code after an edit.
Integration tests, and the smokes that drive a connected flow, use a real server
over a real transport rather than a mock. **For those, load the `test-servers`
skill to pick, build and run the fixture** — which showcase config covers the
feature, which protocol era to connect with, how to add a combination that does
not exist yet, and why a fixture can keep serving stale code after an edit. A
boot-only smoke needs none of it (see the tier list above).