modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

Review round 3: consuming a fixture is a trigger too; drop the brittle count

Browse source 
Copilot, both nits accepted — the first is a correctness hole, not a nit.

1. `smoke:tui` never asserts a round trip, so round 2 filed it under
   "boot-only, needs nothing". It calls
   `ensureTestServers({ requires: ["stdio"] })` and hands the built
   fixture to the TUI as its catalog's stdio command
   (scripts/smoke-tui.mjs:105-124) — which means the staleness hazard,
   the single most expensive thing in `test-servers`, lands on it in
   full. Connecting was the wrong sole condition.

   The trigger is now "does this test **connect to, or build and
   consume**, an MCP fixture?", with `smoke:tui` named at all three
   decision points and the fixture-consumption case given its own
   paragraph in the closing section.

2. The "29 of 74" figure was brittle and I could not reproduce it
   against a different pattern: 29 by my grep, 32 by a direct
   `@modelcontextprotocol/inspector-test-server` import, 39 by a wider
   union. A number whose value is a function of the grep does not belong
   in a skill body that nobody re-runs. Dropped; the concrete
   counter-examples stay, which is what actually carries the point —
   `storage/store-id.test.ts` validates a string and `mcp/import/*`
   parses config files, both sitting beside live-connection tests.

Re-measured after broadening the condition: 100% / 100% hand-off,
63/63 first-move at 100%, full suite at RUNS=5. Unchanged across all
three rounds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-05 17:06:54 -04:00
commit 0894ab32ef
1 changed files with 40 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/skill-authoring.md
View file

@ -305,7 +305,7 @@ points apart on the first case.
Rewriting that classification into an imperative first step ("load the
`test-servers` skill now — that is step one"), and repeating it at the two later
points where the model actually decides it is writing a connecting test, took
points where the model actually decides it is writing a fixture-backed test, took
them to **100% / 100% at `RUNS=5`** with the prompts unchanged (#2247). Nothing
else moved: the two descriptions were not touched, and the same suite scored
**63/63** first-move cases at 100%. (An intermediate build of that change