modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

Review round 8: using a fixture, not connecting, is the trigger

Browse source 
Copilot, all three accepted.

1. Connecting still is not the condition. A few integration tests
   deliberately hand-roll a JSON-RPC server because the composable
   fixture *cannot* produce what they assert on —
   `inspectorClient-malformed-list.test.ts` says so in its own header:
   "the SDK's own server cannot emit these shapes: they are exactly what
   its types forbid". Same for `listSalvage-era.test.ts`. Real
   transport, real client, zero `test-servers/` dependency.

   Round 4 already stated the right condition at the top — "does this
   test depend on a fixture from `test-servers/`" — and then three
   downstream sites went on saying "connects". Those are now scoped to
   fixture use, and the hand-rolled exception is stated where the
   trigger is defined rather than left to be discovered, with both files
   named. Connecting is kept as a hint, since it usually does imply a
   fixture, with the instruction to check whether one can express the
   case first.

2. "description-strength measurement" was the wrong surface, and this
   PR is what made it wrong: a chained prompt is deliberately written so
   the target's *description* cannot fire on it, so the number measures
   the pointer in the first skill's body and nothing else. Left as-is it
   would send the next person tuning a red hand-off at the description.

3. "an 0.8 bar" -> "a 0.8 bar".

Re-measured: 100% / 100% hand-off, 63/63 first-move at 100%, RUNS=5.

Staged by explicit path, not `git add -A` — see ec6895a5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EQVdemwgUhh45yzeaRSLhW
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-05 18:03:37 -04:00
commit 0d947ad569
1 changed files with 31 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

2
scripts/skill-eval.mjs
View file

@ -62,7 +62,7 @@ const THRESHOLD = Number(process.env.THRESHOLD ?? 0.8);
// `test-servers` cases measured 33% (RUNS=3) when `testing` merely classified
// which work belonged to `test-servers`, and 100%/100% (RUNS=5) once #2247
// reshaped that into an imperative step. (An intermediate build of that change
// measured 100%/80%. That is NOT an example of clearing an 0.8 bar — this
// measured 100%/80%. That is NOT an example of clearing a 0.8 bar — this
// threshold is compared strictly, so 80% would fail one — but it is worth
// knowing as the residual noise still present at RUNS=5.) What 0.5 buys
// is a column that still separates an adequate pointer from a broken one — a