modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 3

chore: tighten the testing skill's description

Browse source 
Measured before and after at the same sample size (n=4), with the six other
model-invoked skills unchanged:

  which command runs the TUI suite      25% → 100%
  asserting on a Mantine transition     50% → 100%
  which tier spawns the built binary    25% → 100%

The lever is the description's SHAPE, not its length: leading with the actions
and then enumerating concrete situations ("Use when … ; when … ; when …") beats
a noun-phrase list of contents ending in a generic "use for any question
about …". The same edit took `pre-push-gate` from 3/5 to 5/5 earlier.

Re-ran the full eval to check the change did not simply move attention off
another skill, which is the failure the previous commit recorded: 40/46, up from
38/46, and the failing set is strictly smaller — `testing` went from three
failures to one, nothing else moved. That remaining case measured 100% in
isolation and 67% in the full run, which is sampling variance rather than a
finding.

AGENTS.md now extends the re-run rule to cover a description edit, not just a
flip, and records the shape that works.

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-30 00:53:07 -04:00
commit f95abb2715
1 changed files with 8 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

9
AGENTS.md
View file

@ -187,8 +187,13 @@ that from happening:
measured 100% and every negative case stayed clean. So the ceiling is
attention, not characters — we were at 2.8k of a 4k budget throughout. Adding
a skill therefore has a cost paid by the *existing* ones, which only
`skills:eval` can see. **Re-run the full eval after any flip**, not just the
new skill's own cases.
`skills:eval` can see. **Re-run the full eval after any flip _or description
edit_**, not just the changed skill's own cases.
The lever that works is the description's *shape*. Leading with the actions
and then enumerating concrete situations ("Use when … ; when … ; when …")
beats a noun-phrase list of contents: it took `pre-push-gate` from 3/5 to 5/5
and `testing`'s three cases from 25/50/25% to 100% each (n=4), displacing
nothing else.
⚠️ **`paths` is not a free win.** It looks like the deterministic option, and
it does gate loading to matching files — but measured against the `testing`
skill's own eval cases, adding it roughly **halved** the rate at which the same