docs: accept several skill names in skills:eval and re-aim overlapping cases
Adds a multi-name filter to `npm run skills:eval` so a set of related skills can be iterated together, and makes an unknown name a hard error rather than an empty run — a typo would otherwise report a green 0/0 that reads exactly like a clean pass of the skill that was meant. Re-aims two `local-dev` cases and one `test-servers` case that partially overlapped `AGENTS.md`. Partial overlap oscillates across runs rather than sitting at zero, so it reads as eval noise instead of a defect in the case; `docs/skill-authoring.md` now records that failure mode and the observed scores. Widens the `test-servers` description to the "Use when …" shape and points the `testing` skill at it, since anything needing a real server to run against is a procedure `testing` does not carry. That growth moves the skill listing from 2,921 to 3,144 of 4,000 characters, so the figure recorded in AGENTS.md is updated to match. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
6dd43e78ba
commit
a344225e23
4 changed files with 100 additions and 12 deletions
|
|
@ -4,7 +4,7 @@
|
|||
"expect": "local-dev"
|
||||
},
|
||||
{
|
||||
"prompt": "I added a dependency and the TUI bundle broke at import time. Where should it have gone?",
|
||||
"prompt": "I added a package and now one of the client bundles fails at run time. What do I check?",
|
||||
"expect": "local-dev"
|
||||
},
|
||||
{
|
||||
|
|
@ -16,7 +16,7 @@
|
|||
"expect": "local-dev"
|
||||
},
|
||||
{
|
||||
"prompt": "I just pulled and a client's dependencies look out of sync. What do I need to run?",
|
||||
"prompt": "Do I need to do anything special to set this repo up inside a git worktree?",
|
||||
"expect": "local-dev"
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
name: test-servers
|
||||
description: Run a composable MCP test server by hand — pick the showcase config that reproduces a given feature or bug, build test-servers, and connect with the right protocol era. Covers the stale-build hazard.
|
||||
description: Run a composable MCP test server by hand — pick the showcase config for a feature or bug, build it, and connect with the right protocol era. Use when a change, a PR or a smoke test needs a real server to exercise it; when reproducing a reported bug by hand; when choosing which fixture or protocol era to run; when a fixture keeps serving stale code after an edit; or when the config or preset you need does not exist yet.
|
||||
disable-model-invocation: false
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
"expect": "test-servers"
|
||||
},
|
||||
{
|
||||
"prompt": "I renamed a file under test-servers/src and the fixture is still running the old code. Why?",
|
||||
"prompt": "I need a real server to exercise this change against by hand. What do I run?",
|
||||
"expect": "test-servers"
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -11,6 +11,12 @@ statements, functions, and branches.** That rule and the React/Mantine
|
|||
conventions live in [`AGENTS.md`](../../../AGENTS.md); this skill is where a
|
||||
test goes, how to run it, and how to clear the gate.
|
||||
|
||||
⚠️ **Anything that needs a real server to run against — an integration test, a
|
||||
smoke, reproducing a bug by hand — is `/test-servers`, and you have to load it.**
|
||||
Integration and smoke tests here drive a real server over a real transport
|
||||
rather than a mock, so picking, building and connecting to a fixture is a
|
||||
procedure of its own that this skill does not carry.
|
||||
|
||||
## Where the test file goes
|
||||
|
||||
**Side-by-side by default; `src/test/` only for what can't be co-located; and
|
||||
|
|
|
|||
Loading…
Reference in a new issue