Adds a multi-name filter to `npm run skills:eval` so a set of related skills can be iterated together, and makes an unknown name a hard error rather than an empty run — a typo would otherwise report a green 0/0 that reads exactly like a clean pass of the skill that was meant. Re-aims two `local-dev` cases and one `test-servers` case that partially overlapped `AGENTS.md`. Partial overlap oscillates across runs rather than sitting at zero, so it reads as eval noise instead of a defect in the case; `docs/skill-authoring.md` now records that failure mode and the observed scores. Widens the `test-servers` description to the "Use when …" shape and points the `testing` skill at it, since anything needing a real server to run against is a procedure `testing` does not carry. That growth moves the skill listing from 2,921 to 3,144 of 4,000 characters, so the figure recorded in AGENTS.md is updated to match. Signed-off-by: cliffhall <cliff@futurescale.com>
30 lines
871 B
JSON
30 lines
871 B
JSON
[
|
|
{
|
|
"prompt": "I want to reproduce the OAuth token revocation behaviour by hand. Which test server do I run?",
|
|
"expect": "test-servers"
|
|
},
|
|
{
|
|
"prompt": "How do I start one of the composable MCP test servers with a config file?",
|
|
"expect": "test-servers"
|
|
},
|
|
{
|
|
"prompt": "I need a real server to exercise this change against by hand. What do I run?",
|
|
"expect": "test-servers"
|
|
},
|
|
{
|
|
"prompt": "Which protocol era should I connect with when I run one of these fixtures by hand?",
|
|
"expect": "test-servers"
|
|
},
|
|
{
|
|
"prompt": "I need a fixture combination that doesn't exist yet. How do I add one?",
|
|
"expect": "test-servers"
|
|
},
|
|
{
|
|
"prompt": "Sort this list alphabetically: banana, apple, cherry.",
|
|
"expect": null
|
|
},
|
|
{
|
|
"prompt": "What is the boiling point of water in Fahrenheit?",
|
|
"expect": null
|
|
}
|
|
]
|