inspector/.claude/skills/test-servers/evals/evals.json
cliffhall 0febaa0f6f docs: address Copilot review round 7 on #2203
Strip YAML properties (tags, anchors, in either order) before deciding
whether a description scalar is quoted — testing the first character alone
read `&copy "Counts # safely"` as unquoted and rejected a `#` the quotes
make harmless.

Count the positive-case floor by distinct prompt rather than by entry: the
floor exists for breadth, so five copies of one prompt cleared it while
exercising exactly the single trigger it is meant to look past. Compare on
collapsed, case-folded text so trivial edits do not read as a new situation.

Re-aim the two positive prompts that measured the wrong thing — the
project-structure case now poses a concrete "which piece handles what"
question, and the test-servers case names running a real MCP server.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SyBPs7caoMcyvnp9ArMBhV
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 12:21:09 -04:00

30 lines
910 B
JSON

[
{
"prompt": "I want to reproduce the OAuth token revocation behaviour by hand. Which test server do I run?",
"expect": "test-servers"
},
{
"prompt": "How do I start one of the composable MCP test servers with a config file?",
"expect": "test-servers"
},
{
"prompt": "I need to run a real MCP server locally to exercise this change by hand. Which one do I run, and how do I start it?",
"expect": "test-servers"
},
{
"prompt": "Which protocol era should I connect with when I run one of these fixtures by hand?",
"expect": "test-servers"
},
{
"prompt": "I need a fixture combination that doesn't exist yet. How do I add one?",
"expect": "test-servers"
},
{
"prompt": "Sort this list alphabetically: banana, apple, cherry.",
"expect": null
},
{
"prompt": "What is the boiling point of water in Fahrenheit?",
"expect": null
}
]