Corrects the rationale for the five-positive floor, which was wrong. The claim was that five prompts make a skill's rate flake-tolerant. The evaluator scores each prompt on its own `passes / RUNS`, so the number of prompts never enters any rate and cannot affect a threshold crossing — `RUNS` is the only knob that does. The floor is worth keeping for breadth: one prompt measures one phrasing, and a description that fires on a single narrow shape would pass on its one lucky case. The bad reasoning had reached five places — AGENTS.md, the authoring checklist, the JSDoc, the check-site comment and the regression test — plus the emitted message, which asserted it too. All corrected. Moves `MIN_POSITIVE_CASES` above `validateEvalCases`'s JSDoc. It had been inserted between the two, so the `@param`/`@returns` block documented the constant and the function had none. Replaces the two remaining cases answerable from `AGENTS.md`: `local-dev`'s web-client-startup case (the rules file gives the exact command) and `project-structure`'s clients/web-versus-core case (its tree answers it). Both replacements probed 3/3 and measure 100%. RUNS=5 skills:eval: 63/63, every local-dev and project-structure case at 100%. local:gate passes, 34/34. Signed-off-by: cliffhall <cliff@futurescale.com>
30 lines
971 B
JSON
30 lines
971 B
JSON
[
|
|
{
|
|
"prompt": "This repo has three clients over a shared core. How should I orient myself in it before changing anything?",
|
|
"expect": "project-structure"
|
|
},
|
|
{
|
|
"prompt": "I am adding a module that both the browser side and the Node side need to run. How do I decide where it goes in this repo?",
|
|
"expect": "project-structure"
|
|
},
|
|
{
|
|
"prompt": "Does browser-only transport code belong in core or in the web client?",
|
|
"expect": "project-structure"
|
|
},
|
|
{
|
|
"prompt": "I have a new file and I'm not sure where it goes. How do I decide in this repo?",
|
|
"expect": "project-structure"
|
|
},
|
|
{
|
|
"prompt": "I need to find where a feature is implemented but I do not know which client owns it. How do I work that out in this repo?",
|
|
"expect": "project-structure"
|
|
},
|
|
{
|
|
"prompt": "Convert this array of strings to uppercase.",
|
|
"expect": null
|
|
},
|
|
{
|
|
"prompt": "What is the capital of France?",
|
|
"expect": null
|
|
}
|
|
]
|