docs: address Copilot review round 3 on #2203
Replaces the last `project-structure` case that `AGENTS.md` answers — a helper used by the CLI and TUI belongs in `core/` per its own directory tree. This was also the only case below 100% in the previous suite, so the static reading and the score agreed. The replacement targets the skill's first stated purpose, locating code, which `AGENTS.md` gives no method for; it probed 3/3 and now measures 100%. Extends the truncation guard's tests to `when_to_use`. The guard loops over both fields but only `description` was pinned, so dropping the second from that list would have gone unnoticed. The new case pairs a quoted description with an unquoted `when_to_use` and asserts the description is not implicated. Scopes the negative-case requirement in the prose to model-invoked skills; round 2 scoped the checklist but left this sentence saying "every skill file". Refreshes the stale `~2.8k of 4k` listing figure. The other instance is past-tense about the three-to-nine experiment and stays, now said explicitly so two numbers do not look like a disagreement. Records the eval's real blind spot: all three cases review identified as `AGENTS.md`-answerable were scoring 80-100% when flagged, so a high score does not clear a case of overlap. The eval can show a case failing; it cannot show a passing case is well-aimed. RUNS=5 skills:eval: 63/63. local:gate passes, 34/34. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
880be3acdd
commit
104a05dd7f
1 changed files with 54 additions and 5 deletions
|
|
@ -443,3 +443,30 @@ test("a model-invoked skill needs at least MIN_POSITIVE_CASES positives", () =>
|
|||
/no positive case expects/,
|
||||
);
|
||||
});
|
||||
|
||||
test("the truncation guard covers `when_to_use`, not just `description`", () => {
|
||||
// The guard loops over both fields, so a regression that dropped `when_to_use`
|
||||
// from that list would be invisible to the description-only tests above.
|
||||
const withHash = [
|
||||
"---",
|
||||
"name: alpha",
|
||||
'description: "safe because quoted"',
|
||||
"when_to_use: Use when filing against board #28 or board #11.",
|
||||
"disable-model-invocation: false",
|
||||
"---",
|
||||
"body",
|
||||
].join("\n");
|
||||
const parsed = parseSkill("alpha", withHash);
|
||||
assert.equal(parsed.whenToUse, "Use when filing against board");
|
||||
assert.match(parsed.errors.join(" "), /`when_to_use` is truncated/);
|
||||
// The description is quoted, so it must not be implicated.
|
||||
assert.doesNotMatch(parsed.errors.join(" "), /`description` is truncated/);
|
||||
|
||||
const quoted = withHash.replace(
|
||||
/^when_to_use: (.*)$/m,
|
||||
(_m, w) => `when_to_use: "${w}"`,
|
||||
);
|
||||
const ok = parseSkill("alpha", quoted);
|
||||
assert.deepEqual(ok.errors, []);
|
||||
assert.equal(ok.whenToUse, "Use when filing against board #28 or board #11.");
|
||||
});
|
||||
|
|
|
|||
Loading…
Reference in a new issue