docs: address Copilot review round 3 on #2203

Replaces the last `project-structure` case that `AGENTS.md` answers — a
helper used by the CLI and TUI belongs in `core/` per its own directory
tree. This was also the only case below 100% in the previous suite, so
the static reading and the score agreed. The replacement targets the
skill's first stated purpose, locating code, which `AGENTS.md` gives no
method for; it probed 3/3 and now measures 100%.

Extends the truncation guard's tests to `when_to_use`. The guard loops
over both fields but only `description` was pinned, so dropping the
second from that list would have gone unnoticed. The new case pairs a
quoted description with an unquoted `when_to_use` and asserts the
description is not implicated.

Scopes the negative-case requirement in the prose to model-invoked
skills; round 2 scoped the checklist but left this sentence saying
"every skill file".

Refreshes the stale `~2.8k of 4k` listing figure. The other instance is
past-tense about the three-to-nine experiment and stays, now said
explicitly so two numbers do not look like a disagreement.

Records the eval's real blind spot: all three cases review identified as
`AGENTS.md`-answerable were scoring 80-100% when flagged, so a high score
does not clear a case of overlap. The eval can show a case failing; it
cannot show a passing case is well-aimed.

RUNS=5 skills:eval: 63/63. local:gate passes, 34/34.

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-01 10:53:56 -04:00
commit 104a05dd7f

View file

@ -443,3 +443,30 @@ test("a model-invoked skill needs at least MIN_POSITIVE_CASES positives", () =>
/no positive case expects/,
);
});
test("the truncation guard covers `when_to_use`, not just `description`", () => {
// The guard loops over both fields, so a regression that dropped `when_to_use`
// from that list would be invisible to the description-only tests above.
const withHash = [
"---",
"name: alpha",
'description: "safe because quoted"',
"when_to_use: Use when filing against board #28 or board #11.",
"disable-model-invocation: false",
"---",
"body",
].join("\n");
const parsed = parseSkill("alpha", withHash);
assert.equal(parsed.whenToUse, "Use when filing against board");
assert.match(parsed.errors.join(" "), /`when_to_use` is truncated/);
// The description is quoted, so it must not be implicated.
assert.doesNotMatch(parsed.errors.join(" "), /`description` is truncated/);
const quoted = withHash.replace(
/^when_to_use: (.*)$/m,
(_m, w) => `when_to_use: "${w}"`,
);
const ok = parseSkill("alpha", quoted);
assert.deepEqual(ok.errors, []);
assert.equal(ok.whenToUse, "Use when filing against board #28 or board #11.");
});