chore: address Copilot review round 3
- CI now installs a pinned `claude` CLI and runs `claude plugin validate` as its own step. The guard skips the authoritative validator when the CLI is absent, which without this meant "always, in CI" — so the acceptance criterion was aspirational. Verified it needs no authentication (clean HOME). - skill-eval: classify a run from its terminal `result` event, not its exit code. With `--max-turns 1` a run in which a skill FIRES necessarily hits the limit and the CLI exits 1 (`error_max_turns`, `num_turns: 2`), so last round's blanket nonzero rejection threw away exactly the observations the eval counts. A run with no terminal `result`, or an unusable subtype, still rejects. - skill-eval: fail collection on a skill that does not parse, a model-invoked skill with no eval file, unreadable JSON, or invalid cases. Skipping them let `skills:eval` exit 0 while silently omitting the skill just broken. - verify-skills: `main()` takes an optional skills directory, and `verify-skills.main.test.mjs` drives it over fixtures — missing SKILL.md, malformed frontmatter, undeclared invocation mode, missing/invalid evals, budget overflow (and that name-only skills do not count toward it), and multi-offender reporting. Matches the sibling guards' *.main.test.mjs precedent; without it the gate could stop enforcing while the parser tests stayed green. - issue-create: the note claimed an issue arrives unlabeled, contradicting .github/ISSUE_TEMPLATE — blank issues are disabled, the bug form applies `bug`, the feature form applies `enhancement` + `v2`. Now a table of what each form sets and what triage still has to add. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
ca6828916f
commit
7e0bb8b58d
1 changed files with 357 additions and 32 deletions
|
|
@ -41,7 +41,7 @@ import {
|
|||
} from "./lib/skill-manifest.mjs";
|
||||
|
||||
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const SKILLS_DIR = path.join(ROOT, ".claude", "skills");
|
||||
const DEFAULT_SKILLS_DIR = path.join(ROOT, ".claude", "skills");
|
||||
|
||||
/** Directory names under `.claude/skills` that contain a SKILL.md. */
|
||||
export function skillDirs(
|
||||
|
|
@ -54,7 +54,12 @@ export function skillDirs(
|
|||
.sort();
|
||||
}
|
||||
|
||||
function main() {
|
||||
function main(argv = process.argv.slice(2)) {
|
||||
// An explicit directory is the seam the fixture tests drive; without it this
|
||||
// guard could only ever be exercised against the repo's own (green) skills,
|
||||
// so it could stop enforcing anything while every unit test stayed green.
|
||||
const override = argv[0];
|
||||
const SKILLS_DIR = override ? path.resolve(override) : DEFAULT_SKILLS_DIR;
|
||||
const failures = [];
|
||||
|
||||
if (!existsSync(SKILLS_DIR)) {
|
||||
|
|
@ -124,6 +129,8 @@ function main() {
|
|||
`(${modelInvoked.join(", ")}); listing ${cost}/${LISTING_BUDGET} chars.`,
|
||||
);
|
||||
|
||||
if (override) return;
|
||||
|
||||
// The authoritative parse, when it is available — strictly a bonus. An older
|
||||
// CLI does not have `plugin validate` and exits nonzero on it as an unknown
|
||||
// command, which would fail the MANDATORY gate for a contributor whose only
|
||||
|
|
|
|||
Loading…
Reference in a new issue