chore: address Copilot review round 3

- CI now installs a pinned `claude` CLI and runs `claude plugin validate` as its
  own step. The guard skips the authoritative validator when the CLI is absent,
  which without this meant "always, in CI" — so the acceptance criterion was
  aspirational. Verified it needs no authentication (clean HOME).
- skill-eval: classify a run from its terminal `result` event, not its exit
  code. With `--max-turns 1` a run in which a skill FIRES necessarily hits the
  limit and the CLI exits 1 (`error_max_turns`, `num_turns: 2`), so last round's
  blanket nonzero rejection threw away exactly the observations the eval counts.
  A run with no terminal `result`, or an unusable subtype, still rejects.
- skill-eval: fail collection on a skill that does not parse, a model-invoked
  skill with no eval file, unreadable JSON, or invalid cases. Skipping them let
  `skills:eval` exit 0 while silently omitting the skill just broken.
- verify-skills: `main()` takes an optional skills directory, and
  `verify-skills.main.test.mjs` drives it over fixtures — missing SKILL.md,
  malformed frontmatter, undeclared invocation mode, missing/invalid evals,
  budget overflow (and that name-only skills do not count toward it), and
  multi-offender reporting. Matches the sibling guards' *.main.test.mjs
  precedent; without it the gate could stop enforcing while the parser tests
  stayed green.
- issue-create: the note claimed an issue arrives unlabeled, contradicting
  .github/ISSUE_TEMPLATE — blank issues are disabled, the bug form applies
  `bug`, the feature form applies `enhancement` + `v2`. Now a table of what each
  form sets and what triage still has to add.

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-29 01:26:19 -04:00
commit 7e0bb8b58d

View file

@ -52,6 +52,21 @@ jobs:
# jobs, but that's a larger restructure and deliberately out of scope.
run: npm run validate
- name: Validate the skills with the authoritative CLI (#2163)
# `npm run validate` above already ran `verify:skills`, whose own parser
# reads each SKILL.md the way Claude Code does. This is the second
# opinion: `claude plugin validate` is the authoritative schema, and the
# guard skips it whenever the CLI is absent — which, without this step,
# would mean always, in CI. Installing it here is what makes the
# acceptance criterion true rather than aspirational.
#
# Pinned rather than @latest: an unpinned validator can start failing a
# PR that changed nothing, which is how a gate loses its credibility.
# It needs no authentication — verified against a clean HOME.
run: |
npm install -g @anthropic-ai/claude-code@2.1.250
claude plugin validate .claude/skills
- name: Verify the browser-externalized-builtin build gate (#1769)
# Runs a real `vite build` with a Node built-in forced into the browser
# graph and asserts the build FAILS via the #1769 gate. The unit tests