chore: address Copilot review round 3

- CI now installs a pinned `claude` CLI and runs `claude plugin validate` as its
  own step. The guard skips the authoritative validator when the CLI is absent,
  which without this meant "always, in CI" — so the acceptance criterion was
  aspirational. Verified it needs no authentication (clean HOME).
- skill-eval: classify a run from its terminal `result` event, not its exit
  code. With `--max-turns 1` a run in which a skill FIRES necessarily hits the
  limit and the CLI exits 1 (`error_max_turns`, `num_turns: 2`), so last round's
  blanket nonzero rejection threw away exactly the observations the eval counts.
  A run with no terminal `result`, or an unusable subtype, still rejects.
- skill-eval: fail collection on a skill that does not parse, a model-invoked
  skill with no eval file, unreadable JSON, or invalid cases. Skipping them let
  `skills:eval` exit 0 while silently omitting the skill just broken.
- verify-skills: `main()` takes an optional skills directory, and
  `verify-skills.main.test.mjs` drives it over fixtures — missing SKILL.md,
  malformed frontmatter, undeclared invocation mode, missing/invalid evals,
  budget overflow (and that name-only skills do not count toward it), and
  multi-offender reporting. Matches the sibling guards' *.main.test.mjs
  precedent; without it the gate could stop enforcing while the parser tests
  stayed green.
- issue-create: the note claimed an issue arrives unlabeled, contradicting
  .github/ISSUE_TEMPLATE — blank issues are disabled, the bug form applies
  `bug`, the feature form applies `enhancement` + `v2`. Now a table of what each
  form sets and what triage still has to add.

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-29 01:26:19 -04:00
commit 7e0bb8b58d

View file

@ -102,8 +102,22 @@ The priority rubric is in `/issue-triage`.
## Note on issues that arrive from elsewhere
An issue opened by hand in the GitHub UI — by an outside reporter *or* by a
maintainer — arrives with no label, no milestone, and no card. That is normal on
arrival, not a defect to fix the moment it lands: it comes into the system
through `/issue-triage` instead, and starts in **Incoming** with no milestone,
because nobody has approved it.
An issue opened through the GitHub UI — by an outside reporter *or* by a
maintainer — arrives with **no milestone and no card**, and that is normal on
arrival rather than a defect to fix the moment it lands: it comes into the
system through `/issue-triage` instead, and starts in **Incoming**, because
nobody has approved it.
Its **labels are partly set already**, which the triage pass has to account for:
blank issues are disabled, so it came through a form in
[`.github/ISSUE_TEMPLATE/`](../../../.github/ISSUE_TEMPLATE) (Copilot).
| Form | Applies | Still missing |
| --- | --- | --- |
| Bug report | `bug` | the version label — the form carries a version-line **dropdown** a maintainer reads at triage, because a `labels:` list is static and GitHub cannot map an answer to a label |
| Feature request | `enhancement`, `v2` | nothing — v1 takes security fixes only and cannot receive a feature, so `v2` is correct by construction |
So a bug report needs its version label applied from the dropdown answer, and a
feature request usually needs no label work at all. ⚠️ GitHub serves the chooser
from the **default branch**, so a form edited on `v2/main` does not take effect
until the next milestone merge.