diff --git a/.claude/skills/issue-create/SKILL.md b/.claude/skills/issue-create/SKILL.md index 2d2b43db..5fe54eca 100644 --- a/.claude/skills/issue-create/SKILL.md +++ b/.claude/skills/issue-create/SKILL.md @@ -1,7 +1,7 @@ --- name: issue-create -description: Create a tracked Inspector issue end to end — version label, type label, milestone, board card, Status and Priority. Use when filing a new issue, or when work discovered mid-task needs an issue of its own. -disable-model-invocation: true +description: Create a tracked issue in this repo end to end — version label, type label, milestone, board card, Status and Priority. Use when asked to file, create, open or raise an issue; when picking its labels or its milestone; when checking whether one already exists; or when work discovered mid-task needs an issue of its own. +disable-model-invocation: false --- # Creating an issue diff --git a/.claude/skills/issue-create/evals/evals.json b/.claude/skills/issue-create/evals/evals.json new file mode 100644 index 00000000..b701c81d --- /dev/null +++ b/.claude/skills/issue-create/evals/evals.json @@ -0,0 +1,22 @@ +[ + { + "prompt": "Create an issue titled 'TUI crashes on an empty catalog' and set it up the way this repo wants.", + "expect": "issue-create" + }, + { + "prompt": "Which milestone should a new issue go in, and how do I find the current one?", + "expect": "issue-create" + }, + { + "prompt": "Before I file this, how do I check nobody has reported it already?", + "expect": "issue-create" + }, + { + "prompt": "What is the chemical symbol for potassium?", + "expect": null + }, + { + "prompt": "Reverse this string: stressed", + "expect": null + } +] diff --git a/.claude/skills/issue-triage/SKILL.md b/.claude/skills/issue-triage/SKILL.md index e2137b8d..0511119e 100644 --- a/.claude/skills/issue-triage/SKILL.md +++ b/.claude/skills/issue-triage/SKILL.md @@ -1,7 +1,7 @@ --- name: issue-triage description: Triage unboarded Inspector issues — the two-pass sweep, the priority rubric and its score comment, and the board audit that checks the whole board against its invariants. Use for "triage new issues" or when scoring an issue's Priority. -disable-model-invocation: true +disable-model-invocation: false --- # Triaging issues diff --git a/.claude/skills/issue-triage/evals/evals.json b/.claude/skills/issue-triage/evals/evals.json new file mode 100644 index 00000000..067302f1 --- /dev/null +++ b/.claude/skills/issue-triage/evals/evals.json @@ -0,0 +1,22 @@ +[ + { + "prompt": "Triage the new issues.", + "expect": "issue-triage" + }, + { + "prompt": "What Priority should issue 2189 be given?", + "expect": "issue-triage" + }, + { + "prompt": "Audit the project board for anything that has drifted.", + "expect": "issue-triage" + }, + { + "prompt": "Who wrote the novel Moby-Dick?", + "expect": null + }, + { + "prompt": "Convert 45 degrees Celsius to Fahrenheit.", + "expect": null + } +] diff --git a/AGENTS.md b/AGENTS.md index 0e730f71..f1ba56a1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,8 +19,8 @@ users invoke them by name. | [`local-dev`](.claude/skills/local-dev/SKILL.md) | Install and run each client; the `@inspector/core` alias; and the **reasoning** behind Dependency placement below — what each rule defends against and how to tell you have hit one (the rules themselves stay here) | Model-invoked, or `/local-dev` | | [`project-structure`](.claude/skills/project-structure/SKILL.md) | Which client owns which surface, what is in `core/`, where a new file belongs | Model-invoked only | | [`testing`](.claude/skills/testing/SKILL.md) | Where a test file goes, which command runs it, the tiers, clearing the coverage gate, `renderWithMantine` | Model-invoked, or `/testing` | -| [`issue-create`](.claude/skills/issue-create/SKILL.md) | The five-step create flow: version label, type label, milestone, board card, Status + Priority | `/issue-create` | -| [`issue-triage`](.claude/skills/issue-triage/SKILL.md) | The two-pass sweep of unboarded issues, the priority rubric and its score comment, the board audit | `/issue-triage` | +| [`issue-create`](.claude/skills/issue-create/SKILL.md) | The five-step create flow: version label, type label, milestone, board card, Status + Priority | Model-invoked, or `/issue-create` | +| [`issue-triage`](.claude/skills/issue-triage/SKILL.md) | The two-pass sweep of unboarded issues, the priority rubric and its score comment, the board audit | Model-invoked, or `/issue-triage` | | [`board-ops`](.claude/skills/board-ops/SKILL.md) | `gh project` recipes and the field/option IDs for boards #28 and #11; the option-deletion hazard and its recovery | Model-invoked, or `/board-ops` | | [`pr-flow`](.claude/skills/pr-flow/SKILL.md) | Branch naming, DCO signoff, screenshots, opening the PR, requesting a Copilot review, responding, closing out | Model-invoked, or `/pr-flow` | | [`pre-push-gate`](.claude/skills/pre-push-gate/SKILL.md) | Running `npm run local:gate` and diagnosing a failing stage | Model-invoked, or `/pre-push-gate` | @@ -174,11 +174,21 @@ that from happening: skill at all: it is absent from the listing and the Skill tool refuses it. The costs are asymmetric — a spurious load costs ~250 characters, a missed one costs a wrong base branch or an unsigned commit — and the budget is not tight - (all ten skills would total ~2.8k of 4k). Reserve `true` for a procedure that - is genuinely only ever started deliberately, as `/release` is. + (nine of the ten are model-invoked today and total ~2.8k of 4k). Reserve + `true` for a procedure that is genuinely only ever started deliberately — + `release` is the only one left, because nobody cuts a release by implication. ⚠️ **A `true` skill cannot be reached by another skill either.** If a model-invocable skill says "see `/board-ops`", that pointer is a dead end for the model unless `board-ops` is model-invocable too. + ⚠️ **Flipping is not free, and the listing budget is not what costs.** Going + from three model-invoked skills to nine measurably *lowered* the trigger rate + of the ones already there: `project-structure` fell from 100% to 0% on two + cases (n=4) and `testing` from 3/5 to 2/5, while the six new skills all + measured 100% and every negative case stayed clean. So the ceiling is + attention, not characters — we were at 2.8k of a 4k budget throughout. Adding + a skill therefore has a cost paid by the *existing* ones, which only + `skills:eval` can see. **Re-run the full eval after any flip**, not just the + new skill's own cases. ⚠️ **`paths` is not a free win.** It looks like the deterministic option, and it does gate loading to matching files — but measured against the `testing` skill's own eval cases, adding it roughly **halved** the rate at which the same @@ -212,7 +222,7 @@ that from happening: overflows, and drops the least-invoked entries **first** — which are exactly the model-invoked skills that must fire on their own. `verify:skills` prints the current cost against the budget recorded in `scripts/lib/skill-manifest.mjs` - (2,256/4,000 characters as of this writing) and fails when it is exceeded. Raise + (2,834/4,000 characters as of this writing) and fails when it is exceeded. Raise the budget deliberately, or tighten a description; each entry is capped at 1,536 characters regardless, so **put the key use case first**.