chore: extract AGENTS.md recipes into agent skills
Closes #2163 `AGENTS.md` was one 1,223-line contract loaded in full on every turn, whether the task was a one-line CSS fix or a release cut. This splits it: rules stay resident, procedures move to `.claude/skills/`. - Ten skills under `.claude/skills/`, seven `disable-model-invocation: true` (invoked by name only, so they cost no listing context and have no trigger question) and three model-invoked with committed eval cases. - `AGENTS.md` 1,223 → 380 lines: rules, a skills index, and a project tree reduced to one line per entry. The removed recipes are not duplicated. - `README.md` 811 → 105 lines, split into `docs/{architecture,quality-gate, test-servers,publishing,docker}.md`, and dropped from `CLAUDE.md`'s auto-load. - `.github/copilot-instructions.md` and its mirroring rule removed — Copilot reads `AGENTS.md`. - `npm run verify:skills` (in `validate`, so in `local:gate` and CI) parses each SKILL.md the way Claude Code does, requires an explicit invocation mode and eval cases for model-invoked skills, and budgets the listing. - `npm run skills:eval` runs the committed trigger cases headless. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall
parent
8c91ae8820
commit
73447c72d1
2 changed files with 3291 additions and 1927 deletions
157
.claude/skills/testing/SKILL.md
Normal file
157
.claude/skills/testing/SKILL.md
Normal file
|
|
@ -0,0 +1,157 @@
|
|||
---
|
||||
name: testing
|
||||
description: Testing procedures for this repo — which npm command runs which suite (web unit, web integration, Storybook, cli, tui, launcher, scripts), the test tiers from unit through smoke and pack:verify, where a new test file goes (side-by-side vs src/test/ vs __tests__/), diagnosing the per-file 90% coverage gate, and the renderWithMantine / renderWithMantineTransitions rules for React and Mantine tests. Use for any question about running, placing, or fixing a test here.
|
||||
disable-model-invocation: false
|
||||
---
|
||||
|
||||
# Testing
|
||||
|
||||
**Every change needs tests, and every file must clear ≥90% on lines,
|
||||
statements, functions, and branches.** That rule and the React/Mantine
|
||||
conventions live in [`AGENTS.md`](../../../AGENTS.md); this skill is where a
|
||||
test goes, how to run it, and how to clear the gate.
|
||||
|
||||
## Where the test file goes
|
||||
|
||||
**Side-by-side by default; `src/test/` only for what can't be co-located; and
|
||||
the Node clients are different.**
|
||||
|
||||
### `clients/web` — side-by-side
|
||||
|
||||
`<Name>.test.tsx` (or `.test.ts` for non-React modules) **next to the source**.
|
||||
Components, hooks, `lib/`, `utils/`. This is the overwhelming majority; a
|
||||
web-owned test living under `src/test/` instead is a bug.
|
||||
|
||||
`clients/web/src/test/` is for the three things that *cannot* be co-located:
|
||||
|
||||
1. **Tests of the repo-root `core/` package** → `src/test/core/…`, mirroring the
|
||||
`core/` folder layout. `core/` physically lives outside `clients/web/`, is
|
||||
consumed via the `@inspector/core` alias, and has no test harness of its own.
|
||||
This includes `core/json/*` and `core/client/*`.
|
||||
2. **The `integration` project** → `src/test/integration/…`, mirroring the
|
||||
`core/` source layout (`mcp/`, `mcp/node/`, `mcp/remote/`, `auth/`,
|
||||
`auth/node/`, `storage/`). **Placement is the manifest** — any file under that
|
||||
folder is picked up by the integration project (node env, 30s timeouts) via a
|
||||
folder glob; there is no enumeration to keep in sync.
|
||||
3. **Shared test infrastructure** — `renderWithMantine.tsx`, `setup.ts`,
|
||||
`fixtures/`, `scrollAreaStoryAssertions.ts`.
|
||||
|
||||
### `clients/cli`, `clients/tui`, `clients/launcher` — a top-level `__tests__/`
|
||||
|
||||
**All** their tests, not beside their source. Their `tsconfig.json` excludes
|
||||
`**/*.test.*` and their `tsconfig.test.json` includes `__tests__/**/*`, so a
|
||||
co-located `src/**/*.test.*` lands in **no** tsconfig project and fails
|
||||
`npm run verify:typecheck-coverage` (#1791).
|
||||
|
||||
### Root tooling — `scripts/*.test.mjs`
|
||||
|
||||
A new `scripts/*.mjs` helper with pure logic gets a sibling `*.test.mjs`, run by
|
||||
`npm run test:scripts` (node's built-in runner; the root has no vitest harness by
|
||||
design). ⚠️ Keep the filename `*.test.mjs` — `node --test` silently **skips** a
|
||||
file its glob misses and still exits 0.
|
||||
|
||||
## Running them
|
||||
|
||||
| Scope | From | Command |
|
||||
| --- | --- | --- |
|
||||
| Web unit | `clients/web` | `npm run test` (`test:watch` while iterating) |
|
||||
| Web integration | `clients/web` | `npm run test:integration` |
|
||||
| Web Storybook play fns | `clients/web` | `npm run test:storybook` |
|
||||
| CLI | `clients/cli` | `npm run test` (`pretest` builds test-servers + the bin) |
|
||||
| TUI | `clients/tui` | `npm run test` |
|
||||
| Launcher | `clients/launcher` | `npm run test` |
|
||||
| Root tooling | repo root | `npm run test:scripts` |
|
||||
| Everything, fast | repo root | `npm run validate` |
|
||||
| The coverage gate | repo root | `npm run coverage` |
|
||||
|
||||
There is **no aggregate root `test` script** — each client self-validates.
|
||||
|
||||
In unit tests that expect error output, **suppress it from the console**.
|
||||
|
||||
## The tiers, shallowest first
|
||||
|
||||
unit (`test`, per client) → web integration (`test:integration`, real
|
||||
transports/servers) → out-of-process (`clients/cli/__tests__/e2e.test.ts`,
|
||||
spawns the built binary) → smokes through the built launcher (`npm run smoke`) →
|
||||
Storybook play functions (`test:storybook`) → the published-tarball check
|
||||
(`npm run pack:verify`, local/release only — needs network).
|
||||
|
||||
`validate` runs the per-client `test` scripts — so web **unit** plus cli's
|
||||
out-of-process `e2e.test.ts`, but **not** web's integration project, which runs
|
||||
inside the `coverage` gate. CI therefore has no separate `test:integration` step.
|
||||
|
||||
## The coverage gate
|
||||
|
||||
**Per-file ≥90 on all four dimensions**, CI-enforced, across web, cli, tui and
|
||||
launcher. New code must clear 90 on every dimension.
|
||||
|
||||
Scope notes:
|
||||
|
||||
- The **web** coverage `include` (in `clients/web/vite.config.ts`) also covers
|
||||
the shared `core/` runtime the browser consumes — `core/mcp`, `core/react`,
|
||||
`core/auth`, `core/storage`, `core/logging`, `core/node`, `core/json`,
|
||||
`core/client`.
|
||||
- ⚠️ That `include` is a **whitelist** naming `components`/`hooks`/`theme`/
|
||||
`lib`/`utils`/`server`. A module placed **outside** those directories falls out
|
||||
of the gate entirely, silently. The documented exceptions are `src/App.tsx`
|
||||
(a composition root at ~42% branch coverage — gating it is a dedicated
|
||||
decomposition effort) and the `src/main.tsx` / `src/index.ts` bootstraps.
|
||||
- **CLI** tests run **in-process** by importing `runCli()`
|
||||
(`__tests__/helpers/cli-runner.ts`) so `src` is measured; `src/index.ts` is the
|
||||
only exclusion. `commander` uses `.exitOverride()` so a parse error throws
|
||||
instead of tearing down the test worker.
|
||||
- **TUI** covers **all of `src/**`, React surface included**. Components mount
|
||||
through `ink-testing-library` with the passthrough doubles in
|
||||
`__tests__/helpers/`; keypresses are driven through stdin. The only exclusion
|
||||
is `src/tui-servers.ts` (a pure re-export, excluded so it doesn't surface as a
|
||||
misleading 0/0 row).
|
||||
|
||||
### When a `v8 ignore` is justified
|
||||
|
||||
A genuinely-unreachable branch is annotated at the source rather than waved
|
||||
through by lowering the gate. The acceptable reasons are enumerated in
|
||||
[`AGENTS.md`](../../../AGENTS.md) — do not reach for one that is not on that
|
||||
list.
|
||||
|
||||
## React tests: `renderWithMantine`
|
||||
|
||||
**Always render through `renderWithMantine`** (`src/test/renderWithMantine.tsx`)
|
||||
— it wraps in `MantineProvider` with the project theme. Do not hand-roll a bare
|
||||
`MantineProvider`; it skips the project theme and the helper's options and drifts
|
||||
from every other test.
|
||||
|
||||
Note the justification has changed and the old one is wrong: it sets
|
||||
`env="test"`, which makes Mantine skip the animated **render** — but it does
|
||||
**not** stop the timers. `env` is read only at `Transition.mjs`'s render branch,
|
||||
while `useTransition()` runs before that check and still schedules real
|
||||
`window.setTimeout`s (opening a `<Modal>` schedules three 200ms timers). What
|
||||
prevents a timer outliving its file — and throwing an uncaught
|
||||
`ReferenceError: window is not defined` that fails the **whole run**, attributed
|
||||
to an innocent file (#1760) — is the **leaked-timer safety net in
|
||||
`src/test/setup.ts`**, which is global and covers every unit test however it
|
||||
renders. The rule stands on consistency, not on timer safety.
|
||||
|
||||
- **Forced color scheme:** pass the option —
|
||||
`renderWithMantine(ui, { colorScheme: "dark" })` — rather than hand-rolling
|
||||
`defaultColorScheme="dark"`.
|
||||
- **Mid-flight transition state** (e.g. asserting a `data-anim="out"` cell during
|
||||
an exit crossfade) is the only reason to use `renderWithMantineTransitions`.
|
||||
Pass `settleMs` derived from the component's real animation duration
|
||||
(`HEADER_ANIM_MS + 200`), do **not** also use `vi.useFakeTimers()` in that test
|
||||
(the auto-settle no-ops under fake timers), and if the test unmounts the tree
|
||||
itself use the `unmount()` the helper returns. The mechanism is documented at
|
||||
length on the helper — read there before changing it.
|
||||
|
||||
## Storybook play functions
|
||||
|
||||
Every screen and element component has a `*.stories.tsx`; play functions double
|
||||
as interaction tests, run headless in CI and in the local gate.
|
||||
|
||||
⚠️ **`expect(...)` from `storybook/test` returns a promise.** Storybook
|
||||
instruments it, so every `expect` in a play function is awaited — as is any
|
||||
shared helper that wraps one.
|
||||
|
||||
## Test servers, not mocks
|
||||
|
||||
Integration and smoke tests drive a real server over a real transport. See
|
||||
`/test-servers` for picking and building one.
|
||||
22
.claude/skills/testing/evals/evals.json
Normal file
22
.claude/skills/testing/evals/evals.json
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
[
|
||||
{
|
||||
"prompt": "I just added clients/tui/__tests__/serverList.test.ts. What command runs only the TUI suite?",
|
||||
"expect": "testing"
|
||||
},
|
||||
{
|
||||
"prompt": "In clients/web/src/components/Foo.test.tsx I need to assert on a Mantine transition mid-flight. How?",
|
||||
"expect": "testing"
|
||||
},
|
||||
{
|
||||
"prompt": "Which test tier in this repo spawns the built binary rather than importing it?",
|
||||
"expect": "testing"
|
||||
},
|
||||
{
|
||||
"prompt": "What does this regex do? /^\\d{3}-\\d{4}$/",
|
||||
"expect": null
|
||||
},
|
||||
{
|
||||
"prompt": "Rename the variable `foo` to `bar` in this snippet.",
|
||||
"expect": null
|
||||
}
|
||||
]
|
||||
Loading…
Reference in a new issue