Two factual corrections, both suppressed comments.
- `.claude/skills/local-dev/SKILL.md`: the react/react-dom exception cited
"19.2.8 today, against the root's 19.2.7" — a skew this PR removes. Dropped
the version pair; the point being made is about which install resolves the
pair, and it stands without a number that goes stale on the next bump.
- `verify-dep-lockstep.mjs`: the `devDependencies` bullet claimed all four
surfaced skews were a devDependency somewhere. `react` is a root runtime
dependency whose client copies are peer-installed, so it is three of four —
which is a better illustration of why the union spans fields as well as
installs, and the bullet now says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J86a6vMcA9csPtdsg7ZcCZ
Signed-off-by: cliffhall <cliff@futurescale.com>
- Include `optionalDependencies` in the declared tier's candidate union. A
peer range constrains the consumer's host; an optional range is a direct
declaration npm attempts to install, so omitting it left an optional-only
package free to skew unseen. `peerDependencies` stays excluded, with the
reason stated.
- Make the end-to-end fixture's client copy UNDECLARED — only the root
manifest names `solo`, while `clients/web` merely holds a top-level copy.
That is #2226's defining shape, and it is what makes the candidate set a
union across installs rather than a per-install intersection; the old
fixture would have passed a per-install implementation. Adds the mirror
case (declared only by a client, held by the root) so the union is pinned
in both directions.
- Update the two docs that still described the single-tier guard:
`.claude/skills/local-dev/SKILL.md` said "nothing gates either of those,
and `verify:dep-lockstep` is not it" and cited the live `@types/node`
drift in the present tense; `docs/quality-gate.md` described only the
program tier. Both now state what each tier covers and the two limits
that remain.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J86a6vMcA9csPtdsg7ZcCZ
Signed-off-by: cliffhall <cliff@futurescale.com>
Corrects the rationale for the five-positive floor, which was wrong. The
claim was that five prompts make a skill's rate flake-tolerant. The
evaluator scores each prompt on its own `passes / RUNS`, so the number of
prompts never enters any rate and cannot affect a threshold crossing —
`RUNS` is the only knob that does. The floor is worth keeping for
breadth: one prompt measures one phrasing, and a description that fires
on a single narrow shape would pass on its one lucky case.
The bad reasoning had reached five places — AGENTS.md, the authoring
checklist, the JSDoc, the check-site comment and the regression test —
plus the emitted message, which asserted it too. All corrected.
Moves `MIN_POSITIVE_CASES` above `validateEvalCases`'s JSDoc. It had been
inserted between the two, so the `@param`/`@returns` block documented the
constant and the function had none.
Replaces the two remaining cases answerable from `AGENTS.md`: `local-dev`'s
web-client-startup case (the rules file gives the exact command) and
`project-structure`'s clients/web-versus-core case (its tree answers it).
Both replacements probed 3/3 and measure 100%.
RUNS=5 skills:eval: 63/63, every local-dev and project-structure case at
100%. local:gate passes, 34/34.
Signed-off-by: cliffhall <cliff@futurescale.com>
Adds a multi-name filter to `npm run skills:eval` so a set of related
skills can be iterated together, and makes an unknown name a hard error
rather than an empty run — a typo would otherwise report a green 0/0
that reads exactly like a clean pass of the skill that was meant.
Re-aims two `local-dev` cases and one `test-servers` case that partially
overlapped `AGENTS.md`. Partial overlap oscillates across runs rather
than sitting at zero, so it reads as eval noise instead of a defect in
the case; `docs/skill-authoring.md` now records that failure mode and
the observed scores.
Widens the `test-servers` description to the "Use when …" shape and
points the `testing` skill at it, since anything needing a real server
to run against is a procedure `testing` does not carry. That growth
moves the skill listing from 2,921 to 3,144 of 4,000 characters, so the
figure recorded in AGENTS.md is updated to match.
Signed-off-by: cliffhall <cliff@futurescale.com>
`skills:eval` was at 38/46 with several skills measuring 0-33%. The low
scores were mostly bad eval cases rather than bad descriptions, for two
reasons that were nowhere written down:
- The harness runs each case with `--max-turns 1`, so the skill has to be
the model's FIRST tool call. A prompt asking "how does X work here", or
naming a concrete file, sends the model to Read/Grep first — a better
answer to the question, scored as a miss. Every 0% case was one of these.
- `AGENTS.md` is resident on every turn, so a prompt it already answers is
answered without any skill and the case can never pass.
Replace the cases that measured the wrong thing, expand every model-invoked
skill from 3 to 5 positives (negatives kept) so one flake cannot cross the
threshold, and reshape `local-dev`'s description to the action-first form.
Add `docs/skill-authoring.md` with the case shapes that fire, the ones that
cannot, and the probe-then-measure loop.
RUNS=5 npm run skills:eval: 63/63 at or above threshold, 60 at 100%,
all 18 negatives clean.
Closes#2202
Signed-off-by: cliffhall <cliff@futurescale.com>
Two suppressed comments, both accurate.
- Pin zod, zod/v4 and open to the root in vitest.shared.mts, and add matching
paths to the three clients/web tsconfigs. The claim they replace — that these
resolve unambiguously by walk-up — was wrong: both still sit at the top level
of clients/web/node_modules as transitive copies (zod under
eslint-plugin-react-hooks, open under Storybook), so an unpinned bare import
from a web test resolved the client copy while core/ and the SDK packages
resolved the root's. For open that is only wasteful; for zod it is a schema
built by one copy and an instanceof check made by the other across the whole
@modelcontextprotocol/* surface. zod is deduped as well, and zod/v4 gets its
own entry because first-party code imports both specifiers and pinning one
would split the package rather than collapse it.
- Update the consumer comments that #2195 falsified. clients/web/vite.config.ts
(three places) and server/start-vite-dev-server.ts all still said the shared
aliases resolve pino/hono and friends from clients/web/node_modules; they now
state the split — react and react-dom from the client install, everything
root-declared from the repo root.
- local-dev skill: replace the wrong 'resolves unambiguously' list with the
distinction that actually matters. Deleting a declaration is not the same as
the package resolving from the root, because chokidar, zod, open and react
all sit in a client install transitively; check the install, not the manifest.
ajv, commander and undici carry no pin because no client install has a copy.
npm run local:gate passes from a cold tsc build.
Signed-off-by: cliffhall <cliff@futurescale.com>
- clients/web/tsconfig.{app,node,test}.json: repoint the pino, hono,
hono/streaming, @hono/node-server, atomically, chokidar and @napi-rs/keyring
path mappings from ./node_modules to ../../node_modules. The stale mappings
were not failing the build — an unresolvable paths entry falls back to
ordinary walk-up resolution, which finds the root copy — but chokidar is the
case where the fallback never runs: Vite's transitive copy still sits under
clients/web, so tsc was type-checking against the client's copy while Vitest
and the shipped bundle both resolved the root's. Same version today, silent
skew tomorrow. Both path prefixes now carry a comment saying which ownership
each one asserts.
- local-dev skill: the consolidation rationale no longer groups commander, open
and @hono/node-server (client-imported) with the core/ set, no longer claims
#2195 moved undici (it was already root-only), and no longer implies every
root-declared package has an alias.
npm run local:gate passes with clients/web/node_modules/.tmp cleared, so the
tsc results are from a cold build rather than an incremental cache.
Signed-off-by: cliffhall <cliff@futurescale.com>
- Complete the bundler external lists against the AGENTS.md invariant rather
than against today's reachability: chokidar, hono and react to clients/cli,
chokidar and hono to clients/tui, react to the web runner. Every root-declared
package core/ imports at runtime is now named in all three, so a new import in
core/ cannot silently inline one.
- AGENTS.md: split the single over-broad bullet in two. commander, open,
@hono/node-server, vite and @vitejs/plugin-react are root dependencies reached
only from client code, so they are not part of the core/ set and need naming
only in the external list of the client that imports them. Give the
vitest.shared.mts pins their own rule, stating the react/react-dom exception
instead of implying every root-owned package is aliased to the root.
- vitest.shared.mts: the umbrella comment now says root-OWNED rather than root
runtime dependency — express is test-only and lives in the root
devDependencies — and notes that the list is deliberately not the complete
root set, since a package that resolves unambiguously needs no pin.
- local-dev skill: document why react/react-dom stay client-pinned, which the
previous text contradicted.
npm run local:gate and npm run verify:bundle-externals pass.
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 3. Docs only.
The round-2 wording said every surviving copy is pinned by its holder's peer
range, in the same paragraph that documents cli's @types/node 24.13.1 against
the root's 24.13.3 — a transitive, which no peer range constrains. The rationale
contradicted its own example.
Both files now split the two mechanisms and grade them:
- A peer copy is constrained by its holder's peer range, which is a real pin only
when that range is exact. @vitest/browser-playwright pins vitest to a single
version (hence the exact trio). eslint-plugin-react-refresh accepts ^9 || ^10,
so the eslint copies agree only because npm resolves the same latest in both
installs — a coincidence, not a guarantee.
- A transitive copy is constrained by nothing of ours, and cli's @types/node has
already diverged. It was 24.13.1 on v2/main too: a declared ^24.12.4 still
loses to a nearer transitive.
Nothing gates either, and verify:dep-lockstep is not that gate.
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 2. Docs only; no manifest or lockfile change.
Two claims were wrong:
- "A tool one client runs stays in that client" is contradicted by the
manifests — tsup is declared in web, cli and tui, and vite in web and tui on
top of the root runtime dependency that --web --dev needs. The boundary is
"used by every client", not "used by one"; anything narrower stays put whether
one client declares it or several do. Both are out of scope here and say so.
- verify:dep-lockstep was named as the guard that the surviving copies agree. It
is not: it derives its candidate set from what each tsc program resolves, so a
tool binary never enters a program and is outside the set no matter how far it
drifts — and cli's @types/node 24.13.1 against the root's 24.13.3 is a live
skew that passes for the same reason. Both files now state the narrower
guarantee, say that nothing gates the binaries automatically, note that each
surviving copy is pinned by its holder's peer range, and give
`npm exec -- which` as the manual check.
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 1.
@vitest/browser-playwright declares an EXACT peer on vitest ("vitest":
"4.1.10"), so that package — not the root's range — decides which vitest lands in
clients/web. Dropping web's own vitest declaration left the root free to resolve
4.1.11 while web's peer stayed at 4.1.10, so web's tests ran on one vitest while
loading a @vitest/coverage-v8 provider built against another. Both still passed,
which is the bad part.
vitest and @vitest/coverage-v8 at the root and @vitest/browser-playwright in
clients/web are now pinned exactly to 4.1.10 and move together, the same
discipline as the exact prettier pin (#1790).
The docs also claimed more than the change delivers. npm auto-installs an unmet
peer into the install that needs it, and a client install cannot see the root's
tree — so web's eslint-plugin-react-refresh / -storybook and the TUI's
eslint-plugin-react-hooks each pull a client-local eslint, and web's
Storybook/Vitest stack pulls a local typescript and vitest; @types/express
hoists an @types/node into web and cli. Those copies sit nearer than the root's
and win. AGENTS.md and the local-dev skill now say so, and state what the
consolidation actually buys: one declaration and one place to bump, with
verify:dep-lockstep as the guard that the copies agree.
Signed-off-by: cliffhall <cliff@futurescale.com>
Every runtime dependency core/ imports is now declared once, in the repo-root
package.json, and nowhere else. Deleting the client-side copies does not merely
align them: a package installs only into an install root that declares it, so
with no second declaration a second copy cannot be created at all — the failure
class #1896 was opened for and #1965/#2029 built verify:dep-lockstep to detect.
- clients/{web,cli,tui,launcher}: dropped ajv, atomically, chokidar, commander,
open, pino, zod, @napi-rs/keyring, hono, @hono/node-server and react, as each
applied. clients/cli and clients/launcher now declare no runtime dependencies.
- Root hono floor raised ^4.12.18 -> ^4.13.1, keeping the higher of the two
ranges that were in the tree rather than silently lowering web's.
- tsup external lists: added ajv and zod to all three bundles, plus atomically
and open (cli) and atomically (tui). tsup externalizes only what the nearest
manifest declares, so a package that moves to the root is inlined unless it is
named explicitly.
- vitest.shared.mts: repointed the pino, hono, @hono/node-server, atomically,
chokidar and @napi-rs/keyring pins from <client>/node_modules to the repo
root, joining express, yaml and proper-lockfile. Left behind they would
resolve to a directory that no longer exists, or to a transitive copy an
unrelated dependency dragged in. react and react-dom stay client-pinned so
the pair stays version-matched within an install.
- Docs: AGENTS.md dependency placement, the root README and the local-dev skill
describe the consolidated layout.
verify:dep-lockstep candidate set: 4 -> 2 (hono and pino fell out; only
@standard-schema/spec and zod still reach one tsc program from two installs).
npm run local:gate and npm run pack:verify both pass.
Signed-off-by: cliffhall <cliff@futurescale.com>
Declare the toolchain every client uses once, at the repo root, and delete the
per-client copies: eslint, @eslint/js, typescript-eslint, globals, prettier and
typescript (all five manifests), plus vitest, @vitest/coverage-v8 and
@types/node (four). Adds the latter three to the root manifest; the first six
were already there.
Every client reaches the root copy by walk-up — npm run puts each ancestor
node_modules/.bin on PATH, and Node and TypeScript resolve modules and @types
the same way — so nothing about the client scripts changes. clients/launcher now
declares no devDependencies at all and its validate is unchanged.
The copies had already drifted: globals sat at ^17.7.0 at the root against
^17.4.0 in all four clients, and typescript-eslint at ^8.65.0 against ^8.56.1.
Consolidating takes the root's version in both cases, and makes the exact
prettier pin (#1790) a single pin rather than five that can diverge.
Tools only one client runs stay where they are — tsup, tsx, vite, vite-node,
playwright, storybook, happy-dom, ink-testing-library, and each client's own
@types/*.
Documents the rule in AGENTS.md's Dependency placement section and the reasoning
in the local-dev skill.
Signed-off-by: cliffhall <cliff@futurescale.com>
- The CI wiring guard matched a substring, and last round's justification for
that was wrong: it accepts a longer script name
(`npm run verify:skills:cli:disabled`) and a mere mention (`echo npm run
verify:skills:cli`), either of which lets the real validator be removed while
the guard stays green. `runsCommand` now splits the script on the separators
that begin a new command and requires a segment to equal the invocation
exactly, with both counterexamples as regression cases.
- `local-dev`'s own description and its AGENTS.md index row both still promised
"where a new dependency must be declared" — the rules round 13 moved out of it
entirely. Both now describe what the skill actually carries: the reasoning
behind those rules. Recorded listing share updated to 1,199/4,000.
Signed-off-by: cliffhall <cliff@futurescale.com>
- skill-manifest: an eval file may only expect its own skill. A foreign
`expect: "other-skill"` passed whenever that other skill fired, so the file
reported a measurement of something it does not describe — and it satisfied
the positive-case requirement while doing so, masking the absence of a real
positive. Both halves covered by a regression case.
- local-dev: stop restating the dependency-placement rules. They are normative
and belong only in AGENTS.md; the skill now points there and carries what a
rule cannot — what each one is defending against, what it looked like when it
was violated (#1970, #2082, #2067, #1952, #1896, #2058), and how to tell you
have hit it. Including the two traps that make this class hard to see: the
ordinary test tiers cannot catch a bundling miss, and probing one by hand with
`node -e` falsely succeeds.
Signed-off-by: cliffhall <cliff@futurescale.com>
- verify:skills:cli — the authoritative validator is now a GUARANTEED step in
`local:gate` and CI, not a best-effort hand-off. It resolves the CLI itself
(an installed one at 2.1.233+, else a pinned `npx -y`), so both tiers run the
same script and cannot drift. `verify:skills` keeps skipping when the CLI is
absent, which is right for `validate`: fast, offline, no Claude Code required.
- skill-manifest: the 1,536-char listing-entry cap covers `description` +
`when_to_use` together; checking the description alone passed an over-cap
entry. `when_to_use` is now parsed, validated and counted, with boundary tests
at 1,536/1,537.
- verify-skills: report the budget as *this repo's contribution*. The real
listing budget is shared with bundled skills and the contributor's own
~/.claude/skills, none of which is visible from here, so the number was
claiming more than it measured.
- skill-eval: a negative case now means "no skill OF THIS REPO'S fired". It
failed on any Skill invocation, so a contributor with an unrelated global
skill got a false failure about their environment rather than these skills.
- docs/architecture + local-dev: the Node backend runner gets its
`@inspector/core` alias from `clients/web/tsup.runner.config.ts`, not from
Vite. The old text sent runner work to the wrong config.
- local-dev: the tarball does not ship "only each client's build/" — the
allowlist also carries `clients/web/dist`, `clients/web/static` and the
install helper. States the invariant that actually matters instead.
- issue-triage: the test for needing triage is "no board card". Adding "and no
milestone" would skip the approved-but-unboarded case the same section then
handles. Also corrected the `$M` note: the lookup is `--state all`, so closed
issues DO have entries — `isopen` is about the invariants, not the lookup.
- issue-create: replaced "all five" with a per-version table. Steps 3 and 6
(milestone, Priority) do not exist for v1, so the old list opened with a rule
the same skill later tells you to violate.
Signed-off-by: cliffhall <cliff@futurescale.com>
Closes#2163
`AGENTS.md` was one 1,223-line contract loaded in full on every turn, whether
the task was a one-line CSS fix or a release cut. This splits it: rules stay
resident, procedures move to `.claude/skills/`.
- Ten skills under `.claude/skills/`, seven `disable-model-invocation: true`
(invoked by name only, so they cost no listing context and have no trigger
question) and three model-invoked with committed eval cases.
- `AGENTS.md` 1,223 → 380 lines: rules, a skills index, and a project tree
reduced to one line per entry. The removed recipes are not duplicated.
- `README.md` 811 → 105 lines, split into `docs/{architecture,quality-gate,
test-servers,publishing,docker}.md`, and dropped from `CLAUDE.md`'s auto-load.
- `.github/copilot-instructions.md` and its mirroring rule removed — Copilot
reads `AGENTS.md`.
- `npm run verify:skills` (in `validate`, so in `local:gate` and CI) parses each
SKILL.md the way Claude Code does, requires an explicit invocation mode and
eval cases for model-invoked skills, and budgets the listing.
- `npm run skills:eval` runs the committed trigger cases headless.
Signed-off-by: cliffhall <cliff@futurescale.com>
