Commit graph inspector/scripts
Author SHA1 Message Date
cliffhall
e846dec491 chore(deps): address Copilot review round 14 on #2243
- Reconcile on (package, manifest, GHSA) tuples, not GHSAs alone.
  Dependabot alerts are per manifest and this repo has five lockfiles,
  so the same advisory legitimately covers the root install and a
  client. Keyed on the GHSA alone, another manifest's still-filable
  alert could vouch for this one and clear an issue whose own alert is
  open with no bump available.
- Rename the cleared-date test: it asserts the NEW date is taken when
  the reason changes, which is the opposite of what its name said.
- AGENTS.md no longer implies both sweeps select a dated milestone.
  Only the security sweep filters undated buckets; the monthly one's
  selection is raised on #2239.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:09:24 -04:00
cliffhall
c0d1d56782
chore(deps): address Copilot review round 13 on #2243
Three findings, all of them the same invariant reached by new routes: a
still-open advisory must never stand itself down, and a cleared issue
must be written once.

- Never clear an advisory that is open but unpatched. groupAlerts drops
  an alert with no first_patched_version, so it has no replacement
  group and no replacement issue — calling it "superseded" was false
  and clearing it removed the only thing tracking a live exposure. Such
  an issue is now left exactly as it is, and said so in the log.
- Stop re-editing cleared issues daily. A cleared issue stays open, so
  the next run regenerated its body with a new date and edited it
  again, forever. The date is now read back off the existing body, so
  only a real change writes — and a real change takes today's date.
- Distinguish an absent manifest from an unparseable one. Both returned
  null, so a malformed lockfile was treated as "the manifest is gone"
  and cleared the issue. A read error is evidence of nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
bc47fcbcf0
chore(deps): address Copilot review round 12 on #2243
Build the open-GHSA set from the raw alert feed, not from the grouped
alerts. groupAlerts deliberately drops an alert with no
first_patched_version, so an advisory that stays OPEN but loses its
patched version vanished from both the keys and that set — and
reconciliation would have called it "fixed or dismissed".

Same wrong direction as the superseded case fixed last round, reached a
different way: a still-open advisory must never stand itself down.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
233647163f
chore(deps): address Copilot review round 11 on #2243
- Filter to the npm ecosystem. Dependabot alerts are not npm-only and
  this repo has a Dockerfile, so an alert against it would have been
  parsed as a lockfile, thrown, and aborted the whole daily sweep
  before any npm group ran. A non-npm alert is now reported loudly with
  its GHSAs so a human can file it, and readManifest no longer lets one
  unparseable manifest take the run down.
- Distinguish a superseded bump from a closed one. GitHub can revise
  first_patched_version, moving an advisory to a different key while it
  stays open; reporting that as "fixed or dismissed" would stand down a
  live exposure. The reason now reads from the open GHSA set.
- Only claim security-update PRs are off when the run actually read the
  setting. The guard degrades to UNVERIFIED, and an issue asserting
  what the run could not confirm is worse than one that stays quiet.
- README.md and AGENTS.md: scripts/ now holds repo automation run from
  CI, not only build/verify tooling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
9e516ae0a1
chore(deps): address Copilot review rounds 8-10 on #2243
Three reviews I missed: my poll called the reviews endpoint without
--paginate, so it only ever read page one — the same pagination bug
this PR was reviewed for.

- Reconcile marked issues whose bump has left the open alert feed
  entirely. openAlerts asks for state=open, so a fixed or dismissed
  alert simply vanishes: its group is never built, the loop never
  visits it, and its issue kept asserting a vulnerability with a live
  Todo/High card. Both cases are covered — the zero-alert run, which
  used to return before loading issues at all, and a vanished group
  while other groups remain.
- Ask for a PARENT-SCOPED overrides entry when the package is also
  declared directly. npm rejects an override contradicting a direct
  dependency with EOVERRIDE, so the guidance for the mixed case would
  not have applied. The issue now prints the exact nested JSON.
- Correct the body's NOTE: severity, range, GHSA and CVE are the
  advisory's own. Only the installed versions, their paths and range
  applicability are verified against v2/main.
- Stop the new-advisory comment claiming the body marker "now covers"
  the advisory; it is posted before the edit, so that is false at
  posting time. It speaks for its own marker instead.
- Cover the successful two-field board placement, not only its failure
  modes.
- AGENTS.md: the monthly sweep NEVER boards; only the security sweep
  does, and only with a PAT. The old wording promised a card the
  monthly job cannot create.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
1d6ea1a8f6
chore(deps): address Copilot review round 7 on #2243
Both findings are consequences of round 6's staleness fix, in the two
places it did not reach.

- Resolve the matching open issue BEFORE the skip paths, and rewrite it
  to a cleared state when the exposure is gone. Yesterday's issue is
  still open today, so a manifest that has since been removed or copies
  that have moved out of range left its body asserting a vulnerability
  that no longer exists and its Todo/High card live indefinitely. The
  marker is retained so the issue is reused if the advisory returns; the
  issue is not auto-closed, because whether the card belongs in Done or
  should be deleted depends on why the exposure went away.
- The body's prose counted the marker's monotonic history rather than
  the advisories that apply, so it could claim two open alerts while the
  title and table correctly showed one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
ff339dc7c3
chore(deps): address Copilot review round 6 on #2243
Both were suppressed "previously missed" findings, and both were real.

- isPermissionDenied matched on status alone, so a rate limit (also a
  403) and a bad token (401) were waved through as "missing scope" —
  contradicting the comment right above it, which said those must stop
  the sweep. It now excludes rate-limit wording and drops 401.
- The no-op path asked "were advisories ADDED?" when the question is
  "did the issue CHANGE?". An issue filed for A+B whose branch moved so
  only B applies has nothing added, yet its table, severity, affected
  copies and remediation are all stale. The rendered title and body are
  now compared against the issue, and a comment stays reserved for
  genuinely new advisories.
- The title counts the advisories that APPLY, matching the body, so it
  tracks shrinking exposure as well as growth. The marker's GHSA list
  stays monotonic — its job is to remember what has been announced.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
a17eace0bb
chore(deps): address Copilot review round 5 on #2243
Both findings were "previously missed" suppressed ones; the two against
scripts/dependency-refresh.mjs and its workflow are relayed to #2239.

- Serialize the sweep with a fixed concurrency group and
  cancel-in-progress: false. The marker check is a read-before-write and
  a workflow_dispatch can land on top of the scheduled run, so two runs
  could both see no open issue and both file one — the duplicate the
  whole idempotency design exists to prevent. The queued run must wait
  and re-read, never be cancelled.
- Pick the milestone in JS, not in jq. jq sorts null before every
  string, so sort_by(.due_on) | .[0] returns an UNDATED open milestone
  in preference to every dated one. An undated bucket has no due date
  and so cannot be the nearest; pickMilestone drops it, and files the
  issue unmilestoned if nothing dated is open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
1e58d2fe8a
chore(deps): address Copilot review round 4 on #2243
- Narrow each group to the advisories the installed copies are actually
  in range of. Two advisories can share a package, manifest and patched
  version while having different vulnerable ranges, so a group-level
  "does any match?" left the marker, title, severity and table all
  claiming an advisory that does not apply on this branch.
- Escape every free-form Markdown cell, the vulnerable range included.
  A semver range may contain `||`, which is also the column separator.
- Paginate the open-issue lookup instead of capping it at 100. The
  marker lookup is what makes the sweep idempotent, so a truncated list
  would file a duplicate for every issue it could not see — at the same
  scale the alert fetch is built to handle. Pull requests, which the
  issues endpoint also returns, are dropped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
5c65bd2bfa
chore(deps): address Copilot review round 3 on #2243
Derive the remediation from WHICH copies are vulnerable, not from
whether the package is declared. A manifest can declare a safe `pkg@4`
while a dependency drags a vulnerable `pkg@3` into a nested folder; the
old boolean called that "direct" and asked for a range bump that would
have changed nothing, omitting the override the nested copy needs.

- `lockfileEntries` keeps each copy's tree path and whether it is the
  hoisted one; `lockfileVersions` is now derived from it.
- `remediation(affected, declared)` returns both flags, so the issue can
  ask for a range bump, an overrides pin, or explicitly both. The body
  lists the vulnerable copies and their paths.
- An undeclared hoisted copy counts as transitive: it got there the same
  way any other transitive copy did, and no declared range reaches it.
  (Found by a test written for this change.)

The `vulnerability-alerts: read` finding is declined — verified against
a real runner, which reports `VulnerabilityAlerts: read` and reads the
alerts successfully. See the PR comment for the log.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
761bdd128d
chore(deps): address Copilot review round 2 on #2243
- Remove the literal NUL bytes that were separating the grouping key's
  fields; they classified the whole source file as binary, so repo
  searches skipped it. The key is a JSON array now, with no separator
  left to justify.
- De-duplicate advisory comments per GHSA rather than per whole set. A
  run that comments and then fails before the marker edit left the next
  run computing a different set, which matched nothing and announced the
  same advisory twice.
- Refresh the issue title on edit; it carries the advisory count, so it
  went stale as soon as an issue grew past what it was filed with.
- Correct the test file's header, which still claimed main() was left
  to workflow_dispatch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
b53266bc72
chore(deps): address Copilot review round 1 on #2243
- `--slurp` the paginated alert feed; a bare `--paginate` emits one JSON
  array per page and `JSON.parse` rejects it past 100 open alerts.
- Put `fixedIn` in the marker and in the existing-issue lookup, so a
  second bump of one package cannot merge into the first one's issue.
- Only an authorization-shaped failure of the `automated-security-fixes`
  read becomes UNVERIFIED; a rate limit or 5xx now throws.
- Distinguish "card never added" (benign, triage picks it up) from
  "card added, field not set" — the latter finishes every group, then
  fails the run.
- Don't board an unmilestoned issue at Todo; `Incoming` <=> no milestone.
- Comment before rewriting the marker, and give the comment its own
  marker, so a failed comment cannot be skipped forever.
- Ask a direct dependency's range to be raised, not widened to `>=`.
- Test `main()` through an injected spawn, as the sibling sweep does.
- Correct the docs that named the removed release-time `npm audit fix`,
  the ones promising an unconditional guard, and the sibling sweep's
  issue body claiming security updates remain enabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
66a3332f1d
chore(deps): sweep Dependabot alerts into board-tracked issues
Closes #2233 — the alert-consuming half of #2229. #2235 removed
`.github/dependabot.yml`, ending Dependabot's version-update PRs; this
ends its security-update PRs and replaces them with a daily sweep that
turns the alerts into ordinary board items.

- `scripts/dependabot-alerts.mjs` groups alerts by
  `(package, manifest_path, first_patched_version)` — one issue per
  BUMP, not per advisory — re-checks each vulnerable range against
  `v2/main`'s own lockfile before filing, and keys idempotency on a
  marker comment naming every GHSA the issue covers.
- `.github/workflows/dependabot-alerts.yml` runs it daily and on
  `workflow_dispatch`, with `vulnerability-alerts: read`.
- `semver` is declared at the repo root, per Dependency placement:
  `scripts/` is root-owned code with no manifest of its own.
- The sibling `dependency-refresh` comments no longer claim security
  updates are unaffected, and AGENTS.md gains the flow both halves
  now follow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013H6Cew3KB3jVmQ5x4Dq1sq
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:32 -04:00
cliffhall
0cb12a19ca
chore(deps): paginate the release list so the highest version is actual
Addresses Copilot review round 7 on #2239.

The lookup read one page of 100 and the doc comment justified it with an
assumption rather than a guarantee: "the greatest version is within it
for any real action". The list is ordered by release DATE, not version,
so an action that keeps cutting maintenance releases on lower majors
pushes the genuine maximum off page 1. Ranking what remained would
report a lower major as highest — the exact false-current result that
dropping `releases/latest` in round 3 was meant to prevent, arrived at
by a different route.

Now passes `--paginate`. Cost is unchanged for every action this repo
uses (57 and 64 releases are the largest, so still one request each);
it only costs more where correctness actually required it.

The command-shape test now requires `--paginate` as well as the list
endpoint. It has to: pagination is invisible in the result — one page
and every page return an identical-looking tag list until the day they
do not — so nothing about the returned value can detect its absence.
Verified by mutation: removing `--paginate` fails exactly one test.

Adding the flag also moved the URL out of args[1], which the test's own
filter had hardcoded — it caught that itself. Both the filter and
fakeSpawn's matcher are now argument-position agnostic.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGtzPg3UxMLszysQXXfqax
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:31 -04:00
cliffhall
8414685ad9
docs(scripts): correct the release-lookup contract left stale by round 4
Addresses Copilot review round 5 on #2239.

Round 4 made every non-zero release-list response fatal and deleted
`isMissingRelease`, but `latestReleaseTag`'s JSDoc still promised
`@throws when the lookup fails for any reason other than a 404` — the
exact behavior that change removed. The PR description carried the same
stale claim.

A contract that documents the opposite of the code is worse than none:
the next reader reasonably trusts it, and here it would tell them a
missing action repo is silently tolerated when it now fails the sweep.

The contract now reserves `null` for a successful response carrying no
usable release — an action that has never cut one, or whose tags are all
unparseable — and states that a failed lookup is never `null`.

Documentation only; no behavior change, and the 29 tests are untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGtzPg3UxMLszysQXXfqax
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:31 -04:00
cliffhall
dde1736702
chore(deps): treat every failed release-list response as fatal
Addresses Copilot review round 4 on #2239. Both findings are regressions
I introduced in round 3 and both are correct.

Round 3 moved the lookup from `releases/latest` to the release LIST, to
rank by version rather than by GitHub's designated latest — but kept the
404 suppression that only made sense for the old endpoint. The list
endpoint answers "this action cuts no releases" with a successful empty
array, so a 404 there does not mean that at all: it means the repository
is missing or inaccessible, i.e. a `uses:` ref the sweep cannot check.
Converting it to null silently dropped a broken or renamed action from
the flow that replaced Dependabot — the same silent-success shape this
PR has been closing everywhere else, reintroduced one layer down.

Every non-zero status is now fatal and `isMissingRelease` is gone
entirely; the benign no-releases case is the successful empty array,
covered by its own test.

`buildClearedBody` also overstated its all-clear. Refs pinned to a commit
SHA or a branch return null from `parseVersionRef` and are never ranked,
so "no workflow `uses:` ref is behind" asserted a check that had not
happened for them. It now says version-pinned refs and names the
exclusion explicitly.

Both guards are mutation-checked: restoring the 404 suppression, and
dropping the SHA/branch qualification, each fail exactly one test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGtzPg3UxMLszysQXXfqax
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:31 -04:00
cliffhall
6af74424d8
chore(deps): remove dependabot.yml, fold action bumps into the sweep
Closes #2235.

Switches the old dependency flow off now that #2232 landed the
replacement. Removes the five npm ecosystem entries (root plus each
client under clients/*) and, resolving the question #2229 left open, the
github-actions entry as well — so .github/dependabot.yml goes away
outright rather than being emptied, which its schema does not allow.

The github-actions entry had exactly the property #2229 exists to
remove: it opened a grouped monthly PR carrying no `Closes #N` and no
board card, the one standing exception to "every PR references an
issue". Deleting it unreplaced would have left 9 actions unwatched, and
`npm outdated` says nothing about actions, so the monthly sweep now also
checks every `uses:` ref under .github/workflows and renders the stale
ones as one more section of the same tracking issue.

Ranking comes from the release LIST, not `releases/latest`. That
endpoint returns the release GitHub designates most recent, not the
greatest version, so an action publishing a maintenance release for an
older major (a v6.9.1 cut after v8.0.0) would make a workflow pinned to
v7 compare against v6 and read as current — silently missing a whole
major upgrade, the one thing this check exists to catch.

Staleness is compared only to the precision the ref specifies. `v7` is a
moving major tag that GitHub repoints at every v7.x release, so `v7`
against a highest of `v7.0.1` is current and only `v8` makes it stale;
an exactly-pinned `v7.0.0` is behind `v7.0.1`; a SHA pin is deliberately
immovable and is never reported.

A release lookup suppresses only a 404 — the legitimate "this action has
never cut a release" answer — and throws on anything else. Treating a
rate limit or an expired token as "no release" is indistinguishable from
"not stale", and since every action here already sits on its latest
major, the resulting empty section is byte-identical to a healthy run.

`buildClearedBody` now speaks for both halves: once actions are in scope,
its npm-only wording would assert a clean bill of health the sweep never
checked.

Dependabot security updates are unaffected — they are configured in repo
settings, not in this file, and kept working while it was missing
entirely (#1833, #1840). That note moves into the workflow header rather
than dying with the file; #2233 is where they are turned off
deliberately. The script header, the workflow header and the generated
issue body all say version-update PRs rather than claiming Dependabot is
replaced wholesale.

Also renames the workflow's npm-outdated job to dependency-sweep now
that the sweep is no longer npm-only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGtzPg3UxMLszysQXXfqax
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:31 -04:00
cliffhall
ddf2307c4c
test(scripts): stop the repo-guard test depending on the ambient env
`main refuses to run without a repo` called `main(undefined, …)` so the default
parameter would fire, then asserted it throws. The default reads
`process.env.GITHUB_REPOSITORY` — which GitHub Actions sets on every run — so
the assertion held only where the variable happened to be unset. It passed
locally and failed CI, the one environment where the default is always
populated.

The test now clears the variable and restores it, and a second test covers the
other half of the default: with it set, `main(undefined, …)` uses it instead of
throwing. Between them the default's behavior is pinned in both environments
rather than inherited from whichever one is running.

Verified both ways: 14 pass with the variable unset and with it set, and the
full `test:scripts` suite is 411/411 under the CI environment shape.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWMJENjs1mExXE5XPJTEei
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:30 -04:00
cliffhall
07bd987129
chore(deps): address Copilot review round 1 on #2232
Five findings, all valid.

`runOutdated` accepted every exit status and returned `result.stdout ?? ""`.
A failed `npm outdated` (exit 2 on a registry or config error) also prints
nothing, so the empty string parsed to an empty package list — and across five
installs a total outage produced output byte-identical to a clean sweep. It now
throws on any status other than the documented 0/1.

`main()` was untested, which is how that reached review: the helper-only suite
passed while the silent-success path was live. It now takes its spawn function
as a parameter and is driven by a fake, covering npm failure, create vs. edit,
milestone handling, both no-op paths and the missing-repo guard.

The no-op path returned before looking for the marker issue, so once every
install caught up, a still-open tracking issue kept its obsolete package table
indefinitely — contradicting the body's own promise to update in place. It now
rewrites through `buildClearedBody()`. It deliberately does not close the
issue: the sweep takes no board actions, and closing one whose card a
maintainer may have moved would make the board assert work shipped that this
script cannot verify shipped.

The release step mandated `npm audit fix`, which AGENTS.md forbids. The reason
is not `--force`: plain `audit fix` resolves an advisory with no upward escape
in range by silently downgrading, as it did to esbuild across three installs in
 #2058, and `local:gate` has no version-regression check to catch it. The step
is now `npm audit` report-only, with fixes applied deliberately via a direct
bump or an `overrides` entry.

An unmilestoned issue is swept into Incoming, not Todo — Todo asserts a
maintainer signed off. Message corrected and pinned by a test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWMJENjs1mExXE5XPJTEei
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:30 -04:00
cliffhall
ee14a9fc1c
chore(deps): move the dependabot.yml change out to #2235
The npm-entry removal in `.github/dependabot.yml` is split out of this PR so
that every change to that one file lands in a single reviewable piece —
alongside the `github-actions` entry, which #2229 had declared out of scope but
which opens the same issue-less, board-cardless PR the decision exists to
remove.

Reverts `.github/dependabot.yml` to its `v2/main` state, leaving this PR with
the release-time `npm audit fix` and the monthly `npm outdated` sweep: the
replacement flow, not the switch-off.

Both new files' header comments claimed the switch-off had already happened,
which is false until #2235 lands. They now say #2235 does it, and that until
then Dependabot's npm PRs and this sweep overlap — duplicate signal rather than
conflicting action.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWMJENjs1mExXE5XPJTEei
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:30 -04:00
cliffhall
7dd9f47ca2
chore(deps): npm audit fix before release bump; monthly npm-outdated issue
Adds npm audit fix (root + every client) as the first sub-step of the
release skill's bump step, before npm version, so a release is never
gated on remembering to check separately. Never --force; anything
audit fix can't resolve is left to the dependabot-alert pipeline
(#2229) or a follow-up issue.

Disables Dependabot npm version updates in dependabot.yml (a
version-update PR carries no issue and no board card) and replaces
them with a monthly scheduled sweep that runs npm outdated across the
root install and every client, filing or updating one idempotent
tracking issue instead of an auto-generated PR.

Closes #2231
Part of #2229

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017N9ha27Qg6LHpBQP7G7zPY
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 09:06:30 -04:00
cliffhall
afca8faaea docs: address Copilot review round 9 on #2203
Restrict the truncation guard to PLAIN scalars. `yaml` attaches a legitimate
trailing comment to quoted and block scalars too, so keying on `.comment` alone
rejected valid frontmatter — `description: "Complete value" # note` was reported
as truncated, telling the author to quote a value that was already quoted. The
regex this replaced tested the same thing by reading the first character; moving
to the node dropped the check rather than encoding it. Verified all three
shapes (double-quoted, single-quoted, block) against the parser, and added the
regression cases.

Also correct the `--max-turns 1` bullet in docs/skill-authoring.md, which
claimed the skill must be the model's first tool CALL. `collectSkillInvocations`
scans every tool_use block in the assistant event without regard to order, so a
turn calling Read and Skill together scores a hit. What actually loses the
sample is needing a tool RESULT first — that decision lands in turn two, which
never runs. The practical advice is unchanged; the mechanism now matches the
collector.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SyBPs7caoMcyvnp9ArMBhV
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 13:02:14 -04:00
cliffhall
c66793bfd8 docs: address Copilot review round 8 on #2203
Ask the YAML parser what it discarded instead of re-deriving it from the raw
frontmatter text. `yaml` hangs the comment it stripped on the scalar node, so
`parseDocument(...).get(field, true).comment` IS the truncation signal: a plain
scalar that lost text has one, a quoted or block scalar never does, and
`Issue#28` (no preceding space, so no comment) keeps its full value.

That closes both findings at once. The line-locating regex only ever matched
the canonical `description:` key, so `"description":` and `description :`
populated the mapping while the guard skipped them and left verify:skills green
on a truncated listing; and the continuation scan stopped at the first blank
line, inspecting only the opening paragraph of a plain scalar that YAML happily
continues past one.

It also retires the scanning itself — three rounds of regex here each fixed one
shape and missed the next (tags and anchors, then indented continuations, now
key forms and blank lines). The node carries all of them for free, so 65 lines
of hand-rolled parsing come out. Parsing once with `parseDocument` keeps the
node view and the value view from ever disagreeing.

Tests cover the three noncanonical key forms, the blank-line continuation, and
the standalone comments that must NOT read as truncation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SyBPs7caoMcyvnp9ArMBhV
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 12:46:22 -04:00
cliffhall
0febaa0f6f docs: address Copilot review round 7 on #2203
Strip YAML properties (tags, anchors, in either order) before deciding
whether a description scalar is quoted — testing the first character alone
read `&copy "Counts # safely"` as unquoted and rejected a `#` the quotes
make harmless.

Count the positive-case floor by distinct prompt rather than by entry: the
floor exists for breadth, so five copies of one prompt cleared it while
exercising exactly the single trigger it is meant to look past. Compare on
collapsed, case-folded text so trivial edits do not read as a new situation.

Re-aim the two positive prompts that measured the wrong thing — the
project-structure case now poses a concrete "which piece handles what"
question, and the test-servers case names running a real MCP server.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SyBPs7caoMcyvnp9ArMBhV
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 12:21:09 -04:00
cliffhall
5d3571597a docs: address Copilot review round 6 on #2203
The truncation guard rejected valid frontmatter. It assumed the only way
a raw scalar can exceed its parsed value is a comment, but YAML also
strips presentation syntax, so `description: &copy Some text` and
`description: !!str Some text` were both reported as truncated — naming
an unquoted `#` the value does not contain.

The length check is now gated on an actual comment marker: `#` at the
start of the scalar or preceded by whitespace, which is where YAML begins
a comment. Anchors and tags fall out before the length is consulted.
Regression cases cover both, and assert the genuine `#` case still fails
so the fix cannot pass by disarming the guard.

Also corrects the checklist, which still said an unquoted `#` lets every
check pass. That predated the verifier; verify:skills rejects it now, and
the sentence was telling authors the gate is blind to something it
catches.

No skill description or eval case changed, so skills:eval is unchanged
from round 4's 63/63. local:gate passes, 34/34. 391/391 script tests.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 11:44:57 -04:00
cliffhall
ec2ef5656d docs: address Copilot review round 5 on #2203
The truncation guard read only the first physical line of the scalar, so
a multiline plain scalar slipped past it:

    description: Covers boards
      #28 and #11

YAML parses that as `Covers boards`, and the first-line-only comparison
saw `Covers boards` on both sides and reported no loss — passing the
exact silent truncation the guard exists to reject.

It now walks the whole plain scalar: the value line plus every indented
continuation, whitespace-collapsed to match YAML's folding of newlines to
spaces. Quoted and block scalars are still skipped, since neither loses
text to a comment.

Two regression cases: the multiline truncation, and a multiline scalar
with no `#`, which must stay clean — the fix cannot be "reject anything
multiline".

No skill description or eval case changed, so skills:eval is unchanged
from round 4's 63/63. local:gate passes, 34/34. 390/390 script tests.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 11:31:22 -04:00
cliffhall
4acab97e63 docs: address Copilot review round 4 on #2203
Corrects the rationale for the five-positive floor, which was wrong. The
claim was that five prompts make a skill's rate flake-tolerant. The
evaluator scores each prompt on its own `passes / RUNS`, so the number of
prompts never enters any rate and cannot affect a threshold crossing —
`RUNS` is the only knob that does. The floor is worth keeping for
breadth: one prompt measures one phrasing, and a description that fires
on a single narrow shape would pass on its one lucky case.

The bad reasoning had reached five places — AGENTS.md, the authoring
checklist, the JSDoc, the check-site comment and the regression test —
plus the emitted message, which asserted it too. All corrected.

Moves `MIN_POSITIVE_CASES` above `validateEvalCases`'s JSDoc. It had been
inserted between the two, so the `@param`/`@returns` block documented the
constant and the function had none.

Replaces the two remaining cases answerable from `AGENTS.md`: `local-dev`'s
web-client-startup case (the rules file gives the exact command) and
`project-structure`'s clients/web-versus-core case (its tree answers it).
Both replacements probed 3/3 and measure 100%.

RUNS=5 skills:eval: 63/63, every local-dev and project-structure case at
100%. local:gate passes, 34/34.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 11:14:06 -04:00
cliffhall
104a05dd7f docs: address Copilot review round 3 on #2203
Replaces the last `project-structure` case that `AGENTS.md` answers — a
helper used by the CLI and TUI belongs in `core/` per its own directory
tree. This was also the only case below 100% in the previous suite, so
the static reading and the score agreed. The replacement targets the
skill's first stated purpose, locating code, which `AGENTS.md` gives no
method for; it probed 3/3 and now measures 100%.

Extends the truncation guard's tests to `when_to_use`. The guard loops
over both fields but only `description` was pinned, so dropping the
second from that list would have gone unnoticed. The new case pairs a
quoted description with an unquoted `when_to_use` and asserts the
description is not implicated.

Scopes the negative-case requirement in the prose to model-invoked
skills; round 2 scoped the checklist but left this sentence saying
"every skill file".

Refreshes the stale `~2.8k of 4k` listing figure. The other instance is
past-tense about the three-to-nine experiment and stays, now said
explicitly so two numbers do not look like a disagreement.

Records the eval's real blind spot: all three cases review identified as
`AGENTS.md`-answerable were scoring 80-100% when flagged, so a high score
does not clear a case of overlap. The eval can show a case failing; it
cannot show a passing case is well-aimed.

RUNS=5 skills:eval: 63/63. local:gate passes, 34/34.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 10:53:56 -04:00
cliffhall
880be3acdd docs: address Copilot review round 2 on #2203
Enforces the five-positive standard this PR established rather than only
describing it. `validateEvalCases` checked `positives === 0`, so a skill
could regress to a single prompt and still pass the gate. At RUNS=5 one
case is 20 points of that skill's reading, so a one- or two-case skill
reports a rate a single flake can move across the threshold — well-formed
to the checker, meaningless as a measurement. Zero positives keeps its
own clearer message so the common mistake keeps its diagnosis.

Scopes the eval requirement to model-invoked skills in the checklist. A
name-only skill has no eval file because nothing can measure a trigger
only a human types, so an eval there is unrunnable rather than redundant.

Aligns checklist item 1 with the `#` hazard from round 1; it named only
the unquoted colon, which is the louder of the two failure modes.

Four test fixtures asserted on one-positive eval files and legitimately
went red; they now build positives from the exported MIN_POSITIVE_CASES.
The all-positive row keeps testing the missing-negative path, with a new
row for the below-floor case. 388/388 script tests pass.

No skill description or eval case changed, so trigger behaviour is
unaffected and skills:eval is unchanged from the previous round.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 10:32:41 -04:00
cliffhall
50f3360d65 docs: address Copilot review round 1 on #2203
Fixes an unquoted `#` in `board-ops`'s description. YAML reads ` #28`
onward as a comment, so the parser — and Claude Code, which parses the
same way — only ever saw the description up to `Covers board`. Both
board numbers and the option-deletion hazard were absent from the
listing that decides whether the skill fires, and every check stayed
green because they look for an *empty* description and a truncated one
is still non-empty. Quoting restores all 271 characters, which is the
whole of the 3,144 vs 3,234 discrepancy the review flagged.

`parseSkill` now compares the raw scalar against what YAML kept and
errors on any loss, so the next one fails the gate rather than shipping.
Tests cover the truncated case and a legitimately quoted `#`.

Re-aims the `project-structure` Node-backend case, which `AGENTS.md`
answers via its tree and which had oscillated 80 -> 100% across runs —
the partial-overlap signature this PR documents. The replacement names
no internals and probed 4/4.

Makes the documented probe reproduce the harness: the same
`--disallowedTools` flags as `runPrompt` rather than `--allowedTools`,
`printf` instead of zsh's `print -r`, and `jq` over `tool_use` blocks
instead of a `grep` that also matched MCP server names in the init
event. The first-tool-call table is relabelled as an unrestricted probe,
since the eval cannot reach for Bash at all.

RUNS=5 skills:eval: 63/63, 62 at 100%; restoring those 90 characters
displaced nothing.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 10:10:06 -04:00
cliffhall
a344225e23 docs: accept several skill names in skills:eval and re-aim overlapping cases
Adds a multi-name filter to `npm run skills:eval` so a set of related
skills can be iterated together, and makes an unknown name a hard error
rather than an empty run — a typo would otherwise report a green 0/0
that reads exactly like a clean pass of the skill that was meant.

Re-aims two `local-dev` cases and one `test-servers` case that partially
overlapped `AGENTS.md`. Partial overlap oscillates across runs rather
than sitting at zero, so it reads as eval noise instead of a defect in
the case; `docs/skill-authoring.md` now records that failure mode and
the observed scores.

Widens the `test-servers` description to the "Use when …" shape and
points the `testing` skill at it, since anything needing a real server
to run against is a procedure `testing` does not carry. That growth
moves the skill listing from 2,921 to 3,144 of 4,000 characters, so the
figure recorded in AGENTS.md is updated to match.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-01 00:18:43 -04:00
cliffhall
a2ddece7a5 chore: address Copilot review round 20
All three findings are the same shape: a check that accepts a command which
exists but cannot gate.

- Splitting on `|` and `&` conflated `||`, so both failure-masking forms counted:
  `npm run X || true` swallows a rejection and `true || npm run X` never runs
  the validator. Both `runsCommand` and the chain walk now split on `\n`, `;`
  and `&&` only, which refuses each without special-casing either.
- A CI step (or job) carrying `continue-on-error` asserts nothing, since a
  rejected skill leaves the workflow green. Both are now skipped, treating a
  string `"true"` and an unevaluable `${{ }}` expression as masking on the same
  conservative principle as the `if:` checks; an explicit `false` still counts.
- `rootReachesScript` was substring-based, so a `validate` reading `echo npm run
  verify:skills` satisfied the sibling vouch while the guard never executed —
  the cycle protecting nothing. It now uses the exact chain walk, which is
  therefore shared in `lib/npm-scripts.mjs` and fixes the same hole in all four
  vouches, not just this PR's. `reachableScripts` keeps its substring behavior
  for the harvesting callers, where over-inclusion is safe.

The chain walk also follows npm's implicit pre/post hooks, with a test — a guard
invoked from a `prevalidate` is genuinely run, and reporting it missing would be
a false failure introduced by the fix.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 05:32:12 -04:00
cliffhall
b91b746867 chore: address Copilot review round 19
The `local:gate` half of the wiring vouch used `reachableScripts`, which
extracts any `npm run …` substring — so `echo npm run verify:skills:cli && npm
run coverage` satisfied it while the validator never ran. Same over-match the CI
half had in round 15, from the other side.

`scriptChainRuns` walks the script graph instead, splitting each body on the
separators that begin a new command and requiring an exact invocation of the
target. A segment naming some other script is followed with or without trailing
flags, since `npm run coverage --silent` still runs it, and `seen` makes a
cyclic graph terminate. Regression cases for the mention, a longer script name,
and the cycle; verified by mutation.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 05:18:18 -04:00
cliffhall
1203246537 chore: address Copilot review round 17
- ciRunsUnconditionally never looked at the workflow-level `on:` trigger, so a
  workflow switched to `on: release` still contained the step and reported as
  wired while no PR ever ran the validator. It now requires `push` or
  `pull_request`, handling all three shapes `on:` takes. The positive fixtures
  had no trigger at all, so they could not assert the "on every push" half of
  the contract; they carry one now, alongside a release-only negative. Verified
  by mutation.
- Stale JSDoc from the round-16 version change: `probeClaudeVersion` is now
  generic in its parser's result rather than hard-coding `number[]`,
  `validatorCommand` documents `{ parts, prerelease }`, and `runValidator`
  documents the `platform` option it already accepted. Annotations that lie make
  valid call sites look type-invalid.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 04:54:04 -04:00
cliffhall
c26995bf82 chore: address Copilot review round 16
`parseClaudeVersion` dropped SemVer suffixes, so an installed `2.1.250-beta.1`
parsed as `[2, 1, 250]` and compared EQUAL to the stable pin — a different
validator schema running locally while the log claimed an exact match, which is
the cross-machine disagreement the pin exists to prevent.

It now returns `{ parts, prerelease }`, because the two callers want different
things: `isPinnedVersion` rejects any prerelease (build metadata still matches,
per SemVer precedence), while the eval's availability probe only needs some
usable CLI. `formatClaudeVersion` keeps the suffix in the messages, so a
prerelease is reported as what it is rather than as the pin.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 04:40:48 -04:00
cliffhall
3905ceef06 chore: address Copilot review round 15
- The CI wiring guard matched a substring, and last round's justification for
  that was wrong: it accepts a longer script name
  (`npm run verify:skills:cli:disabled`) and a mere mention (`echo npm run
  verify:skills:cli`), either of which lets the real validator be removed while
  the guard stays green. `runsCommand` now splits the script on the separators
  that begin a new command and requires a segment to equal the invocation
  exactly, with both counterexamples as regression cases.
- `local-dev`'s own description and its AGENTS.md index row both still promised
  "where a new dependency must be declared" — the rules round 13 moved out of it
  entirely. Both now describe what the skill actually carries: the reasoning
  behind those rules. Recorded listing share updated to 1,199/4,000.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 04:24:48 -04:00
cliffhall
0798ecf5fa chore: address Copilot review round 14
- The CI wiring vouch accepted any syntactically-executable `run:`, so a step
  moved into a release-only job or put behind `if: false` still reported as
  wired while PR CI validated nothing. It now requires an UNCONDITIONAL step:
  any `if:` on the job or the step disqualifies it, since this guard cannot
  evaluate an Actions expression and "runs sometimes" is not the property being
  asserted. Seven predicate cases, including the two conditional shapes and a
  multi-line `run:` that must still count.
- The `local:gate` stage list now has one owner, `docs/quality-gate.md`. It had
  three copies, and they drifted apart across rounds 5, 6 and 8 of this review —
  which is the argument the PR itself is making, applied to itself.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 04:11:25 -04:00
cliffhall
f13d415eae chore: address Copilot review round 13
- skill-manifest: an eval file may only expect its own skill. A foreign
  `expect: "other-skill"` passed whenever that other skill fired, so the file
  reported a measurement of something it does not describe — and it satisfied
  the positive-case requirement while doing so, masking the absence of a real
  positive. Both halves covered by a regression case.
- local-dev: stop restating the dependency-placement rules. They are normative
  and belong only in AGENTS.md; the skill now points there and carries what a
  rule cannot — what each one is defending against, what it looked like when it
  was violated (#1970, #2082, #2067, #1952, #1896, #2058), and how to tell you
  have hit it. Including the two traps that make this class hard to see: the
  ordinary test tiers cannot catch a bundling miss, and probing one by hand with
  `node -e` falsely succeeds.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 03:57:25 -04:00
cliffhall
97122b508b chore: address Copilot review round 11
- skill-manifest: accept CRLF on the opening fence. A Windows checkout with
  `core.autocrlf` has `---\r\n`, the fence is still the first line, and the
  authoritative validator accepts it — so `npm run validate` was failing a valid
  manifest, and failing it *before* that validator ever ran. The closing fence
  now strips a preceding CR too, so the YAML handed to the parser is clean.
  Regression cases for CRLF, mixed endings, and CRLF-with-no-body, plus cases
  proving the first-line rule and the whole-line terminator rule still hold with
  CRLF. Verified by mutation.
- docs/quality-gate: `verify:skills` requires eval cases only from model-invoked
  skills; the seven name-only ones intentionally have none, and the summary read
  as an instruction to add them everywhere.
- docs/quality-gate: the `test:scripts` inventory said four new suites;
  `claude-cli.test.mjs` makes five.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 03:31:39 -04:00
cliffhall
23c59d7a17 chore: address Copilot review round 10
The CI half of the wiring vouch matched the workflow's raw text, so a YAML
comment, a step `name:`, or an unused anchor mentioning `npm run
verify:skills:cli` satisfied it — including this guard's own docblock wording.
Deleting the real step would then have left `verify:skills` green, which is the
vacuous-guard failure the vouch exists to prevent.

It now reads the workflow's executable regions via `extractExecutableRegions`,
the same helper the #2146 workflow gate is built on, and matches only a `run:`.
Covered by three should-not-count cases (comment, step name, workflow input)
plus a multi-line `run:` that must count; verified by mutation — restoring the
substring check fails them.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 03:17:37 -04:00
cliffhall
17068fc89e chore: address Copilot review round 9
- Enrol both skill gates in the repo's gate-integrity vouch cycle. A gate that
  stops being invoked fails silently with every fixture test green, which is the
  failure the existing cycle exists to prevent. `verify:format-coverage` now
  vouches for `verify:skills`, `verify:skills` vouches back, and it additionally
  asserts the two links nothing else could see: `verify:skills:cli` present in
  `local:gate` and in the workflow — neither is reachable from `validate` or
  from any test. `checkWiring` is pure and unit-tested, with a live case against
  the real package.json and workflow so the fixtures cannot drift. All four
  links verified by mutation.
- The existing sibling-vouch fixture is now a table over every sibling, so
  adding a fourth guard later does not need a new test.
- release skill: stop hard-coding `2.3.0`, which has already shipped — the tag
  command aborted on an existing tag and the branch template named the wrong
  release. Both now read the version from `package.json` (the tag from
  `origin/main`'s), with a warning never to copy a version out of the file.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 03:04:03 -04:00
cliffhall
5078a5bc50 chore: address Copilot review round 8
- skill-eval: `only` no longer narrows `ours`. A negative case means "no skill
  of this repo fired", so filtering the repo's other model-invoked skills out of
  that set let `skills:eval -- testing` score a `local-dev` invocation as
  somebody else's skill and pass a negative case it should fail. Every
  model-invoked skill is now inspected; `only` applies when enqueueing.
  `collectCases` takes a skills directory so the focused-mode regression is
  actually testable — verified by mutation: restoring the early filter fails it.
- The workflow's `validate` step comment said two coverage guards; it runs four.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 02:47:06 -04:00
cliffhall
09b8f0e834 chore: address Copilot review round 7 (Windows .cmd shims)
Node has refused shell-free `.cmd`/`.bat` spawns since CVE-2024-27980, and an
npm-installed `claude` is a `.cmd` shim on Windows — so every shell-free spawn
of it reported ENOENT there, each in a quiet way: `verify:skills` skipped its
authoritative hand-off, `verify:skills:cli` downloaded the pinned package it
already had (and failed offline), and `skills:eval` said "not on PATH".

- New `scripts/lib/claude-cli.mjs` is the one place that decides how `claude` is
  spawned: `shell` on win32, arguments through the existing `winShellArgs`
  quoting so `cmd.exe` cannot re-parse them as syntax (#1939). Unit-tested with
  the platform injected, since a win32-only bug no runner here executes is
  exactly the kind that ships.
- `verify-skills`, `verify-skills-cli` and `skill-eval` all route their probes
  and spawns through it.
- skill-eval sends the prompt on **stdin** rather than in argv. `claude -p` reads
  a piped prompt (verified end to end), which sidesteps `cmd.exe` metacharacter
  re-parsing entirely rather than relying on quoting a model-visible string, and
  keeps the prompt out of the process table.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 02:32:30 -04:00
cliffhall
c3c8a33e2c chore: address Copilot review round 6
- verify-skills: the best-effort `plugin validate` hand-off now also requires
  the EXACT pinned CLI. `local:gate` runs `validate` before the pinned
  `verify:skills:cli` step, so a newer local CLI could reject skills that CI's
  pinned validator accepts — and the gate would exit there, never reaching the
  reproducible check. One schema everywhere, or none.
- Dropped `PLUGIN_VALIDATE_MIN_VERSION`: with both validators on the exact pin
  it was read only by its own test, and a constant nothing enforces is a claim
  nothing enforces. Its test now pins what actually matters — that equality is
  distinguishable from newer as well as older.
- The CI step's comment described the old floor behavior.
- issue-create: the duplicate check searched only open issues, so a completed or
  rejected match was invisible to a procedure that says never to duplicate.
  Searches all states and reports state/stateReason.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 02:15:02 -04:00
cliffhall
75ab1b069f chore: address Copilot review round 5
- verify-skills-cli: use a local CLI only when it matches the pin EXACTLY. A
  floor let a maintainer on a newer CLI validate against a different schema than
  CI's, so the same `local:gate` could disagree across machines — the failure a
  pin exists to prevent.
- verify-skills-cli: extract `runValidator({probe, spawn, log, error})` and
  cover the orchestration — validator accepts, validator rejects, spawn fails,
  and the three unusable-probe shapes. On a machine with a working CLI an actual
  run only ever walks the happy path.
- pr-flow: stop putting the `gh` token in curl's argv, where any local process
  can read it off the process table. Feed the header through `curl --config -`.
- docs/quality-gate: three places had drifted — the CI tier row, the
  `local:gate` chain, and the `test:scripts` inventory (which now names all four
  suites this PR adds, not two).

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 02:02:51 -04:00
cliffhall
d05e76d470 chore: address Copilot review round 4
- verify:skills:cli — the authoritative validator is now a GUARANTEED step in
  `local:gate` and CI, not a best-effort hand-off. It resolves the CLI itself
  (an installed one at 2.1.233+, else a pinned `npx -y`), so both tiers run the
  same script and cannot drift. `verify:skills` keeps skipping when the CLI is
  absent, which is right for `validate`: fast, offline, no Claude Code required.
- skill-manifest: the 1,536-char listing-entry cap covers `description` +
  `when_to_use` together; checking the description alone passed an over-cap
  entry. `when_to_use` is now parsed, validated and counted, with boundary tests
  at 1,536/1,537.
- verify-skills: report the budget as *this repo's contribution*. The real
  listing budget is shared with bundled skills and the contributor's own
  ~/.claude/skills, none of which is visible from here, so the number was
  claiming more than it measured.
- skill-eval: a negative case now means "no skill OF THIS REPO'S fired". It
  failed on any Skill invocation, so a contributor with an unrelated global
  skill got a false failure about their environment rather than these skills.
- docs/architecture + local-dev: the Node backend runner gets its
  `@inspector/core` alias from `clients/web/tsup.runner.config.ts`, not from
  Vite. The old text sent runner work to the wrong config.
- local-dev: the tarball does not ship "only each client's build/" — the
  allowlist also carries `clients/web/dist`, `clients/web/static` and the
  install helper. States the invariant that actually matters instead.
- issue-triage: the test for needing triage is "no board card". Adding "and no
  milestone" would skip the approved-but-unboarded case the same section then
  handles. Also corrected the `$M` note: the lookup is `--state all`, so closed
  issues DO have entries — `isopen` is about the invariants, not the lookup.
- issue-create: replaced "all five" with a per-version table. Steps 3 and 6
  (milestone, Priority) do not exist for v1, so the old list opened with a rule
  the same skill later tells you to violate.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 01:47:43 -04:00
cliffhall
7e0bb8b58d chore: address Copilot review round 3
- CI now installs a pinned `claude` CLI and runs `claude plugin validate` as its
  own step. The guard skips the authoritative validator when the CLI is absent,
  which without this meant "always, in CI" — so the acceptance criterion was
  aspirational. Verified it needs no authentication (clean HOME).
- skill-eval: classify a run from its terminal `result` event, not its exit
  code. With `--max-turns 1` a run in which a skill FIRES necessarily hits the
  limit and the CLI exits 1 (`error_max_turns`, `num_turns: 2`), so last round's
  blanket nonzero rejection threw away exactly the observations the eval counts.
  A run with no terminal `result`, or an unusable subtype, still rejects.
- skill-eval: fail collection on a skill that does not parse, a model-invoked
  skill with no eval file, unreadable JSON, or invalid cases. Skipping them let
  `skills:eval` exit 0 while silently omitting the skill just broken.
- verify-skills: `main()` takes an optional skills directory, and
  `verify-skills.main.test.mjs` drives it over fixtures — missing SKILL.md,
  malformed frontmatter, undeclared invocation mode, missing/invalid evals,
  budget overflow (and that name-only skills do not count toward it), and
  multi-offender reporting. Matches the sibling guards' *.main.test.mjs
  precedent; without it the gate could stop enforcing while the parser tests
  stayed green.
- issue-create: the note claimed an issue arrives unlabeled, contradicting
  .github/ISSUE_TEMPLATE — blank issues are disabled, the bug form applies
  `bug`, the feature form applies `enhancement` + `v2`. Now a table of what each
  form sets and what triage still has to add.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 01:26:19 -04:00
cliffhall
ca6828916f chore: address Copilot review round 2
- verify-skills: version-gate the optional `claude plugin validate`. Probing
  only for the executable failed the MANDATORY gate for anyone on a CLI older
  than 2.1.233, which has no such subcommand and exits nonzero on it. Skips
  below the floor; a failure now means a supported validator rejected the skills.
- skill-eval: match the invoked skill structurally. A substring test counted
  `{"skill":"not-testing"}` as a hit for `testing` and inflated the rate.
- skill-eval: `runOnce` → exported `runPrompt` with an injectable `spawnFn`, so
  the nonzero-exit branch is actually pinned. Reverting it now fails a test
  (verified by mutation) rather than leaving the suite green.
- board-ops: filter the item lookups by repository. Issue numbers are repo-local
  and these are org boards — #11 really does carry a `servers` card — so an
  unfiltered `select` can move or delete the wrong card, or pass two ids.
- issue-triage: the audit's `own()` filter dropped draft cards, which carry no
  `.content.repository`, so the "non-Issue on a board" check reported 0 while
  the invariant was being violated. Verified against a fixture.
- pr-flow: cut the branch from the base it will target. The blanket "cut from
  v2/main" was wrong for v1, whose history is unrelated.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 01:07:41 -04:00
cliffhall
4520e6aa2c docs: state the gap the gate cannot cover
`verify:skills` checks that a skill is well-formed and that its cases exist;
only `skills:eval` observes whether it actually fires, and that cannot be
gated. Say so in the script header and in AGENTS.md rather than leaving the
split to be inferred.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 00:53:49 -04:00
cliffhall
a51e62dfa1 chore: address Copilot review round 1
- skill-manifest: the closing `---` fence must occupy a whole line. `---oops`
  was accepted as a terminator, silently truncating the frontmatter and passing
  validation — the exact failure the guard exists to catch. Regression cases
  added (prefix, trailing whitespace, EOF, CRLF).
- skill-eval: reject a nonzero `claude -p` exit instead of resolving it as an
  empty observation. An auth error or rate limit was scoring as "no skill
  invoked", which passes every negative case and reads as a trigger miss on
  every positive one.
- skill-eval: extract `collectSkillInvocations` / `sampleHit` behind a `main()`
  guard and cover them in `skill-eval.test.mjs` — the stream parsing and the
  process-outcome handling are unreachable from an eval run's happy path.
- docs/test-servers: annotate the era on every catalogue row. Six legacy rows
  carried no annotation under a "unless noted, Modern" default, so the doc sent
  readers to the wrong era for the feature they were trying to see.
- docs/quality-gate: `validate` runs four guards, not three; add the
  `verify:skills` row and the two new `test:scripts` suites; fix the `#publishing`
  fragment left dangling by the split.
- issue-create / pr-flow: delegate the card steps to `/board-ops` rather than
  restating its option IDs, which are explicitly unstable.
- issue-triage: the audit counted presence, not cardinality — an issue carrying
  both `v1` and `v2` passed a check that claims "exactly one". Same for the five
  type labels. Also scope the two milestone checks to board #28.
- AGENTS.md: codify the v1 milestone exception. Every milestone is a v2 release
  bucket, so a v1 issue has none to take and the Incoming-milestone invariant
  cannot apply to board #11.
- clients/launcher/README: its pointer promised an explanation of why the cli/tui
  manifests ship, which the extracted guide does not make.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 00:52:45 -04:00