`local:gate` ran every client's unit suite bare inside `validate` and
again instrumented inside `coverage` — ~80s of a ~370s run re-executing
exactly the files the coverage pass runs a few minutes later. CI pays
nothing for that duplication because its two jobs run in parallel; the
gate runs serially, so it paid all of it.
The gate's first stage is now `local:validate`: the same `validate:guards`
and `validate:core`, then each client's new `check` script (`format:check`
+ `lint` + `typecheck` + `build`) instead of its `validate`, which is now
literally `check && test`. `npm run validate` and both CI jobs are
unchanged; `local:validate` sits in the `local:` namespace so the
workflow guard keeps it out of CI by construction.
The "strict superset" claim in AGENTS.md is reworded to what is now
true — every CI *check*, not every CI *invocation* — with the reasoning:
`@vitest/coverage-v8` reads V8's precise-coverage profiler and rewrites
no source, so the instrumented pass runs byte-identical code and is only
slower, the stricter direction for the failure class this repo sees.
Measured on the same worktree: 370s -> 262s wall (EXIT=0 both), the
four removed legs summing to 78.5s in the baseline run's own profile.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsmUAZ6aLUAFqenjJKSSRp
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, eleventh review:
- `Set-Item Env:SMOKE_BROWSER firefox` — PowerShell's Env: DRIVE, the other
standard way to write the environment there — matched none of the patterns.
Covered now, positional or via -Path/-Value, for Set-Item/New-Item and their
-ItemProperty forms. A `Get-Item` READ is still not a finding, which has its
own negative case: reading the variable is not setting it, the same
distinction as round 5's `echo "SMOKE_BROWSER: firefox"`.
- Two scope summaries had drifted behind the traversal, both saying the guard
reads only `run` and `env`/`with`: the module header's "ONLY EXECUTABLE
POSITIONS" paragraph, and the copilot-instructions mirror. Both list the real
set now — `run`, the `shell` command template at all three levels, and
`env`/`container.env`/`with`. The mirror one is the AGENTS.md rule this repo
states about itself, so it is a rule violation rather than a typo.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, ninth and tenth reviews:
- A custom `shell:` is a COMMAND TEMPLATE that Actions runs around every
script (`{0}` is the script path), so `shell: npm run local:gate && bash {0}`
invoked the gate while the `run:` value beside it looked harmless. Step
`shell`, and `defaults.run.shell` at both workflow and job level, are
command regions now. An ordinary `bash`/`pwsh`/`cmd` matches no rule, so
naming them costs nothing.
- GitHub's multiline environment-file syntax (`SMOKE_BROWSER<<EOF`, value,
delimiter) bypassed the assignment patterns. It is reported as an override
whose value the guard cannot READ rather than parsed: the value can arrive
through any number of shapes — a brace group redirected to $GITHUB_ENV, a
heredoc nested in another — and a parser handling some of them would be a
coverage claim that holds only for the shapes someone thought of. Nobody
spells `chromium` that way, and if they did, `SMOKE_BROWSER=chromium` is the
fix.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, ninth review: the .NET arm required a QUOTED second argument, so
`SetEnvironmentVariable("SMOKE_BROWSER", $engine)` — or a ${{ }} expression
there — set the environment with the guard silent. An engine the guard cannot
read is forbidden for the same reason `smoke:web:engine` is, so the arm now
accepts an unquoted value too. Bare values stop at `,` or `)`, so a third
argument ("Process") is not swallowed — pinned by an allowed three-argument
case beside the dynamic one.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, eighth review: the bare-value pattern ran to whitespace, so a QUOTED
assignment swallowed the closing quote — `echo "SMOKE_BROWSER=chromium" >>
$GITHUB_ENV` captured `chromium"`, which is not `chromium`, and the guard
rejected a PERMITTED assignment. A false positive on the allowed value is the
worse direction to be wrong in: it fails a workflow that is doing the right
thing, and there is no way to satisfy the guard except to stop using the
supported spelling.
The value is now a quoted string or a bare run stopping at a quote or a `;`
(PowerShell's separator), so all three dialects unwrap the same way.
Regression cases for the permitted quoted `chromium` in $GITHUB_ENV, in a
cmd-style `set`, and single-quoted in PowerShell — each paired with the
firefox spelling that must still fail.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, seventh review:
- The unreadable-name rules only knew GitHub expression syntax, so
`ENGINE: firefox` plus `npm run smoke:web:$ENGINE` ran a forbidden pass with
the guard silent — the shell resolves the name, which makes it exactly as
unreadable as a ${{ }} expression and forbidden for the same reason. The
pattern keys off a leading $ or %, covering $VAR, ${VAR}, $env:VAR and cmd's
%VAR% in one, for both the engine and `local:` families.
- The env-key comparison was case-sensitive, but Windows environment variables
are not: `env: { smoke_browser: firefox }` on a windows-latest job sets the
very variable Node reads as SMOKE_BROWSER. Compared case-insensitively now,
as is the POSIX assignment pattern. On a POSIX runner that spelling would be
a different variable and harmless, so this is stricter there than it needs to
be — the right direction for a guard, and the alternative is a verdict that
depends on a `runs-on` which may itself be an expression.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, sixth review: the run-script rule matched POSIX `NAME=value` only.
PowerShell is the DEFAULT shell on a Windows runner, so
`$env:SMOKE_BROWSER = "firefox"; npm run smoke` redirected the suite with the
guard silent — on exactly the workflow least likely to be read closely.
Three patterns now, one per dialect: POSIX (which also covers cmd's
`set NAME=…` and the GitHub-native `>> $GITHUB_ENV` append, since both carry
the same substring), PowerShell's `$env:NAME =` and `${env:NAME} =`, and the
.NET SetEnvironmentVariable call reachable from PowerShell. Regression cases
for all five spellings, plus a negative for a PowerShell assignment of
chromium.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, fifth review:
- The SMOKE_BROWSER rule was a blanket text match across every region, so a
script that merely PRINTS the variable — `echo "SMOKE_BROWSER: firefox"` —
or a `with:` input mentioning it was reported as an override. It is
kind-aware now: an `env:` entry counts only when SMOKE_BROWSER is literally
its key (recognized structurally, not by regex), a `run:` script only on a
shell assignment, and a `with:` value not at all — whatever it names is
already covered by the script rules. A shell assignment inside an `echo` is
indistinguishable without parsing the shell, and inside `run:` erring toward
the finding is the safe direction; that is stated in the header.
- `jobs.<id>.container.env` is job-wide too — every step in the container sees
it — so an override there redirected `npm run smoke` with the guard silent.
Now traversed, and a `container:` given as a bare image string is ignored
rather than tripping over.
- Nothing asserted the FIRST acceptance criterion: reintroducing a `ci` or
`ci:*` root script left test:scripts green. Asserted directly over the
manifest's keys now, and checked to actually bite by reviving `ci` and
watching the suite fail.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, fourth review — four findings, all real:
- Matching pairs by KEY NAME anywhere in the tree scanned things that execute
nothing. Workflow input ids are user-defined, so an input legitimately named
`env` put its own description and default in front of the rules — failing
the gate on documentation OF the gate. The walk is structural now: workflow
`env`, job `env`/`with`, step `run`/`env`/`with`, and nothing else.
- YAML aliases were read as scalars, which have no `.value` on an alias node —
so `run: *command`, with the command anchored in a metadata scalar the walk
never visits, invoked anything at all while the guard saw an empty string.
Aliases are resolved, and reported at the line where they are USED.
- `file` was accepted for diagnostics and then not forwarded, so a malformed
real workflow was reported as `<workflow>`.
- One negative fixture was a YAML comment, which the parser strips — so it
asserted nothing. Inside a `run:` block the comment text really does survive
into the scalar, so the "a comment invokes nothing" rule is now applied
there, where it actually has to be, and the fixture is a real block scalar.
The table's fixtures are schema-valid workflows now rather than bare `- run:`
fragments, which a structural walk correctly cannot see.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, third review — and the second finding is really a verdict on the
approach, so the fix is the approach:
- `SMOKE_BROWSER:` with no value is valid YAML that sets the variable to the
empty string, which `resolveBrowserName` rejects as present-but-empty rather
than treating as unset. The value pattern required \S+, so the guard said
nothing. It is \S* now, with its own message.
- The hand-rolled opener could not see `run: | # explanation` or the flow
mapping `env: { SMOKE_BROWSER: firefox }`. That is the third spelling it has
missed in three rounds, each one a forbidden invocation that would have
passed while test:scripts stayed green — worse than no guard, because it
reports a coverage that is not there. YAML has too many spellings to
recognize by indentation, so extractExecutableRegions now parses with the
`yaml` package. It is already a root dependency, and this file is not in the
published files allowlist, so nothing about the tarball changes.
A parse error now throws rather than yielding no regions: a workflow the guard
cannot read must not pass as a workflow with nothing in it.
Verified by injecting a flow-mapping SMOKE_BROWSER override into the real
main.yml — the suite fails, and goes green again when removed.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, second review — both findings were real:
- A `${{ … }}` body can itself contain braces, so the `[^}]*\}\}` arm added in
the previous round stopped at the first `}` and missed
`smoke:web:${{ format('{0}', matrix.browser) }}` — the very case it was
added for. It now keys off the `${{` prefix, which nesting cannot defeat.
- Scanning every non-comment line made ordinary metadata a finding:
`name: Explain why smoke:web:firefox stays local` executes nothing and was
reported as an engine invocation, contradicting the header's "what it
forbids, and nothing more". The scan now walks executable positions only —
`run:` scalars (inline and block) and `env:`/`with:` values — via an
exported `extractExecutableRegions`, hand-rolled because `test:scripts` is
dependency-free by design. The narrowing is a stated limit, not a hidden
one: an execution vector outside those three positions is not seen.
Verified by injecting a nested-brace SMOKE_BROWSER expression into the real
main.yml — the suite fails, and goes green again when removed.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot, first review:
- The guard read only literal names, so `npm run smoke:web:${{ matrix.browser }}`
and `npm run local:${{ matrix.task }}` walked past it — the one form in which
a forbidden engine reaches CI without anyone typing its name. It already
refused an expression-valued SMOKE_BROWSER for that exact reason; this applies
the reasoning consistently. Its limit is now stated: a fully opaque
`npm run ${{ matrix.script }}` names no family and is not detectable.
- The module header claimed the gate adds "the non-Chromium engine passes";
it adds Firefox only, and WebKit belongs to neither tier.
- eslint.config.js:33 still said `npm run ci` — the last stale reference.
- AGENTS.md read "everything from `smoke` rightward is `local:gate` only",
which is two claims wrong: smoke and Storybook run in GitHub CI too, and
`pack:verify` is in neither tier.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
The root `ci` script was never what runs in GitHub CI, and the name worked
against us three ways: it collides with the built-in `npm ci` (which does
something else entirely and is one keystroke away), it invites the
cross-engine passes into a workflow by looking like the thing a workflow
should call, and it blurs "GitHub CI" and "the local gate" in every doc
that names it.
- Rename `ci` → `local:gate` and `ci:storybook` → `local:storybook`, with
no back-compat alias: `npm run ci` now fails with npm's missing-script
error, which prints the available scripts.
- Add `scripts/lib/workflow-gate.mjs` + its `test:scripts` suite, which
runs over the repo's real `.github/workflows/**` and fails on an
invocation of a `local:*` script, of a non-Chromium engine pass (derived
from SUPPORTED_BROWSERS), or of `smoke:web:engine`, or on `SMOKE_BROWSER`
set to anything but a literal `chromium`. It does not forbid
`npm run smoke`, `smoke:web:chromium` or `smoke:tui` — those belong in CI
and are there today. Comments are not scanned, so the workflow can still
explain itself. This is the durable half; the rest is prose.
- Document the two-tier model once, in the root README ("Two tiers: GitHub
CI and the local gate"), and point the other docs at it rather than
restating the split.
- Sweep the ~40 stale references across README, AGENTS.md,
.github/copilot-instructions.md, the client READMEs, the roadmap doc and
the script headers. AGENTS.md and copilot-instructions.md change
together, per the mirror rule.
No gate changes what it actually runs.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
