ci: drop the Firefox smoke job; keep the engines as an on-demand tool

The job was trialled and did not earn its place. It was genuinely cheap —
~2 minutes, running in parallel with the 15-minute build job, so zero
added wall-clock — but across a dozen runs Firefox never once disagreed
with Chromium. It was paying runner time and carrying a real flake
surface (playwright install --with-deps runs apt-get update, which fails
whenever a third-party repo in the runner image breaks; that is what the
one red run was) to re-confirm a result already in hand.

Everything that made cross-engine checking possible stays: SMOKE_BROWSER,
the shared headless-browser helper, the installer, and the tests. So
"SMOKE_BROWSER=firefox npm run smoke:web:engine" is there for the moment
it is worth running — before touching the sandbox, the CSP builder or the
proxy page — without taxing every push to re-derive the same answer.

The workflow is now byte-identical to v2/main again; the publish jobs go
back to needing only build.

Docs say what is true rather than what was intended, and AGENTS.md
records the reasoning so the job is not re-added on the general argument
that cross-engine coverage is good — that argument was already accepted,
and the on-demand path is what serves it. Re-add it on evidence: a real
cross-engine regression a gate would have caught.

Also swept the now-stale "matrix" language out of the smokes and the
resolver, including the empty-SMOKE_BROWSER rationale, which no longer
leans on a matrix key that does not exist.

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-25 23:54:51 -04:00
commit 8a7ade7b08

View file

@ -5,8 +5,8 @@
* per process and then spends its whole run inside the happy path, so the branch
* that matters most — an unrecognized `SMOKE_BROWSER` — is dead code from their
* point of view. It is also the branch whose failure is silent rather than loud:
* a fallback to Chromium there would report a green run under a job labelled
* "webkit", claiming coverage that never ran.
* a fallback to Chromium there would report a green run to a caller who asked
* for another engine — coverage claimed that never ran.
*
* `loadBrowser`'s failure branches are covered here too, through its injectable
* `loadPlaywright`. The smokes cannot reach them by construction —
@ -36,20 +36,20 @@ describe("resolveBrowserName", () => {
});
it("rejects a present-but-empty value instead of defaulting", () => {
// This is the case worth being strict about. A CI expression that resolves
// to nothing (`SMOKE_BROWSER: ${{ matrix.browsr }}`, an undefined key) sets
// the variable to "" rather than removing it — so defaulting here would run
// Chromium inside a job named for another engine, which is the very
// false-coverage outcome the unrecognized-value branch exists to stop. Only
// an ABSENT variable may select the default.
// This is the case worth being strict about. `SMOKE_BROWSER="$ENGINE"` with
// ENGINE unset — or a CI expression naming a key that does not exist — sets
// the variable to "" rather than removing it, so defaulting here would
// silently run Chromium for a caller who asked for something else. That is
// the same false-coverage outcome the unrecognized-value branch exists to
// stop. Only an ABSENT variable may select the default.
for (const raw of ["", " ", "\t\n"]) {
assert.throws(
() => resolveBrowserName({ [BROWSER_ENV_VAR]: raw }),
(err) => {
assert.match(err.message, /set but empty/);
// Names the likely cause and the actual remedy — "unset it", which is
// different from "set it to nothing" and is the whole point here.
assert.match(err.message, /matrix/);
// Names the actual remedy — "unset it", which is different from
// "set it to nothing" and is the whole point here.
assert.match(err.message, /did not resolve/);
assert.match(err.message, /Unset it/);
return true;
},