fix(smoke): route the Chromium tier through ENGINE_SMOKES too

I claimed ENGINE_SMOKES was the single home for "which smokes are
engine-sensitive". It was not: npm run smoke still hard-coded the same
three, so a fourth entry would have reached the Firefox and on-demand runs
and silently skipped the DEFAULT Chromium run -- which is the one GitHub
CI executes. The exact drift the list was introduced to remove, left in
the one tier that matters most.

npm run smoke now ends in smoke:web:chromium, which goes through the
runner like the other two tiers.

Two things fall out of that:

- The Chromium tier passes its engine as a literal argument, so an
  ambient SMOKE_BROWSER can no longer redirect it. Without that,
  SMOKE_BROWSER=firefox npm run ci would have run Firefox twice and never
  exercised Chromium at all -- a gate silently testing something other
  than what it claims.

- run-engine-smokes.test.mjs now asserts that no tier names an individual
  smoke, that both gated tiers name their engine explicitly, and that the
  on-demand entry point is the one that follows the environment. This is
  the durable half: a tier running fewer smokes is invisible at runtime,
  because it still passes. Mutation-verified -- restoring the old hard-
  coded chain fails the test.

Docs updated to state the wider claim rather than the one that was true
of only two tiers.

Addresses the suppressed comment on Copilot review 5026921914 of #2133.

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-26 08:55:51 -04:00
commit 1cebe71761

View file

@ -12,12 +12,15 @@
*/
import assert from "node:assert/strict";
import { existsSync } from "node:fs";
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
import { describe, it } from "node:test";
import { ENGINE_SMOKES } from "./run-engine-smokes.mjs";
const scriptDir = import.meta.dirname;
const scripts = JSON.parse(
readFileSync(join(scriptDir, "..", "package.json"), "utf8"),
).scripts;
describe("ENGINE_SMOKES", () => {
it("names every browser-driven smoke, and each one exists", () => {
@ -43,3 +46,48 @@ describe("ENGINE_SMOKES", () => {
assert.equal(ENGINE_SMOKES[0], "smoke-web-browser.mjs");
});
});
describe("every engine tier consumes ENGINE_SMOKES", () => {
// The point of the list is defeated if any tier enumerates the smokes itself.
// `npm run smoke` used to do exactly that (Copilot, #2133): a fourth entry in
// ENGINE_SMOKES would have reached the Firefox and on-demand runs and silently
// skipped the DEFAULT Chromium run — which is the one GitHub CI executes. That
// is invisible at runtime, because a tier running fewer smokes still passes.
it("the Chromium tier goes through the runner, not its own list", () => {
assert.match(scripts["smoke:web:chromium"], /run-engine-smokes\.mjs/);
assert.match(scripts.smoke, /smoke:web:chromium/);
for (const individual of [
"smoke:web:browser",
"smoke:web:app",
"smoke:web:elicit",
]) {
assert.ok(
!scripts.smoke.includes(individual),
`\`smoke\` names ${individual} directly — it must route through ` +
"`smoke:web:chromium` so every tier reads one list",
);
}
});
it("the Firefox tier goes through the runner and is in the pre-push gate", () => {
assert.match(scripts["smoke:web:firefox"], /run-engine-smokes\.mjs/);
assert.match(scripts.ci, /smoke:web:firefox/);
});
it("each gated tier names its engine explicitly rather than reading the env", () => {
// An ambient SMOKE_BROWSER must not be able to redirect a gate: without the
// literal engine, `SMOKE_BROWSER=firefox npm run ci` would run Firefox twice
// and never exercise Chromium at all.
assert.match(
scripts["smoke:web:chromium"],
/run-engine-smokes\.mjs chromium$/,
);
assert.match(
scripts["smoke:web:firefox"],
/run-engine-smokes\.mjs firefox$/,
);
// The on-demand entry point is the one that SHOULD follow the environment.
assert.match(scripts["smoke:web:engine"], /run-engine-smokes\.mjs$/);
});
});