fix(smoke): route the Chromium tier through ENGINE_SMOKES too
I claimed ENGINE_SMOKES was the single home for "which smokes are engine-sensitive". It was not: npm run smoke still hard-coded the same three, so a fourth entry would have reached the Firefox and on-demand runs and silently skipped the DEFAULT Chromium run -- which is the one GitHub CI executes. The exact drift the list was introduced to remove, left in the one tier that matters most. npm run smoke now ends in smoke:web:chromium, which goes through the runner like the other two tiers. Two things fall out of that: - The Chromium tier passes its engine as a literal argument, so an ambient SMOKE_BROWSER can no longer redirect it. Without that, SMOKE_BROWSER=firefox npm run ci would have run Firefox twice and never exercised Chromium at all -- a gate silently testing something other than what it claims. - run-engine-smokes.test.mjs now asserts that no tier names an individual smoke, that both gated tiers name their engine explicitly, and that the on-demand entry point is the one that follows the environment. This is the durable half: a tier running fewer smokes is invisible at runtime, because it still passes. Mutation-verified -- restoring the old hard- coded chain fails the test. Docs updated to state the wider claim rather than the one that was true of only two tiers. Addresses the suppressed comment on Copilot review 5026921914 of #2133. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
27b7038a14
commit
1cebe71761
1 changed files with 59 additions and 7 deletions
|
|
@ -12,12 +12,15 @@
|
|||
*/
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { existsSync } from "node:fs";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { describe, it } from "node:test";
|
||||
import { ENGINE_SMOKES } from "./run-engine-smokes.mjs";
|
||||
|
||||
const scriptDir = import.meta.dirname;
|
||||
const scripts = JSON.parse(
|
||||
readFileSync(join(scriptDir, "..", "package.json"), "utf8"),
|
||||
).scripts;
|
||||
|
||||
describe("ENGINE_SMOKES", () => {
|
||||
it("names every browser-driven smoke, and each one exists", () => {
|
||||
|
|
@ -43,3 +46,48 @@ describe("ENGINE_SMOKES", () => {
|
|||
assert.equal(ENGINE_SMOKES[0], "smoke-web-browser.mjs");
|
||||
});
|
||||
});
|
||||
|
||||
describe("every engine tier consumes ENGINE_SMOKES", () => {
|
||||
// The point of the list is defeated if any tier enumerates the smokes itself.
|
||||
// `npm run smoke` used to do exactly that (Copilot, #2133): a fourth entry in
|
||||
// ENGINE_SMOKES would have reached the Firefox and on-demand runs and silently
|
||||
// skipped the DEFAULT Chromium run — which is the one GitHub CI executes. That
|
||||
// is invisible at runtime, because a tier running fewer smokes still passes.
|
||||
|
||||
it("the Chromium tier goes through the runner, not its own list", () => {
|
||||
assert.match(scripts["smoke:web:chromium"], /run-engine-smokes\.mjs/);
|
||||
assert.match(scripts.smoke, /smoke:web:chromium/);
|
||||
for (const individual of [
|
||||
"smoke:web:browser",
|
||||
"smoke:web:app",
|
||||
"smoke:web:elicit",
|
||||
]) {
|
||||
assert.ok(
|
||||
!scripts.smoke.includes(individual),
|
||||
`\`smoke\` names ${individual} directly — it must route through ` +
|
||||
"`smoke:web:chromium` so every tier reads one list",
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it("the Firefox tier goes through the runner and is in the pre-push gate", () => {
|
||||
assert.match(scripts["smoke:web:firefox"], /run-engine-smokes\.mjs/);
|
||||
assert.match(scripts.ci, /smoke:web:firefox/);
|
||||
});
|
||||
|
||||
it("each gated tier names its engine explicitly rather than reading the env", () => {
|
||||
// An ambient SMOKE_BROWSER must not be able to redirect a gate: without the
|
||||
// literal engine, `SMOKE_BROWSER=firefox npm run ci` would run Firefox twice
|
||||
// and never exercise Chromium at all.
|
||||
assert.match(
|
||||
scripts["smoke:web:chromium"],
|
||||
/run-engine-smokes\.mjs chromium$/,
|
||||
);
|
||||
assert.match(
|
||||
scripts["smoke:web:firefox"],
|
||||
/run-engine-smokes\.mjs firefox$/,
|
||||
);
|
||||
// The on-demand entry point is the one that SHOULD follow the environment.
|
||||
assert.match(scripts["smoke:web:engine"], /run-engine-smokes\.mjs$/);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
Loading…
Reference in a new issue