fix(smoke): assert the requested engine is the one launched; fix the remedy

Three findings, all from Copilot's suppressed block.

1. The tests never checked that loadBrowser launches the engine it was
   asked for. The failure-path stand-in rejects identically for every
   engine, so loadBrowser(root, "firefox") could have called
   chromium.launch() and still produced a correctly Firefox-labelled
   error. The smokes could not catch it either: the Firefox gate runs
   after Chromium is already installed, so a mis-dispatched launch would
   SUCCEED and report Firefox coverage that never happened.

   Added a success-path test with distinct per-engine spies asserting the
   requested engine is invoked, that nothing else is, and that its handle
   is what comes back. Mutation-verified: hard-coding playwright.chromium
   fails it.

2. The launch-failure message told the reader to run
   `npx playwright install --with-deps <engine>`, which is wrong now that
   the npm scripts run from the repo root rather than clients/web -- I
   introduced that when I removed the `cd`. Playwright is pinned in
   clients/web, so a bare root-level npx can fetch a different version and
   install a browser revision the pinned one still cannot launch. It now
   points at `npm run smoke:web:<engine>` first, and scopes the raw
   command to clients/web with the reason.

   That message named smoke:web:webkit, which did not exist -- only the
   chromium and firefox scripts did. Added it, and a test asserting every
   supported engine has one, since a remedy nobody can run is worse than
   no remedy. Mutation-verified.

3. AGENTS.md said npm run ci "mirrors" the workflow and then, two
   sentences later, that it is "a superset, not a mirror". It now leads
   with strict superset, which is the topology this PR is documenting.

The README and AGENTS.md remediation wording is corrected the same way as
the runtime message, so a reader gets the same advice wherever they find
it.

Addresses Copilot review 5030668906 on #2133.

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-26 09:33:21 -04:00
commit 93bc7df360

View file

@ -76,6 +76,7 @@
"smoke:web:engine": "node scripts/run-engine-smokes.mjs",
"smoke:web:chromium": "node scripts/run-engine-smokes.mjs chromium",
"smoke:web:firefox": "node scripts/run-engine-smokes.mjs firefox",
"smoke:web:webkit": "node scripts/run-engine-smokes.mjs webkit",
"smoke:launcher": "node scripts/smoke-launcher.mjs",
"pack:verify": "node scripts/install-smoke-browser.mjs chromium && node scripts/pack-and-verify.mjs",
"prepack": "npm run build",