fix(smoke): assert the requested engine is the one launched; fix the remedy

Three findings, all from Copilot's suppressed block.

1. The tests never checked that loadBrowser launches the engine it was
   asked for. The failure-path stand-in rejects identically for every
   engine, so loadBrowser(root, "firefox") could have called
   chromium.launch() and still produced a correctly Firefox-labelled
   error. The smokes could not catch it either: the Firefox gate runs
   after Chromium is already installed, so a mis-dispatched launch would
   SUCCEED and report Firefox coverage that never happened.

   Added a success-path test with distinct per-engine spies asserting the
   requested engine is invoked, that nothing else is, and that its handle
   is what comes back. Mutation-verified: hard-coding playwright.chromium
   fails it.

2. The launch-failure message told the reader to run
   `npx playwright install --with-deps <engine>`, which is wrong now that
   the npm scripts run from the repo root rather than clients/web -- I
   introduced that when I removed the `cd`. Playwright is pinned in
   clients/web, so a bare root-level npx can fetch a different version and
   install a browser revision the pinned one still cannot launch. It now
   points at `npm run smoke:web:<engine>` first, and scopes the raw
   command to clients/web with the reason.

   That message named smoke:web:webkit, which did not exist -- only the
   chromium and firefox scripts did. Added it, and a test asserting every
   supported engine has one, since a remedy nobody can run is worse than
   no remedy. Mutation-verified.

3. AGENTS.md said npm run ci "mirrors" the workflow and then, two
   sentences later, that it is "a superset, not a mirror". It now leads
   with strict superset, which is the topology this PR is documenting.

The README and AGENTS.md remediation wording is corrected the same way as
the runtime message, so a reader gets the same advice wherever they find
it.

Addresses Copilot review 5030668906 on #2133.

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-26 09:33:21 -04:00
commit 93bc7df360

View file

@ -187,8 +187,11 @@ export async function loadBrowser(
return await playwright[browserName].launch({ headless: true });
} catch (err) {
throw new Error(
`${browserName} failed to launch — run \`npx playwright install --with-deps ${browserName}\`, ` +
`which fetches the browser and (on a bare Linux box) its system libraries ` +
`${browserName} failed to launch — run \`npm run smoke:web:${browserName}\`, ` +
`or from clients/web run \`npx playwright install --with-deps ${browserName}\` ` +
`if you also need its system libraries. Playwright is pinned in ` +
`clients/web, so running that from the repo root can fetch a different ` +
`version and install a browser revision this one still cannot launch ` +
`(${err instanceof Error ? err.message : String(err)})`,
);
}