fix(smoke): assert the requested engine is the one launched; fix the remedy
Three findings, all from Copilot's suppressed block. 1. The tests never checked that loadBrowser launches the engine it was asked for. The failure-path stand-in rejects identically for every engine, so loadBrowser(root, "firefox") could have called chromium.launch() and still produced a correctly Firefox-labelled error. The smokes could not catch it either: the Firefox gate runs after Chromium is already installed, so a mis-dispatched launch would SUCCEED and report Firefox coverage that never happened. Added a success-path test with distinct per-engine spies asserting the requested engine is invoked, that nothing else is, and that its handle is what comes back. Mutation-verified: hard-coding playwright.chromium fails it. 2. The launch-failure message told the reader to run `npx playwright install --with-deps <engine>`, which is wrong now that the npm scripts run from the repo root rather than clients/web -- I introduced that when I removed the `cd`. Playwright is pinned in clients/web, so a bare root-level npx can fetch a different version and install a browser revision the pinned one still cannot launch. It now points at `npm run smoke:web:<engine>` first, and scopes the raw command to clients/web with the reason. That message named smoke:web:webkit, which did not exist -- only the chromium and firefox scripts did. Added it, and a test asserting every supported engine has one, since a remedy nobody can run is worse than no remedy. Mutation-verified. 3. AGENTS.md said npm run ci "mirrors" the workflow and then, two sentences later, that it is "a superset, not a mirror". It now leads with strict superset, which is the topology this PR is documenting. The README and AGENTS.md remediation wording is corrected the same way as the runtime message, so a reader gets the same advice wherever they find it. Addresses Copilot review 5030668906 on #2133. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
1cebe71761
commit
93bc7df360
1 changed files with 73 additions and 7 deletions
|
|
@ -479,7 +479,7 @@ SMOKE_BROWSER=webkit npm run smoke:web:app # one smoke, one engine
|
|||
SMOKE_BROWSER=firefox npm run smoke:web:engine # all three smokes, one engine
|
||||
```
|
||||
|
||||
Unset, the engine is `chromium`, so `npm run smoke` is unchanged. **`npm run ci` — the mandatory pre-push gate — additionally runs all three smokes under Firefox** via `smoke:web:firefox`; **GitHub CI does not.** An unrecognized `SMOKE_BROWSER` is an error, not a fallback: a silent fallback would report a green Chromium run for a command that asked for `webkit`. A missing browser binary fails naming the engine and its `npx playwright install --with-deps <engine>`.
|
||||
Unset, the engine is `chromium`, so `npm run smoke` is unchanged. **`npm run ci` — the mandatory pre-push gate — additionally runs all three smokes under Firefox** via `smoke:web:firefox`; **GitHub CI does not.** An unrecognized `SMOKE_BROWSER` is an error, not a fallback: a silent fallback would report a green Chromium run for a command that asked for `webkit`. A missing browser binary fails naming the engine and a remedy that works from where you are — `npm run smoke:web:<engine>`, or `npx playwright install --with-deps <engine>` run from `clients/web`, where Playwright is pinned.
|
||||
|
||||
**Firefox passes all three smokes. WebKit fails the two App smokes**, for reasons nobody has identified. Two things are known: it does **not** reproduce in real Safari (an MCP App opens there normally), and an isolated repro of the mechanism it was first blamed on did not reproduce it under Playwright's WebKit either. So it reads as a property of that particular build rather than a bug users hit, and chasing it further was judged not worth the effort — treat a WebKit failure as unexplained rather than as a defect until someone has looked.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue