Two findings from Copilot's suppressed block.
1. resolveBrowserName treated a present-but-empty SMOKE_BROWSER as unset
and returned chromium. The test that covered it even documented WHY
that is dangerous — GitHub Actions renders an undefined matrix key as
"" rather than omitting the variable — and then asserted the unsafe
behavior. A misspelled matrix key would run chromium inside a job named
for another engine and report green: the same false-coverage outcome
the unrecognized-value branch exists to prevent, reached by a typo
instead of a bad value.
Only an absent variable now selects the default. Empty throws, naming
the likely cause and the actual remedy ('unset it', which is not the
same as setting it to nothing). The workflow always renders a real
value, so nothing in CI is affected.
2. install-smoke-browser.mjs had no tests. Its I/O half runs on every
smoke and fails loudly; its argument half does not — every caller
passes nothing or the literal 'chromium', so the precedence rule is
asserted by nothing at runtime. It matters: pack:verify passes chromium
BECAUSE it launches chromium, and if the environment won,
SMOKE_BROWSER=webkit npm run pack:verify would install one engine and
launch another, surfacing as a misleading "Executable doesn't exist".
resolveRequestedBrowser is now exported and pure; execution moved
behind main() with the entrypoint guard verify-typecheck-coverage.mjs
uses, so importing the module for tests cannot trigger an install.
Both mutation-verified: restoring either behavior fails the tests that
claim to cover it.
Addresses the suppressed comments on Copilot's fifth review of #2133.
Signed-off-by: cliffhall <cliff@futurescale.com>
Two findings from Copilot's suppressed block.
1. The launch-failure message is #2086's acceptance criterion — a missing
browser must name THAT engine and its own install command — and it had
no test. The smokes cannot supply one: install-smoke-browser fetches
the binary first, so a passing smoke says nothing about what a failing
one prints, and that message is the deliverable. loadBrowser now takes
an injectable loadPlaywright purely to open that seam.
The test asserts per engine that the message leads with that engine,
names --with-deps <that engine> and no other, and preserves the
underlying cause. Verified by mutation: hard-coding 'chromium' in the
message fails it. It also pins the two branches apart — an unresolvable
package must say
> @modelcontextprotocol/inspector@2.3.0 postinstall
> node scripts/install-clients.mjs
[install-clients] Installing dependencies for clients/web...
up to date, audited 583 packages in 614ms
198 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
[install-clients] Installing dependencies for clients/cli...
up to date, audited 237 packages in 496ms
69 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
[install-clients] Installing dependencies for clients/tui...
up to date, audited 314 packages in 518ms
102 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
[install-clients] Installing dependencies for clients/launcher...
up to date, audited 160 packages in 449ms
50 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
up to date, audited 281 packages in 3s
108 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities and must NOT say ,
which fetches binaries and cannot fix a missing package.
The unsupported-engine test now also asserts loadPlaywright was never
called, pinning the ORDER rather than only the message: validating
first is what makes the error name the real mistake.
2. prod-web-server.mjs's header still described the prefix this PR removed. Corrected, keeping why
deriving paths from import.meta.url is still right, and adding the
elicitation smoke to the list of callers it had also outgrown.
Addresses the suppressed comments on Copilot's review of #2133.
Signed-off-by: cliffhall <cliff@futurescale.com>
#2132 was filed asserting that a Safari user opening an MCP App gets a
permanently 'loading' widget. That was never tested. A manual check in
real Safari renders the App normally, so the claim is unsupported and is
removed from all six places it reached.
The failure under Playwright's WebKit is real and still keeps that engine
out of the CI matrix. What is not established is that any shipping
browser behaves this way — and Playwright ships its own WebKit build
whose networking stack is not Safari's, which is exactly the layer the
failure lives in.
headless-browser.mjs already carried the caveat that a green WebKit run
is not a Safari guarantee. The same sentence forbids the inverse
inference, which is the one that was made; it now says so explicitly, and
AGENTS.md carries the rule: reproduce in the real browser before claiming
user impact in one.
No behavior change.
Signed-off-by: cliffhall <cliff@futurescale.com>
Every browser-driven check in this repo launched Chromium and nothing else.
That is fine for most of the web client, whose behavior is React and Mantine.
It is not fine for the MCP Apps sandbox, which is built out of the primitives
that genuinely diverge between engines: a CSP <meta> injected as the first
<head> child of a srcdoc document, a nested sandboxed iframe, a
Permissions-Policy allow attribute, and postMessage origin discipline across
those two frames. Nothing else covers that surface — sandbox-csp.test.ts
asserts which policy STRING is built (environment-independent by
construction), and no Storybook story reaches the sandbox at all, since all
three App stories use a data: placeholder iframe and a mock bridge.
The engine is now a parameter:
SMOKE_BROWSER=webkit npm run smoke:web:app
SMOKE_BROWSER=firefox npm run smoke:web:engine # all three smokes
Unset means chromium, so `npm run ci` and `npm run smoke` are unchanged. An
unrecognized value is an error rather than a fallback — falling back would
report a green Chromium run under a job labelled "webkit", claiming coverage
that never ran — and a launch failure names the engine that failed and its own
`playwright install` invocation.
Selection, launching, and the fatal-vs-benign page-diagnostics split now live
in one shared scripts/lib/headless-browser.mjs. The three smokes had each
hand-rolled the diagnostics split and two had their own copy of loadChromium;
pack:verify goes through the same helper, pinned to Chromium explicitly since
it is a packaging check.
The npm scripts drop their `cd clients/web && npx playwright install chromium`
prefix for scripts/install-smoke-browser.mjs. The engine is a variable now, and
a ${VAR:-default} expansion does not expand under Windows' cmd.exe; npx is also
a .cmd shim there that a shell-free spawn cannot start, so this resolves the
Playwright CLI through resolveNodeBin — the #1939 problem solved the #1939 way.
CI gains a `Sandbox smokes (<engine>)` matrix job rather than matrixing `build`,
which also runs validate, the coverage gate, two verify gates and Storybook —
all engine-independent. Both publish jobs now need it.
WebKit is supported by the tooling but deliberately NOT in the CI matrix yet:
pointing the smokes at it immediately found #2132, a pre-existing Safari
incompatibility in the web client's SSE transport that strands the last message
on /api/mcp/events, so an MCP App never leaves "loading". That is a product bug
affecting real Safari users, with its own diagnosis and a verified fix, and it
is not a matrix change. Adding `webkit` here is a one-word diff once it lands.
An engine is either green in CI or absent from it; a continue-on-error job would
report coverage nobody is held to.
Signed-off-by: cliffhall <cliff@futurescale.com>
