modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

Merge branch 'v2/main' into v2/fix/2123-root-union-schemas

Browse source
This commit is contained in:
Cliff Hall 2026-08-26 16:49:03 -04:00 • committed by GitHub
commit f01ec8047c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
16 changed files with 1181 additions and 229 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.github/copilot-instructions.md vendored
View file

@ -102,9 +102,11 @@ Both exist and do different jobs. Theme files (`src/theme/<Component>.ts`) custo
## Gates and PR hygiene
- `npm run format` before committing; **`npm run ci` before pushing** (`validate` → `coverage` → `verify:build-gate` → `verify:bundle-externals` → `smoke` → Storybook). `npm run validate` is the fast inner-loop check and is **not** a substitute — it runs `test`, not `test:coverage`, so it does zero coverage gating.
- `npm run format` before committing; **`npm run ci` before pushing** (`validate` → `coverage` → `verify:build-gate` → `verify:bundle-externals` → `smoke` → `smoke:web:firefox` → Storybook; the Firefox step is local-only and not mirrored in GitHub CI). `npm run validate` is the fast inner-loop check and is **not** a substitute — it runs `test`, not `test:coverage`, so it does zero coverage gating.
- **A dependency bump must land in every install that declares it.** v2 is not a workspace — the root and each `clients/*` have their own `node_modules`, and a client's test project compiles `core/` and `test-servers/src` (which resolve from the **root**) alongside the client's own sources. Bumping a shared dependency in one manifest only puts two versions of it in one `tsc` program; for a recursive-generic surface like zod that exhausts the tsc heap (#1896). `verify:dep-lockstep` fails the build on this — it derives its candidates from what each `tsc` program actually resolves (`tsc --listFilesOnly`, keeping packages that reach one program from two installs), so a package reached only through another package's `.d.ts` counts too (#1965) — so a PR bumping a package the shared sources pull in should update the root **and every client that already lists it** — not every client unconditionally, since a package absent from an install can't skew and adding it there would be a spurious dependency.
- **A test or smoke must not touch real user state.** The web smokes run against a throwaway catalog via the shared `scripts/lib/prod-web-server.mjs` helper, never the developer's `~/.mcp-inspector/mcp.json` (#1977); the cli/tui smokes drive a temp `--catalog`; `pack:verify`'s `--web` child sets its own `MCP_CATALOG_PATH` for the same reason (#2003 — its App deep link persists a server row). Anything that boots the web backend and then *navigates* it needs that isolation, not just the scripts named `smoke:*`. A new smoke spawning its own server, or teardown that removes a work dir without first awaiting `stopChild` (the #1801 race — `child-cleanup.mjs` exports both halves and both are required), should be flagged.
- **The headless web smokes are engine-parameterized, not Chromium-only.** `smoke:web:browser` / `smoke:web:app` / `smoke:web:elicit` take their engine from `SMOKE_BROWSER` (chromium — the default — firefox, webkit). **GitHub CI runs Chromium only; Firefox runs in the local pre-push gate** via `smoke:web:firefox` (#2086 — a CI job was trialled and removed for never disagreeing with Chromium across a dozen runs, and moved into `npm run ci` instead). They exist because the MCP Apps sandbox is built out of the primitives that diverge between engines: `srcdoc` CSP inheritance, nested sandboxed iframes, `Permissions-Policy`, cross-frame `postMessage`. Nothing else covers that — `sandbox-csp.test.ts` asserts a policy *string*, and no Storybook story reaches the sandbox at all. Flag a new browser-driven script that launches Playwright itself instead of going through `scripts/lib/headless-browser.mjs`, an unrecognized-engine path that falls back to Chromium rather than failing (it would claim coverage that never ran), or a launch-failure message naming the wrong engine. `pack:verify` is pinned to Chromium on purpose — it is a packaging check.
- **Build output is never a gate target.** Lint, format, and typecheck read first-party source only; everything a build writes (`clients/*/build`, `clients/web/dist`, `storybook-static`, `coverage`, `test-servers/build`, `core/**/{build,dist}`, `*.tsbuildinfo`) stays out via each scope's `globalIgnores`, `format` globs, and tsconfig `include`. Gating generated code reports defects in vendored third-party source that nobody can fix, and a rule promotion turns that warning into a `validate` failure (#2043). Flag a PR that adds a build location without ignoring it in the same change, that widens an ignore to silence a finding in first-party code, or that adds a build directory to a tsconfig `include` to make a generated `.d.ts` resolve. Note the coverage guards don't catch this — they assert source is _covered_, not that output is _excluded_.
- **Lint has no warning tier.** Every `lint` script runs `--max-warnings 0`, so a warning fails `validate` exactly as an error does (#2085) — a `warn`-level `react-hooks/exhaustive-deps` finding otherwise let a stale-closure bug pass the pre-push gate and reach review. Flag a PR that silences a finding to make the gate pass (widening a `globalIgnores`, dropping a rule, or an inline disable with no justification comment); the fix is the defect, not the message. A rule meant to be enforced should be set to `error` rather than left at `warn` and carried by the flag.
- **Every PR references an issue**, first body line `Closes #<ISSUE_NUMBER>`.

40
AGENTS.md
View file

File diff suppressed because one or more lines are too long

27
README.md
View file

@ -486,14 +486,14 @@ Each client self-validates from its own folder; the root scripts chain them. The
| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `npm run validate` | Runs the three durable guards first — `verify:format-coverage` (every tracked source file is format-gated), `verify:typecheck-coverage` (every one lands in a tsconfig project), `verify:dep-lockstep` (no dependency reaching one `tsc` program from two installs skews across them) — then `test:scripts` (the guards' own parser unit tests), then `validate:core` (the shared `core/` `format:check` + `lint` gate), then per client: `format:check` + `lint` + **`typecheck`** (cli/tui/launcher; web typechecks via `tsc -b` inside its `build`) + `build` + fast unit tests. The quick inner-loop check. |
| `npm run coverage` | The **per-file ≥90% gate** (lines/statements/functions/branches) under v8 instrumentation, per client. CI-enforced. For web this also runs the integration project and covers the shared `core/` runtime (including `core/json` and `core/client`). |
| `npm run smoke` | End-to-end smokes through the built launcher (`--help` dispatch + prod cli/tui/web), plus three headless-Chromium smokes: a boot smoke that runs the prod web bundle and asserts a clean first render (no uncaught error — sync exception or unhandled rejection, how a Node built-in reaching the browser bundle manifests), and an **MCP Apps** smoke (`smoke:web:app`) that drives connect → open app → `data-app-status="ready"` against a composable App server, covering the sandbox proxy and UI-protocol bridge, and an **app-rendered elicitation** smoke (`smoke:web:elicit`) that drives one end to end — call the tool, answer inside the sandboxed app, see the app's `ElicitResult` reach the server — and then the same tool against a server that never advertised the capability, which must fall back to the native elicitation form. |
| `npm run smoke` | End-to-end smokes through the built launcher (`--help` dispatch + prod cli/tui/web), plus three headless-browser smokes: a boot smoke that runs the prod web bundle and asserts a clean first render (no uncaught error — sync exception or unhandled rejection, how a Node built-in reaching the browser bundle manifests), and an **MCP Apps** smoke (`smoke:web:app`) that drives connect → open app → `data-app-status="ready"` against a composable App server, covering the sandbox proxy and UI-protocol bridge, and an **app-rendered elicitation** smoke (`smoke:web:elicit`) that drives one end to end — call the tool, answer inside the sandboxed app, see the app's `ElicitResult` reach the server — and then the same tool against a server that never advertised the capability, which must fall back to the native elicitation form. All three take their engine from `SMOKE_BROWSER` — see [Supported browsers](#supported-browsers). |
| `npm run verify:build-gate` | Runs a real `vite build` with a Node built-in forced into the browser graph and asserts the build **fails** via the #1769 gate (which turns Vite's browser-externalization warning into a hard error). Guards against the warning phrasing drifting in a Vite bump and silently disabling the gate. Part of `npm run ci`. |
| `npm run verify:bundle-externals` | Guards the must-not-bundle invariant (#2067): for each tsup-bundled client it reads the **built** `build/` output and fails if any package that must stay external was inlined anyway. Candidates are the union of the client's own `external` array **and the root manifest's `dependencies`** — the latter because #2067 was a *missing* `external` entry, which a self-referential check would have passed. Detection is via esbuild's `// <path>` module banners, so it covers both shapes — a separate `<pkg>-HASH.js` chunk (what a dynamically `import()`ed CommonJS package produces) and a statically-imported package folded straight into `index.js`, which emits no chunk at all. A build with no banners fails as such rather than passing clean, so enabling `minify` cannot silently retire the check. `undici` was declared only in the root and `clients/cli` manifests, and tsup auto-externalizes only what the _nearest_ manifest declares, so the web and TUI bundles inlined 1.05MB of it — and CommonJS inlined into an ESM bundle throws `Dynamic require of "assert" is not supported` on first use, from a specifier no user-side install can satisfy. Reads the output rather than the config because those two disagreed for four releases. Part of `npm run ci`. |
| `npm run verify:format-coverage` | Parses the `format:check` globs out of every `package.json` (only those reachable from `validate`), enumerates all tracked source files, and **fails** listing any not covered by a glob — the durable guard for the "every first-party source file is format-gated" invariant (#1792). Runs first in `validate`. |
| `npm run test:scripts` | Table-driven unit tests (`node --test`) for the guard's own pure parsers (`scripts/lib/npm-scripts.mjs`, `scripts/lib/tsc-program.mjs` + the exported helpers of `verify-typecheck-coverage.mjs` and `verify-dep-lockstep.mjs`), one case per rule they encode, plus two suites over shared `scripts/lib` helpers that no smoke can check itself: `resolve-node-bin.test.mjs` — the cross-platform bin resolver (#1939), pinned against the real `bin`/`exports` shapes of the packages the scripts actually spawn — and `announced-child.test.mjs` — the spawn/readiness ownership helper (#2000), which drives real `node -e` children to prove a child that never announces is still published to the caller before the timeout throws, and so is reachable by teardown rather than orphaned. Two more do the same: `mcp-app-flow.test.mjs` covers the shared MCP Apps flow (#2003) — the deep link's two CSRF gates and `appArgs` encoding, plus `driveAppFlow`'s failure branches against a stand-in page, all of which are dead code from the happy-path smokes' point of view and would otherwise surface only as opaque timeouts; and `ensure-test-servers.test.mjs` pins the [#2111](https://github.com/modelcontextprotocol/inspector/issues/2111) invariant — that `test-servers/build` is rebuilt **even when it already exists** — which no smoke can assert about itself, since one driving a stale fixture reports a product failure rather than a staleness one. Runs in `validate` — and `verify:typecheck-coverage` guards *this* gate in turn (reachable from `validate`, non-empty test set, every test file matched by the `test:scripts` glob), since `node --test` silently skips a file its glob misses and still exits 0. |
| `npm run verify:typecheck-coverage` | The typecheck-coverage analog of the above (#1791): for each Node client (auto-discovered from disk — enrolled via its `typecheck` script's projects, or for a `tsc -b` client like `clients/web` via its `tsconfig.json` `references`) it runs those projects with `tsc --listFilesOnly`, unions them, and **fails** listing any tracked `.ts`/`.tsx`/`.mts`/`.cts` under the client that lands in no project (so a new top-level config/helper can't silently go untypechecked). It also requires, deny-by-default, the first-party TS no client owns (`test-servers/src`, the root `vitest.shared.mts`, all of `core/`, and any new top-level location) to land in some client project's tsc pass — so a `core` `*.tsx` web's projects don't reach is caught too. Also asserts the gate is wired (each client's typecheck pass — its `typecheck` script, or web's `tsc -b` — is reachable from its `validate`, and the root chain runs each client's `validate`). Runs in `validate`. |
| `npm run verify:dep-lockstep` | Guards the "one version per install-crossing dependency" invariant (#1896). v2 is not a workspace, so a client's test project compiles the shared first-party TypeScript — `core/`, `test-servers/src`, and the root-owned `vitest.shared.mts`, all of which resolve their dependencies from the **root** install — alongside the client's own sources, putting the same package in one `tsc` program twice. At the same version that's harmless; skewed, TypeScript must relate two structurally-distinct copies of every type, which for a recursive-generic surface is exponential (zod `4.3.6` vs `4.4.3` exhausted the 4GB tsc heap in `clients/web`). Derives its candidate set from **what actually enters each program** (#1965) — every client tsconfig project listed with `tsc --listFilesOnly` via the shared `scripts/lib/tsc-program.mjs`, each resolved `node_modules` file mapped to its owning install, keeping the packages that reach one program from two installs (a package whose declarations arrive only through another package's `.d.ts`, as `@modelcontextprotocol/sdk`'s do, is invisible to a scan of first-party imports). Prices each copy from the lockfile entry for the exact install path the program resolved, compares only the installs that met in one program, and **fails deny-by-default** on any disagreement not in the annotated `TOLERATED_SKEW` allowlist — empty today — with an allowlisted package tolerated only *within a major version*. Runs in `validate`.
| `npm run ci` | **Mandatory pre-push command.** `validate` → `coverage` → `verify:build-gate` → `verify:bundle-externals` → `smoke` → Storybook. A true superset of GitHub CI. |
| `npm run ci` | **Mandatory pre-push command.** `validate` → `coverage` → `verify:build-gate` → `verify:bundle-externals` → `smoke` → `smoke:web:firefox` → Storybook. A strict superset of GitHub CI — `smoke:web:firefox` (and `smoke:tui`) run only here. |
| `npm run pack:verify` | Publish smoke — see [Publishing](#publishing). |
Per-client scripts exist too (`validate:web`, `coverage:cli`, `smoke:tui`, …), plus root `validate:core` / `format:core` for the shared `core/` package, `format:scripts` for the root `scripts/` tooling, and `format:shared` / `lint:shared` for the root "shared" surface (`test-servers/src/**`, `vitest.shared.mts`, the root `eslint.config.js`). Run `npm run format` before committing — the root `format` fixes `core/`, the root `scripts/`, the shared surface, and every client; `validate` runs the non-fixing `format:check` and fails CI on any unformatted file.
@ -504,6 +504,29 @@ Per-client scripts exist too (`validate:web`, `coverage:cli`, `smoke:tui`, …),
For the full testing rules — the ≥90% per-file gate, where test files live, the unit vs. integration vs. storybook projects, and the `v8 ignore` policy — see [`AGENTS.md`](./AGENTS.md).
### Supported browsers
The three headless web smokes — `smoke:web:browser`, `smoke:web:app`, `smoke:web:elicit` — take their browser engine from `SMOKE_BROWSER`, which accepts **`chromium`, `firefox` and `webkit`** ([#2086](https://github.com/modelcontextprotocol/inspector/issues/2086)):
```bash
SMOKE_BROWSER=webkit npm run smoke:web:app # one smoke, one engine
SMOKE_BROWSER=firefox npm run smoke:web:engine # all three smokes, one engine
```
Unset, the engine is `chromium`, so `npm run smoke` is unchanged. **`npm run ci` — the mandatory pre-push gate — additionally runs all three smokes under Firefox** via `smoke:web:firefox`; **GitHub CI does not.** An unrecognized `SMOKE_BROWSER` is an error, not a fallback: a silent fallback would report a green Chromium run for a command that asked for `webkit`. A missing browser binary fails naming the engine and a remedy that works from where you are — `npm run smoke:web:<engine>`, or `npx playwright install --with-deps <engine>` run from `clients/web`, where Playwright is pinned.
**Firefox passes all three smokes. WebKit fails the two App smokes**, for reasons nobody has identified. Two things are known: it does **not** reproduce in real Safari (an MCP App opens there normally), and an isolated repro of the mechanism it was first blamed on did not reproduce it under Playwright's WebKit either. So it reads as a property of that particular build rather than a bug users hit, and chasing it further was judged not worth the effort — treat a WebKit failure as unexplained rather than as a defect until someone has looked.
**Why Firefox is in the pre-push gate rather than in CI.** A GitHub Actions job was trialled and was cheap — about two minutes, in parallel with the 15-minute `build` job, so no added wall-clock. It was dropped anyway, on the honest count: across a dozen runs it never once disagreed with Chromium, so it spent runner minutes on every push from every branch to re-confirm a result already in hand, and carried a real flake surface (`playwright install --with-deps` runs `apt-get update`, which fails whenever a third-party repo in the runner image breaks).
Putting it in `npm run ci` instead keeps the check where it is worth most — in front of a human about to push a change they can still reason about — and pays for it once per push rather than once per CI run. If a cross-engine regression ever reaches `v2/main` because someone skipped the gate, that is the evidence for restoring the CI job; it is a one-job diff.
**Why these smokes specifically.** Most of the web client's behavior is React and Mantine, where a second engine buys little. The MCP Apps sandbox is the exception — it is built out of the primitives that genuinely diverge between engines: a CSP `<meta>` injected as the first `<head>` child of a `srcdoc` document, a nested sandboxed iframe, a `Permissions-Policy` `allow` attribute, and `postMessage` origin discipline across those two frames. Nothing else covers that: `sandbox-csp.test.ts` asserts which policy _string_ is built, which passes identically on an engine that ignores `<meta>` CSP entirely, and no Storybook story reaches the sandbox at all (all three App stories point the iframe at a `data:` placeholder and hand the renderer a mock bridge). Storybook itself remains Chromium-only — broadening it covers a much larger and differently-shaped surface, and is a separate decision.
> ⚠️ **Playwright's WebKit is a WebKit build, not Safari.** It is close enough to catch engine-level CSP and iframe divergence, and not close enough to certify Safari specifically. A green run here is not a Safari guarantee.
`pack:verify` stays Chromium-only on purpose — it is a _packaging_ check, and the engine question belongs where the sandbox is under test.
## Publishing
The root `@modelcontextprotocol/inspector` package ships as **one tarball with a single version number** — no separate `-web` / `-cli` / `-tui` / `-core` packages. `npm run build` builds every client, then `prepack` runs before `npm publish`. Runtime dependencies are declared on the root `package.json`; client builds bundle `@inspector/core` and externalize npm packages resolved from the root install.

16
package.json
View file

@ -38,7 +38,7 @@
"build:tui": "cd clients/tui && npm run build",
"build:web": "cd clients/web && npm run build",
"build:launcher": "cd clients/launcher && npm run build",
"ci": "npm run validate && npm run coverage && npm run verify:build-gate && npm run verify:bundle-externals && npm run smoke && npm run ci:storybook",
"ci": "npm run validate && npm run coverage && npm run verify:build-gate && npm run verify:bundle-externals && npm run smoke && npm run smoke:web:firefox && npm run ci:storybook",
"ci:storybook": "cd clients/web && npx playwright install chromium && npm run test:storybook",
"verify:build-gate": "node scripts/verify-build-gate.mjs",
"verify:bundle-externals": "node scripts/verify-bundle-externals.mjs",
@ -66,15 +66,19 @@
"coverage:tui": "cd clients/tui && npm run test:coverage",
"coverage:web": "cd clients/web && npm run test:coverage",
"coverage:launcher": "cd clients/launcher && npm run test:coverage",
"smoke": "npm run smoke:launcher && npm run smoke:cli && npm run smoke:tui && npm run smoke:web && npm run smoke:web:browser && npm run smoke:web:app && npm run smoke:web:elicit",
"smoke": "npm run smoke:launcher && npm run smoke:cli && npm run smoke:tui && npm run smoke:web && npm run smoke:web:chromium",
"smoke:cli": "node scripts/smoke-cli.mjs",
"smoke:tui": "node scripts/smoke-tui.mjs",
"smoke:web": "node scripts/smoke-web.mjs",
"smoke:web:browser": "cd clients/web && npx playwright install chromium && node ../../scripts/smoke-web-browser.mjs",
"smoke:web:app": "cd clients/web && npx playwright install chromium && node ../../scripts/smoke-web-app.mjs",
"smoke:web:elicit": "cd clients/web && npx playwright install chromium && node ../../scripts/smoke-web-elicitation.mjs",
"smoke:web:browser": "node scripts/install-smoke-browser.mjs && node scripts/smoke-web-browser.mjs",
"smoke:web:app": "node scripts/install-smoke-browser.mjs && node scripts/smoke-web-app.mjs",
"smoke:web:elicit": "node scripts/install-smoke-browser.mjs && node scripts/smoke-web-elicitation.mjs",
"smoke:web:engine": "node scripts/run-engine-smokes.mjs",
"smoke:web:chromium": "node scripts/run-engine-smokes.mjs chromium",
"smoke:web:firefox": "node scripts/run-engine-smokes.mjs firefox",
"smoke:web:webkit": "node scripts/run-engine-smokes.mjs webkit",
"smoke:launcher": "node scripts/smoke-launcher.mjs",
"pack:verify": "cd clients/web && npx playwright install chromium && cd ../.. && node scripts/pack-and-verify.mjs",
"pack:verify": "node scripts/install-smoke-browser.mjs chromium && node scripts/pack-and-verify.mjs",
"prepack": "npm run build",
"postinstall": "node scripts/install-clients.mjs"
},

110
scripts/install-smoke-browser.mjs Normal file
View file

@ -0,0 +1,110 @@
#!/usr/bin/env node
/**
* Fetch the Playwright browser binary a headless web smoke is about to launch
* (#2086).
*
* This replaces the `cd clients/web && npx playwright install chromium` prefix
* each smoke's npm script used to carry. Two reasons it is a script rather than
* a longer npm-script line:
*
* - The engine is now a variable. `npx playwright install ${SMOKE_BROWSER:-…}`
* is a POSIX-shell expansion that does not expand under Windows' `cmd.exe`,
* which npm uses there — it would silently try to install a browser named
* `${SMOKE_BROWSER:-chromium}`.
* - `npx` is a `.cmd` shim on Windows, which a shell-free spawn cannot start.
* `resolveNodeBin` walks to the JS entry behind the package's `bin` and runs
* it with `process.execPath` instead — the same fix #1939 made for the
* verify scripts, and it keeps the resolution pinned to the repo's own
* Playwright rather than whatever `npx` might fetch.
*
* Usage: `node scripts/install-smoke-browser.mjs [engine]`. With no argument the
* engine comes from `SMOKE_BROWSER` (default `chromium`); pass one explicitly
* where the consumer's engine is fixed regardless of that variable, as
* `pack:verify` does.
*
* Execution lives behind `main()` so importing this module for tests does not
* run an install — the same shape `verify-typecheck-coverage.mjs` uses.
*/
import { spawnSync } from "node:child_process";
import { join, resolve } from "node:path";
import { pathToFileURL } from "node:url";
import {
SUPPORTED_BROWSERS,
resolveBrowserName,
} from "./lib/headless-browser.mjs";
import { resolveNodeBin } from "./lib/resolve-node-bin.mjs";
/**
* Which engine to install: the explicit argument if given, else `SMOKE_BROWSER`.
*
* The precedence is the load-bearing part, and it is why this is a function
* rather than three lines inline. `pack:verify` passes `chromium` explicitly
* *because* it launches Chromium explicitly — if the argument ever lost to the
* environment, `SMOKE_BROWSER=webkit npm run pack:verify` would install WebKit
* and then launch Chromium, and the mismatch would surface as a confusing
* "Executable doesn't exist" rather than as the wiring error it is.
*
* An explicit argument is validated here rather than deferred, so a typo in an
* npm script fails naming itself instead of reaching the Playwright CLI.
*/
export function resolveRequestedBrowser(argv = [], env = process.env) {
const requested = argv[0];
if (requested === undefined) return resolveBrowserName(env);
if (!SUPPORTED_BROWSERS.includes(requested)) {
throw new Error(
`unsupported browser "${requested}" — expected one of ${SUPPORTED_BROWSERS.join(", ")}`,
);
}
return requested;
}
function main() {
const repoRoot = resolve(import.meta.dirname, "..");
const webDir = join(repoRoot, "clients", "web");
let browserName;
try {
browserName = resolveRequestedBrowser(process.argv.slice(2), process.env);
} catch (err) {
console.error(
`install-smoke-browser: ${err instanceof Error ? err.message : String(err)}`,
);
process.exit(1);
}
let cli;
try {
cli = resolveNodeBin("playwright", "playwright", webDir);
} catch (err) {
console.error(
"install-smoke-browser: could not resolve the Playwright CLI from clients/web — " +
`run \`npm install\` at the repo root (${err instanceof Error ? err.message : String(err)})`,
);
process.exit(1);
}
// `install` is a no-op when the binary is already present, so this is cheap on
// a warm machine and on a CI runner whose Playwright cache was restored.
const result = spawnSync(process.execPath, [cli, "install", browserName], {
cwd: webDir,
stdio: "inherit",
});
if (result.error) {
console.error(
`install-smoke-browser: could not run the Playwright CLI: ${result.error.message}`,
);
process.exit(1);
}
if (result.status !== 0) {
console.error(
`install-smoke-browser: \`playwright install ${browserName}\` exited ${result.status}`,
);
process.exit(result.status ?? 1);
}
}
// Only when run as a script, never on import (see the header). Same guard as
// verify-typecheck-coverage.mjs — `pathToFileURL` so it holds on Windows paths.
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href)
main();

88
scripts/install-smoke-browser.test.mjs Normal file
View file

@ -0,0 +1,88 @@
/**
* Unit tests for the smoke browser installer's argument resolution (#2086).
*
* The installer's I/O half — resolving the Playwright CLI and spawning it — is
* exercised on every `npm run smoke:web:*`, so it fails loudly the moment it
* breaks. Its *argument* half is not: every caller in the repo passes either
* nothing or the literal `chromium`, so the precedence rule below is asserted by
* nothing at runtime, and getting it wrong produces a mismatch (install one
* engine, launch another) that surfaces as a misleading "Executable doesn't
* exist" rather than as the wiring error it is.
*
* Importing this module must not run an install; `main()` is behind an
* entrypoint guard for exactly that reason, and these tests are also what would
* catch that guard regressing — an unguarded module would try to install a
* browser the moment the suite imported it.
*/
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { resolveRequestedBrowser } from "./install-smoke-browser.mjs";
import {
BROWSER_ENV_VAR,
DEFAULT_BROWSER,
SUPPORTED_BROWSERS,
} from "./lib/headless-browser.mjs";
describe("resolveRequestedBrowser", () => {
it("falls back to the environment when no argument is given", () => {
assert.equal(resolveRequestedBrowser([], {}), DEFAULT_BROWSER);
assert.equal(
resolveRequestedBrowser([], { [BROWSER_ENV_VAR]: "webkit" }),
"webkit",
);
});
it("lets an explicit argument beat the environment", () => {
// The `pack:verify` invariant. It passes `chromium` because it LAUNCHES
// chromium; if the environment won here, `SMOKE_BROWSER=webkit npm run
// pack:verify` would install WebKit and then launch Chromium.
assert.equal(
resolveRequestedBrowser(["chromium"], { [BROWSER_ENV_VAR]: "webkit" }),
"chromium",
);
for (const name of SUPPORTED_BROWSERS) {
assert.equal(
resolveRequestedBrowser([name], { [BROWSER_ENV_VAR]: "firefox" }),
name,
);
}
});
it("rejects an unsupported argument, naming it and the allowed set", () => {
assert.throws(
() => resolveRequestedBrowser(["safari"], {}),
(err) => {
assert.match(err.message, /unsupported browser "safari"/);
SUPPORTED_BROWSERS.forEach((name) =>
assert.ok(err.message.includes(name)),
);
return true;
},
);
});
it("does not let a bad argument fall through to the environment", () => {
// Silently using SMOKE_BROWSER after ignoring a typo'd argument would
// install an engine nobody asked for, and the npm script's typo would
// survive unnoticed.
assert.throws(
() =>
resolveRequestedBrowser(["chrome"], { [BROWSER_ENV_VAR]: "firefox" }),
/unsupported browser "chrome"/,
);
});
it("propagates the environment's own validation, empty included", () => {
// Delegation, not a second copy of the rules — so the deny-by-default
// behavior cannot drift between the two entry points.
assert.throws(
() => resolveRequestedBrowser([], { [BROWSER_ENV_VAR]: "safari" }),
/is not a supported browser/,
);
assert.throws(
() => resolveRequestedBrowser([], { [BROWSER_ENV_VAR]: "" }),
/set but empty/,
);
});
});

198
scripts/lib/headless-browser.mjs Normal file
View file

@ -0,0 +1,198 @@
/**
* The headless browser the web smokes drive: which engine, how it is launched,
* and the two error channels a page reports on (#2086).
*
* All three smokes (`smoke:web:browser`, `smoke:web:app`, `smoke:web:elicit`)
* and `pack:verify` go through here, so the engine is a parameter in exactly
* one place and the diagnostics split is stated once rather than hand-rolled
* four times.
*
* Every browser-driven check in this repo used to be Chromium-only. That is
* fine for most of the web client, whose behavior is React and Mantine — and it
* is NOT fine for the MCP Apps sandbox, which is built out of exactly the
* primitives that diverge between engines: a CSP `<meta>` injected into a
* `srcdoc` document, a nested sandboxed iframe, a `Permissions-Policy` `allow`
* attribute, and `postMessage` origin discipline across those two frames. A
* regression in any of those is invisible unless it also reproduces in Chromium.
*
* The unit tests cannot cover it either: `sandbox-csp.test.ts` asserts which
* policy STRING is built, which would pass identically on an engine that
* ignores `<meta>` CSP entirely. And no Storybook story reaches the sandbox at
* all — all three App stories point the iframe at a `data:` placeholder and hand
* the renderer a mock bridge. So the smokes are the only place the sandbox is
* genuinely exercised, and this module is what lets them be pointed at another
* engine.
*
* ⚠️ Playwright's WebKit is a WebKit build, not Safari. It is close enough to
* catch engine-level CSP and iframe divergence, and not close enough to certify
* Safari specifically — a green run here is not a Safari guarantee.
*
* **And that cuts both ways: a RED run here is not a Safari indictment.** The
* WebKit App-smoke failure was once written up as a Safari bug on the strength
* of a failure in this build alone; a manual check in Safari did not reproduce
* it, an isolated repro of the claimed mechanism did not reproduce it here
* either, and the whole line of investigation was dropped. Reproduce in the real
* browser before describing a finding as one users hit.
*
* Playwright is resolved with a `createRequire` based at
* clients/web/package.json rather than a bare `import("playwright")`: a bare ESM
* specifier resolves relative to `scripts/`, not the cwd, so a `cd clients/web`
* in the npm script would NOT make it resolvable. See smoke-web-browser.mjs's
* header for the full gotcha.
*/
import { createRequire } from "node:module";
import { join } from "node:path";
/**
* Console messages that are the async half of the uncaught-crash class (an
* unhandled rejection or a failed dynamic import). Hard failures; every other
* console error is a diagnostic, so benign font-CDN / React-warning noise can't
* flake CI. smoke-web-browser.mjs's header documents the reasoning at length.
*/
export const FATAL_CONSOLE =
/^Uncaught\b|Failed to fetch dynamically imported module/;
/**
* Attach the two error channels a headless page reports on.
*
* An uncaught *synchronous* page error fires `pageerror`; its async twin — an
* unhandled rejection or a failed dynamic import — is not a `pageerror` at all
* and arrives on the console channel instead. Both are captured, and both are
* read on every engine rather than branching per browser: which channel an
* engine picks is exactly the kind of detail that differs between them, and
* reading both makes that difference irrelevant. Only `fatal()` is a failure, so
* ordinary console noise stays a diagnostic.
*/
export function attachPageDiagnostics(page) {
const pageErrors = [];
const consoleErrors = [];
page.on("pageerror", (err) =>
pageErrors.push(err instanceof Error ? err.message : String(err)),
);
page.on("console", (msg) => {
if (msg.type() === "error") consoleErrors.push(msg.text());
});
return {
pageErrors,
consoleErrors,
fatalConsole: () => consoleErrors.filter((m) => FATAL_CONSOLE.test(m)),
benignConsole: () => consoleErrors.filter((m) => !FATAL_CONSOLE.test(m)),
fatal: () => [
...pageErrors,
...consoleErrors.filter((m) => FATAL_CONSOLE.test(m)),
],
};
}
/** The supported engine set. Anything else is a typo, not a request. */
export const SUPPORTED_BROWSERS = ["chromium", "firefox", "webkit"];
/** The engine used when nothing asks for another one. */
export const DEFAULT_BROWSER = "chromium";
/** The env var each smoke reads to pick its engine. */
export const BROWSER_ENV_VAR = "SMOKE_BROWSER";
/**
* Resolve the engine name from the environment, deny-by-default.
*
* An unrecognized value FAILS rather than falling back to Chromium: a silent
* fallback would report a green Chromium run under a job labelled "webkit",
* which is worse than no coverage — it claims coverage that never ran.
*
* **Only an ABSENT variable selects the default. A present-but-empty one is an
* error**, because empty is what a variable set from something that did not
* resolve looks like — `SMOKE_BROWSER="$ENGINE"` with `ENGINE` unset, or a CI
* expression naming a key that does not exist (GitHub Actions renders those as
* the empty string rather than omitting the variable). Treating that as "not
* set" would silently run Chromium for a caller who asked for something else,
* which is the same false-coverage failure the unrecognized-value branch exists
* to prevent, arriving through a typo instead of a bad value. Unsetting the
* variable is how you ask for the default; setting it to nothing is not.
*/
export function resolveBrowserName(env = process.env) {
const raw = env[BROWSER_ENV_VAR];
if (raw === undefined) return DEFAULT_BROWSER;
const name = raw.trim().toLowerCase();
if (name === "") {
throw new Error(
`${BROWSER_ENV_VAR} is set but empty — a variable set from something ` +
`that did not resolve looks exactly like this. ` +
`Unset it to use ${DEFAULT_BROWSER}, or set one of ${SUPPORTED_BROWSERS.join(", ")}.`,
);
}
if (!SUPPORTED_BROWSERS.includes(name)) {
throw new Error(
`${BROWSER_ENV_VAR}="${raw}" is not a supported browser — expected one of ${SUPPORTED_BROWSERS.join(", ")}`,
);
}
return name;
}
/**
* Resolve the Playwright package from the web client's install.
*
* Separate from `loadBrowser` so it can be substituted in tests — see the
* `loadPlaywright` option there for why that seam has to exist.
*/
export function requirePlaywright(repoRoot) {
const requireFromWeb = createRequire(
join(repoRoot, "clients", "web", "package.json"),
);
return requireFromWeb("playwright");
}
/**
* Launch a headless browser of `browserName`, resolved from the web client's
* install.
*
* Both failure branches produce a message that names the actual remedy, and they
* are different remedies — which is the whole reason they are separate branches:
*
* - **not resolvable** means the Playwright *npm package* is missing, fixed by
* `npm install` at the repo root. `playwright install` (which fetches
* browser binaries) would not help and is the wrong thing to suggest.
* - **launch rejected** means the package is there but that engine's *binary*
* is not, fixed by `playwright install --with-deps <that engine>`. Naming
* "chromium" while WebKit was the missing one sends the reader off to
* install a browser they already have — the reason the engine is
* interpolated rather than hard-coded (#2086's acceptance criterion).
*
* `loadPlaywright` is injectable purely so those branches can be unit-tested.
* They are unreachable from the smokes by construction: `install-smoke-browser`
* runs first and fetches the binary, so a passing smoke proves nothing about the
* message a failing one would print — and that message IS the deliverable here,
* since its whole job is to be read by someone whose setup is broken.
*/
export async function loadBrowser(
repoRoot,
browserName = DEFAULT_BROWSER,
{ loadPlaywright = requirePlaywright } = {},
) {
if (!SUPPORTED_BROWSERS.includes(browserName)) {
throw new Error(
`unsupported browser "${browserName}" — expected one of ${SUPPORTED_BROWSERS.join(", ")}`,
);
}
let playwright;
try {
playwright = loadPlaywright(repoRoot);
} catch (err) {
throw new Error(
`could not resolve the Playwright package from clients/web — run \`npm install\` at the repo root (${err instanceof Error ? err.message : String(err)})`,
);
}
try {
return await playwright[browserName].launch({ headless: true });
} catch (err) {
throw new Error(
`${browserName} failed to launch — run \`npm run smoke:web:${browserName}\`, ` +
`or from clients/web run \`npx playwright install --with-deps ${browserName}\` ` +
`if you also need its system libraries. Playwright is pinned in ` +
`clients/web, so running that from the repo root can fetch a different ` +
`version and install a browser revision this one still cannot launch ` +
`(${err instanceof Error ? err.message : String(err)})`,
);
}
}

284
scripts/lib/headless-browser.test.mjs Normal file
View file

@ -0,0 +1,284 @@
/**
* Unit tests for the headless-browser engine selection (#2086).
*
* The smokes cannot check this themselves: each one resolves exactly one engine
* per process and then spends its whole run inside the happy path, so the branch
* that matters most — an unrecognized `SMOKE_BROWSER` — is dead code from their
* point of view. It is also the branch whose failure is silent rather than loud:
* a fallback to Chromium there would report a green run to a caller who asked
* for another engine — coverage claimed that never ran.
*
* `loadBrowser`'s failure branches are covered here too, through its injectable
* `loadPlaywright`. The smokes cannot reach them by construction —
* `install-smoke-browser` fetches the binary before the smoke runs, so a passing
* smoke says nothing about what a failing one would print. And that message is
* the deliverable: #2086's acceptance criterion is that a missing browser fails
* naming *that engine* and its own `playwright install` command, which is
* exactly the kind of string that rots into naming the wrong one.
*
* The successful path is covered here too, with distinct per-engine spies — the
* failure-path stand-in rejects identically for every engine, so it cannot tell
* a correct dispatch from a mis-dispatch that happens to print the right label.
* Only a launch against a REAL binary is left to the smokes.
*/
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import {
BROWSER_ENV_VAR,
DEFAULT_BROWSER,
SUPPORTED_BROWSERS,
attachPageDiagnostics,
loadBrowser,
resolveBrowserName,
} from "./headless-browser.mjs";
describe("resolveBrowserName", () => {
it("defaults to chromium when the variable is unset", () => {
assert.equal(resolveBrowserName({}), DEFAULT_BROWSER);
});
it("rejects a present-but-empty value instead of defaulting", () => {
// This is the case worth being strict about. `SMOKE_BROWSER="$ENGINE"` with
// ENGINE unset — or a CI expression naming a key that does not exist — sets
// the variable to "" rather than removing it, so defaulting here would
// silently run Chromium for a caller who asked for something else. That is
// the same false-coverage outcome the unrecognized-value branch exists to
// stop. Only an ABSENT variable may select the default.
for (const raw of ["", " ", "\t\n"]) {
assert.throws(
() => resolveBrowserName({ [BROWSER_ENV_VAR]: raw }),
(err) => {
assert.match(err.message, /set but empty/);
// Names the actual remedy — "unset it", which is different from
// "set it to nothing" and is the whole point here.
assert.match(err.message, /did not resolve/);
assert.match(err.message, /Unset it/);
return true;
},
`expected ${JSON.stringify(raw)} to be rejected`,
);
}
});
it("accepts every supported engine", () => {
for (const name of SUPPORTED_BROWSERS) {
assert.equal(resolveBrowserName({ [BROWSER_ENV_VAR]: name }), name);
}
});
it("normalizes surrounding whitespace and case", () => {
assert.equal(
resolveBrowserName({ [BROWSER_ENV_VAR]: " WebKit " }),
"webkit",
);
});
it("rejects an unrecognized engine rather than falling back", () => {
assert.throws(
() => resolveBrowserName({ [BROWSER_ENV_VAR]: "safari" }),
// The message must quote what was asked for AND list what is accepted —
// "safari" is the most likely typo here and its fix is not guessable.
(err) =>
/safari/.test(err.message) &&
SUPPORTED_BROWSERS.every((name) => err.message.includes(name)),
);
});
});
describe("loadBrowser", () => {
/** A Playwright stand-in whose every engine rejects on launch. */
const launchAlwaysFails = (message) => () => ({
chromium: { launch: () => Promise.reject(new Error(message)) },
firefox: { launch: () => Promise.reject(new Error(message)) },
webkit: { launch: () => Promise.reject(new Error(message)) },
});
it("rejects an unsupported engine before touching Playwright", async () => {
let loaded = false;
await assert.rejects(
() =>
loadBrowser("/nonexistent-repo-root", "safari", {
loadPlaywright: () => {
loaded = true;
return {};
},
}),
/unsupported browser "safari"/,
);
// Pins the ORDER, not just the message: validating first is what makes the
// error name the real mistake instead of a downstream resolution failure.
assert.equal(loaded, false);
});
it("launches the engine that was asked for, and returns it", async () => {
// The load-bearing assertion, and the one the failure-path tests below
// CANNOT make (Copilot, #2133): their stand-in rejects identically for every
// engine, so `loadBrowser(root, "firefox")` could call `chromium.launch()`
// and still produce a correctly Firefox-labelled error. The smokes cannot
// catch that either — the Firefox gate runs after Chromium is already
// installed, so a mis-dispatched launch would succeed and report Firefox
// coverage that never happened. Only distinct per-engine spies pin it.
const launched = [];
const spyPlaywright = () =>
Object.fromEntries(
SUPPORTED_BROWSERS.map((name) => [
name,
{
launch: async (options) => {
launched.push({ name, options });
return { engine: name };
},
},
]),
);
for (const name of SUPPORTED_BROWSERS) {
launched.length = 0;
const browser = await loadBrowser("/repo", name, {
loadPlaywright: spyPlaywright,
});
assert.deepEqual(
launched.map((l) => l.name),
[name],
`loadBrowser(…, "${name}") must launch ${name} and nothing else`,
);
// The returned handle is that engine's, not some other engine's.
assert.deepEqual(browser, { engine: name });
assert.deepEqual(launched[0].options, { headless: true });
}
});
it("names the engine that failed, and its own install command", async () => {
// The whole point of #2086's acceptance criterion: a reader whose WebKit
// binary is missing must not be sent to install chromium.
for (const name of SUPPORTED_BROWSERS) {
await assert.rejects(
() =>
loadBrowser("/repo", name, {
loadPlaywright: launchAlwaysFails("Executable doesn't exist"),
}),
(err) => {
assert.match(err.message, new RegExp(`^${name} failed to launch`));
// Names a remedy that works from where the caller is: the npm
// script, and the raw command scoped to clients/web (Playwright is
// pinned there; a bare root-level `npx playwright` can fetch a
// different version and install a revision this cannot launch).
assert.match(err.message, new RegExp(`npm run smoke:web:${name}`));
assert.match(err.message, /from clients\/web/);
assert.match(
err.message,
new RegExp(
`--with-deps ${name}\\\`(?![\\s\\S]*--with-deps (?!${name}))`,
),
);
// The underlying cause survives, so the reader can tell a missing
// binary from a sandbox/permissions problem.
assert.match(err.message, /Executable doesn't exist/);
// And no OTHER engine is named anywhere in the message.
for (const other of SUPPORTED_BROWSERS.filter((b) => b !== name)) {
assert.ok(
!err.message.includes(other),
`message for ${name} must not mention ${other}: ${err.message}`,
);
}
return true;
},
);
}
});
it("sends an unresolvable Playwright to `npm install`, not `playwright install`", async () => {
// Two different failures with two different remedies. Suggesting
// `playwright install` here would be actively wrong — it fetches browser
// binaries and cannot install the missing npm package.
await assert.rejects(
() =>
loadBrowser("/repo", "firefox", {
loadPlaywright: () => {
throw new Error("Cannot find module 'playwright'");
},
}),
(err) => {
assert.match(err.message, /could not resolve the Playwright package/);
assert.match(err.message, /npm install/);
assert.ok(
!/playwright install/.test(err.message),
`must not suggest \`playwright install\` for a missing package: ${err.message}`,
);
assert.match(err.message, /Cannot find module 'playwright'/);
return true;
},
);
});
});
describe("attachPageDiagnostics", () => {
/** Minimal Playwright `page` stand-in: records handlers, replays events. */
function fakePage() {
const handlers = {};
return {
on: (event, fn) => {
(handlers[event] ??= []).push(fn);
},
emit: (event, arg) => handlers[event]?.forEach((fn) => fn(arg)),
};
}
const consoleMessage = (type, text) => ({
type: () => type,
text: () => text,
});
it("splits fatal console errors from benign noise", () => {
const page = fakePage();
const diagnostics = attachPageDiagnostics(page);
page.emit("console", consoleMessage("error", "Uncaught (in promise) boom"));
page.emit(
"console",
consoleMessage("error", "Failed to fetch dynamically imported module x"),
);
// The two shapes that used to flake CI, and must stay diagnostics.
page.emit(
"console",
consoleMessage("error", "Failed to load resource: net::ERR_FAILED"),
);
page.emit(
"console",
consoleMessage("error", "Warning: each child needs a key"),
);
// Non-error console output is ignored entirely.
page.emit("console", consoleMessage("log", "Uncaught looking but a log"));
assert.deepEqual(diagnostics.fatalConsole(), [
"Uncaught (in promise) boom",
"Failed to fetch dynamically imported module x",
]);
assert.deepEqual(diagnostics.benignConsole(), [
"Failed to load resource: net::ERR_FAILED",
"Warning: each child needs a key",
]);
});
it("counts every pageerror as fatal, whatever it was thrown as", () => {
const page = fakePage();
const diagnostics = attachPageDiagnostics(page);
page.emit("pageerror", new Error("sync boom"));
page.emit("pageerror", "thrown as a string");
page.emit("console", consoleMessage("error", "benign"));
assert.deepEqual(diagnostics.pageErrors, [
"sync boom",
"thrown as a string",
]);
assert.deepEqual(diagnostics.fatal(), ["sync boom", "thrown as a string"]);
});
it("reports nothing on a clean page", () => {
const diagnostics = attachPageDiagnostics(fakePage());
assert.deepEqual(diagnostics.fatal(), []);
assert.deepEqual(diagnostics.benignConsole(), []);
});
});

73
scripts/lib/mcp-app-flow.mjs
View file

@ -22,14 +22,12 @@
* **test server** is always a repo fixture — it is not in the tarball and
* should not be.
*
* Playwright is resolved with a `createRequire` based at
* clients/web/package.json rather than a bare `import("playwright")`: a bare ESM
* specifier resolves relative to `scripts/`, not the cwd, so a `cd clients/web`
* in the npm script would NOT make it resolvable. Same gotcha as
* smoke-web-browser.mjs; see its header.
* Launching the browser is NOT this module's job — `lib/headless-browser.mjs`
* owns that, so the engine is a parameter rather than a hard-coded Chromium
* (#2086) and so `smoke:web:browser`, which does not drive an App at all, can
* share it without importing this file.
*/
import { createRequire } from "node:module";
import { join } from "node:path";
import { startAnnouncedChild } from "./announced-child.mjs";
import {
@ -40,16 +38,6 @@ import {
/** The App tool the `mcp-app-http.json` fixture serves. */
export const APP_TOOL = "mcp_app_demo";
/**
* Console messages that are the async half of the uncaught-crash class (an
* unhandled rejection or a failed dynamic import). Hard failures; every other
* console error is a diagnostic, so benign font-CDN / React-warning noise can't
* flake CI. Kept identical to smoke-web-browser.mjs, which documents the
* reasoning at length.
*/
export const FATAL_CONSOLE =
/^Uncaught\b|Failed to fetch dynamically imported module/;
/** Path to the composable test server build, relative to the repo root. */
export function composableServerPath(repoRoot) {
return testServerEntryPath(repoRoot, "composable");
@ -143,59 +131,6 @@ export function buildAppDeepLink({
);
}
/**
* Attach the two error channels a headless page reports on.
*
* An uncaught *synchronous* page error fires `pageerror`; its async twin — an
* unhandled rejection or a failed dynamic import — is not a `pageerror` at all,
* Chromium reports it on the console channel instead. Both are captured; only
* `fatal()` is a failure, so ordinary console noise stays a diagnostic.
*/
export function attachPageDiagnostics(page) {
const pageErrors = [];
const consoleErrors = [];
page.on("pageerror", (err) =>
pageErrors.push(err instanceof Error ? err.message : String(err)),
);
page.on("console", (msg) => {
if (msg.type() === "error") consoleErrors.push(msg.text());
});
return {
pageErrors,
consoleErrors,
fatalConsole: () => consoleErrors.filter((m) => FATAL_CONSOLE.test(m)),
benignConsole: () => consoleErrors.filter((m) => !FATAL_CONSOLE.test(m)),
fatal: () => [
...pageErrors,
...consoleErrors.filter((m) => FATAL_CONSOLE.test(m)),
],
};
}
/** Launch headless Chromium, resolved from the web client's install. */
export async function loadChromium(repoRoot) {
const requireFromWeb = createRequire(
join(repoRoot, "clients", "web", "package.json"),
);
let chromium;
try {
({ chromium } = requireFromWeb("playwright"));
} catch (err) {
// Not resolvable means devDependencies are missing — fixed by `npm install`
// at the repo root, NOT by `playwright install` (which fetches binaries).
throw new Error(
`could not resolve the Playwright package from clients/web — run \`npm install\` at the repo root (${err instanceof Error ? err.message : String(err)})`,
);
}
try {
return await chromium.launch({ headless: true });
} catch (err) {
throw new Error(
`chromium failed to launch — on a bare Linux box run \`npx playwright install --with-deps chromium\` for the system libraries (${err instanceof Error ? err.message : String(err)})`,
);
}
}
/**
* Drive **connect → open app → widget ready** on an already-open page.
*

22
scripts/lib/prod-web-server.mjs
View file

@ -2,15 +2,21 @@
* Shared boot/readiness helper for the prod web smokes.
*
* `scripts/smoke-web.mjs` (serves-the-HTML check), `scripts/smoke-web-browser.mjs`
* (runs-the-bundle check, #1615), and `scripts/smoke-web-app.mjs` (MCP Apps
* end-to-end, #1859) all boot the *same* prod `mcp-inspector --web` server, so the
* spawn + readiness-poll boilerplate lives here once instead of being copy-pasted
* (and drifting) in each script. Catalog isolation (#1977) lives here for the same
* reason — it is a property every web smoke needs, not one script's concern.
* (runs-the-bundle check, #1615), `scripts/smoke-web-app.mjs` (MCP Apps
* end-to-end, #1859) and `scripts/smoke-web-elicitation.mjs` (app-rendered
* elicitations, #1854) all boot the *same* prod `mcp-inspector --web` server, so
* the spawn + readiness-poll boilerplate lives here once instead of being
* copy-pasted (and drifting) in each script. Catalog isolation (#1977) lives here
* for the same reason — it is a property every web smoke needs, not one script's
* concern.
*
* Repo-root paths are derived from import.meta.url, so a caller's cwd (e.g.
* `smoke:web:browser` does `cd clients/web` first so its `npx playwright
* install` finds the local bin) doesn't affect which launcher/build tree runs.
* Repo-root paths are derived from import.meta.url rather than the cwd, so it
* does not matter where a caller is run from. (Until #2086 the browser-driven
* smokes were invoked through a `cd clients/web && npx playwright install …`
* prefix, which is what originally made that independence load-bearing; they now
* run from the repo root via `scripts/install-smoke-browser.mjs`. Deriving from
* import.meta.url is still the right call — it is what lets a smoke be run by
* hand from any directory.)
*/
import { spawn } from "node:child_process";

9
scripts/pack-and-verify.mjs
View file

@ -60,12 +60,11 @@ import {
ensureTestServers,
testServerEntryPath,
} from "./lib/ensure-test-servers.mjs";
import { attachPageDiagnostics, loadBrowser } from "./lib/headless-browser.mjs";
import {
APP_TOOL,
attachPageDiagnostics,
buildAppDeepLink,
driveAppFlow,
loadChromium,
startMcpAppServer,
} from "./lib/mcp-app-flow.mjs";
import { winShellArgs } from "./lib/win-shell-args.mjs";
@ -567,7 +566,11 @@ async function verifyAppRender(baseUrl, token, whenWebServerExits) {
},
label: LABEL,
});
browser = await loadChromium(repoRoot);
// Chromium explicitly, not `resolveBrowserName()`: this is a *packaging*
// check, and the engine question (#2086) belongs to the smokes, where the
// sandbox surface is what is under test. Pinning it also means `pack:verify`
// cannot be pointed at an engine its npm script never installed.
browser = await loadBrowser(repoRoot, "chromium");
const page = await browser.newPage();
const diagnostics = attachPageDiagnostics(page);

86
scripts/run-engine-smokes.mjs Normal file
View file

@ -0,0 +1,86 @@
#!/usr/bin/env node
/**
* Run the three headless web smokes in one browser engine (#2086).
*
* `SMOKE_BROWSER=firefox npm run smoke:web:engine` needs the variable to reach
* three child processes, and a POSIX `VAR=x npm run …` prefix does not work
* under Windows' `cmd.exe`, which npm uses there. Setting it in the child env
* from Node does, on every platform — the same reason `install-smoke-browser`
* exists rather than an inline shell expansion.
*
* It also gives the smoke list ONE home. `ENGINE_SMOKES` is that list, and
* EVERY tier reads it — the default Chromium run inside `npm run smoke`
* (`smoke:web:chromium`), the Firefox pass in the pre-push gate
* (`smoke:web:firefox`), and the on-demand `smoke:web:engine`. Getting that
* wrong is silent: a tier that enumerated the smokes itself would simply run
* fewer of them and still pass, so `run-engine-smokes.test.mjs` asserts that no
* tier names an individual smoke.
*
* Usage: `node scripts/run-engine-smokes.mjs [engine]`. With no argument the
* engine comes from `SMOKE_BROWSER` (default `chromium`).
*/
import { spawnSync } from "node:child_process";
import { join, resolve } from "node:path";
import { pathToFileURL } from "node:url";
import { resolveRequestedBrowser } from "./install-smoke-browser.mjs";
/**
* The smokes that are engine-sensitive, in run order.
*
* Cheapest first, so a bundle that cannot even boot on the engine fails in
* seconds rather than after two full App flows have timed out.
*/
export const ENGINE_SMOKES = [
"smoke-web-browser.mjs",
"smoke-web-app.mjs",
"smoke-web-elicitation.mjs",
];
function main() {
const scriptDir = resolve(import.meta.dirname);
let browserName;
try {
browserName = resolveRequestedBrowser(process.argv.slice(2), process.env);
} catch (err) {
console.error(
`run-engine-smokes: ${err instanceof Error ? err.message : String(err)}`,
);
process.exit(1);
}
// Children inherit this rather than reading the ambient variable, so an
// explicit argument wins over SMOKE_BROWSER all the way down.
const env = { ...process.env, SMOKE_BROWSER: browserName };
const install = spawnSync(
process.execPath,
[join(scriptDir, "install-smoke-browser.mjs"), browserName],
{ stdio: "inherit", env },
);
if (install.status !== 0) process.exit(install.status ?? 1);
for (const smoke of ENGINE_SMOKES) {
const result = spawnSync(process.execPath, [join(scriptDir, smoke)], {
stdio: "inherit",
env,
});
if (result.error) {
console.error(
`run-engine-smokes: could not run ${smoke}: ${result.error.message}`,
);
process.exit(1);
}
if (result.status !== 0) {
// The smoke has already printed its own diagnosis; add only the engine,
// so a failure in the pre-push gate names which run it came from.
console.error(`run-engine-smokes: ${smoke} failed under ${browserName}`);
process.exit(result.status ?? 1);
}
}
}
// Only when run as a script, never on import — see install-smoke-browser.mjs.
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href)
main();

155
scripts/run-engine-smokes.test.mjs Normal file
View file

@ -0,0 +1,155 @@
/**
* The engine-smoke runner's list (#2086).
*
* `ENGINE_SMOKES` is the single home for "which smokes are engine-sensitive",
* and the thing that can silently rot about it is a smoke going missing — the
* suite still passes, just covering less. Nothing at runtime notices, because a
* shorter list is a *successful* run.
*
* Importing the module must not spawn anything; `main()` is behind an entrypoint
* guard, and this test would catch that regressing by running three browser
* smokes the moment the suite imported it.
*/
import assert from "node:assert/strict";
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
import { describe, it } from "node:test";
import { ENGINE_SMOKES } from "./run-engine-smokes.mjs";
import { SUPPORTED_BROWSERS } from "./lib/headless-browser.mjs";
const scriptDir = import.meta.dirname;
const scripts = JSON.parse(
readFileSync(join(scriptDir, "..", "package.json"), "utf8"),
).scripts;
describe("ENGINE_SMOKES", () => {
it("names every browser-driven smoke, and each one exists", () => {
// Deliberately spelled out rather than derived: this list IS the coverage
// claim, so a test that computed it from disk would agree with any mistake.
assert.deepEqual(ENGINE_SMOKES, [
"smoke-web-browser.mjs",
"smoke-web-app.mjs",
"smoke-web-elicitation.mjs",
]);
for (const smoke of ENGINE_SMOKES) {
assert.ok(
existsSync(join(scriptDir, smoke)),
`${smoke} is listed but missing — a rename would silently shrink the run`,
);
}
});
it("runs the cheapest smoke first", () => {
// smoke-web-browser only boots the bundle; the other two drive a whole App
// flow with 45s waits. Ordering it first is what makes "this engine cannot
// run the bundle at all" fail in seconds rather than after two timeouts.
assert.equal(ENGINE_SMOKES[0], "smoke-web-browser.mjs");
});
});
describe("every engine tier consumes ENGINE_SMOKES", () => {
// The point of the list is defeated if any tier enumerates the smokes itself.
// `npm run smoke` used to do exactly that (Copilot, #2133): a fourth entry in
// ENGINE_SMOKES would have reached the Firefox and on-demand runs and silently
// skipped the DEFAULT Chromium run — which is the one GitHub CI executes. That
// is invisible at runtime, because a tier running fewer smokes still passes.
it("the Chromium tier goes through the runner, not its own list", () => {
assert.match(scripts["smoke:web:chromium"], /run-engine-smokes\.mjs/);
assert.match(scripts.smoke, /smoke:web:chromium/);
for (const individual of [
"smoke:web:browser",
"smoke:web:app",
"smoke:web:elicit",
]) {
assert.ok(
!scripts.smoke.includes(individual),
`\`smoke\` names ${individual} directly — it must route through ` +
"`smoke:web:chromium` so every tier reads one list",
);
}
});
it("the Firefox tier goes through the runner and is in the pre-push gate", () => {
assert.match(scripts["smoke:web:firefox"], /run-engine-smokes\.mjs/);
assert.match(scripts.ci, /smoke:web:firefox/);
});
it("every supported engine has a `smoke:web:<engine>` script", () => {
// loadBrowser's launch-failure message tells the reader to run
// `npm run smoke:web:<engine>`. If a supported engine has no such script,
// that remedy is unrunnable — which is how the message shipped naming
// `smoke:web:webkit` before it existed (Copilot, #2133).
for (const name of SUPPORTED_BROWSERS) {
assert.match(
scripts[`smoke:web:${name}`] ?? "",
new RegExp(`run-engine-smokes\\.mjs ${name}$`),
`npm run smoke:web:${name} is promised by the launch-failure message`,
);
}
});
it("each gated tier names its engine explicitly rather than reading the env", () => {
// An ambient SMOKE_BROWSER must not be able to redirect a gate: without the
// literal engine, `SMOKE_BROWSER=firefox npm run ci` would run Firefox twice
// and never exercise Chromium at all.
assert.match(
scripts["smoke:web:chromium"],
/run-engine-smokes\.mjs chromium$/,
);
assert.match(
scripts["smoke:web:firefox"],
/run-engine-smokes\.mjs firefox$/,
);
// The on-demand entry point is the one that SHOULD follow the environment.
assert.match(scripts["smoke:web:engine"], /run-engine-smokes\.mjs$/);
});
});
describe("each smoke's own docs point at its own command", () => {
/** The npm script that runs each engine-sensitive smoke on its own. */
const OWN_SCRIPT = {
"smoke-web-browser.mjs": "smoke:web:browser",
"smoke-web-app.mjs": "smoke:web:app",
"smoke-web-elicitation.mjs": "smoke:web:elicit",
};
it("never tells the reader to run a sibling smoke", () => {
// `smoke-web-elicitation.mjs` shipped an on-demand example invoking
// `smoke:web:app` (Copilot, #2133) — a direct cost of my deciding the two
// App headers should be word-for-word identical so they could not drift.
// The shared PROSE should be identical; the example command is the one line
// that must not be, and copy-paste does not distinguish them.
//
// Following such an example does not fail — it runs a real smoke and passes.
// It just never exercises the file you were reading about, which is why
// nothing else catches this.
for (const [file, own] of Object.entries(OWN_SCRIPT)) {
const source = readFileSync(join(scriptDir, file), "utf8");
const siblings = Object.values(OWN_SCRIPT).filter((s) => s !== own);
for (const sibling of siblings) {
assert.ok(
!source.includes(`npm run ${sibling}`),
`${file} tells the reader to run \`npm run ${sibling}\` — a sibling ` +
`smoke. Its examples must use \`${own}\`, or the reader never ` +
`exercises the path this file documents.`,
);
}
}
});
it("every command it does name is a real script", () => {
// A command that does not exist is worse than none, and the header is the
// one place nothing executes to find out.
for (const file of Object.keys(OWN_SCRIPT)) {
const source = readFileSync(join(scriptDir, file), "utf8");
for (const [, name] of source.matchAll(/npm run ([\w:]+)/g)) {
assert.ok(
Object.hasOwn(scripts, name),
`${file} names \`npm run ${name}\`, which is not a script`,
);
}
}
});
});

50
scripts/smoke-web-app.mjs
View file

@ -41,6 +41,31 @@
* renamed* without its reader being updated, a repo-tree failure pack:verify
* would only find later.
*
* ── Which engine ────────────────────────────────────────────────────────────
*
* `SMOKE_BROWSER` picks the engine (`chromium` — the default — `firefox`, or
* `webkit`). Three tiers, deliberately (#2086):
*
* - **GitHub CI** runs this smoke in **Chromium** only.
* - **`npm run ci`**, the local pre-push gate, runs it in **Chromium and
* Firefox** — the Firefox pass is `smoke:web:firefox`, and it is the one
* gate step with no GitHub CI counterpart.
* - **WebKit is on demand only**: `SMOKE_BROWSER=webkit npm run smoke:web:app`
* for this smoke alone, or `npm run smoke:web:webkit` for all three.
*
* Firefox passes. WebKit fails this smoke for reasons nobody has identified and
* nobody is investigating: it does not reproduce in real Safari, and an isolated
* SSE repro did not reproduce it under Playwright's WebKit either, so it reads
* as a property of that build rather than a browser bug. Do not read a WebKit
* failure here as a defect until someone has actually looked.
*
* This smoke is one of the two places the
* MCP Apps sandbox is genuinely exercised, and the sandbox is built out of the
* primitives that actually diverge between engines — `srcdoc` CSP inheritance,
* nested sandboxed iframes, `Permissions-Policy`, cross-frame `postMessage`. See
* `lib/headless-browser.mjs`, including why a green WebKit run is not a Safari
* guarantee.
*
* Expects `clients/web/dist` and `clients/launcher/build` to be built first —
* the validate / CI ordering guarantees this. `test-servers/build` is rebuilt on
* every run, as in smoke:cli — see `scripts/lib/ensure-test-servers.mjs` for why
@ -54,17 +79,34 @@ import { join, resolve } from "node:path";
import { startProdWebServer } from "./lib/prod-web-server.mjs";
import { stopChild } from "./lib/child-cleanup.mjs";
import {
APP_TOOL,
attachPageDiagnostics,
loadBrowser,
resolveBrowserName,
} from "./lib/headless-browser.mjs";
import {
APP_TOOL,
buildAppDeepLink,
driveAppFlow,
loadChromium,
sandboxProxyPageFor,
startMcpAppServer,
} from "./lib/mcp-app-flow.mjs";
const repoRoot = resolve(import.meta.dirname, "..");
const LABEL = "smoke:web:app";
// Resolved before anything is started, so an unsupported SMOKE_BROWSER fails
// immediately rather than after a web server and two MCP servers are up. Every
// message this smoke prints carries the engine, so a failure names which one
// broke rather than leaving the reader to remember what they invoked it with.
let BROWSER;
try {
BROWSER = resolveBrowserName();
} catch (err) {
console.error(
`smoke:web:app FAILED — ${err instanceof Error ? err.message : String(err)}`,
);
process.exit(1);
}
const LABEL = `smoke:web:app [${BROWSER}]`;
// Resolved exactly as the runtime does, from the built runner's directory.
const sandboxProxyPage = sandboxProxyPageFor(
@ -141,7 +183,7 @@ try {
label: LABEL,
});
await server.waitForReady();
browser = await loadChromium(repoRoot);
browser = await loadBrowser(repoRoot, BROWSER);
const page = await browser.newPage();
const diagnostics = attachPageDiagnostics(page);

131
scripts/smoke-web-browser.mjs
View file

@ -29,12 +29,13 @@
* Two channels carry the failure. A *synchronous* uncaught exception (the
* CASE-1 shape above — a stub call during module init) fires `pageerror`. Its
* *async* twin — the same `TypeError` reached through an `await`/`.then()`, or a
* failed dynamic import (this app lazy-loads chunks) — is NOT a `pageerror`;
* Chromium logs it on the **console** channel as `Uncaught (in promise) …` /
* `Failed to fetch dynamically imported module`. Both are hard failures.
* failed dynamic import (this app lazy-loads chunks) — is NOT a `pageerror`; it
* arrives on the **console** channel as `Uncaught (in promise) …` /
* `Failed to fetch dynamically imported module`. Both are hard failures, and
* both channels are read on every engine rather than branching per browser.
*
* Every *other* `console.error` is NOT a hard failure: the console is where
* Chromium also reports benign things a boot smoke shouldn't fail on — a failed
* Every *other* `console.error` is NOT a hard failure: the console is also where
* benign things a boot smoke shouldn't fail on land — a failed
* subresource load (e.g. the Google-Fonts `<link>` in index.html on a
* network-restricted box) or a React key/prop warning. Those are printed as
* diagnostics. The `Uncaught` / dynamic-import prefixes are unambiguous — a
@ -42,29 +43,48 @@
* never starts with `Uncaught` — so hard-failing on them can't reintroduce that
* flake.
*
* Playwright lives in clients/web's node_modules, so it's resolved with a
* `createRequire` based at clients/web/package.json rather than a bare
* `import("playwright")`. A bare ESM specifier resolves relative to *this
* script's* directory (scripts/), not the cwd — so `cd clients/web` in the npm
* script would NOT make it resolvable (it only appeared to work locally when an
* ancestor node_modules happened to carry playwright; it fails in CI, which has
* none). createRequire pins resolution to clients/web regardless of cwd.
* Launching the browser (and resolving Playwright from clients/web, which has
* its own gotcha — see `lib/headless-browser.mjs`) is delegated to that module,
* which is also where `SMOKE_BROWSER` picks the engine: `chromium` (the
* default), `firefox`, or `webkit` (#2086). GitHub CI runs this in **Chromium**
* only; `npm run ci`, the local pre-push gate, also runs it in **Firefox** via
* `smoke:web:firefox`; **WebKit is on demand only**. This smoke passes in all
* three — it is the two App smokes that fail under WebKit (see their headers).
*
* The engine question here is narrower than in the App smokes — this asserts a clean
* first paint, i.e. that the shipped bundle's syntax and API level are
* *reachable* on the engine at all, rather than anything about the sandbox — but
* it is real, and it is nearly free to include.
*
* Expects `clients/web/dist` and `clients/launcher/build` to be built first —
* the validate / CI ordering guarantees this.
*/
import { createRequire } from "node:module";
import { setTimeout as delay } from "node:timers/promises";
import { fileURLToPath } from "node:url";
import { dirname, resolve } from "node:path";
import { startProdWebServer } from "./lib/prod-web-server.mjs";
import {
attachPageDiagnostics,
loadBrowser,
resolveBrowserName,
} from "./lib/headless-browser.mjs";
const scriptDir = dirname(fileURLToPath(import.meta.url));
// Resolve playwright from clients/web (where it's installed) no matter the cwd.
const requireFromWeb = createRequire(
resolve(scriptDir, "..", "clients/web/package.json"),
);
const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
// Resolved before the web server is started, so an unsupported SMOKE_BROWSER
// fails immediately. Every message carries the engine, so a failure names which
// one broke.
let BROWSER;
try {
BROWSER = resolveBrowserName();
} catch (err) {
console.error(
`smoke:web:browser FAILED — ${err instanceof Error ? err.message : String(err)}`,
);
process.exit(1);
}
const LABEL = `smoke:web:browser [${BROWSER}]`;
const HOST = "127.0.0.1";
// Distinct from smoke:web's SMOKE_WEB_PORT so overriding one doesn't make both
@ -72,17 +92,11 @@ const HOST = "127.0.0.1";
const PORT = process.env.SMOKE_WEB_BROWSER_PORT ?? "6298";
const TOKEN = "smoke-web-browser-token";
// Console messages that are the async half of the uncaught-crash class (an
// unhandled promise rejection or a failed dynamic import). Hard failures, unlike
// benign console noise. See the header comment for why this can't reintroduce
// the font/CDN flake.
const FATAL_CONSOLE = /^Uncaught\b|Failed to fetch dynamically imported module/;
const server = startProdWebServer({
host: HOST,
port: PORT,
token: TOKEN,
label: "smoke:web:browser",
label: LABEL,
});
let browser = null;
@ -99,51 +113,22 @@ async function shutdown() {
}
async function fail(message) {
console.error(`smoke:web:browser FAILED — ${message}`);
console.error(`${LABEL} FAILED — ${message}`);
await shutdown();
process.exit(1);
}
async function loadChromium() {
let chromium;
try {
({ chromium } = requireFromWeb("playwright"));
} catch (err) {
// A failure here means the playwright *npm package* isn't resolvable from
// clients/web — fixed by installing devDependencies, NOT by `playwright
// install` (which fetches browser binaries). `npm install` at the repo root
// cascades into clients/web via postinstall.
throw new Error(
`could not resolve the Playwright package from clients/web — run \`npm install\` at the repo root (${err instanceof Error ? err.message : String(err)})`,
);
}
try {
return await chromium.launch({ headless: true });
} catch (err) {
throw new Error(
`chromium failed to launch — on a bare Linux box run \`npx playwright install --with-deps chromium\` for the system libraries (${err instanceof Error ? err.message : String(err)})`,
);
}
}
try {
await server.waitForReady();
browser = await loadChromium();
browser = await loadBrowser(repoRoot, BROWSER);
const page = await browser.newPage();
// Uncaught (synchronous) page errors are a hard failure — a Node-only module
// reaching the browser bundle surfaces here as a TypeError when its empty stub
// is called during module init.
const pageErrors = [];
// Console errors are diagnostic only EXCEPT those matching FATAL_CONSOLE (the
// async half of the same crash class) — see the header comment.
const consoleErrors = [];
page.on("pageerror", (err) =>
pageErrors.push(err instanceof Error ? err.message : String(err)),
);
page.on("console", (msg) => {
if (msg.type() === "error") consoleErrors.push(msg.text());
});
// is called during module init. Console errors are diagnostic only EXCEPT the
// async half of that same crash class; see the header comment, and the shared
// helper for the split.
const diagnostics = attachPageDiagnostics(page);
const render = async () => {
// Token is injected into index.html by the prod server, so a bare `/` load
@ -175,39 +160,35 @@ try {
try {
await Promise.race([server.whenChildExits(), render()]);
} catch (err) {
const diagnostics = [
...pageErrors,
...consoleErrors.map((m) => `console: ${m}`),
const notes = [
...diagnostics.pageErrors,
...diagnostics.consoleErrors.map((m) => `console: ${m}`),
];
await fail(
`${err instanceof Error ? err.message : String(err)}${
diagnostics.length
? ` — page diagnostics: ${diagnostics.join("; ")}`
: ""
notes.length ? ` — page diagnostics: ${notes.join("; ")}` : ""
}`,
);
}
// Hard failures: any uncaught (sync) page error, plus console errors that are
// the async half of the class (unhandled rejection / failed dynamic import).
const fatalConsole = consoleErrors.filter((m) => FATAL_CONSOLE.test(m));
if (pageErrors.length > 0 || fatalConsole.length > 0) {
await fail(
`app logged uncaught error(s): ${[...pageErrors, ...fatalConsole].join("; ")}`,
);
const fatal = diagnostics.fatal();
if (fatal.length > 0) {
await fail(`app logged uncaught error(s): ${fatal.join("; ")}`);
}
// Non-fatal console errors: surface them so a real problem isn't invisible,
// without failing the smoke on benign subresource/warning noise.
const benignConsole = consoleErrors.filter((m) => !FATAL_CONSOLE.test(m));
if (benignConsole.length > 0) {
const benign = diagnostics.benignConsole();
if (benign.length > 0) {
console.log(
`smoke:web:browser note — ${benignConsole.length} non-fatal console error(s): ${benignConsole.join("; ")}`,
`${LABEL} note — ${benign.length} non-fatal console error(s): ${benign.join("; ")}`,
);
}
console.log(
`smoke:web:browser OK — app booted at ${server.baseUrl}, rendered "Add Servers" with no uncaught errors (sync page error or unhandled rejection)`,
`${LABEL} OK — app booted at ${server.baseUrl}, rendered "Add Servers" with no uncaught errors (sync page error or unhandled rejection)`,
);
await shutdown();
process.exit(0);

117
scripts/smoke-web-elicitation.mjs
View file

@ -22,9 +22,29 @@
* therefore needs `frameLocator(...).frameLocator(...)`, not one hop.
*
* Set `SMOKE_SCREENSHOT_DIR` to capture PNGs of each state (used to attach
* proof to a PR); unset, it asserts only. Playwright is resolved with a
* `createRequire` based at clients/web/package.json for the reason documented at
* length in smoke-web-browser.mjs.
* proof to a PR); unset, it asserts only.
*
* `SMOKE_BROWSER` picks the engine (`chromium` — the default — `firefox`, or
* `webkit`). Three tiers, deliberately (#2086):
*
* - **GitHub CI** runs this smoke in **Chromium** only.
* - **`npm run ci`**, the local pre-push gate, runs it in **Chromium and
* Firefox** — the Firefox pass is `smoke:web:firefox`, and it is the one
* gate step with no GitHub CI counterpart.
* - **WebKit is on demand only**: `SMOKE_BROWSER=webkit npm run smoke:web:elicit`
* for this smoke alone, or `npm run smoke:web:webkit` for all three.
*
* Firefox passes. WebKit fails this smoke for reasons nobody has identified and
* nobody is investigating: it does not reproduce in real Safari, and an isolated
* SSE repro did not reproduce it under Playwright's WebKit either, so it reads
* as a property of that build rather than a browser bug. Do not read a WebKit
* failure here as a defect until someone has actually looked.
*
* Along with `smoke:web:app` this is one
* of the two places the MCP Apps sandbox is actually loaded, and the two nested
* frames below are precisely the surface that diverges between engines. See
* `lib/headless-browser.mjs`, including why a green WebKit run is not a Safari
* guarantee.
*
* Expects `clients/web/dist` and `clients/launcher/build` to be built first.
* `test-servers/build` is rebuilt on every run, as in smoke:web:app — see
@ -33,20 +53,35 @@
import { spawn } from "node:child_process";
import { mkdirSync } from "node:fs";
import { createRequire } from "node:module";
import { setTimeout as delay } from "node:timers/promises";
import { join, resolve } from "node:path";
import { startProdWebServer } from "./lib/prod-web-server.mjs";
import { stopChild } from "./lib/child-cleanup.mjs";
import {
attachPageDiagnostics,
loadBrowser,
resolveBrowserName,
} from "./lib/headless-browser.mjs";
import {
ensureTestServers,
testServerEntryPath,
} from "./lib/ensure-test-servers.mjs";
const repoRoot = resolve(import.meta.dirname, "..");
const requireFromWeb = createRequire(
resolve(repoRoot, "clients/web/package.json"),
);
// Resolved before anything is started, so an unsupported SMOKE_BROWSER fails
// immediately rather than after a web server and two MCP servers are up. Every
// message carries the engine, so a failure names which one broke.
let BROWSER;
try {
BROWSER = resolveBrowserName();
} catch (err) {
console.error(
`smoke:web:elicit FAILED — ${err instanceof Error ? err.message : String(err)}`,
);
process.exit(1);
}
const LABEL = `smoke:web:elicit [${BROWSER}]`;
const composableServer = testServerEntryPath(repoRoot, "composable");
const configPath = (name) =>
@ -59,9 +94,6 @@ const PORT = process.env.SMOKE_WEB_ELICIT_PORT ?? "6296";
const TOKEN = "smoke-web-elicit-token";
const TOOL = "app_choose_option";
const SHOT_DIR = process.env.SMOKE_SCREENSHOT_DIR;
// The async half of the uncaught-crash class. Kept identical to
// smoke-web-browser.mjs / smoke-web-app.mjs.
const FATAL_CONSOLE = /^Uncaught\b|Failed to fetch dynamically imported module/;
const servers = [];
let browser = null;
@ -69,7 +101,7 @@ const web = startProdWebServer({
host: HOST,
port: PORT,
token: TOKEN,
label: "smoke:web:elicit",
label: LABEL,
});
async function shutdown() {
@ -84,14 +116,14 @@ async function shutdown() {
await web.stop();
while (servers.length) {
await stopChild(servers.pop(), {
label: "smoke:web:elicit",
label: LABEL,
what: "MCP test server",
});
}
}
async function fail(message) {
console.error(`smoke:web:elicit FAILED — ${message}`);
console.error(`${LABEL} FAILED — ${message}`);
await shutdown();
process.exit(1);
}
@ -134,24 +166,6 @@ async function startMcpServer(configName) {
throw new Error(`MCP test server did not start within 30s:\n${out}`);
}
async function loadChromium() {
let chromium;
try {
({ chromium } = requireFromWeb("playwright"));
} catch (err) {
throw new Error(
`could not resolve the Playwright package from clients/web — run \`npm install\` at the repo root (${err instanceof Error ? err.message : String(err)})`,
);
}
try {
return await chromium.launch({ headless: true });
} catch (err) {
throw new Error(
`chromium failed to launch — on a bare Linux box run \`npx playwright install --with-deps chromium\` for the system libraries (${err instanceof Error ? err.message : String(err)})`,
);
}
}
async function shot(page, name) {
if (!SHOT_DIR) return;
mkdirSync(SHOT_DIR, { recursive: true });
@ -159,7 +173,7 @@ async function shot(page, name) {
path: join(SHOT_DIR, `${name}.png`),
fullPage: false,
});
console.log(`smoke:web:elicit — captured ${name}.png`);
console.log(`${LABEL} — captured ${name}.png`);
}
/** Connect to `mcpUrl` through the deep link and wait for the Tools list. */
@ -207,31 +221,24 @@ try {
// Rebuilt on every run — presence is not freshness (#2111).
ensureTestServers({
repoRoot,
label: "smoke:web:elicit",
label: LABEL,
requires: ["composable"],
});
const appUrl = await startMcpServer("app-elicitation-http");
const nativeUrl = await startMcpServer("app-elicitation-native-http");
await web.waitForReady();
browser = await loadChromium();
browser = await loadBrowser(repoRoot, BROWSER);
const page = await browser.newPage({
viewport: { width: 1280, height: 900 },
});
// Uncaught *synchronous* page errors, and their *async* twin (an unhandled
// rejection or a failed dynamic import), which Chromium reports on the
// console channel instead. Both are hard failures; every other console error
// is only a diagnostic, so benign font/React noise cannot flake CI. Same
// split as smoke:web:browser and smoke:web:app, which document it at length.
const pageErrors = [];
const consoleErrors = [];
page.on("pageerror", (err) =>
pageErrors.push(err instanceof Error ? err.message : String(err)),
);
page.on("console", (msg) => {
if (msg.type() === "error") consoleErrors.push(msg.text());
});
const fatalConsole = () => consoleErrors.filter((m) => FATAL_CONSOLE.test(m));
// Uncaught *synchronous* page errors, plus their *async* twin (an unhandled
// rejection or a failed dynamic import), which arrives on the console channel
// instead. Both are hard failures; every other console error is only a
// diagnostic, so benign font/React noise cannot flake CI. Shared with
// smoke:web:app rather than re-hand-rolled — the split is subtle enough that
// two copies would drift, and it is documented at length on the helper.
const diagnostics = attachPageDiagnostics(page);
const drive = async () => {
// ── 1. Negotiated: the server's app answers the elicitation ────────────
@ -297,27 +304,25 @@ try {
try {
await Promise.race([web.whenChildExits(), drive()]);
} catch (err) {
const diagnostics = [
...pageErrors,
...fatalConsole().map((m) => `console: ${m}`),
const notes = [
...diagnostics.pageErrors,
...diagnostics.fatalConsole().map((m) => `console: ${m}`),
];
await fail(
`${err instanceof Error ? err.message : String(err)}${
diagnostics.length
? ` — page diagnostics: ${diagnostics.join("; ")}`
: ""
notes.length ? ` — page diagnostics: ${notes.join("; ")}` : ""
}`,
);
}
// A drive that reached all of its assertions still fails if the page threw on
// the way: without this the smoke prints OK over a broken bundle.
const fatal = [...pageErrors, ...fatalConsole()];
const fatal = diagnostics.fatal();
if (fatal.length > 0) {
await fail(`page logged uncaught error(s): ${fatal.join("; ")}`);
}
console.log("smoke:web:elicit OK — app-rendered and native paths both drive");
console.log(`${LABEL} OK — app-rendered and native paths both drive`);
await shutdown();
} catch (err) {
await fail(err instanceof Error ? err.message : String(err));