modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

chore(ci): render an MCP App from the installed tarball in pack:verify (#2003)

Browse source 
Nothing covered the intersection #1859 fell through. `smoke:web:app` drives
the App path but against the repo build tree; `pack:verify` runs against the
installed tarball but only asserted the sandbox proxy page *exists*. So a
packaging regression that ships the file present-but-unreachable — a rename
with a stale reader, a wrong position relative to clients/web/build, a future
"files"/.npmignore edit — passes both.

Extract the drive-the-App-flow half of the smoke into
scripts/lib/mcp-app-flow.mjs and call it from both, rather than copying it:
the deep-link shape is the part that rots silently, and a drifted link fails
as an opaque timeout rather than a mismatch. pack:verify then drives
connect -> open app -> data-app-status="ready" in headless Chromium against
the installed `--web` boot it already starts.

The client under test comes from the install; the App test server stays a
repo fixture.

Also isolates pack:verify's `--web` catalog (the #1977 treatment the smokes
already have). Untidy while the step only did `GET /`; the App deep link
persists a server row, which would make it destructive and non-deterministic
against the developer's real ~/.mcp-inspector/mcp.json.

Unit tests cover what neither consumer can check itself — both CSRF gates and
the appArgs encoding on the deep link, and driveAppFlow's failure branches
against a stand-in page, all dead code from a happy-path smoke's view.

Verified: pack:verify passes end to end against the real tarball, and fails
at the new step alone when sandbox-controller.ts is pointed at a renamed
proxy page (the tarball and on-disk assertions both still pass — the exact
gap this closes).

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-24 07:44:58 -04:00
commit 17c33080f9
7 changed files with 676 additions and 242 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AGENTS.md
View file

@ -814,7 +814,7 @@ The ⚠️ option-deletion hazard, the snapshot rule, and the recovery recipe ab
**Every web smoke runs against a throwaway catalog (#1977).** The helper mints a temp dir per run and passes it as `MCP_CATALOG_PATH`; without it the web backend falls back to the developer's real `~/.mcp-inspector/mcp.json`, which made these smokes both destructive and non-deterministic — `smoke:web:app`'s deep link persists a `deep-link` server row, so a *second* run found it already on disk, raced hydration, and drew a spurious (swallowed, non-fatal) 409 that was really just residue from the previous run. CI never saw it: a fresh `HOME` per run made every CI run look like a first run. This matches `smoke:cli` / `smoke:tui`, which have always driven a temp `--catalog`. Only the **catalog** is redirected — other per-user state under `~/.mcp-inspector` (OAuth tokens, `storage/`) stays shared, because isolating it means redirecting `HOME` wholesale, which would also move the npx and Playwright caches these smokes depend on. Teardown uses **both** halves of `scripts/lib/child-cleanup.mjs` (`stopChild` to await the child's exit, then `removeSafe` to delete the dir) — a bare `kill()` only *delivers* the signal, so removing synchronously re-enters the #1801 ENOTEMPTY race. That makes `stop()` **async**, so every caller must `await` it (and a caller's own `fail()`/`shutdown()` becomes async in turn, or execution runs past the intended exit). The isolation contract is unit-tested in `scripts/lib/prod-web-server.test.mjs` via `test:scripts`, since the smokes exit immediately after teardown and so cannot detect a regression that silently reshared the catalog or stopped cleaning up: `createTempCatalog` and `buildWebServerEnv` cover *which* catalog the server gets, and `teardownWebServer` — extracted from `stop()` for exactly this reason — is driven against a stand-in child process so the teardown asserts on the real directory rather than a spy.
- `smoke:web:browser` (`scripts/smoke-web-browser.mjs`, #1615) goes a step further than `smoke:web`: it boots the same prod `--web` server and then actually **runs** the bundle in headless Chromium (Playwright — already a `clients/web` devDependency for the Storybook tests), asserting the app renders its first meaningful frame (the "Add Servers" control) with **no uncaught error**. `smoke:web` only checks the served HTML, so a Node built-in reaching the browser bundle slipped through it; this smoke catches that regression as a _class_ (e.g. #1612). The mechanism is the uncaught error, not a magic string: under Vite the excluded module becomes an empty stub and the first _call_ into it (e.g. `fs.readFileSync(...)` during a transitive module's init) throws a `TypeError` that aborts app mount. A _synchronous_ such throw fires `pageerror`; its _async_ twin (the same `TypeError` via `await`/`.then()`, or a failed dynamic import) is logged on the console channel as `Uncaught (in promise) …` / `Failed to fetch dynamically imported module` — the smoke hard-fails on both. The literal `Module "…" has been externalized` text is, **in a prod build**, a build-time warning (`vite build` / `npm run build`), not a runtime message, so the browser never sees it (under `npm run dev` Vite's stub is instead a `Proxy` that `console.warn`s that string at runtime); and an externalized import that is never _called_ ships a harmless `{}` and is invisible here by design. Every _other_ console error is printed as a diagnostic, not a failure (so a benign font-CDN or React-warning `console.error` doesn't flake CI). Playwright is resolved via `createRequire` based at `clients/web/package.json` — a bare `import("playwright")` would resolve relative to `scripts/`, not the cwd, so it can't be reached that way (it only appears to work when an ancestor `node_modules` carries playwright, and fails in CI, which has none). The npm script's `cd clients/web` exists only so `npx playwright install chromium` finds the local playwright bin (a no-op when already installed).
- `smoke:web:app` (`scripts/smoke-web-app.mjs`, #1859) goes one step further again: `smoke:web:browser` stops at first paint and never connects to a server, so the Apps tab, the sandbox controller, and the UI-protocol bridge were unexercised by any smoke. This one boots the same prod `--web` server, spawns the `mcp-app-http.json` composable test server (the `mcp_app_demo` tool + its `mcp_app_demo_widget` UI resource), and drives the whole **connect → open app → widget ready** chain through a single deep-link navigate (`?serverUrl=…&autoConnect=<token>&openApp=…&appArgs=…&autoOpen=<token>`). The assertion is the `data-app-status="ready"` contract from [clients/web/README.md](clients/web/README.md) — the renderer reports `ready` only once the widget has loaded inside the sandbox iframe _and_ fired `notifications/initialized` back through the bridge, so one attribute covers the sandbox proxy being served, the UI resource loading, and the handshake completing. Two mechanics are load-bearing and easy to get wrong: the test server announces readiness on **stderr** (`console.error` in `server-composable.ts`), so both child streams are piped and scanned — watching stdout alone times out with an empty diagnostic; and its bound port is **not** the config's, because `createTestServerHttp` resolves through `findAvailablePort()`, which walks upward when the configured port is taken — so the smoke parses the announced URL rather than assuming `3130`. Both mechanics now live in `scripts/lib/announced-child.mjs` rather than in the smoke, so the failure path is testable: this smoke's happy path always receives the announcement, so nothing it could assert would prove that a child alive *through* the 30s timeout is still reachable by `shutdown()` — the case that orphaned a live server (#2000). The helper publishes the child via `onSpawn` before waiting, and `scripts/lib/announced-child.test.mjs` drives real `node -e` children (not spies) to assert it is published, still alive when the throw lands, and actually killable. Same reason `teardownWebServer` was extracted from `prod-web-server.mjs`'s `stop()`. **Scope note:** this runs against the repo build tree like every other smoke, so it would _not_ have caught #1859 itself (a packaging failure — the file is always present in-repo); `pack:verify` owns that dimension. It does carry a cheap structural pre-check that the proxy page exists at the path `sandbox-controller.ts` resolves, so a move/rename fails fast with a clear cause instead of an opaque render timeout.
- `smoke:web:app` (`scripts/smoke-web-app.mjs`, #1859) goes one step further again: `smoke:web:browser` stops at first paint and never connects to a server, so the Apps tab, the sandbox controller, and the UI-protocol bridge were unexercised by any smoke. This one boots the same prod `--web` server, spawns the `mcp-app-http.json` composable test server (the `mcp_app_demo` tool + its `mcp_app_demo_widget` UI resource), and drives the whole **connect → open app → widget ready** chain through a single deep-link navigate (`?serverUrl=…&autoConnect=<token>&openApp=…&appArgs=…&autoOpen=<token>`). The assertion is the `data-app-status="ready"` contract from [clients/web/README.md](clients/web/README.md) — the renderer reports `ready` only once the widget has loaded inside the sandbox iframe _and_ fired `notifications/initialized` back through the bridge, so one attribute covers the sandbox proxy being served, the UI resource loading, and the handshake completing. Two mechanics are load-bearing and easy to get wrong: the test server announces readiness on **stderr** (`console.error` in `server-composable.ts`), so both child streams are piped and scanned — watching stdout alone times out with an empty diagnostic; and its bound port is **not** the config's, because `createTestServerHttp` resolves through `findAvailablePort()`, which walks upward when the configured port is taken — so the smoke parses the announced URL rather than assuming `3130`. Both mechanics now live in `scripts/lib/announced-child.mjs` rather than in the smoke, so the failure path is testable: this smoke's happy path always receives the announcement, so nothing it could assert would prove that a child alive *through* the 30s timeout is still reachable by `shutdown()` — the case that orphaned a live server (#2000). The helper publishes the child via `onSpawn` before waiting, and `scripts/lib/announced-child.test.mjs` drives real `node -e` children (not spies) to assert it is published, still alive when the throw lands, and actually killable. Same reason `teardownWebServer` was extracted from `prod-web-server.mjs`'s `stop()`. **Scope note:** this runs against the repo build tree like every other smoke, so it would _not_ have caught #1859 itself (a packaging failure — the file is always present in-repo); `pack:verify` owns that dimension, and since #2003 it owns it *properly* — it drives this same flow against the installed tarball rather than only asserting the proxy page exists. The flow therefore lives in `scripts/lib/mcp-app-flow.mjs` (deep-link construction, the three staged assertions, the Chromium/diagnostics plumbing) and is **shared, not copied**: two copies of the deep-link shape would drift, and a drifted link fails as a silent timeout rather than a mismatch. Keep both consumers — `pack:verify` is network-bound and local/release-only, so it does not run in `npm run ci`, where this smoke is the only thing exercising the App path at all. It does carry a cheap structural pre-check that the proxy page exists at the path `sandbox-controller.ts` resolves, so a move/rename fails fast with a clear cause instead of an opaque render timeout.
- `smoke:web:elicit` (`scripts/smoke-web-elicitation.mjs`, #1854) is the app-rendered **elicitation** counterpart of `smoke:web:app`: same prod `--web` server and the same deep-link connect, but it then calls `app_choose_option` from the Tools tab, waits for `[data-testid="app-elicitation"][data-app-elicitation-status="ready"]`, clicks a choice **inside the sandboxed app** (two `frameLocator` hops — the trusted sandbox proxy, then the untrusted app), and asserts the app's standard `ElicitResult` comes back in the *tool result*, i.e. that it reached the server rather than merely the host. It then repeats against `app-elicitation-native-http.json` — the same tool and app on a server that never advertised the nested MCP Apps `elicitation` capability — and asserts the **native** elicitation dialog takes it and no app modal is rendered. That second half is the more valuable one: the failure this feature can produce is not "the app doesn't render" but "an app renders when it should not have been offered one", which strands every user of a server that never opted in. Set `SMOKE_SCREENSHOT_DIR` to capture PNGs of the three states (used for PR proof); unset, it asserts only. Two mechanics worth knowing: the main-view tabs are a Mantine `SegmentedControl`, so there is no `role="tab"` — the clickable element is the sibling `label[for$="-Tools"]`; and the prompt string also appears in the (hidden) Protocol-tab payload, so the fallback assertion is scoped to the dialog rather than a bare text lookup.
- **The build gate for the browser-externalized-builtin class (#1769)** is the earlier, more complete companion to `smoke:web:browser`. A Vite plugin in `clients/web/vite.config.ts` (logic in `clients/web/server/browser-externalized-builtin-gate.ts`, unit-tested) turns Vite 8's _browser-externalization warning_ (`Module "node:*" has been externalized for browser compatibility`) into a hard `vite build` error, so a Node built-in in the browser graph now **fails `npm run build` / `validate`** instead of shipping a `{}` stub. This catches **both** the _called-at-init_ case (which `smoke:web:browser` also catches, but later/at runtime) **and** the _imported-but-never-called_ case (the `{}` stub that is invisible to the runtime smoke "by design" — see above). Because rolldown **swallows a throw inside `onLog`** (the one hook where a thrown error doesn't abort — verified against vite@8.0.0), the plugin _records_ the warning in `onLog` and re-throws in `buildEnd`. There is **no stable log `code`**, so the gate keys off the documented message phrasing; `npm run verify:build-gate` (`scripts/verify-build-gate.mjs`, in `npm run ci` and the GitHub workflow) runs a real build with a `node:fs` probe forced into `src/main.tsx` and asserts the build fails via the gate — the only check that catches the message phrasing **drifting** in a future Vite bump and silently disabling the gate. The gate is scoped to `vite build` (`apply: 'build'`) — never `vite dev` or the vitest projects — **and** to the browser (`client`) environment (`applyToEnvironment`), so a future SSR/node environment built from this config isn't failed for a legitimate `node:*` import; the Node runner build (tsup, `build:runner`) is a separate config where built-ins are legitimate. `smoke:web:browser` stays as the runtime backstop for crashes the build can't reason about.

6
README.md
View file

@ -369,7 +369,7 @@ Each client self-validates from its own folder; the root scripts chain them. The
| `npm run verify:build-gate` | Runs a real `vite build` with a Node built-in forced into the browser graph and asserts the build **fails** via the #1769 gate (which turns Vite's browser-externalization warning into a hard error). Guards against the warning phrasing drifting in a Vite bump and silently disabling the gate. Part of `npm run ci`. |
| `npm run verify:bundle-externals` | Guards the must-not-bundle invariant (#2067): for each tsup-bundled client it reads the **built** `build/` output and fails if any package that must stay external was inlined anyway. Candidates are the union of the client's own `external` array **and the root manifest's `dependencies`** — the latter because #2067 was a *missing* `external` entry, which a self-referential check would have passed. Detection is via esbuild's `// <path>` module banners, so it covers both shapes — a separate `<pkg>-HASH.js` chunk (what a dynamically `import()`ed CommonJS package produces) and a statically-imported package folded straight into `index.js`, which emits no chunk at all. A build with no banners fails as such rather than passing clean, so enabling `minify` cannot silently retire the check. `undici` was declared only in the root and `clients/cli` manifests, and tsup auto-externalizes only what the _nearest_ manifest declares, so the web and TUI bundles inlined 1.05MB of it — and CommonJS inlined into an ESM bundle throws `Dynamic require of "assert" is not supported` on first use, from a specifier no user-side install can satisfy. Reads the output rather than the config because those two disagreed for four releases. Part of `npm run ci`. |
| `npm run verify:format-coverage` | Parses the `format:check` globs out of every `package.json` (only those reachable from `validate`), enumerates all tracked source files, and **fails** listing any not covered by a glob — the durable guard for the "every first-party source file is format-gated" invariant (#1792). Runs first in `validate`. |
| `npm run test:scripts` | Table-driven unit tests (`node --test`) for the guard's own pure parsers (`scripts/lib/npm-scripts.mjs`, `scripts/lib/tsc-program.mjs` + the exported helpers of `verify-typecheck-coverage.mjs` and `verify-dep-lockstep.mjs`), one case per rule they encode, plus two suites over shared `scripts/lib` helpers that no smoke can check itself: `resolve-node-bin.test.mjs` — the cross-platform bin resolver (#1939), pinned against the real `bin`/`exports` shapes of the packages the scripts actually spawn — and `announced-child.test.mjs` — the spawn/readiness ownership helper (#2000), which drives real `node -e` children to prove a child that never announces is still published to the caller before the timeout throws, and so is reachable by teardown rather than orphaned. Runs in `validate` — and `verify:typecheck-coverage` guards *this* gate in turn (reachable from `validate`, non-empty test set, every test file matched by the `test:scripts` glob), since `node --test` silently skips a file its glob misses and still exits 0. |
| `npm run test:scripts` | Table-driven unit tests (`node --test`) for the guard's own pure parsers (`scripts/lib/npm-scripts.mjs`, `scripts/lib/tsc-program.mjs` + the exported helpers of `verify-typecheck-coverage.mjs` and `verify-dep-lockstep.mjs`), one case per rule they encode, plus two suites over shared `scripts/lib` helpers that no smoke can check itself: `resolve-node-bin.test.mjs` — the cross-platform bin resolver (#1939), pinned against the real `bin`/`exports` shapes of the packages the scripts actually spawn — and `announced-child.test.mjs` — the spawn/readiness ownership helper (#2000), which drives real `node -e` children to prove a child that never announces is still published to the caller before the timeout throws, and so is reachable by teardown rather than orphaned. A third, `mcp-app-flow.test.mjs`, covers the shared MCP Apps flow (#2003) — the deep link's two CSRF gates and `appArgs` encoding, plus `driveAppFlow`'s failure branches against a stand-in page, all of which are dead code from the happy-path smokes' point of view and would otherwise surface only as opaque timeouts. Runs in `validate` — and `verify:typecheck-coverage` guards *this* gate in turn (reachable from `validate`, non-empty test set, every test file matched by the `test:scripts` glob), since `node --test` silently skips a file its glob misses and still exits 0. |
| `npm run verify:typecheck-coverage` | The typecheck-coverage analog of the above (#1791): for each Node client (auto-discovered from disk — enrolled via its `typecheck` script's projects, or for a `tsc -b` client like `clients/web` via its `tsconfig.json` `references`) it runs those projects with `tsc --listFilesOnly`, unions them, and **fails** listing any tracked `.ts`/`.tsx`/`.mts`/`.cts` under the client that lands in no project (so a new top-level config/helper can't silently go untypechecked). It also requires, deny-by-default, the first-party TS no client owns (`test-servers/src`, the root `vitest.shared.mts`, all of `core/`, and any new top-level location) to land in some client project's tsc pass — so a `core` `*.tsx` web's projects don't reach is caught too. Also asserts the gate is wired (each client's typecheck pass — its `typecheck` script, or web's `tsc -b` — is reachable from its `validate`, and the root chain runs each client's `validate`). Runs in `validate`. |
| `npm run verify:dep-lockstep` | Guards the "one version per install-crossing dependency" invariant (#1896). v2 is not a workspace, so a client's test project compiles the shared first-party TypeScript — `core/`, `test-servers/src`, and the root-owned `vitest.shared.mts`, all of which resolve their dependencies from the **root** install — alongside the client's own sources, putting the same package in one `tsc` program twice. At the same version that's harmless; skewed, TypeScript must relate two structurally-distinct copies of every type, which for a recursive-generic surface is exponential (zod `4.3.6` vs `4.4.3` exhausted the 4GB tsc heap in `clients/web`). Derives its candidate set from **what actually enters each program** (#1965) — every client tsconfig project listed with `tsc --listFilesOnly` via the shared `scripts/lib/tsc-program.mjs`, each resolved `node_modules` file mapped to its owning install, keeping the packages that reach one program from two installs (a package whose declarations arrive only through another package's `.d.ts`, as `@modelcontextprotocol/sdk`'s do, is invisible to a scan of first-party imports). Prices each copy from the lockfile entry for the exact install path the program resolved, compares only the installs that met in one program, and **fails deny-by-default** on any disagreement not in the annotated `TOLERATED_SKEW` allowlist — empty today — with an allowlisted package tolerated only *within a major version*. Runs in `validate`.
| `npm run ci` | **Mandatory pre-push command.** `validate` → `coverage` → `verify:build-gate` → `verify:bundle-externals` → `smoke` → Storybook. A true superset of GitHub CI. |
@ -393,13 +393,13 @@ The root `package.json` `"files"` allowlist is the source of truth for the tarba
- **No source maps.** The client bundlers set `sourcemap: false` (`clients/{cli,tui}/tsup.config.ts`, `clients/web/tsup.runner.config.ts`); Vite and the launcher's `tsc` already emit none. Maps are ~half the unpacked size and aren't needed at runtime — debug via `npm run dev` on the source.
- **`clients/web/build` ships via `clients/web/.npmignore`.** `clients/web/.gitignore` lists `build/`, and npm's packlist honors that nested `.gitignore` over the root `"files"` allowlist — so the prod web-server runner was silently missing from the tarball while `clients/web/dist` slipped through (its `.gitignore` only lists `dist-ssr`). `clients/web/.npmignore` overrides the `.gitignore` for publishing so both `build/` (runner) and `dist/` (SPA) ship. The other clients don't need this — none ship a nested `.gitignore`.
- **`clients/web/static` ships the MCP Apps sandbox proxy.** `clients/web/static/sandbox_proxy.html` is a committed source file (not a build artifact), read from disk at runtime by `clients/web/server/sandbox-controller.ts` as `<runner dir>/../static/sandbox_proxy.html`. It was missing from the root `"files"` allowlist entirely, so every published build failed the Apps tab with **"Sandbox not loaded"** ([#1859](https://github.com/modelcontextprotocol/inspector/issues/1859)) while working fine in the repo. Because the path is resolved _relative to_ `clients/web/build`, the directory must ship at that exact location — `pack:verify` asserts both the tarball entry and the installed-on-disk path.
- **`clients/web/static` ships the MCP Apps sandbox proxy.** `clients/web/static/sandbox_proxy.html` is a committed source file (not a build artifact), read from disk at runtime by `clients/web/server/sandbox-controller.ts` as `<runner dir>/../static/sandbox_proxy.html`. It was missing from the root `"files"` allowlist entirely, so every published build failed the Apps tab with **"Sandbox not loaded"** ([#1859](https://github.com/modelcontextprotocol/inspector/issues/1859)) while working fine in the repo. Because the path is resolved _relative to_ `clients/web/build`, the directory must ship at that exact location — `pack:verify` asserts the tarball entry, the installed-on-disk path, and (since [#2003](https://github.com/modelcontextprotocol/inspector/issues/2003)) that a widget actually *loads through* it on the installed bin. Presence and reachability are different properties: a rename with a stale reader ships a file that is there and unusable.
- **A dependency that renders React is bundled, not externalized.** An externalized package resolves its own `react` from wherever npm placed **it** in the consumer's tree, which is not necessarily where the bundle resolves ours — npm places a package beside a React satisfying _its_ peer range, and those ranges are looser than ours. `ink-form` and `ink-scroll-view` declare `">=18"`, so a project holding React 18 satisfies them and gets them hoisted while the Inspector's React 19 nests underneath: two React copies, and the TUI dies with `TypeError: Cannot read properties of null (reading 'useState')` the moment a tool test form or a scroll view mounts ([#1952](https://github.com/modelcontextprotocol/inspector/issues/1952)). Both are therefore inlined by `clients/tui/tsup.config.ts` and are **not** root dependencies: the tarball ships their code inside `clients/tui/build/index.js` rather than having consumers install them. Bundling also pins their transitive deps to what this repo's install resolved (notably `ink-select-input@6` via `overrides`, which npm ignores for a package installed as a dependency). **`ink` is the one exception, on cost:** bundling it works but adds ~1.4 MB (`react-reconciler` and `yoga-layout` come along, plus a `createRequire` banner for the inlined CJS), so it stays external — _not_ because its `">=19"` peer makes it safe, which it does not. What keeps that tolerable is the root `react` range: `"^19.0.0"` is deliberately open to the whole major so npm can dedupe our React with whatever React 19 a consumer pins, leaving an external `ink` on the same copy the bundle uses. **Narrowing that range reopens the bug for the renderer itself** — `clients/tui/__tests__/tsupConfig.test.ts` pins it to `ink`'s peer floor, and guards the rest of the split; see the [TUI README](./clients/tui/README.md#bundling-react-rendering-dependencies-must-be-inlined-1952).
- **A single version number, read from the root `package.json`.** The Inspector ships as one package with one version, so only the **root** `package.json` carries a `version` — the four `clients/*/package.json`s deliberately have none. Every Node client (CLI, TUI, and the web backend) resolves the version through the shared `readInspectorVersion()` reader in `core/node/version.ts`, which walks up to the root manifest (always present in the tarball). No client `package.json` is read at runtime, so none needs to ship. The web **browser** can't read the filesystem; it gets its version from the backend via `GET /api/config` (see [#1639](https://github.com/modelcontextprotocol/inspector/issues/1639)).
### `npm run pack:verify` — publish smoke against the real tarball
The `smoke:*` scripts run against the in-repo build tree, which is **not** the published package. `npm run pack:verify` (`scripts/pack-and-verify.mjs`) closes that gap: it builds, `npm pack`s the publishable tarball (asserting no source maps ship and that the runtime-required files are present), installs the tarball into a **clean throwaway consumer** — a fresh temp directory where it runs a real `npm install <tgz>` (pulls runtime deps, runs `postinstall`), exactly as `npx @modelcontextprotocol/inspector` would — and drives the installed `mcp-inspector` bin end to end: `--help` dispatch, a real `--cli tools/list` over stdio, and a prod `--web` boot that must serve `/` from the shipped `dist`. It catches "works in `--dev`, breaks under `npx …`" path/packaging failures. It requires network access (the install pulls deps), so it is a local / release check, **not** part of the fast `validate`/`ci` loop.
The `smoke:*` scripts run against the in-repo build tree, which is **not** the published package. `npm run pack:verify` (`scripts/pack-and-verify.mjs`) closes that gap: it builds, `npm pack`s the publishable tarball (asserting no source maps ship and that the runtime-required files are present), installs the tarball into a **clean throwaway consumer** — a fresh temp directory where it runs a real `npm install <tgz>` (pulls runtime deps, runs `postinstall`), exactly as `npx @modelcontextprotocol/inspector` would — and drives the installed `mcp-inspector` bin end to end: `--help` dispatch, a real `--cli tools/list` over stdio, a prod `--web` boot that must serve `/` from the shipped `dist`, and — riding that same boot — an **MCP App rendered in headless Chromium** through the shipped sandbox proxy, connect → open app → `data-app-status="ready"` ([#2003](https://github.com/modelcontextprotocol/inspector/issues/2003)). That last step shares its flow with `smoke:web:app` via `scripts/lib/mcp-app-flow.mjs` rather than copying it, since the deep-link shape is the part that rots silently; the *client* comes from the install while the App test server stays a repo fixture. It catches "works in `--dev`, breaks under `npx …`" path/packaging failures. It requires network access (the install pulls deps), so it is a local / release check, **not** part of the fast `validate`/`ci` loop.
### Cutting a release

2
package.json
View file

@ -74,7 +74,7 @@
"smoke:web:app": "cd clients/web && npx playwright install chromium && node ../../scripts/smoke-web-app.mjs",
"smoke:web:elicit": "cd clients/web && npx playwright install chromium && node ../../scripts/smoke-web-elicitation.mjs",
"smoke:launcher": "node scripts/smoke-launcher.mjs",
"pack:verify": "node scripts/pack-and-verify.mjs",
"pack:verify": "cd clients/web && npx playwright install chromium && cd ../.. && node scripts/pack-and-verify.mjs",
"prepack": "npm run build",
"postinstall": "node scripts/install-clients.mjs"
},

282
scripts/lib/mcp-app-flow.mjs Normal file
View file

@ -0,0 +1,282 @@
/**
* The MCP Apps drive-the-App-flow half of `smoke:web:app`, extracted so
* `pack:verify` can run the same flow against the **installed tarball** (#2003).
*
* Two checks bracketed the bug that motivated all of this (#1859, the missing
* `clients/web/static/sandbox_proxy.html`) without either one catching it:
*
* | check | runs against | drives the App path? |
* | --------------- | ------------------ | -------------------- |
* | `smoke:web:app` | the repo build tree | yes |
* | `pack:verify` | the installed tarball | no — only `GET /` |
*
* So a packaging regression that leaves the file present-but-unreachable — a
* rename with a stale reader, a wrong relative position, a future `.npmignore`
* or `"files"` edit — passes both: the smoke never installs, and `pack:verify`
* only asserted the path *exists*. Sharing the flow is what closes it, and it
* has to be *shared* rather than copied: the deep-link shape is the part that
* silently rots, and two copies would drift without either failing.
*
* Note the split of responsibilities that survives here. The **client** under
* test comes from wherever the caller booted it (repo tree or install); the
* **test server** is always a repo fixture — it is not in the tarball and
* should not be.
*
* Playwright is resolved with a `createRequire` based at
* clients/web/package.json rather than a bare `import("playwright")`: a bare ESM
* specifier resolves relative to `scripts/`, not the cwd, so a `cd clients/web`
* in the npm script would NOT make it resolvable. Same gotcha as
* smoke-web-browser.mjs; see its header.
*/
import { spawnSync } from "node:child_process";
import { existsSync } from "node:fs";
import { createRequire } from "node:module";
import { join } from "node:path";
import { startAnnouncedChild } from "./announced-child.mjs";
import { resolveNodeBin } from "./resolve-node-bin.mjs";
/** The App tool the `mcp-app-http.json` fixture serves. */
export const APP_TOOL = "mcp_app_demo";
/**
* Console messages that are the async half of the uncaught-crash class (an
* unhandled rejection or a failed dynamic import). Hard failures; every other
* console error is a diagnostic, so benign font-CDN / React-warning noise can't
* flake CI. Kept identical to smoke-web-browser.mjs, which documents the
* reasoning at length.
*/
export const FATAL_CONSOLE =
/^Uncaught\b|Failed to fetch dynamically imported module/;
/** Path to the composable test server build, relative to the repo root. */
export function composableServerPath(repoRoot) {
return join(repoRoot, "test-servers", "build", "server-composable.js");
}
/** Path to a `test-servers/configs/<name>.json` fixture. */
export function testServerConfigPath(repoRoot, name) {
return join(repoRoot, "test-servers", "configs", `${name}.json`);
}
/**
* The path `clients/web/server/sandbox-controller.ts` resolves at runtime, given
* the directory holding the built web runner. Kept in sync with the
* `join(__dirname, "../static/sandbox_proxy.html")` there — the position
* *relative to* the runner is what matters, which is why callers pass the
* runner dir rather than a package root.
*/
export function sandboxProxyPageFor(webBuildDir) {
return join(webBuildDir, "..", "static", "sandbox_proxy.html");
}
/**
* Build the composable test server bundle if it isn't present yet.
*
* The root-installed tsc is run via this Node — `npx` is a `.cmd` shim on
* Windows that a shell-free spawnSync can't start (ENOENT — #1939).
*/
export function ensureComposableTestServer(repoRoot, label) {
const entry = composableServerPath(repoRoot);
if (existsSync(entry)) return entry;
console.log(`${label} — building test-servers (missing build output)...`);
const r = spawnSync(
process.execPath,
[
resolveNodeBin("typescript", "tsc", repoRoot),
"-p",
"test-servers",
"--noCheck",
],
{ cwd: repoRoot, stdio: "inherit" },
);
if (r.status !== 0 || !existsSync(entry)) {
throw new Error(
"could not build the test servers (test-servers/build/server-composable.js). " +
"Run `npm run test-servers:build` from clients/web.",
);
}
return entry;
}
/**
* Spawn a composable test server and wait for it to announce its URL.
*
* The announced line is authoritative rather than the config's port:
* `createTestServerHttp` resolves through `findAvailablePort()`, which walks
* upward when the configured value is taken — so the config port is a starting
* hint, not a guarantee, and assuming it fails whenever anything else holds it.
* The announcement goes to **stderr** (`console.error` in
* server-composable.ts), which is why `startAnnouncedChild` scans both channels.
*
* `onSpawn` publishes the child before the readiness wait, so the caller's
* teardown reaches it on every throw path — the timeout included (#2000).
*/
export async function startMcpAppServer({
repoRoot,
config = "mcp-app-http",
onSpawn,
label,
}) {
const entry = ensureComposableTestServer(repoRoot, label);
const { match } = await startAnnouncedChild({
command: process.execPath,
args: [entry, "--config", testServerConfigPath(repoRoot, config)],
cwd: repoRoot,
pattern: /listening at (http:\/\/\S+)/i,
onSpawn,
what: `MCP test server (${entry})`,
});
return match[1];
}
/** base64url(JSON) — the `appArgs` encoding the deep link expects. */
export function encodeAppArgs(args) {
return Buffer.from(JSON.stringify(args))
.toString("base64")
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, "");
}
/**
* The deep link that connects, switches to the Apps tab, pre-selects the app
* tool, and fires "Open App". `autoConnect`/`autoOpen` must equal the session
* token (CSRF gate). Shape owned by clients/web/README.md#deep-link-auto-connect.
*/
export function buildAppDeepLink({
baseUrl,
mcpUrl,
token,
appTool = APP_TOOL,
appArgs = {},
}) {
return (
`${baseUrl}/?serverUrl=${encodeURIComponent(mcpUrl)}` +
`&transport=http&autoConnect=${token}&openApp=${appTool}` +
`&appArgs=${encodeAppArgs(appArgs)}&autoOpen=${token}`
);
}
/**
* Attach the two error channels a headless page reports on.
*
* An uncaught *synchronous* page error fires `pageerror`; its async twin — an
* unhandled rejection or a failed dynamic import — is not a `pageerror` at all,
* Chromium reports it on the console channel instead. Both are captured; only
* `fatal()` is a failure, so ordinary console noise stays a diagnostic.
*/
export function attachPageDiagnostics(page) {
const pageErrors = [];
const consoleErrors = [];
page.on("pageerror", (err) =>
pageErrors.push(err instanceof Error ? err.message : String(err)),
);
page.on("console", (msg) => {
if (msg.type() === "error") consoleErrors.push(msg.text());
});
return {
pageErrors,
consoleErrors,
fatalConsole: () => consoleErrors.filter((m) => FATAL_CONSOLE.test(m)),
benignConsole: () => consoleErrors.filter((m) => !FATAL_CONSOLE.test(m)),
fatal: () => [
...pageErrors,
...consoleErrors.filter((m) => FATAL_CONSOLE.test(m)),
],
};
}
/** Launch headless Chromium, resolved from the web client's install. */
export async function loadChromium(repoRoot) {
const requireFromWeb = createRequire(
join(repoRoot, "clients", "web", "package.json"),
);
let chromium;
try {
({ chromium } = requireFromWeb("playwright"));
} catch (err) {
// Not resolvable means devDependencies are missing — fixed by `npm install`
// at the repo root, NOT by `playwright install` (which fetches binaries).
throw new Error(
`could not resolve the Playwright package from clients/web — run \`npm install\` at the repo root (${err instanceof Error ? err.message : String(err)})`,
);
}
try {
return await chromium.launch({ headless: true });
} catch (err) {
throw new Error(
`chromium failed to launch — on a bare Linux box run \`npx playwright install --with-deps chromium\` for the system libraries (${err instanceof Error ? err.message : String(err)})`,
);
}
}
/**
* Drive **connect → open app → widget ready** on an already-open page.
*
* The load-bearing assertion is the `data-app-status="ready"` contract
* documented in clients/web/README.md ("MCP Apps screen automation contract"):
* the renderer reports `ready` only once the widget has loaded inside the
* sandbox iframe *and* fired `notifications/initialized` back through the
* bridge. So one attribute covers the whole path — the sandbox controller
* serving the proxy page, the proxy loading the UI resource, and the bridge
* completing its handshake.
*
* Throws with the last observed `data-app-status` / `data-app-error` rather than
* a bare timeout, so a failure names its stage.
*
* Takes only the small slice of Playwright's `page` it uses (`goto`, `locator`
* → `waitFor`/`getAttribute`/`count`), which is what lets its failure paths be
* unit-tested against a stand-in — a smoke only ever exercises its happy path.
*/
export async function driveAppFlow({
page,
url,
gotoTimeoutMs = 30_000,
connectTimeoutMs = 45_000,
readyTimeoutMs = 45_000,
}) {
const response = await page.goto(url, {
waitUntil: "domcontentloaded",
timeout: gotoTimeoutMs,
});
if (!response || !response.ok()) {
throw new Error(
`GET / returned HTTP ${response ? response.status() : "no response"}`,
);
}
// 1. The deep link must be accepted (not rejected by the token gate).
const status = page.locator('[data-testid="connection-status"]');
await status.waitFor({ state: "attached", timeout: gotoTimeoutMs });
const deeplink = await status.getAttribute("data-deeplink");
if (deeplink !== "parsed") {
throw new Error(
`deep link was not accepted (data-deeplink="${deeplink}") — expected "parsed"`,
);
}
// 2. Connected to the test server.
await page
.locator('[data-testid="connection-status"][data-status="connected"]')
.waitFor({ state: "attached", timeout: connectTimeoutMs });
// 3. The widget rendered inside the sandbox and completed its handshake.
try {
await page
.locator('[data-testid="apps-form"][data-app-status="ready"]')
.waitFor({ state: "attached", timeout: readyTimeoutMs });
} catch {
const form = page.locator('[data-testid="apps-form"]');
const present = (await form.count()) > 0;
const appStatus = present
? await form.getAttribute("data-app-status")
: "(no apps-form)";
const appError = present ? await form.getAttribute("data-app-error") : null;
throw new Error(
`app never reached data-app-status="ready" (last: "${appStatus}"` +
`${appError ? `, data-app-error="${appError}"` : ""}) — the sandbox proxy ` +
`or the UI-protocol bridge failed to complete`,
);
}
}

166
scripts/lib/mcp-app-flow.test.mjs Normal file
View file

@ -0,0 +1,166 @@
/**
* Unit tests for the shared MCP Apps flow (#2003).
*
* These cover the halves neither consumer can check itself. `smoke:web:app` and
* `pack:verify` only ever exercise the happy path — the widget reaches `ready`,
* so every failure branch below is dead code from their point of view, and the
* deep-link shape is asserted by nothing at all once it is built rather than
* inlined. Both are exactly the parts that rot silently: a mangled `appArgs`
* encoding or a dropped `autoOpen` token produces a page that simply never
* connects, reported as an opaque timeout.
*
* `driveAppFlow` is driven against a stand-in page rather than Chromium — it
* takes only `goto` and `locator(...)` → `waitFor`/`getAttribute`/`count`,
* which is what makes that possible.
*/
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import {
buildAppDeepLink,
driveAppFlow,
encodeAppArgs,
sandboxProxyPageFor,
} from "./mcp-app-flow.mjs";
/** Minimal Playwright `page` stand-in. `plan` maps a selector to its behavior. */
function fakePage(plan, { gotoStatus = 200 } = {}) {
return {
goto: async () => ({
ok: () => gotoStatus === 200,
status: () => gotoStatus,
}),
locator: (selector) => {
const entry = plan[selector] ?? {};
return {
waitFor: async () => {
if (entry.waitFails) throw new Error(`timeout: ${selector}`);
},
getAttribute: async (name) => entry.attrs?.[name] ?? null,
count: async () => entry.count ?? 0,
};
},
};
}
const STATUS = '[data-testid="connection-status"]';
const CONNECTED = '[data-testid="connection-status"][data-status="connected"]';
const FORM = '[data-testid="apps-form"]';
const READY = '[data-testid="apps-form"][data-app-status="ready"]';
/** The plan under which every stage succeeds. */
const happyPlan = () => ({
[STATUS]: { attrs: { "data-deeplink": "parsed" } },
[CONNECTED]: {},
[READY]: {},
});
describe("encodeAppArgs", () => {
it("encodes JSON as base64url with no padding", () => {
const encoded = encodeAppArgs({ title: "a/b+c?" });
assert.doesNotMatch(encoded, /[+/=]/);
assert.deepEqual(JSON.parse(Buffer.from(encoded, "base64url").toString()), {
title: "a/b+c?",
});
});
});
describe("buildAppDeepLink", () => {
const link = buildAppDeepLink({
baseUrl: "http://127.0.0.1:6399",
mcpUrl: "http://127.0.0.1:3130/mcp",
token: "tok",
appArgs: { title: "t" },
});
const params = new URL(link).searchParams;
it("percent-encodes the server URL so its own query/scheme can't leak", () => {
assert.ok(link.includes(encodeURIComponent("http://127.0.0.1:3130/mcp")));
assert.equal(params.get("serverUrl"), "http://127.0.0.1:3130/mcp");
});
it("carries the session token on BOTH gates", () => {
// autoConnect and autoOpen are separate CSRF gates; dropping either leaves
// the page silently idle rather than erroring.
assert.equal(params.get("autoConnect"), "tok");
assert.equal(params.get("autoOpen"), "tok");
});
it("defaults to the fixture's app tool and round-trips appArgs", () => {
assert.equal(params.get("openApp"), "mcp_app_demo");
assert.equal(params.get("transport"), "http");
assert.deepEqual(
JSON.parse(Buffer.from(params.get("appArgs"), "base64url").toString()),
{ title: "t" },
);
});
});
describe("sandboxProxyPageFor", () => {
it("resolves relative to the runner dir, as sandbox-controller.ts does", () => {
assert.match(
sandboxProxyPageFor("/pkg/clients/web/build").replace(/\\/g, "/"),
/\/pkg\/clients\/web\/(build\/\.\.\/)?static\/sandbox_proxy\.html$/,
);
});
});
describe("driveAppFlow", () => {
it("resolves when every stage succeeds", async () => {
await driveAppFlow({ page: fakePage(happyPlan()), url: "http://x/" });
});
it("reports a non-200 document instead of waiting on selectors", async () => {
await assert.rejects(
driveAppFlow({
page: fakePage(happyPlan(), { gotoStatus: 500 }),
url: "http://x/",
}),
/GET \/ returned HTTP 500/,
);
});
it("names a rejected deep link rather than timing out on connect", async () => {
// The token gate refusing the link is the failure a bad `autoConnect`
// produces; without this branch it surfaces as a 45s connect timeout.
const plan = happyPlan();
plan[STATUS] = { attrs: { "data-deeplink": "rejected" } };
await assert.rejects(
driveAppFlow({ page: fakePage(plan), url: "http://x/" }),
/deep link was not accepted \(data-deeplink="rejected"\)/,
);
});
it("surfaces the last app status and error when ready never arrives", async () => {
const plan = happyPlan();
plan[READY] = { waitFails: true };
plan[FORM] = {
count: 1,
attrs: {
"data-app-status": "error",
"data-app-error": "Sandbox not loaded",
},
};
await assert.rejects(
driveAppFlow({ page: fakePage(plan), url: "http://x/" }),
(err) => {
// The #1859 shape: the widget never renders because the proxy page the
// runtime reads is not where it was shipped. The message has to name
// that, not just "timeout".
assert.match(err.message, /last: "error"/);
assert.match(err.message, /data-app-error="Sandbox not loaded"/);
return true;
},
);
});
it("says so when the apps form never mounted at all", async () => {
const plan = happyPlan();
plan[READY] = { waitFails: true };
plan[FORM] = { count: 0 };
await assert.rejects(
driveAppFlow({ page: fakePage(plan), url: "http://x/" }),
/last: "\(no apps-form\)"/,
);
});
});

178
scripts/pack-and-verify.mjs
View file

@ -29,7 +29,19 @@
* 4. runs the installed `mcp-inspector` bin: `--help`, `--cli`/`--tui` help
* dispatch, a real `--cli` `tools/list` over stdio, and a prod `--web` boot
* that must serve `/` (HTTP 200) with the injected auth-token global from
* the shipped `dist` — all from the INSTALLED location, not the repo.
* the shipped `dist` — all from the INSTALLED location, not the repo;
* 5. drives the **MCP Apps** path in headless Chromium against that same
* installed `--web` server — connect → open app → `data-app-status="ready"`
* (#2003). Asserting the sandbox proxy page merely *exists* (step 2/3) is
* not the same as loading a widget through it: a rename with a stale
* reader, or a wrong position relative to `clients/web/build`, ships a file
* that is present and unreachable. The flow is shared with `smoke:web:app`
* via `lib/mcp-app-flow.mjs` so the deep-link shape cannot drift between
* the repo-tree check and this one.
*
* The **client** under test comes from the install; the **test server** stays a
* repo fixture (`test-servers/build/server-composable.js`) — it is not in the
* tarball and should not be.
*
* Exits non-zero on the first failure. Requires network access (step 3 pulls the
* package's runtime dependencies from the registry) — it is a local / release
@ -44,6 +56,15 @@ import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { hasExited, removeSafe, stopChild } from "./lib/child-cleanup.mjs";
import {
APP_TOOL,
attachPageDiagnostics,
buildAppDeepLink,
composableServerPath,
driveAppFlow,
loadChromium,
startMcpAppServer,
} from "./lib/mcp-app-flow.mjs";
import { winShellArgs } from "./lib/win-shell-args.mjs";
const repoRoot = resolve(import.meta.dirname, "..");
@ -53,6 +74,7 @@ const testServer = join(
"build",
"test-server-stdio.js",
);
const composableServer = composableServerPath(repoRoot);
// Mirrors INSPECTOR_API_TOKEN_GLOBAL in core/mcp/remote/constants.ts; kept as a
// literal because this plain .mjs script can't import the TS source.
@ -70,6 +92,10 @@ let workDir = null;
// the port and could even serve a stale false-200 to a later run (verifyWeb's
// own `finally { stop() }` is skipped when fail() calls process.exit()).
let webChild = null;
// The composable MCP App test server spawned for the step-5 App render, if any.
// Same reasoning as `webChild`: fail() calls process.exit(), which skips the
// `finally` that would otherwise stop it, and an orphan holds its port.
let appServerChild = null;
const LABEL = "pack:verify";
@ -82,8 +108,10 @@ function fail(message) {
// exiting 1, so an ENOTEMPTY could only bury the real diagnostic under an
// rmSync stack (and skip the "tarball retained" hint below), never turn a
// green run red.
if (webChild && !hasExited(webChild)) {
webChild.kill("SIGTERM");
for (const child of [webChild, appServerChild]) {
if (child && !hasExited(child)) {
child.kill("SIGTERM");
}
}
if (workDir) {
removeSafe(workDir, { label: LABEL });
@ -117,14 +145,22 @@ function runInherit(command, args, cwd = repoRoot) {
return r.status;
}
/** Build the bundled stdio test server if it isn't present yet. */
/**
* Build the bundled test servers if they aren't present yet — both the stdio
* entry (step 4b) and the composable one (step 5). One `tsc -p test-servers`
* emits both, so this builds once and checks each entry.
*/
function ensureTestServer() {
if (existsSync(testServer)) return;
if (existsSync(testServer) && existsSync(composableServer)) return;
step("building test-servers (missing build output)...");
const status = runInherit("npx", ["tsc", "-p", "test-servers", "--noCheck"]);
if (status !== 0 || !existsSync(testServer)) {
if (
status !== 0 ||
!existsSync(testServer) ||
!existsSync(composableServer)
) {
fail(
"could not build the stdio test server (test-servers/build/test-server-stdio.js)",
"could not build the test servers (test-servers/build/{test-server-stdio,server-composable}.js)",
);
}
}
@ -376,7 +412,8 @@ try {
console.log(
"\npack:verify OK — published tarball installs clean and the real bin drives " +
"web (prod / served dist), cli (stdio tools/list), and tui (help) end to end.",
"web (prod / served dist + an MCP App rendered through the shipped sandbox " +
"proxy), cli (stdio tools/list), and tui (help) end to end.",
);
} catch (err) {
// Unexpected throw (not via fail()) — route through fail() for consistent
@ -404,6 +441,15 @@ async function verifyWeb(bin, cwd) {
CLIENT_PORT: port,
HOST: host,
MCP_INSPECTOR_API_TOKEN: token,
// Isolate the catalog, for the reason #1977 gave the web smokes: without
// it the backend falls back to the developer's real
// ~/.mcp-inspector/mcp.json. That was merely untidy while this step only
// did `GET /`; the App deep link below *persists a server row*, which
// would make this both destructive and non-deterministic (a second run
// finds the row already there and races hydration). Assigned after the
// spread so an inherited MCP_CATALOG_PATH is overridden, not obeyed. The
// file lives in the throwaway consumer dir, removed with it.
MCP_CATALOG_PATH: join(cwd, "web-catalog.json"),
MCP_AUTO_OPEN_ENABLED: "false",
},
stdio: ["ignore", "inherit", "inherit"],
@ -464,7 +510,123 @@ async function verifyWeb(bin, cwd) {
if (!body.includes(token)) {
fail("`--web` served HTML is missing the injected auth-token value");
}
// Step 5, riding the same boot: drive the MCP Apps path in a real browser
// against this installed server. Throws rather than calling fail() so the
// `finally` below still stops the `--web` child — the outer catch routes
// the throw back through fail() for the usual cleanup and exit.
//
// `whenExits` lets a mid-run server death be reported as the real cause
// instead of two stacked 45s selector timeouts. Promise.race attaches a
// handler to it either way, so a rejection arriving later (during stop())
// is never an unhandled one.
const whenExits = new Promise((_, reject) => {
child.on("exit", (code) =>
reject(
new Error(
`\`--web\` exited (code ${code}) during the App render — see output above`,
),
),
);
});
await verifyAppRender(`http://${host}:${port}`, token, whenExits);
} finally {
await stop();
}
}
/**
* Drive **connect → open app → widget ready** against the installed `--web`
* server (#2003).
*
* This is what makes the sandbox proxy's *reachability* a checked property
* rather than an inferred one. Steps 2 and 3 assert the file is in the tarball
* and on disk at the right path; neither loads a widget through it, so a
* renamed file with a stale reader, or a future `"files"` / `.npmignore` edit
* that moves it relative to `clients/web/build`, ships present-but-unreachable
* and passes both — exactly the shape of #1859.
*
* The App test server is a repo fixture spawned here; only the *client* comes
* from the install. Everything downstream of the deep link is shared with
* `smoke:web:app` through `lib/mcp-app-flow.mjs`, so the two cannot drift.
*/
async function verifyAppRender(baseUrl, token, whenWebServerExits) {
step(
"verifying an MCP App renders end to end against the installed `--web` server...",
);
ensureTestServer();
let browser = null;
let mcpUrl = null;
try {
// `startMcpAppServer` publishes the child through `onSpawn` before it waits
// for readiness, so teardown reaches it on every throw path — the readiness
// timeout included (#2000).
mcpUrl = await startMcpAppServer({
repoRoot,
onSpawn: (child) => {
appServerChild = child;
},
label: LABEL,
});
browser = await loadChromium(repoRoot);
const page = await browser.newPage();
const diagnostics = attachPageDiagnostics(page);
try {
await Promise.race([
whenWebServerExits,
driveAppFlow({
page,
url: buildAppDeepLink({
baseUrl,
mcpUrl,
token,
appArgs: { title: LABEL },
}),
}),
]);
} catch (err) {
const notes = [
...diagnostics.pageErrors,
...diagnostics.fatalConsole().map((m) => `console: ${m}`),
];
throw new Error(
`${err instanceof Error ? err.message : String(err)}${
notes.length ? ` — page diagnostics: ${notes.join("; ")}` : ""
}`,
);
}
// Hard failures: any uncaught sync page error, plus the console errors that
// are the async half of the same class. Benign console noise stays a note.
const fatal = diagnostics.fatal();
if (fatal.length > 0) {
throw new Error(`app logged uncaught error(s): ${fatal.join("; ")}`);
}
const benign = diagnostics.benignConsole();
if (benign.length > 0) {
console.log(
`pack:verify note — ${benign.length} non-fatal console error(s): ${benign.join("; ")}`,
);
}
} finally {
if (browser) {
try {
await browser.close();
} catch {
// best-effort
}
}
if (appServerChild) {
const child = appServerChild;
appServerChild = null;
await stopChild(child, { label: LABEL, what: "MCP App test server" });
}
}
console.log(
`pack:verify — installed \`--web\` opened "${APP_TOOL}" from ${mcpUrl}; ` +
`widget reached data-app-status="ready" through the shipped sandbox proxy`,
);
}

286
scripts/smoke-web-app.mjs
View file

@ -8,12 +8,9 @@
* unexercised by any smoke. This closes that gap: it drives the full
* **connect → open app → widget ready** chain against a real MCP App server.
*
* The assertion is the `data-app-status="ready"` contract documented in
* clients/web/README.md ("MCP Apps screen automation contract"): the renderer
* only reports `ready` once the widget has loaded inside the sandbox iframe and
* fired `notifications/initialized` back through the bridge. So a single
* attribute covers the whole path — sandbox controller serving the proxy page,
* the proxy loading the UI resource, and the bridge completing its handshake.
* The flow itself lives in `lib/mcp-app-flow.mjs`, shared with `pack:verify`
* (#2003) — see that header for why it is shared rather than copied, and for
* the `data-app-status="ready"` contract the assertion rests on.
*
* ── What this does and does NOT catch ───────────────────────────────────────
*
@ -24,11 +21,11 @@
* smoke would have stayed green through that entire bug.
*
* The packaging dimension is owned by `npm run pack:verify`, which asserts the
* file both in the tarball packlist and on disk after a real install. Keep both:
* pack:verify proves the file *ships*, this proves the App path *works*. Neither
* subsumes the other, and the failure this one is positioned to catch is a
* regression in the sandbox/bridge code itself — which pack:verify, driving only
* `GET /`, would not notice.
* file in the tarball packlist and on disk after a real install — and, since
* #2003, drives this same App flow against the **installed** bin. Keep both:
* pack:verify covers the published artifact, this covers the repo tree on every
* `npm run ci`, where pack:verify (network-bound, local/release-only) does not
* run. Neither subsumes the other.
*
* As a cheap extra, this does assert the proxy page exists at the location the
* runtime resolves it from (`clients/web/build/../static/…`, see
@ -36,51 +33,31 @@
* renamed* without its reader being updated, a repo-tree failure pack:verify
* would only find later.
*
* Playwright is resolved with a `createRequire` based at clients/web/package.json
* rather than a bare `import("playwright")` — a bare ESM specifier resolves
* relative to scripts/, not the cwd, so `cd clients/web` in the npm script would
* NOT make it resolvable. Same gotcha as smoke:web:browser; see its header.
*
* Expects `clients/web/dist` and `clients/launcher/build` to be built first —
* the validate / CI ordering guarantees this. `test-servers/build` is built on
* demand if missing, as in smoke:cli.
*/
import { spawnSync } from "node:child_process";
import { existsSync } from "node:fs";
import { createRequire } from "node:module";
import { join, resolve } from "node:path";
import { startProdWebServer } from "./lib/prod-web-server.mjs";
import { stopChild } from "./lib/child-cleanup.mjs";
import { startAnnouncedChild } from "./lib/announced-child.mjs";
import { resolveNodeBin } from "./lib/resolve-node-bin.mjs";
import {
APP_TOOL,
attachPageDiagnostics,
buildAppDeepLink,
driveAppFlow,
loadChromium,
sandboxProxyPageFor,
startMcpAppServer,
} from "./lib/mcp-app-flow.mjs";
const repoRoot = resolve(import.meta.dirname, "..");
const requireFromWeb = createRequire(
resolve(repoRoot, "clients/web/package.json"),
);
const LABEL = "smoke:web:app";
const composableServer = join(
repoRoot,
"test-servers",
"build",
"server-composable.js",
);
const appConfig = join(
repoRoot,
"test-servers",
"configs",
"mcp-app-http.json",
);
// The path clients/web/server/sandbox-controller.ts resolves at runtime, from
// the built runner at clients/web/build/. Kept in sync with the `join(__dirname,
// "../static/sandbox_proxy.html")` there.
const sandboxProxyPage = join(
repoRoot,
"clients",
"web",
"static",
"sandbox_proxy.html",
// Resolved exactly as the runtime does, from the built runner's directory.
const sandboxProxyPage = sandboxProxyPageFor(
join(repoRoot, "clients", "web", "build"),
);
const HOST = "127.0.0.1";
@ -89,18 +66,8 @@ const HOST = "127.0.0.1";
// can't EADDRINUSE this one. The three run back-to-back in `npm run smoke`.
const PORT = process.env.SMOKE_WEB_APP_PORT ?? "6297";
const TOKEN = "smoke-web-app-token";
const APP_TOOL = "mcp_app_demo";
// Console messages that are the async half of the uncaught-crash class (an
// unhandled rejection or a failed dynamic import). Hard failures; every other
// console error is a diagnostic, so benign font-CDN / React-warning noise can't
// flake CI. Kept identical to smoke-web-browser.mjs, which documents the
// reasoning at length.
const FATAL_CONSOLE = /^Uncaught\b|Failed to fetch dynamically imported module/;
// The URL the test server announces on startup. NOT derived from the config's
// port: createTestServerHttp resolves its port with findAvailablePort(), which
// walks UPWARD from the configured value when it's taken — so the config port is
// a starting hint, not a guarantee, and assuming it makes this smoke fail
// whenever anything else holds that port. The announced line is authoritative.
// The URL the test server announces on startup — authoritative over the
// config's port; see startMcpAppServer.
let mcpUrl = null;
let mcpServer = null;
@ -109,7 +76,7 @@ const server = startProdWebServer({
host: HOST,
port: PORT,
token: TOKEN,
label: "smoke:web:app",
label: LABEL,
});
async function shutdown() {
@ -125,104 +92,20 @@ async function shutdown() {
if (mcpServer) {
const child = mcpServer;
mcpServer = null;
await stopChild(child, { label: "smoke:web:app", what: "MCP test server" });
await stopChild(child, { label: LABEL, what: "MCP test server" });
}
}
async function fail(message) {
console.error(`smoke:web:app FAILED — ${message}`);
console.error(`${LABEL} FAILED — ${message}`);
await shutdown();
process.exit(1);
}
/** Build the composable test server bundle if it isn't present yet. */
function ensureTestServer() {
if (existsSync(composableServer)) return;
console.log(
"smoke:web:app — building test-servers (missing build output)...",
);
// The root-installed tsc, run via this Node — `npx` is a `.cmd` shim on
// Windows that a shell-free spawnSync can't start (ENOENT — #1939).
const r = spawnSync(
process.execPath,
[
resolveNodeBin("typescript", "tsc", repoRoot),
"-p",
"test-servers",
"--noCheck",
],
{ cwd: repoRoot, stdio: "inherit" },
);
if (r.status !== 0 || !existsSync(composableServer)) {
throw new Error(
"could not build the test servers (test-servers/build/server-composable.js). " +
"Run `npm run test-servers:build` from clients/web.",
);
}
}
/**
* Spawn the MCP App test server and wait for it to announce its URL.
*
* The spawn/readiness ownership lives in `lib/announced-child.mjs` so the
* failure path is testable: this smoke's happy path always gets the
* announcement, so nothing here could prove that a child which stays alive
* through the readiness timeout is still reachable by `shutdown()` — the case
* that used to orphan a live server holding its port (#2000). `onSpawn`
* publishes the child to `mcpServer` before the wait, so every throw path is
* covered by teardown.
*
* The announced URL is authoritative rather than the config's port:
* createTestServerHttp resolves through findAvailablePort(), which walks upward
* when the configured value is taken.
*/
async function startMcpServer() {
const { match } = await startAnnouncedChild({
command: process.execPath,
args: [composableServer, "--config", appConfig],
cwd: repoRoot,
pattern: /listening at (http:\/\/\S+)/i,
onSpawn: (child) => {
mcpServer = child;
},
what: `MCP test server (${composableServer})`,
});
return { url: match[1] };
}
/** base64url(JSON) — the appArgs encoding the deep link expects. */
function encodeAppArgs(args) {
return Buffer.from(JSON.stringify(args))
.toString("base64")
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, "");
}
async function loadChromium() {
let chromium;
try {
({ chromium } = requireFromWeb("playwright"));
} catch (err) {
// Not resolvable means devDependencies are missing — fixed by `npm install`
// at the repo root, NOT by `playwright install` (which fetches binaries).
throw new Error(
`could not resolve the Playwright package from clients/web — run \`npm install\` at the repo root (${err instanceof Error ? err.message : String(err)})`,
);
}
try {
return await chromium.launch({ headless: true });
} catch (err) {
throw new Error(
`chromium failed to launch — on a bare Linux box run \`npx playwright install --with-deps chromium\` for the system libraries (${err instanceof Error ? err.message : String(err)})`,
);
}
}
try {
// Cheap structural check first: the sandbox proxy page must exist where the
// runtime looks for it. Fails fast with a clear cause instead of surfacing as
// an opaque "app never reached ready" 30s timeout below.
// an opaque "app never reached ready" 45s timeout below.
if (!existsSync(sandboxProxyPage)) {
await fail(
`sandbox proxy page missing at ${sandboxProxyPage} — clients/web/server/sandbox-controller.ts ` +
@ -231,120 +114,61 @@ try {
);
}
ensureTestServer();
// `startMcpServer` publishes the child to `mcpServer` itself, so teardown
// reaches it even when this throws before returning.
({ url: mcpUrl } = await startMcpServer());
await server.waitForReady();
browser = await loadChromium();
const page = await browser.newPage();
// Uncaught *synchronous* page errors. Their *async* twin — an unhandled
// rejection or a failed dynamic import — is not a `pageerror`; Chromium
// reports it on the console channel instead, so both are captured and both
// are hard failures. Same split as smoke:web:browser; see FATAL_CONSOLE there.
const pageErrors = [];
const consoleErrors = [];
page.on("pageerror", (err) =>
pageErrors.push(err instanceof Error ? err.message : String(err)),
);
page.on("console", (msg) => {
if (msg.type() === "error") consoleErrors.push(msg.text());
// `startMcpAppServer` publishes the child through `onSpawn` itself, so
// teardown reaches it even when this throws before returning (#2000).
mcpUrl = await startMcpAppServer({
repoRoot,
onSpawn: (child) => {
mcpServer = child;
},
label: LABEL,
});
const fatalConsole = () => consoleErrors.filter((m) => FATAL_CONSOLE.test(m));
await server.waitForReady();
browser = await loadChromium(repoRoot);
const page = await browser.newPage();
const diagnostics = attachPageDiagnostics(page);
// Deep link: connect, switch to the Apps tab, pre-select the app tool, and
// fire "Open App". autoConnect/autoOpen must equal the session token (CSRF
// gate). Shape owned by clients/web/README.md#deep-link-auto-connect.
const url =
`${server.baseUrl}/?serverUrl=${encodeURIComponent(mcpUrl)}` +
`&transport=http&autoConnect=${TOKEN}&openApp=${APP_TOOL}` +
`&appArgs=${encodeAppArgs({ title: "smoke:web:app" })}&autoOpen=${TOKEN}`;
const drive = async () => {
const response = await page.goto(url, {
waitUntil: "domcontentloaded",
timeout: 30_000,
});
if (!response || !response.ok()) {
throw new Error(
`GET / returned HTTP ${response ? response.status() : "no response"}`,
);
}
// 1. The deep link must be accepted (not rejected by the token gate).
const status = page.locator('[data-testid="connection-status"]');
await status.waitFor({ state: "attached", timeout: 30_000 });
const deeplink = await status.getAttribute("data-deeplink");
if (deeplink !== "parsed") {
throw new Error(
`deep link was not accepted (data-deeplink="${deeplink}") — expected "parsed"`,
);
}
// 2. Connected to the test server.
await page
.locator('[data-testid="connection-status"][data-status="connected"]')
.waitFor({ state: "attached", timeout: 45_000 });
// 3. The widget rendered inside the sandbox and completed its handshake.
// This is the load-bearing assertion — see the header comment.
try {
await page
.locator('[data-testid="apps-form"][data-app-status="ready"]')
.waitFor({ state: "attached", timeout: 45_000 });
} catch {
const form = page.locator('[data-testid="apps-form"]');
const appStatus = (await form.count())
? await form.getAttribute("data-app-status")
: "(no apps-form)";
const appError = (await form.count())
? await form.getAttribute("data-app-error")
: null;
throw new Error(
`app never reached data-app-status="ready" (last: "${appStatus}"` +
`${appError ? `, data-app-error="${appError}"` : ""}) — the sandbox proxy ` +
`or the UI-protocol bridge failed to complete`,
);
}
};
const url = buildAppDeepLink({
baseUrl: server.baseUrl,
mcpUrl,
token: TOKEN,
appArgs: { title: LABEL },
});
// Race against launcher death so a mid-run server crash is reported as the
// real cause instead of a downstream timeout.
try {
await Promise.race([server.whenChildExits(), drive()]);
await Promise.race([server.whenChildExits(), driveAppFlow({ page, url })]);
} catch (err) {
const diagnostics = [
...pageErrors,
...fatalConsole().map((m) => `console: ${m}`),
const notes = [
...diagnostics.pageErrors,
...diagnostics.fatalConsole().map((m) => `console: ${m}`),
];
await fail(
`${err instanceof Error ? err.message : String(err)}${
diagnostics.length
? ` — page diagnostics: ${diagnostics.join("; ")}`
: ""
notes.length ? ` — page diagnostics: ${notes.join("; ")}` : ""
}`,
);
}
// Hard failures: any uncaught sync page error, plus the console errors that
// are the async half of the same class.
const fatal = [...pageErrors, ...fatalConsole()];
const fatal = diagnostics.fatal();
if (fatal.length > 0) {
await fail(`app logged uncaught error(s): ${fatal.join("; ")}`);
}
// Non-fatal console errors: surface them so a real problem isn't invisible,
// without failing on benign subresource/warning noise.
const benignConsole = consoleErrors.filter((m) => !FATAL_CONSOLE.test(m));
if (benignConsole.length > 0) {
const benign = diagnostics.benignConsole();
if (benign.length > 0) {
console.log(
`smoke:web:app note — ${benignConsole.length} non-fatal console error(s): ${benignConsole.join("; ")}`,
`${LABEL} note — ${benign.length} non-fatal console error(s): ${benign.join("; ")}`,
);
}
console.log(
`smoke:web:app OK — connected to ${mcpUrl}, opened "${APP_TOOL}", ` +
`${LABEL} OK — connected to ${mcpUrl}, opened "${APP_TOOL}", ` +
`widget reached data-app-status="ready" through the sandbox proxy`,
);
await shutdown();