modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

test: add smoke:web:app — drive a real MCP App end to end in headless Chromium

Browse source 
`smoke:web:browser` stops at first paint and never connects to a server, so
everything downstream of the connect — the Apps tab, the sandbox controller,
the UI-protocol bridge — was unexercised by any smoke. That is the code #1859
broke, and nothing would have caught a regression in it.

smoke:web:app boots the same prod --web server, spawns the mcp-app-http.json
composable server, and drives connect → open app → widget ready through one
deep-link navigate. The assertion is the documented data-app-status="ready"
contract, which the renderer only reports once the widget has loaded inside the
sandbox iframe AND completed its bridge handshake — so a single attribute
covers the proxy being served, the UI resource loading, and the handshake.

Two mechanics found by running it, both silent-failure shaped:
  - server-composable.ts announces readiness on stderr, not stdout, so watching
    stdout alone times out with an empty diagnostic. Both streams are scanned.
  - the bound port is not the configured one — createTestServerHttp resolves via
    findAvailablePort(), which walks upward when the port is taken. The smoke
    parses the announced URL instead of assuming 3130.

Scope: this runs against the repo build tree like every other smoke, so it would
NOT have caught #1859 itself (a packaging failure; the file is always present
in-repo). pack:verify owns that dimension — the two are complements, and the
header comment says so. A cheap structural pre-check does assert the proxy page
sits where sandbox-controller.ts resolves it, so a move/rename fails fast.

Both failure modes verified by hand: with clients/web/static removed, the
pre-check fires; with the pre-check bypassed, the data-app-status assertion
catches it independently.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAR2Nr9kXbrywFNUVoTe9F
This commit is contained in:
cliffhall 2026-08-05 19:00:29 -04:00
commit cf0ed4ac78
5 changed files with 315 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

8
.github/workflows/main.yml vendored
View file

@ -93,9 +93,11 @@ jobs:
# CLI/web). Not part of any client's `validate`: it needs the
# cli/tui/launcher bundles, which `validate` above already built
# (smoke:web builds clients/web/dist on demand — #1486). smoke:web:browser
# boots the prod web bundle in headless chromium (#1615). smoke:tui
# self-skips here — the Ink TUI needs a real TTY (raw mode) that headless
# CI lacks, so its boot/render check is local-only.
# boots the prod web bundle in headless chromium (#1615); smoke:web:app
# goes further and drives connect → open app → widget ready against a
# composable MCP App server (#1859). Both reuse the chromium installed
# above. smoke:tui self-skips here — the Ink TUI needs a real TTY (raw
# mode) that headless CI lacks, so its boot/render check is local-only.
run: npm run smoke
- name: Run Storybook play-function tests

3
AGENTS.md
View file

File diff suppressed because one or more lines are too long

2
README.md
View file

@ -265,7 +265,7 @@ Each client self-validates from its own folder; the root scripts chain them. The
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `npm run validate` | Runs `verify:format-coverage` (asserts every tracked source file is format-gated) first, then `validate:core` (the shared `core/` `format:check` + `lint` gate), then per client: `format:check` + `lint` + **`typecheck`** (cli/tui only) + `build` + fast unit tests. The quick inner-loop check. |
| `npm run coverage` | The **per-file ≥90% gate** (lines/statements/functions/branches) under v8 instrumentation, per client. CI-enforced. For web this also runs the integration project and covers the shared `core/` runtime (including `core/json` and `core/client`). |
| `npm run smoke` | End-to-end smokes through the built launcher (`--help` dispatch + prod cli/tui/web), plus a headless-Chromium boot smoke that runs the prod web bundle and asserts a clean first render (no uncaught error — sync exception or unhandled rejection, how a Node built-in reaching the browser bundle manifests). |
| `npm run smoke` | End-to-end smokes through the built launcher (`--help` dispatch + prod cli/tui/web), plus two headless-Chromium smokes: a boot smoke that runs the prod web bundle and asserts a clean first render (no uncaught error — sync exception or unhandled rejection, how a Node built-in reaching the browser bundle manifests), and an **MCP Apps** smoke (`smoke:web:app`) that drives connect → open app → `data-app-status="ready"` against a composable App server, covering the sandbox proxy and UI-protocol bridge. |
| `npm run verify:build-gate` | Runs a real `vite build` with a Node built-in forced into the browser graph and asserts the build **fails** via the #1769 gate (which turns Vite's browser-externalization warning into a hard error). Guards against the warning phrasing drifting in a Vite bump and silently disabling the gate. Part of `npm run ci`. |
| `npm run verify:format-coverage` | Parses the `format:check` globs out of every `package.json` (only those reachable from `validate`), enumerates all tracked source files, and **fails** listing any not covered by a glob — the durable guard for the "every first-party source file is format-gated" invariant (#1792). Runs first in `validate`. |
| `npm run test:scripts` | Table-driven unit tests (`node --test`) for the guard's own pure parsers (`scripts/lib/npm-scripts.mjs` + the exported helpers of `verify-typecheck-coverage.mjs`), one case per rule they encode. Runs in `validate` — and `verify:typecheck-coverage` guards *this* gate in turn (reachable from `validate`, non-empty test set, every test file matched by the `test:scripts` glob), since `node --test` silently skips a file its glob misses and still exits 0. |

3
package.json
View file

@ -64,11 +64,12 @@
"coverage:tui": "cd clients/tui && npm run test:coverage",
"coverage:web": "cd clients/web && npm run test:coverage",
"coverage:launcher": "cd clients/launcher && npm run test:coverage",
"smoke": "npm run smoke:launcher && npm run smoke:cli && npm run smoke:tui && npm run smoke:web && npm run smoke:web:browser",
"smoke": "npm run smoke:launcher && npm run smoke:cli && npm run smoke:tui && npm run smoke:web && npm run smoke:web:browser && npm run smoke:web:app",
"smoke:cli": "node scripts/smoke-cli.mjs",
"smoke:tui": "node scripts/smoke-tui.mjs",
"smoke:web": "node scripts/smoke-web.mjs",
"smoke:web:browser": "cd clients/web && npx playwright install chromium && node ../../scripts/smoke-web-browser.mjs",
"smoke:web:app": "cd clients/web && npx playwright install chromium && node ../../scripts/smoke-web-app.mjs",
"smoke:launcher": "node scripts/smoke-launcher.mjs",
"pack:verify": "node scripts/pack-and-verify.mjs",
"prepack": "npm run build",

305
scripts/smoke-web-app.mjs Normal file
View file

@ -0,0 +1,305 @@
#!/usr/bin/env node
/**
* Headless-browser MCP Apps smoke for the prod web client (#1859).
*
* `smoke:web:browser` proves the bundle boots and paints its first frame. It
* stops there — it never connects to a server, so everything downstream of the
* connect (the Apps tab, the sandbox controller, the UI-protocol bridge) is
* unexercised by any smoke. This closes that gap: it drives the full
* **connect → open app → widget ready** chain against a real MCP App server.
*
* The assertion is the `data-app-status="ready"` contract documented in
* clients/web/README.md ("MCP Apps screen automation contract"): the renderer
* only reports `ready` once the widget has loaded inside the sandbox iframe and
* fired `notifications/initialized` back through the bridge. So a single
* attribute covers the whole path — sandbox controller serving the proxy page,
* the proxy loading the UI resource, and the bridge completing its handshake.
*
* ── What this does and does NOT catch ───────────────────────────────────────
*
* This runs against the **repo build tree**, like every other `smoke:*`. That
* matters for the bug that motivated it: #1859 was a *packaging* failure —
* `clients/web/static/sandbox_proxy.html` was missing from the published
* tarball's "files" allowlist. In the repo that file is always present, so this
* smoke would have stayed green through that entire bug.
*
* The packaging dimension is owned by `npm run pack:verify`, which asserts the
* file both in the tarball packlist and on disk after a real install. Keep both:
* pack:verify proves the file *ships*, this proves the App path *works*. Neither
* subsumes the other, and the failure this one is positioned to catch is a
* regression in the sandbox/bridge code itself — which pack:verify, driving only
* `GET /`, would not notice.
*
* As a cheap extra, this does assert the proxy page exists at the location the
* runtime resolves it from (`clients/web/build/../static/…`, see
* server/sandbox-controller.ts) — which catches the file being *moved or
* renamed* without its reader being updated, a repo-tree failure pack:verify
* would only find later.
*
* Playwright is resolved with a `createRequire` based at clients/web/package.json
* rather than a bare `import("playwright")` — a bare ESM specifier resolves
* relative to scripts/, not the cwd, so `cd clients/web` in the npm script would
* NOT make it resolvable. Same gotcha as smoke:web:browser; see its header.
*
* Expects `clients/web/dist` and `clients/launcher/build` to be built first —
* the validate / CI ordering guarantees this. `test-servers/build` is built on
* demand if missing, as in smoke:cli.
*/
import { spawn, spawnSync } from "node:child_process";
import { existsSync } from "node:fs";
import { createRequire } from "node:module";
import { setTimeout as delay } from "node:timers/promises";
import { join, resolve } from "node:path";
import { startProdWebServer } from "./lib/prod-web-server.mjs";
import { stopChild } from "./lib/child-cleanup.mjs";
const repoRoot = resolve(import.meta.dirname, "..");
const requireFromWeb = createRequire(
resolve(repoRoot, "clients/web/package.json"),
);
const composableServer = join(
repoRoot,
"test-servers",
"build",
"server-composable.js",
);
const appConfig = join(
repoRoot,
"test-servers",
"configs",
"mcp-app-http.json",
);
// The path clients/web/server/sandbox-controller.ts resolves at runtime, from
// the built runner at clients/web/build/. Kept in sync with the `join(__dirname,
// "../static/sandbox_proxy.html")` there.
const sandboxProxyPage = join(
repoRoot,
"clients",
"web",
"static",
"sandbox_proxy.html",
);
const HOST = "127.0.0.1";
// Distinct from the other web smokes' ports so back-to-back runs can't collide.
const PORT = process.env.SMOKE_WEB_APP_PORT ?? "6299";
const TOKEN = "smoke-web-app-token";
const APP_TOOL = "mcp_app_demo";
// The URL the test server announces on startup. NOT derived from the config's
// port: createTestServerHttp resolves its port with findAvailablePort(), which
// walks UPWARD from the configured value when it's taken — so the config port is
// a starting hint, not a guarantee, and assuming it makes this smoke fail
// whenever anything else holds that port. The announced line is authoritative.
let mcpUrl = null;
let mcpServer = null;
let browser = null;
const server = startProdWebServer({ host: HOST, port: PORT, token: TOKEN });
async function shutdown() {
if (browser) {
try {
await browser.close();
} catch {
// best-effort
}
browser = null;
}
server.stop();
if (mcpServer) {
const child = mcpServer;
mcpServer = null;
await stopChild(child, { label: "smoke:web:app", what: "MCP test server" });
}
}
async function fail(message) {
console.error(`smoke:web:app FAILED — ${message}`);
await shutdown();
process.exit(1);
}
/** Build the composable test server bundle if it isn't present yet. */
function ensureTestServer() {
if (existsSync(composableServer)) return;
console.log(
"smoke:web:app — building test-servers (missing build output)...",
);
const r = spawnSync("npx", ["tsc", "-p", "test-servers", "--noCheck"], {
cwd: repoRoot,
stdio: "inherit",
});
if (r.status !== 0 || !existsSync(composableServer)) {
throw new Error(
"could not build the test servers (test-servers/build/server-composable.js). " +
"Run `npm run test-servers:build` from clients/web.",
);
}
}
/**
* Spawn the MCP App test server and wait for it to announce its URL.
*
* Both stdio channels are piped and scanned: server-composable.ts announces
* readiness with `console.error`, so watching stdout alone never matches and
* this times out with an empty diagnostic. Piping both also keeps the child's
* noise out of the smoke's own output while still making it available in the
* failure message.
*/
async function startMcpServer() {
const child = spawn(
process.execPath,
[composableServer, "--config", appConfig],
{ cwd: repoRoot, stdio: ["ignore", "pipe", "pipe"] },
);
let out = "";
child.stdout.on("data", (d) => (out += d));
child.stderr.on("data", (d) => (out += d));
let exited = false;
child.on("exit", () => (exited = true));
for (let attempt = 0; attempt < 120; attempt++) {
// Take the port the server actually bound, not the one we asked for.
const announced = out.match(/listening at (http:\/\/\S+)/i);
if (announced) return { child, url: announced[1] };
if (exited) throw new Error(`MCP test server exited early:\n${out}`);
await delay(250);
}
throw new Error(`MCP test server did not start within 30s:\n${out}`);
}
/** base64url(JSON) — the appArgs encoding the deep link expects. */
function encodeAppArgs(args) {
return Buffer.from(JSON.stringify(args))
.toString("base64")
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, "");
}
async function loadChromium() {
let chromium;
try {
({ chromium } = requireFromWeb("playwright"));
} catch (err) {
// Not resolvable means devDependencies are missing — fixed by `npm install`
// at the repo root, NOT by `playwright install` (which fetches binaries).
throw new Error(
`could not resolve the Playwright package from clients/web — run \`npm install\` at the repo root (${err instanceof Error ? err.message : String(err)})`,
);
}
try {
return await chromium.launch({ headless: true });
} catch (err) {
throw new Error(
`chromium failed to launch — on a bare Linux box run \`npx playwright install --with-deps chromium\` for the system libraries (${err instanceof Error ? err.message : String(err)})`,
);
}
}
try {
// Cheap structural check first: the sandbox proxy page must exist where the
// runtime looks for it. Fails fast with a clear cause instead of surfacing as
// an opaque "app never reached ready" 30s timeout below.
if (!existsSync(sandboxProxyPage)) {
await fail(
`sandbox proxy page missing at ${sandboxProxyPage} — clients/web/server/sandbox-controller.ts ` +
`reads it as \`join(__dirname, "../static/sandbox_proxy.html")\`; if it moved, update both ` +
`(and the "files" allowlist in the root package.json, see #1859)`,
);
}
ensureTestServer();
({ child: mcpServer, url: mcpUrl } = await startMcpServer());
await server.waitForReady();
browser = await loadChromium();
const page = await browser.newPage();
const pageErrors = [];
page.on("pageerror", (err) =>
pageErrors.push(err instanceof Error ? err.message : String(err)),
);
// Deep link: connect, switch to the Apps tab, pre-select the app tool, and
// fire "Open App". autoConnect/autoOpen must equal the session token (CSRF
// gate). Shape owned by clients/web/README.md#deep-link-auto-connect.
const url =
`${server.baseUrl}/?serverUrl=${encodeURIComponent(mcpUrl)}` +
`&transport=http&autoConnect=${TOKEN}&openApp=${APP_TOOL}` +
`&appArgs=${encodeAppArgs({ title: "smoke:web:app" })}&autoOpen=${TOKEN}`;
const drive = async () => {
const response = await page.goto(url, {
waitUntil: "domcontentloaded",
timeout: 30_000,
});
if (!response || !response.ok()) {
throw new Error(
`GET / returned HTTP ${response ? response.status() : "no response"}`,
);
}
// 1. The deep link must be accepted (not rejected by the token gate).
const status = page.locator('[data-testid="connection-status"]');
await status.waitFor({ state: "attached", timeout: 30_000 });
const deeplink = await status.getAttribute("data-deeplink");
if (deeplink !== "parsed") {
throw new Error(
`deep link was not accepted (data-deeplink="${deeplink}") — expected "parsed"`,
);
}
// 2. Connected to the test server.
await page
.locator('[data-testid="connection-status"][data-status="connected"]')
.waitFor({ state: "attached", timeout: 45_000 });
// 3. The widget rendered inside the sandbox and completed its handshake.
// This is the load-bearing assertion — see the header comment.
try {
await page
.locator('[data-testid="apps-form"][data-app-status="ready"]')
.waitFor({ state: "attached", timeout: 45_000 });
} catch {
const form = page.locator('[data-testid="apps-form"]');
const appStatus = (await form.count())
? await form.getAttribute("data-app-status")
: "(no apps-form)";
const appError = (await form.count())
? await form.getAttribute("data-app-error")
: null;
throw new Error(
`app never reached data-app-status="ready" (last: "${appStatus}"` +
`${appError ? `, data-app-error="${appError}"` : ""}) — the sandbox proxy ` +
`or the UI-protocol bridge failed to complete`,
);
}
};
// Race against launcher death so a mid-run server crash is reported as the
// real cause instead of a downstream timeout.
try {
await Promise.race([server.whenChildExits(), drive()]);
} catch (err) {
await fail(
`${err instanceof Error ? err.message : String(err)}${
pageErrors.length ? ` — page errors: ${pageErrors.join("; ")}` : ""
}`,
);
}
if (pageErrors.length > 0) {
await fail(`app logged uncaught error(s): ${pageErrors.join("; ")}`);
}
console.log(
`smoke:web:app OK — connected to ${mcpUrl}, opened "${APP_TOOL}", ` +
`widget reached data-app-status="ready" through the sandbox proxy`,
);
await shutdown();
process.exit(0);
} catch (err) {
await fail(err instanceof Error ? err.message : String(err));
}