chore(web): connected-flow smoke for the Tools, Resources and Prompts tabs
The web client had exactly one connected-flow smoke and it was the Apps tab. `smoke:web:browser` proves the bundle boots and paints but never connects; `smoke:web:app` / `smoke:web:elicit` connect but only drive the Apps surface. So the three highest-traffic tabs were unexercised through the prod bundle end to end — everything covering them (unit tests, Storybook, the integration project) mocks or bypasses the prod server, so a regression living only in the built bundle or in the prod Hono backend's wiring of a tab passed all of them. `scripts/smoke-web-tabs.mjs` drives all three against one server on one browser launch (~5s added wall-clock): Tools runs `list_items` and asserts its `structuredContent` section rendered (#1908), Resources reads `events` with the template list asserted separately — `resources/list` and `resources/templates/list` are different calls and either can fail alone — and Prompts fetches the argument-less `simple_prompt`. Every assertion is on a `data-*` attribute, never on copy, mirroring the `data-app-status` precedent: each screen now carries a readiness contract (`data-tool-count` / `data-call-status` and its Resources / Prompts equivalents), tabulated in the web README and pinned by screen unit tests so a rename fails loudly there rather than as an opaque timeout here. The deep-link connect step is extracted to `lib/deep-link-connect.mjs` rather than copied a third time. That matters because `parseDeepLink` ignores a link it cannot validate rather than reporting one, so a drifted copy fails as an opaque connect timeout in whichever smoke was not updated. `smoke:web:elicit`'s hand-rolled copy is retired onto it. Chromium-only and deliberately not in `ENGINE_SMOKES`: the engine tiers exist for the MCP Apps sandbox's genuinely divergent primitives, and these tabs are ordinary React and Mantine. Network, Protocol, Logs and Tasks remain uncovered — a stated gap, not an oversight. Closes #2148 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UL9retfvAXRgvi6EWk4SY4 Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall
parent
833ad40a45
commit
f62410a9b3
18 changed files with 806 additions and 56 deletions
|
|
@ -972,6 +972,8 @@ The ⚠️ option-deletion hazard, the snapshot rule, and the recovery recipe ab
|
|||
- `smoke:web:app` (`scripts/smoke-web-app.mjs`, #1859) goes one step further again: `smoke:web:browser` stops at first paint and never connects to a server, so the Apps tab, the sandbox controller, and the UI-protocol bridge were unexercised by any smoke. This one boots the same prod `--web` server, spawns the `mcp-app-http.json` composable test server (the `mcp_app_demo` tool + its `mcp_app_demo_widget` UI resource), and drives the whole **connect → open app → widget ready** chain through a single deep-link navigate (`?serverUrl=…&autoConnect=<token>&openApp=…&appArgs=…&autoOpen=<token>`). The assertion is the `data-app-status="ready"` contract from [clients/web/README.md](clients/web/README.md) — the renderer reports `ready` only once the widget has loaded inside the sandbox iframe _and_ fired `notifications/initialized` back through the bridge, so one attribute covers the sandbox proxy being served, the UI resource loading, and the handshake completing. Two mechanics are load-bearing and easy to get wrong: the test server announces readiness on **stderr** (`console.error` in `server-composable.ts`), so both child streams are piped and scanned — watching stdout alone times out with an empty diagnostic; and its bound port is **not** the config's, because `createTestServerHttp` resolves through `findAvailablePort()`, which walks upward when the configured port is taken — so the smoke parses the announced URL rather than assuming `3130`. Both mechanics now live in `scripts/lib/announced-child.mjs` rather than in the smoke, so the failure path is testable: this smoke's happy path always receives the announcement, so nothing it could assert would prove that a child alive *through* the 30s timeout is still reachable by `shutdown()` — the case that orphaned a live server (#2000). The helper publishes the child via `onSpawn` before waiting, and `scripts/lib/announced-child.test.mjs` drives real `node -e` children (not spies) to assert it is published, still alive when the throw lands, and actually killable. Same reason `teardownWebServer` was extracted from `prod-web-server.mjs`'s `stop()`. **Scope note:** this runs against the repo build tree like every other smoke, so it would _not_ have caught #1859 itself (a packaging failure — the file is always present in-repo); `pack:verify` owns that dimension, and since #2003 it owns it *properly* — it drives this smoke's first phase against the installed tarball rather than only asserting the proxy page exists. The flow therefore lives in `scripts/lib/mcp-app-flow.mjs` (deep-link construction and the staged assertions; the browser launch and the page-diagnostics split live one level down, in `scripts/lib/headless-browser.mjs`) and is **shared, not copied**: two copies of the deep-link shape would drift, and a drifted link fails as a silent timeout rather than a mismatch. Keep both consumers — `pack:verify` is network-bound and local/release-only, so it does not run in `npm run ci`, where this smoke is the only thing exercising the App path at all, and the **`_meta.ui.domain` phase (#2056) is this smoke's alone**: the dedicated app origin is served by the same runner the packaging checks already cover, so driving it a second time from the tarball would cost a browser launch for no new packaging signal. It does carry a cheap structural pre-check that the proxy page exists at the path `sandbox-controller.ts` resolves, so a move/rename fails fast with a clear cause instead of an opaque render timeout.
|
||||
- `smoke:web:elicit` (`scripts/smoke-web-elicitation.mjs`, #1854) is the app-rendered **elicitation** counterpart of `smoke:web:app`: same prod `--web` server and the same deep-link connect, but it then calls `app_choose_option` from the Tools tab, waits for `[data-testid="app-elicitation"][data-app-elicitation-status="ready"]`, clicks a choice **inside the sandboxed app** (two `frameLocator` hops — the trusted sandbox proxy, then the untrusted app), and asserts the app's standard `ElicitResult` comes back in the *tool result*, i.e. that it reached the server rather than merely the host. It then repeats against `app-elicitation-native-http.json` — the same tool and app on a server that never advertised the nested MCP Apps `elicitation` capability — and asserts the **native** elicitation dialog takes it and no app modal is rendered. That second half is the more valuable one: the failure this feature can produce is not "the app doesn't render" but "an app renders when it should not have been offered one", which strands every user of a server that never opted in. Set `SMOKE_SCREENSHOT_DIR` to capture PNGs of the three states (used for PR proof); unset, it asserts only. Two mechanics worth knowing: the main-view tabs are a Mantine `SegmentedControl`, so there is no `role="tab"` — the clickable element is the sibling `label[for$="-Tools"]`; and the prompt string also appears in the (hidden) Protocol-tab payload, so the fallback assertion is scoped to the dialog rather than a bare text lookup.
|
||||
|
||||
- `smoke:web:tabs` (`scripts/smoke-web-tabs.mjs`, #2148) is the connected-flow smoke for the **core tabs** — the surfaces the App smokes never touch. Before it, the web client's only connected smoke was the Apps tab, so Tools, Resources, and Prompts — what nearly every session actually uses — were unexercised through the prod bundle: unit tests render a screen with fixture props, Storybook does the same in a browser, and the integration project drives `InspectorClient` with no browser at all, so a regression living only in the built bundle or in the prod Hono backend's wiring of a tab passed all of them. It boots the same prod `--web` server, connects once through the shared deep link against `web-tabs-http.json`, and drives three tabs on **one** browser launch (launch and boot dominate the cost; `npm run ci` is already several minutes): Tools runs `list_items` and asserts the `structuredContent` section rendered (#1908 — a panel showing only the `content[]` summary looks right while dropping the payload), Resources reads `events` with both list counts asserted separately (`resources/list` and `resources/templates/list` are different calls and either can fail alone), and Prompts selects the argument-less `simple_prompt`, which auto-fetches. Three things worth knowing: every assertion is on a **`data-*` attribute**, never on copy — the contract is tabulated in [clients/web/README.md](clients/web/README.md#core-tab-automation-contract) and each attribute name is pinned by a screen unit test, so a rename fails loudly there rather than as an opaque 45s timeout here; it is **Chromium-only and deliberately not in `ENGINE_SMOKES`**, because the engine tiers exist for the MCP Apps sandbox's genuinely divergent primitives and these tabs are ordinary React and Mantine, so a second Firefox pass would double the cost for coverage that is not engine-sensitive; and Network / Protocol / Logs / Tasks are **not** covered — each needs its own config and more waiting, and that remains a stated gap rather than an oversight.
|
||||
- **The deep-link connect step is shared, not copied** (`scripts/lib/deep-link-connect.mjs`, #2148). Every browser smoke that touches a server begins with the same `?serverUrl=…&autoConnect=<token>` link, the `data-deeplink` token gate, and the `data-status="connected"` wait; `driveAppFlow` had it inline and `smoke:web:elicit` carried a second hand-rolled copy. Sharing matters more than the line count here: `parseDeepLink` **ignores** a link it cannot validate rather than reporting one, so a drifted copy fails as an opaque connect timeout in whichever smoke was not updated, not as a mismatch. Its failure branches are unit-tested against a stand-in `page` (`deep-link-connect.test.mjs`), since every consumer only ever drives the happy path.
|
||||
- **The build gate for the browser-externalized-builtin class (#1769)** is the earlier, more complete companion to `smoke:web:browser`. A Vite plugin in `clients/web/vite.config.ts` (logic in `clients/web/server/browser-externalized-builtin-gate.ts`, unit-tested) turns Vite 8's _browser-externalization warning_ (`Module "node:*" has been externalized for browser compatibility`) into a hard `vite build` error, so a Node built-in in the browser graph now **fails `npm run build` / `validate`** instead of shipping a `{}` stub. This catches **both** the _called-at-init_ case (which `smoke:web:browser` also catches, but later/at runtime) **and** the _imported-but-never-called_ case (the `{}` stub that is invisible to the runtime smoke "by design" — see above). Because rolldown **swallows a throw inside `onLog`** (the one hook where a thrown error doesn't abort — verified against vite@8.0.0), the plugin _records_ the warning in `onLog` and re-throws in `buildEnd`. There is **no stable log `code`**, so the gate keys off the documented message phrasing; `npm run verify:build-gate` (`scripts/verify-build-gate.mjs`, in `npm run ci` and the GitHub workflow) runs a real build with a `node:fs` probe forced into `src/main.tsx` and asserts the build fails via the gate — the only check that catches the message phrasing **drifting** in a future Vite bump and silently disabling the gate. The gate is scoped to `vite build` (`apply: 'build'`) — never `vite dev` or the vitest projects — **and** to the browser (`client`) environment (`applyToEnvironment`), so a future SSR/node environment built from this config isn't failed for a legitimate `node:*` import; the Node runner build (tsup, `build:runner`) is a separate config where built-ins are legitimate. `smoke:web:browser` stays as the runtime backstop for crashes the build can't reason about.
|
||||
- `smoke:cli` (`scripts/smoke-cli.mjs`) drives `mcp-inspector --cli` through the built launcher against the bundled stdio test server via a temp `--catalog`: it asserts `tools/list` returns the server's tools (real connect over stdio), the default writable catalog is seeded empty on first run, a missing read-only `--config` errors without seeding, and `--catalog` + `--config` is rejected. `smoke:tui` (`scripts/smoke-tui.mjs`) launches `mcp-inspector --tui --catalog <temp>` **under a pseudoterminal**, asserts the Ink app renders its first frame (the "MCP Servers" panel) within a timeout, then waits `SMOKE_TUI_SURVIVE_MS` (default 2s) and asserts it is **still running** before SIGTERMing it — a shallow boot/render/survival check, not full interaction. Both rebuild `test-servers/build` on **every run** — once per process, whether or not it already exists (#2111): presence is not freshness, and a smoke driving a stale fixture reports a product failure rather than a staleness one.
|
||||
|
||||
|
|
|
|||
|
|
@ -486,7 +486,7 @@ Each client self-validates from its own folder; the root scripts chain them. The
|
|||
| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `npm run validate` | Runs the three durable guards first — `verify:format-coverage` (every tracked source file is format-gated), `verify:typecheck-coverage` (every one lands in a tsconfig project), `verify:dep-lockstep` (no dependency reaching one `tsc` program from two installs skews across them) — then `test:scripts` (the guards' own parser unit tests), then `validate:core` (the shared `core/` `format:check` + `lint` gate), then per client: `format:check` + `lint` + **`typecheck`** (cli/tui/launcher; web typechecks via `tsc -b` inside its `build`) + `build` + fast unit tests. The quick inner-loop check. |
|
||||
| `npm run coverage` | The **per-file ≥90% gate** (lines/statements/functions/branches) under v8 instrumentation, per client. CI-enforced. For web this also runs the integration project and covers the shared `core/` runtime (including `core/json` and `core/client`). |
|
||||
| `npm run smoke` | End-to-end smokes through the built launcher (`--help` dispatch + prod cli/tui/web — the TUI one runs under a **pseudoterminal** and asserts the Ink app is still running two seconds after its first frame, not merely that it painted one, [#2147](https://github.com/modelcontextprotocol/inspector/issues/2147)), plus three headless-browser smokes: a boot smoke that runs the prod web bundle and asserts a clean first render (no uncaught error — sync exception or unhandled rejection, how a Node built-in reaching the browser bundle manifests), and an **MCP Apps** smoke (`smoke:web:app`) that drives connect → open app → `data-app-status="ready"` against a composable App server, covering the sandbox proxy and UI-protocol bridge, and an **app-rendered elicitation** smoke (`smoke:web:elicit`) that drives one end to end — call the tool, answer inside the sandboxed app, see the app's `ElicitResult` reach the server — and then the same tool against a server that never advertised the capability, which must fall back to the native elicitation form. All three take their engine from `SMOKE_BROWSER` — see [Supported browsers](#supported-browsers). |
|
||||
| `npm run smoke` | End-to-end smokes through the built launcher (`--help` dispatch + prod cli/tui/web — the TUI one runs under a **pseudoterminal** and asserts the Ink app is still running two seconds after its first frame, not merely that it painted one, [#2147](https://github.com/modelcontextprotocol/inspector/issues/2147)), plus three headless-browser smokes: a boot smoke that runs the prod web bundle and asserts a clean first render (no uncaught error — sync exception or unhandled rejection, how a Node built-in reaching the browser bundle manifests), and an **MCP Apps** smoke (`smoke:web:app`) that drives connect → open app → `data-app-status="ready"` against a composable App server, covering the sandbox proxy and UI-protocol bridge, and an **app-rendered elicitation** smoke (`smoke:web:elicit`) that drives one end to end — call the tool, answer inside the sandboxed app, see the app's `ElicitResult` reach the server — and then the same tool against a server that never advertised the capability, which must fall back to the native elicitation form. Those three take their engine from `SMOKE_BROWSER` — see [Supported browsers](#supported-browsers). A fourth, **`smoke:web:tabs`**, drives the core tabs against one server on one browser launch — Tools runs a tool and asserts its `structuredContent` section, Resources reads a resource with templates listed, Prompts fetches one ([#2148](https://github.com/modelcontextprotocol/inspector/issues/2148)) — asserting the `data-*` contract in [the web README](./clients/web/README.md#core-tab-automation-contract) rather than visible copy. It is **Chromium-only**: those tabs are ordinary React and Mantine, so unlike the sandbox they are not engine-sensitive. |
|
||||
| `npm run verify:build-gate` | Runs a real `vite build` with a Node built-in forced into the browser graph and asserts the build **fails** via the #1769 gate (which turns Vite's browser-externalization warning into a hard error). Guards against the warning phrasing drifting in a Vite bump and silently disabling the gate. Part of `npm run ci`. |
|
||||
| `npm run verify:bundle-externals` | Guards the must-not-bundle invariant (#2067): for each tsup-bundled client it reads the **built** `build/` output and fails if any package that must stay external was inlined anyway. Candidates are the union of the client's own `external` array **and the root manifest's `dependencies`** — the latter because #2067 was a *missing* `external` entry, which a self-referential check would have passed. Detection is via esbuild's `// <path>` module banners, so it covers both shapes — a separate `<pkg>-HASH.js` chunk (what a dynamically `import()`ed CommonJS package produces) and a statically-imported package folded straight into `index.js`, which emits no chunk at all. A build with no banners fails as such rather than passing clean, so enabling `minify` cannot silently retire the check. `undici` was declared only in the root and `clients/cli` manifests, and tsup auto-externalizes only what the _nearest_ manifest declares, so the web and TUI bundles inlined 1.05MB of it — and CommonJS inlined into an ESM bundle throws `Dynamic require of "assert" is not supported` on first use, from a specifier no user-side install can satisfy. Reads the output rather than the config because those two disagreed for four releases. Part of `npm run ci`. |
|
||||
| `npm run verify:format-coverage` | Parses the `format:check` globs out of every `package.json` (only those reachable from `validate`), enumerates all tracked source files, and **fails** listing any not covered by a glob — the durable guard for the "every first-party source file is format-gated" invariant (#1792). Runs first in `validate`. |
|
||||
|
|
|
|||
|
|
@ -91,6 +91,34 @@ The top-level `src/types/` is a separate sibling — ambient `.d.ts` module stub
|
|||
|
||||
Nothing _enforces_ the boundary — no path alias keys off it, and the coverage `include` in `vite.config.ts` lists both directories, so a move between them is coverage-neutral. It's a human-legible import-time signal. See [`AGENTS.md`](../../AGENTS.md) for the full rule (including the whitelist caveat — a module placed outside `components`/`lib`/`utils`/`server` falls out of the ≥90 gate).
|
||||
|
||||
## Core tab automation contract
|
||||
|
||||
The Tools, Resources, and Prompts screens each expose a `data-testid` plus a
|
||||
small set of `data-*` attributes, so a headless driver can `waitForSelector` on
|
||||
a deterministic signal rather than on visible copy. `scripts/smoke-web-tabs.mjs`
|
||||
drives all three against `test-servers/configs/web-tabs-http.json` ([#2148](https://github.com/modelcontextprotocol/inspector/issues/2148)).
|
||||
Treat them as a public contract, for the same reason as the Apps ones below:
|
||||
|
||||
| Attribute | Where | Meaning |
|
||||
| ------------------------------------ | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `data-testid="tools-screen"` | Tools screen root | The element carrying the two attributes below. |
|
||||
| `data-tool-count` | on `tools-screen` | How many tools the list holds. `0` is a *populated screen with an empty list* — distinct from the screen being absent, which is what a driver waits out. |
|
||||
| `data-call-status` | on `tools-screen` | `idle` → `pending` → `ok` / `error` for the current `tools/call`. Always a value; an absent call state reads `idle`, never empty. |
|
||||
| `data-testid="structured-output"` | result panel | The `structuredContent` section of a tool result ([#1908](https://github.com/modelcontextprotocol/inspector/issues/1908)). Absent when a result carries none. |
|
||||
| `data-testid="resources-screen"` | Resources root | The element carrying the three attributes below. |
|
||||
| `data-resource-count` | on `resources-screen` | Entries from `resources/list`. |
|
||||
| `data-template-count` | on `resources-screen` | Entries from `resources/templates/list` — a **separate** call that can fail on its own, so it is reported separately. |
|
||||
| `data-read-status` | on `resources-screen` | `idle` → `pending` → `ok` / `error` for the current `resources/read`. |
|
||||
| `data-testid="prompts-screen"` | Prompts root | The element carrying the two attributes below. |
|
||||
| `data-prompt-count` | on `prompts-screen`| Entries from `prompts/list`. |
|
||||
| `data-get-status` | on `prompts-screen`| `idle` → `pending` → `ok` / `error` for the current `prompts/get`. |
|
||||
|
||||
Why attributes rather than text: a smoke that waited on a label fails the next
|
||||
time the label is reworded, which is noise rather than signal — and it fails as
|
||||
an opaque timeout, because there is nothing to compare against. The screen tests
|
||||
pin each attribute name for the same reason: a rename should fail loudly in a
|
||||
unit test, not silently in a five-minute gate.
|
||||
|
||||
## MCP Apps screen automation contract
|
||||
|
||||
The Apps screen exposes a small, stable set of `data-testid` / `data-*` attributes so an automated driver (deep-link auto-open, CI review harness) can `waitForSelector` on a deterministic signal instead of sleeping. Treat these as a public contract — drivers depend on them staying stable:
|
||||
|
|
|
|||
|
|
@ -85,3 +85,15 @@ describe("StructuredOutputPanel", () => {
|
|||
expect(screen.getByTestId("code-highlight")).toHaveTextContent("{}");
|
||||
});
|
||||
});
|
||||
|
||||
// The `data-testid` the headless tab smoke waits on (#2148). Asserting the
|
||||
// section by its heading copy would make the smoke fail on a rewording, so the
|
||||
// smoke keys off this attribute — pinned here so a rename fails loudly.
|
||||
describe("automation contract (#2148)", () => {
|
||||
it("exposes a stable data-testid on the section", () => {
|
||||
renderWithMantine(
|
||||
<StructuredOutputPanel structuredContent={structuredContent} />,
|
||||
);
|
||||
expect(screen.getByTestId("structured-output")).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -72,7 +72,10 @@ export function StructuredOutputPanel({
|
|||
const [expanded, setExpanded] = useState(defaultExpanded);
|
||||
|
||||
return (
|
||||
<StructuredBox>
|
||||
// `data-testid` so the headless tab smoke (#2148) can assert this section
|
||||
// rendered without matching on its heading copy. A `data-*` attribute is
|
||||
// not part of a component's typed props, so it is passed at the call site.
|
||||
<StructuredBox data-testid="structured-output">
|
||||
<StructuredInner>
|
||||
<HeaderRow>
|
||||
<StructuredHeader>Structured Output</StructuredHeader>
|
||||
|
|
|
|||
|
|
@ -391,3 +391,34 @@ describe("PromptsScreen", () => {
|
|||
});
|
||||
});
|
||||
});
|
||||
|
||||
// See the ToolsScreen counterpart: the `data-*` readiness contract the headless
|
||||
// tab smoke drives (#2148), pinned here so a rename fails loudly.
|
||||
describe("automation contract (#2148)", () => {
|
||||
it("reports the prompt count and an idle get status", () => {
|
||||
renderWithMantine(<PromptsScreen {...baseProps} />);
|
||||
const root = screen.getByTestId("prompts-screen");
|
||||
expect(root).toHaveAttribute(
|
||||
"data-prompt-count",
|
||||
String(promptsWithArgs.length),
|
||||
);
|
||||
expect(root).toHaveAttribute("data-get-status", "idle");
|
||||
});
|
||||
|
||||
it("tracks the get status through to a result", () => {
|
||||
renderWithMantine(
|
||||
<PromptsScreen
|
||||
{...baseProps}
|
||||
getPromptState={{
|
||||
status: "ok",
|
||||
promptName: promptsWithArgs[0].name,
|
||||
result: { messages: [] },
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
expect(screen.getByTestId("prompts-screen")).toHaveAttribute(
|
||||
"data-get-status",
|
||||
"ok",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -274,7 +274,13 @@ export function PromptsScreen({
|
|||
}
|
||||
|
||||
return (
|
||||
<ScreenLayout>
|
||||
// `data-*` readiness contract for the headless tab smoke (#2148); see
|
||||
// clients/web/README.md#core-tab-automation-contract.
|
||||
<ScreenLayout
|
||||
data-testid="prompts-screen"
|
||||
data-prompt-count={prompts.length}
|
||||
data-get-status={getPromptState?.status ?? "idle"}
|
||||
>
|
||||
<Sidebar>
|
||||
<SidebarCard>
|
||||
<PromptControls
|
||||
|
|
|
|||
|
|
@ -403,3 +403,36 @@ describe("ResourcesScreen", () => {
|
|||
).not.toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
// See the ToolsScreen counterpart: the `data-*` readiness contract the headless
|
||||
// tab smoke drives (#2148), pinned here so a rename fails loudly.
|
||||
describe("automation contract (#2148)", () => {
|
||||
it("reports the resource and template counts separately", () => {
|
||||
renderWithMantine(<ResourcesScreen {...baseProps} />);
|
||||
const root = screen.getByTestId("resources-screen");
|
||||
// Two attributes rather than one: `resources/list` and
|
||||
// `resources/templates/list` are separate calls and either can fail alone.
|
||||
expect(root).toHaveAttribute(
|
||||
"data-resource-count",
|
||||
String(resources.length),
|
||||
);
|
||||
expect(root).toHaveAttribute(
|
||||
"data-template-count",
|
||||
String(templates.length),
|
||||
);
|
||||
expect(root).toHaveAttribute("data-read-status", "idle");
|
||||
});
|
||||
|
||||
it("tracks the read status through to a result", () => {
|
||||
renderWithMantine(
|
||||
<ResourcesScreen
|
||||
{...baseProps}
|
||||
readState={{ status: "ok", uri: "file:///x", result: okResult }}
|
||||
/>,
|
||||
);
|
||||
expect(screen.getByTestId("resources-screen")).toHaveAttribute(
|
||||
"data-read-status",
|
||||
"ok",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -319,7 +319,14 @@ export function ResourcesScreen({
|
|||
}
|
||||
|
||||
return (
|
||||
<ScreenLayout>
|
||||
// `data-*` readiness contract for the headless tab smoke (#2148); see
|
||||
// clients/web/README.md#core-tab-automation-contract.
|
||||
<ScreenLayout
|
||||
data-testid="resources-screen"
|
||||
data-resource-count={resources.length}
|
||||
data-template-count={templates.length}
|
||||
data-read-status={readState?.status ?? "idle"}
|
||||
>
|
||||
<Sidebar>
|
||||
<SidebarCard>
|
||||
<ResourceControls
|
||||
|
|
|
|||
|
|
@ -445,3 +445,41 @@ describe("ToolsScreen", () => {
|
|||
expect(screen.getByText("invalid_header_tool")).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
// The `data-*` readiness contract the headless tab smoke drives (#2148). It is
|
||||
// a documented public contract (clients/web/README.md), and a smoke asserting
|
||||
// it fails as an opaque 45s timeout rather than a mismatch — so the attribute
|
||||
// names are pinned here, where a rename fails loudly instead.
|
||||
describe("automation contract (#2148)", () => {
|
||||
it("reports the tool count and an idle call status", () => {
|
||||
renderWithMantine(<ToolsScreen {...baseProps} />);
|
||||
const root = screen.getByTestId("tools-screen");
|
||||
expect(root).toHaveAttribute("data-tool-count", String(tools.length));
|
||||
// Absent call state reads `idle`, not empty: the smoke waits on a value.
|
||||
expect(root).toHaveAttribute("data-call-status", "idle");
|
||||
});
|
||||
|
||||
it("reports an empty list distinctly from a missing one", () => {
|
||||
renderWithMantine(<ToolsScreen {...baseProps} tools={[]} />);
|
||||
expect(screen.getByTestId("tools-screen")).toHaveAttribute(
|
||||
"data-tool-count",
|
||||
"0",
|
||||
);
|
||||
});
|
||||
|
||||
it("tracks the call status through to a result", () => {
|
||||
renderWithMantine(
|
||||
<ToolsScreen
|
||||
{...baseProps}
|
||||
callState={{
|
||||
status: "ok",
|
||||
result: { content: [{ type: "text", text: "done" }] },
|
||||
}}
|
||||
/>,
|
||||
);
|
||||
expect(screen.getByTestId("tools-screen")).toHaveAttribute(
|
||||
"data-call-status",
|
||||
"ok",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -198,7 +198,15 @@ export function ToolsScreen({
|
|||
};
|
||||
|
||||
return (
|
||||
<ScreenLayout>
|
||||
// `data-*` readiness contract for the headless tab smoke (#2148); see
|
||||
// clients/web/README.md#core-tab-automation-contract. Attributes are not
|
||||
// part of a component's typed props, so they are passed here rather than
|
||||
// baked into the `ScreenLayout` constant.
|
||||
<ScreenLayout
|
||||
data-testid="tools-screen"
|
||||
data-tool-count={tools.length}
|
||||
data-call-status={callState?.status ?? "idle"}
|
||||
>
|
||||
<Sidebar>
|
||||
<SidebarCard>
|
||||
<ToolControls
|
||||
|
|
|
|||
|
|
@ -66,13 +66,14 @@
|
|||
"coverage:tui": "cd clients/tui && npm run test:coverage",
|
||||
"coverage:web": "cd clients/web && npm run test:coverage",
|
||||
"coverage:launcher": "cd clients/launcher && npm run test:coverage",
|
||||
"smoke": "npm run smoke:launcher && npm run smoke:cli && npm run smoke:tui && npm run smoke:web && npm run smoke:web:chromium",
|
||||
"smoke": "npm run smoke:launcher && npm run smoke:cli && npm run smoke:tui && npm run smoke:web && npm run smoke:web:chromium && npm run smoke:web:tabs",
|
||||
"smoke:cli": "node scripts/smoke-cli.mjs",
|
||||
"smoke:tui": "node scripts/smoke-tui.mjs",
|
||||
"smoke:web": "node scripts/smoke-web.mjs",
|
||||
"smoke:web:browser": "node scripts/install-smoke-browser.mjs && node scripts/smoke-web-browser.mjs",
|
||||
"smoke:web:app": "node scripts/install-smoke-browser.mjs && node scripts/smoke-web-app.mjs",
|
||||
"smoke:web:elicit": "node scripts/install-smoke-browser.mjs && node scripts/smoke-web-elicitation.mjs",
|
||||
"smoke:web:tabs": "node scripts/install-smoke-browser.mjs && node scripts/smoke-web-tabs.mjs",
|
||||
"smoke:web:engine": "node scripts/run-engine-smokes.mjs",
|
||||
"smoke:web:chromium": "node scripts/run-engine-smokes.mjs chromium",
|
||||
"smoke:web:firefox": "node scripts/run-engine-smokes.mjs firefox",
|
||||
|
|
|
|||
87
scripts/lib/deep-link-connect.mjs
Normal file
87
scripts/lib/deep-link-connect.mjs
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
/**
|
||||
* The **connect** half of a headless web smoke: build the deep link, navigate
|
||||
* to it, and wait until the Inspector reports a live connection (#2148).
|
||||
*
|
||||
* Every browser-driven smoke that touches a server starts the same way — the
|
||||
* `?serverUrl=…&transport=http&autoConnect=<token>` link, the `data-deeplink`
|
||||
* token gate, and the `data-status="connected"` wait. `smoke:web:app` had it
|
||||
* inside `driveAppFlow`, and `smoke:web:elicit` carried a second, hand-rolled
|
||||
* copy; a third copy for the tab smoke is what this exists to prevent.
|
||||
*
|
||||
* Sharing it matters more than the line count. The deep link's shape is owned
|
||||
* by clients/web/README.md#deep-link-auto-connect and enforced by
|
||||
* `parseDeepLink`, which **ignores** a link it cannot validate rather than
|
||||
* reporting one — so a drifted copy does not fail as a mismatch, it fails as an
|
||||
* opaque 45s connect timeout in whichever smoke was not updated. One copy is
|
||||
* one thing to keep right.
|
||||
*
|
||||
* The two gates are separate on purpose and both are worth asserting:
|
||||
*
|
||||
* - `data-deeplink` distinguishes "the link was rejected by the CSRF token
|
||||
* gate" from "no deep link was present at all". Without it a bad token
|
||||
* reads as a connect that simply never happened.
|
||||
* - `data-status` is the connection itself.
|
||||
*
|
||||
* Takes only the slice of Playwright's `page` it uses (`goto`, `locator` →
|
||||
* `waitFor`/`getAttribute`), which is what lets its failure branches be
|
||||
* unit-tested against a stand-in — a smoke only ever drives its happy path.
|
||||
*/
|
||||
|
||||
/**
|
||||
* The deep link that connects to `mcpUrl`.
|
||||
*
|
||||
* `autoConnect` must equal the session's `MCP_INSPECTOR_API_TOKEN`: it is the
|
||||
* CSRF gate, and a link that fails it is silently ignored. Shape owned by
|
||||
* clients/web/README.md#deep-link-auto-connect.
|
||||
*/
|
||||
export function buildConnectDeepLink({
|
||||
baseUrl,
|
||||
mcpUrl,
|
||||
token,
|
||||
transport = "http",
|
||||
}) {
|
||||
return (
|
||||
`${baseUrl}/?serverUrl=${encodeURIComponent(mcpUrl)}` +
|
||||
`&transport=${transport}&autoConnect=${token}`
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Navigate to `url` and wait for a connected Inspector.
|
||||
*
|
||||
* `expectDeepLink: false` skips the token-gate assertion — useful when
|
||||
* re-navigating an already-proven page to a second server, where a rejected
|
||||
* link would fail at the connect wait anyway and the gate is not the subject.
|
||||
*/
|
||||
export async function connectViaDeepLink({
|
||||
page,
|
||||
url,
|
||||
expectDeepLink = true,
|
||||
gotoTimeoutMs = 30_000,
|
||||
connectTimeoutMs = 45_000,
|
||||
}) {
|
||||
const response = await page.goto(url, {
|
||||
waitUntil: "domcontentloaded",
|
||||
timeout: gotoTimeoutMs,
|
||||
});
|
||||
if (!response || !response.ok()) {
|
||||
throw new Error(
|
||||
`GET / returned HTTP ${response ? response.status() : "no response"}`,
|
||||
);
|
||||
}
|
||||
|
||||
const status = page.locator('[data-testid="connection-status"]');
|
||||
if (expectDeepLink) {
|
||||
await status.waitFor({ state: "attached", timeout: gotoTimeoutMs });
|
||||
const deeplink = await status.getAttribute("data-deeplink");
|
||||
if (deeplink !== "parsed") {
|
||||
throw new Error(
|
||||
`deep link was not accepted (data-deeplink="${deeplink}") — expected "parsed"`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
await page
|
||||
.locator('[data-testid="connection-status"][data-status="connected"]')
|
||||
.waitFor({ state: "attached", timeout: connectTimeoutMs });
|
||||
}
|
||||
148
scripts/lib/deep-link-connect.test.mjs
Normal file
148
scripts/lib/deep-link-connect.test.mjs
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
/**
|
||||
* Unit tests for the shared deep-link connect step (#2148).
|
||||
*
|
||||
* These cover what no smoke can check itself. Every consumer only ever drives
|
||||
* the happy path — the page connects — so both failure branches below are dead
|
||||
* code from their point of view, and the link's *shape* is asserted by nothing
|
||||
* once it is built rather than inlined. Those are exactly the parts that rot
|
||||
* silently: `parseDeepLink` **ignores** a link it cannot validate rather than
|
||||
* reporting one, so a dropped `autoConnect` token or a mangled `serverUrl`
|
||||
* produces a page that simply never connects, reported as an opaque 45s
|
||||
* timeout in whichever smoke was not updated.
|
||||
*
|
||||
* `connectViaDeepLink` is driven against a stand-in page rather than a real
|
||||
* browser — it takes only `goto` and `locator(...)` → `waitFor`/`getAttribute`,
|
||||
* which is what makes that possible.
|
||||
*/
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { describe, it } from "node:test";
|
||||
import {
|
||||
buildConnectDeepLink,
|
||||
connectViaDeepLink,
|
||||
} from "./deep-link-connect.mjs";
|
||||
|
||||
/** Minimal Playwright `page` stand-in. `plan` maps a selector to its behavior. */
|
||||
function fakePage(plan, { gotoStatus = 200 } = {}) {
|
||||
const gotos = [];
|
||||
return {
|
||||
gotos,
|
||||
goto: async (url) => {
|
||||
gotos.push(url);
|
||||
return { ok: () => gotoStatus === 200, status: () => gotoStatus };
|
||||
},
|
||||
locator: (selector) => {
|
||||
const entry = plan[selector] ?? {};
|
||||
return {
|
||||
waitFor: async () => {
|
||||
if (entry.waitFails) throw new Error(`timeout: ${selector}`);
|
||||
},
|
||||
getAttribute: async (name) => entry.attrs?.[name] ?? null,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const STATUS = '[data-testid="connection-status"]';
|
||||
const CONNECTED = '[data-testid="connection-status"][data-status="connected"]';
|
||||
|
||||
const happyPlan = () => ({
|
||||
[STATUS]: { attrs: { "data-deeplink": "parsed" } },
|
||||
[CONNECTED]: {},
|
||||
});
|
||||
|
||||
describe("buildConnectDeepLink", () => {
|
||||
const link = buildConnectDeepLink({
|
||||
baseUrl: "http://127.0.0.1:6399",
|
||||
mcpUrl: "http://127.0.0.1:3130/mcp?a=1",
|
||||
token: "tok",
|
||||
});
|
||||
const params = new URL(link).searchParams;
|
||||
|
||||
it("percent-encodes the server URL so its own query can't leak", () => {
|
||||
// Unencoded, the fixture's `?a=1` would terminate `serverUrl` and become a
|
||||
// sibling param — the link would still parse, just against a different
|
||||
// server than the caller asked for.
|
||||
assert.ok(
|
||||
link.includes(encodeURIComponent("http://127.0.0.1:3130/mcp?a=1")),
|
||||
);
|
||||
assert.equal(params.get("serverUrl"), "http://127.0.0.1:3130/mcp?a=1");
|
||||
});
|
||||
|
||||
it("carries the session token on the CSRF gate", () => {
|
||||
assert.equal(params.get("autoConnect"), "tok");
|
||||
});
|
||||
|
||||
it("defaults to streamable HTTP and honors an explicit transport", () => {
|
||||
assert.equal(params.get("transport"), "http");
|
||||
assert.equal(
|
||||
new URL(
|
||||
buildConnectDeepLink({
|
||||
baseUrl: "http://x",
|
||||
mcpUrl: "http://y",
|
||||
token: "t",
|
||||
transport: "sse",
|
||||
}),
|
||||
).searchParams.get("transport"),
|
||||
"sse",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("connectViaDeepLink", () => {
|
||||
it("resolves once the gate passes and the status reads connected", async () => {
|
||||
const page = fakePage(happyPlan());
|
||||
await connectViaDeepLink({ page, url: "http://x/" });
|
||||
assert.deepEqual(page.gotos, ["http://x/"]);
|
||||
});
|
||||
|
||||
it("reports a non-200 document instead of waiting on selectors", async () => {
|
||||
await assert.rejects(
|
||||
connectViaDeepLink({
|
||||
page: fakePage(happyPlan(), { gotoStatus: 500 }),
|
||||
url: "http://x/",
|
||||
}),
|
||||
/GET \/ returned HTTP 500/,
|
||||
);
|
||||
});
|
||||
|
||||
it("distinguishes a rejected deep link from an absent one", async () => {
|
||||
// The whole reason the gate is asserted separately: both `rejected` and
|
||||
// `none` leave the connection idle, and only this attribute says which.
|
||||
for (const value of ["rejected", "none"]) {
|
||||
await assert.rejects(
|
||||
connectViaDeepLink({
|
||||
page: fakePage({
|
||||
...happyPlan(),
|
||||
[STATUS]: { attrs: { "data-deeplink": value } },
|
||||
}),
|
||||
url: "http://x/",
|
||||
}),
|
||||
new RegExp(`data-deeplink="${value}"`),
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it("skips the gate assertion when the caller opts out", async () => {
|
||||
// A re-navigate of an already-proven page: the gate was established on the
|
||||
// first navigate, and a rejected one still fails at the connect wait.
|
||||
await connectViaDeepLink({
|
||||
page: fakePage({
|
||||
...happyPlan(),
|
||||
[STATUS]: { attrs: { "data-deeplink": "none" } },
|
||||
}),
|
||||
url: "http://x/",
|
||||
expectDeepLink: false,
|
||||
});
|
||||
});
|
||||
|
||||
it("propagates a connect timeout", async () => {
|
||||
await assert.rejects(
|
||||
connectViaDeepLink({
|
||||
page: fakePage({ ...happyPlan(), [CONNECTED]: { waitFails: true } }),
|
||||
url: "http://x/",
|
||||
}),
|
||||
/timeout/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
@ -30,6 +30,10 @@
|
|||
|
||||
import { join } from "node:path";
|
||||
import { startAnnouncedChild } from "./announced-child.mjs";
|
||||
import {
|
||||
buildConnectDeepLink,
|
||||
connectViaDeepLink,
|
||||
} from "./deep-link-connect.mjs";
|
||||
import {
|
||||
ensureTestServers,
|
||||
testServerEntryPath,
|
||||
|
|
@ -116,6 +120,9 @@ export function encodeAppArgs(args) {
|
|||
* The deep link that connects, switches to the Apps tab, pre-selects the app
|
||||
* tool, and fires "Open App". `autoConnect`/`autoOpen` must equal the session
|
||||
* token (CSRF gate). Shape owned by clients/web/README.md#deep-link-auto-connect.
|
||||
*
|
||||
* The connect half comes from `deep-link-connect.mjs`, shared with every other
|
||||
* browser smoke; only the three App params (#1859) are added here.
|
||||
*/
|
||||
export function buildAppDeepLink({
|
||||
baseUrl,
|
||||
|
|
@ -125,8 +132,8 @@ export function buildAppDeepLink({
|
|||
appArgs = {},
|
||||
}) {
|
||||
return (
|
||||
`${baseUrl}/?serverUrl=${encodeURIComponent(mcpUrl)}` +
|
||||
`&transport=http&autoConnect=${token}&openApp=${appTool}` +
|
||||
`${buildConnectDeepLink({ baseUrl, mcpUrl, token })}` +
|
||||
`&openApp=${appTool}` +
|
||||
`&appArgs=${encodeAppArgs(appArgs)}&autoOpen=${token}`
|
||||
);
|
||||
}
|
||||
|
|
@ -171,32 +178,14 @@ export async function driveAppFlow({
|
|||
connectTimeoutMs = 45_000,
|
||||
readyTimeoutMs = 45_000,
|
||||
}) {
|
||||
const response = await page.goto(url, {
|
||||
waitUntil: "domcontentloaded",
|
||||
timeout: gotoTimeoutMs,
|
||||
// 1-2. Navigated, the deep link accepted by the token gate, and connected.
|
||||
await connectViaDeepLink({
|
||||
page,
|
||||
url,
|
||||
expectDeepLink,
|
||||
gotoTimeoutMs,
|
||||
connectTimeoutMs,
|
||||
});
|
||||
if (!response || !response.ok()) {
|
||||
throw new Error(
|
||||
`GET / returned HTTP ${response ? response.status() : "no response"}`,
|
||||
);
|
||||
}
|
||||
|
||||
// 1. The deep link must be accepted (not rejected by the token gate).
|
||||
if (expectDeepLink) {
|
||||
const status = page.locator('[data-testid="connection-status"]');
|
||||
await status.waitFor({ state: "attached", timeout: gotoTimeoutMs });
|
||||
const deeplink = await status.getAttribute("data-deeplink");
|
||||
if (deeplink !== "parsed") {
|
||||
throw new Error(
|
||||
`deep link was not accepted (data-deeplink="${deeplink}") — expected "parsed"`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Connected to the test server.
|
||||
await page
|
||||
.locator('[data-testid="connection-status"][data-status="connected"]')
|
||||
.waitFor({ state: "attached", timeout: connectTimeoutMs });
|
||||
|
||||
// 3. The widget rendered inside the sandbox and completed its handshake.
|
||||
try {
|
||||
|
|
|
|||
|
|
@ -66,6 +66,10 @@ import {
|
|||
ensureTestServers,
|
||||
testServerEntryPath,
|
||||
} from "./lib/ensure-test-servers.mjs";
|
||||
import {
|
||||
buildConnectDeepLink,
|
||||
connectViaDeepLink,
|
||||
} from "./lib/deep-link-connect.mjs";
|
||||
|
||||
const repoRoot = resolve(import.meta.dirname, "..");
|
||||
|
||||
|
|
@ -176,31 +180,22 @@ async function shot(page, name) {
|
|||
console.log(`${LABEL} — captured ${name}.png`);
|
||||
}
|
||||
|
||||
/** Connect to `mcpUrl` through the deep link and wait for the Tools list. */
|
||||
/**
|
||||
* Connect to `mcpUrl` through the deep link.
|
||||
*
|
||||
* Shared with every other browser smoke (#2148) rather than hand-rolled here:
|
||||
* a link `parseDeepLink` cannot validate is *ignored* rather than reported, so
|
||||
* a drifted copy surfaces as an opaque connect timeout instead of a mismatch.
|
||||
*/
|
||||
async function connect(page, mcpUrl) {
|
||||
const url =
|
||||
`${web.baseUrl}/?serverUrl=${encodeURIComponent(mcpUrl)}` +
|
||||
`&transport=http&autoConnect=${TOKEN}`;
|
||||
const response = await page.goto(url, {
|
||||
waitUntil: "domcontentloaded",
|
||||
timeout: 30_000,
|
||||
await connectViaDeepLink({
|
||||
page,
|
||||
url: buildConnectDeepLink({
|
||||
baseUrl: web.baseUrl,
|
||||
mcpUrl,
|
||||
token: TOKEN,
|
||||
}),
|
||||
});
|
||||
if (!response || !response.ok()) {
|
||||
throw new Error(
|
||||
`GET / returned HTTP ${response ? response.status() : "no response"}`,
|
||||
);
|
||||
}
|
||||
const status = page.locator('[data-testid="connection-status"]');
|
||||
await status.waitFor({ state: "attached", timeout: 30_000 });
|
||||
const deeplink = await status.getAttribute("data-deeplink");
|
||||
if (deeplink !== "parsed") {
|
||||
throw new Error(
|
||||
`deep link was not accepted (data-deeplink="${deeplink}") — expected "parsed"`,
|
||||
);
|
||||
}
|
||||
await page
|
||||
.locator('[data-testid="connection-status"][data-status="connected"]')
|
||||
.waitFor({ state: "attached", timeout: 45_000 });
|
||||
}
|
||||
|
||||
/** Select the elicitation tool in the Tools tab and run it. */
|
||||
|
|
|
|||
348
scripts/smoke-web-tabs.mjs
Normal file
348
scripts/smoke-web-tabs.mjs
Normal file
|
|
@ -0,0 +1,348 @@
|
|||
#!/usr/bin/env node
|
||||
/**
|
||||
* Headless-browser connected-flow smoke for the core tabs — Tools, Resources,
|
||||
* Prompts (#2148).
|
||||
*
|
||||
* ── What was missing ────────────────────────────────────────────────────────
|
||||
*
|
||||
* The web client had exactly one connected-flow smoke, and it was the Apps tab.
|
||||
* `smoke:web:browser` proves the bundle boots and paints; it never connects.
|
||||
* `smoke:web:app` / `smoke:web:elicit` connect, but only ever drive the Apps
|
||||
* surface. So the three highest-traffic tabs — the ones nearly every session
|
||||
* actually uses — were unexercised through the prod bundle end to end.
|
||||
*
|
||||
* Everything that *did* cover them mocks or bypasses the prod server: unit
|
||||
* tests render a screen with fixture props, Storybook play functions do the
|
||||
* same in a browser, and the integration project drives `InspectorClient`
|
||||
* without a browser at all. A regression living only in the built bundle, or in
|
||||
* the prod Hono backend's wiring of a tab, passes all of them. That is the same
|
||||
* class `smoke:web:browser` was added for (#1615) and `smoke:web:app` extended
|
||||
* (#1859).
|
||||
*
|
||||
* ── What it drives ──────────────────────────────────────────────────────────
|
||||
*
|
||||
* One server, one browser, one page, three tabs:
|
||||
*
|
||||
* 1. **Tools** — the list renders, `list_items` is selected and executed, and
|
||||
* the result panel populates *including* its `structuredContent` section.
|
||||
* That last part is the half #1908 fixed: a tool declaring an
|
||||
* `outputSchema` returns its real payload there and the `content[]` blocks
|
||||
* only summarize it, so a result panel that renders text alone looks
|
||||
* correct while dropping the data.
|
||||
* 2. **Resources** — the list renders, a resource is read, and the preview
|
||||
* populates. Templates are asserted present (the RFC 6570 fixture, #1919),
|
||||
* which is what proves `resources/templates/list` reached the screen —
|
||||
* that list is fetched by a *different* call than `resources/list` and can
|
||||
* fail on its own.
|
||||
* 3. **Prompts** — the list renders and `simple_prompt` is fetched (selecting
|
||||
* an argument-less prompt auto-fetches), and the messages render.
|
||||
*
|
||||
* Deliberately **one script over one browser launch** rather than one per tab:
|
||||
* launching the browser and booting the server dominate the cost, and
|
||||
* `npm run ci` is already several minutes. Measured added wall-clock is ~5s
|
||||
* on a warm checkout (macOS, Chromium), the test-server build included.
|
||||
*
|
||||
* ── What it asserts on ──────────────────────────────────────────────────────
|
||||
*
|
||||
* `data-*` attributes, never copy. Each screen carries a readiness contract
|
||||
* mirroring the `data-app-status` precedent — `data-testid="tools-screen"` with
|
||||
* `data-tool-count` / `data-call-status`, and the Resources / Prompts
|
||||
* equivalents. See clients/web/README.md#core-tab-automation-contract. A smoke
|
||||
* that waited on visible text would fail the next time a label is reworded,
|
||||
* which is noise rather than signal.
|
||||
*
|
||||
* The connect half is `lib/deep-link-connect.mjs`, shared with every other
|
||||
* browser smoke — see that header for why a copied deep link fails as an opaque
|
||||
* timeout rather than a mismatch.
|
||||
*
|
||||
* ── Scope ───────────────────────────────────────────────────────────────────
|
||||
*
|
||||
* **Chromium only, and not in `ENGINE_SMOKES`.** The engine tiers exist for the
|
||||
* MCP Apps sandbox, which is built out of the primitives that genuinely diverge
|
||||
* between engines (`srcdoc` CSP inheritance, nested sandboxed iframes,
|
||||
* cross-frame `postMessage`). These three tabs are ordinary React and Mantine;
|
||||
* running them a second time under Firefox would double their cost for
|
||||
* coverage that is not engine-sensitive. Cross-engine parameterization is
|
||||
* #2031's third bullet and is tracked separately.
|
||||
*
|
||||
* The deeper tabs — Network, Protocol, Logs, Tasks — are **not** covered here.
|
||||
* Each needs its own server config and more waiting, and #2148 explicitly
|
||||
* allows splitting. They remain unsmoked; that is a stated gap, not an
|
||||
* oversight.
|
||||
*
|
||||
* Expects `clients/web/dist` and `clients/launcher/build` to be built first —
|
||||
* the validate / CI ordering guarantees this. `test-servers/build` is rebuilt on
|
||||
* every run; see `scripts/lib/ensure-test-servers.mjs` for why presence is not
|
||||
* freshness (#2111).
|
||||
*/
|
||||
|
||||
import { resolve } from "node:path";
|
||||
import { startProdWebServer } from "./lib/prod-web-server.mjs";
|
||||
import { stopChild } from "./lib/child-cleanup.mjs";
|
||||
import {
|
||||
attachPageDiagnostics,
|
||||
loadBrowser,
|
||||
resolveBrowserName,
|
||||
} from "./lib/headless-browser.mjs";
|
||||
import {
|
||||
buildConnectDeepLink,
|
||||
connectViaDeepLink,
|
||||
} from "./lib/deep-link-connect.mjs";
|
||||
import { startMcpAppServer } from "./lib/mcp-app-flow.mjs";
|
||||
|
||||
const repoRoot = resolve(import.meta.dirname, "..");
|
||||
|
||||
// Resolved before anything starts, so an unsupported SMOKE_BROWSER fails
|
||||
// immediately rather than after a web server and an MCP server are up. Every
|
||||
// message carries the engine so a failure names which run it came from — even
|
||||
// though this smoke is Chromium-only in every gate, it can still be pointed
|
||||
// elsewhere by hand.
|
||||
let BROWSER;
|
||||
try {
|
||||
BROWSER = resolveBrowserName();
|
||||
} catch (err) {
|
||||
console.error(
|
||||
`smoke:web:tabs FAILED — ${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
const LABEL = `smoke:web:tabs [${BROWSER}]`;
|
||||
|
||||
const HOST = "127.0.0.1";
|
||||
// Distinct from the other web smokes (6299 / 6298 / 6297 / 6296) so a prior
|
||||
// run whose port is still in TIME_WAIT can't EADDRINUSE this one.
|
||||
const PORT = process.env.SMOKE_WEB_TABS_PORT ?? "6295";
|
||||
const TOKEN = "smoke-web-tabs-token";
|
||||
|
||||
/** The fixture's tool with an `outputSchema`, so its result carries
|
||||
* `structuredContent` (#1908) as well as a `content[]` summary. */
|
||||
const TOOL = "list_items";
|
||||
/** The fixture's plain resource — `foobar://events`, listed as "events". */
|
||||
const RESOURCE = "events";
|
||||
/** The fixture's argument-less prompt; selecting it auto-fetches. */
|
||||
const PROMPT = "simple_prompt";
|
||||
|
||||
let mcpServer = null;
|
||||
let browser = null;
|
||||
const web = startProdWebServer({
|
||||
host: HOST,
|
||||
port: PORT,
|
||||
token: TOKEN,
|
||||
label: LABEL,
|
||||
});
|
||||
|
||||
async function shutdown() {
|
||||
if (browser) {
|
||||
try {
|
||||
await browser.close();
|
||||
} catch {
|
||||
// best-effort
|
||||
}
|
||||
browser = null;
|
||||
}
|
||||
await web.stop();
|
||||
if (mcpServer) {
|
||||
const child = mcpServer;
|
||||
mcpServer = null;
|
||||
await stopChild(child, { label: LABEL, what: "MCP test server" });
|
||||
}
|
||||
}
|
||||
|
||||
async function fail(message) {
|
||||
console.error(`${LABEL} FAILED — ${message}`);
|
||||
await shutdown();
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Switch to a main-view tab.
|
||||
*
|
||||
* The tabs are a Mantine `SegmentedControl`: a visually-hidden radio plus a
|
||||
* sibling `<label for>`. There is no `role="tab"` here and the radio has no hit
|
||||
* box, so the label is the clickable element.
|
||||
*/
|
||||
async function openTab(page, name) {
|
||||
await page
|
||||
.locator(`label[for$="-${name}"]`)
|
||||
.first()
|
||||
.click({ timeout: 30_000 });
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for `selector`, reporting the screen's own status attributes on failure
|
||||
* instead of a bare timeout — so a failure names the stage it stalled at.
|
||||
*/
|
||||
async function waitForStage(page, { selector, screen, attrs, what }) {
|
||||
try {
|
||||
await page
|
||||
.locator(selector)
|
||||
.waitFor({ state: "attached", timeout: 45_000 });
|
||||
} catch {
|
||||
const root = page.locator(screen);
|
||||
const present = (await root.count()) > 0;
|
||||
const observed = present
|
||||
? (
|
||||
await Promise.all(
|
||||
attrs.map(async (a) => `${a}="${await root.getAttribute(a)}"`),
|
||||
)
|
||||
).join(", ")
|
||||
: `(no ${screen})`;
|
||||
throw new Error(`${what} — last observed: ${observed}`);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
// Reuses the App smoke's spawner: it is the composable test server either
|
||||
// way, and it publishes the child through `onSpawn` before the readiness
|
||||
// wait, so teardown reaches it even when startup throws (#2000). The
|
||||
// announced URL is authoritative over the config's port — `findAvailablePort`
|
||||
// walks upward when the configured one is taken.
|
||||
const mcpUrl = await startMcpAppServer({
|
||||
repoRoot,
|
||||
config: "web-tabs-http",
|
||||
onSpawn: (child) => {
|
||||
mcpServer = child;
|
||||
},
|
||||
label: LABEL,
|
||||
});
|
||||
await web.waitForReady();
|
||||
browser = await loadBrowser(repoRoot, BROWSER);
|
||||
const page = await browser.newPage({
|
||||
viewport: { width: 1280, height: 900 },
|
||||
});
|
||||
|
||||
// Uncaught synchronous page errors plus their async twin (an unhandled
|
||||
// rejection or a failed dynamic import), which arrives on the console channel
|
||||
// instead. Both are hard failures; every other console error stays a
|
||||
// diagnostic so benign font/React noise can't flake CI.
|
||||
const diagnostics = attachPageDiagnostics(page);
|
||||
|
||||
const drive = async () => {
|
||||
await connectViaDeepLink({
|
||||
page,
|
||||
url: buildConnectDeepLink({ baseUrl: web.baseUrl, mcpUrl, token: TOKEN }),
|
||||
});
|
||||
|
||||
// ── Tools ──────────────────────────────────────────────────────────────
|
||||
await openTab(page, "Tools");
|
||||
await waitForStage(page, {
|
||||
// `:not([data-tool-count="0"])` rather than a numeric comparison, which
|
||||
// a CSS attribute selector cannot express — an empty list and a missing
|
||||
// list are the two failures worth telling apart, and both are excluded.
|
||||
selector: '[data-testid="tools-screen"]:not([data-tool-count="0"])',
|
||||
screen: '[data-testid="tools-screen"]',
|
||||
attrs: ["data-tool-count", "data-call-status"],
|
||||
what: "the Tools list never populated",
|
||||
});
|
||||
await page
|
||||
.getByRole("button", { name: TOOL, exact: true })
|
||||
.click({ timeout: 30_000 });
|
||||
await page
|
||||
.getByRole("button", { name: /execute tool/i })
|
||||
.click({ timeout: 30_000 });
|
||||
await waitForStage(page, {
|
||||
selector: '[data-testid="tools-screen"][data-call-status="ok"]',
|
||||
screen: '[data-testid="tools-screen"]',
|
||||
attrs: ["data-call-status"],
|
||||
what: `\`${TOOL}\` did not complete successfully`,
|
||||
});
|
||||
// The half #1908 fixed: the payload lives in `structuredContent`, and a
|
||||
// panel rendering only the `content[]` summary looks right while dropping
|
||||
// the data.
|
||||
await page
|
||||
.locator('[data-testid="structured-output"]')
|
||||
.waitFor({ state: "attached", timeout: 15_000 });
|
||||
|
||||
// ── Resources ──────────────────────────────────────────────────────────
|
||||
await openTab(page, "Resources");
|
||||
await waitForStage(page, {
|
||||
// Both counts, in one selector: `resources/list` and
|
||||
// `resources/templates/list` are separate calls and either can fail
|
||||
// alone, so asserting only the first would pass on a screen with no
|
||||
// templates at all.
|
||||
selector:
|
||||
'[data-testid="resources-screen"]:not([data-resource-count="0"])' +
|
||||
':not([data-template-count="0"])',
|
||||
screen: '[data-testid="resources-screen"]',
|
||||
attrs: ["data-resource-count", "data-template-count", "data-read-status"],
|
||||
what: "the Resources / templates lists never populated",
|
||||
});
|
||||
// The sidebar sections are an accordion whose open set is a persisted
|
||||
// preference, so it may legitimately start collapsed. Open it only when it
|
||||
// is closed rather than clicking blind, which would toggle it shut.
|
||||
const uris = page.getByRole("button", { name: /^URIs/ });
|
||||
if ((await uris.getAttribute("aria-expanded")) !== "true") {
|
||||
await uris.click({ timeout: 15_000 });
|
||||
}
|
||||
// Selecting a resource reads it — there is no separate Read button here.
|
||||
await page
|
||||
.getByRole("button", { name: RESOURCE, exact: true })
|
||||
.click({ timeout: 30_000 });
|
||||
await waitForStage(page, {
|
||||
selector: '[data-testid="resources-screen"][data-read-status="ok"]',
|
||||
screen: '[data-testid="resources-screen"]',
|
||||
attrs: ["data-read-status"],
|
||||
what: `reading \`${RESOURCE}\` did not succeed`,
|
||||
});
|
||||
|
||||
// ── Prompts ────────────────────────────────────────────────────────────
|
||||
await openTab(page, "Prompts");
|
||||
await waitForStage(page, {
|
||||
selector: '[data-testid="prompts-screen"]:not([data-prompt-count="0"])',
|
||||
screen: '[data-testid="prompts-screen"]',
|
||||
attrs: ["data-prompt-count", "data-get-status"],
|
||||
what: "the Prompts list never populated",
|
||||
});
|
||||
// An argument-less prompt is fetched the moment it is selected, so no
|
||||
// submit step follows.
|
||||
await page
|
||||
.getByRole("button", { name: new RegExp(`^${PROMPT}`) })
|
||||
.click({ timeout: 30_000 });
|
||||
await waitForStage(page, {
|
||||
selector: '[data-testid="prompts-screen"][data-get-status="ok"]',
|
||||
screen: '[data-testid="prompts-screen"]',
|
||||
attrs: ["data-get-status"],
|
||||
what: `getting \`${PROMPT}\` did not succeed`,
|
||||
});
|
||||
};
|
||||
|
||||
// Race against launcher death so a mid-run server crash is reported as the
|
||||
// real cause instead of a downstream timeout.
|
||||
try {
|
||||
await Promise.race([web.whenChildExits(), drive()]);
|
||||
} catch (err) {
|
||||
const notes = [
|
||||
...diagnostics.pageErrors,
|
||||
...diagnostics.fatalConsole().map((m) => `console: ${m}`),
|
||||
];
|
||||
await fail(
|
||||
`${err instanceof Error ? err.message : String(err)}${
|
||||
notes.length ? ` — page diagnostics: ${notes.join("; ")}` : ""
|
||||
}`,
|
||||
);
|
||||
}
|
||||
|
||||
// A drive that reached every assertion still fails if the page threw on the
|
||||
// way: without this the smoke prints OK over a broken bundle.
|
||||
const fatal = diagnostics.fatal();
|
||||
if (fatal.length > 0) {
|
||||
await fail(`page logged uncaught error(s): ${fatal.join("; ")}`);
|
||||
}
|
||||
|
||||
const benign = diagnostics.benignConsole();
|
||||
if (benign.length > 0) {
|
||||
console.log(
|
||||
`${LABEL} note — ${benign.length} non-fatal console error(s): ${benign.join("; ")}`,
|
||||
);
|
||||
}
|
||||
|
||||
console.log(
|
||||
`${LABEL} OK — connected to ${mcpUrl}; Tools ran "${TOOL}" with structured ` +
|
||||
`output, Resources read "${RESOURCE}" with templates listed, and Prompts ` +
|
||||
`fetched "${PROMPT}"`,
|
||||
);
|
||||
await shutdown();
|
||||
process.exit(0);
|
||||
} catch (err) {
|
||||
await fail(err instanceof Error ? err.message : String(err));
|
||||
}
|
||||
14
test-servers/configs/web-tabs-http.json
Normal file
14
test-servers/configs/web-tabs-http.json
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"serverInfo": {
|
||||
"name": "web-tabs-showcase",
|
||||
"version": "1.0.0"
|
||||
},
|
||||
"tools": [{ "preset": "list_items" }, { "preset": "echo" }],
|
||||
"resources": [{ "preset": "rfc6570_base" }],
|
||||
"resourceTemplates": [{ "preset": "rfc6570_templates" }],
|
||||
"prompts": [{ "preset": "simple_prompt" }, { "preset": "args_prompt" }],
|
||||
"transport": {
|
||||
"type": "streamable-http",
|
||||
"port": 6605
|
||||
}
|
||||
}
|
||||
Loading…
Reference in a new issue