Three of the six Vitest projects ran on Vitest's own testTimeout: 5000 and
five on its hookTimeout/teardownTimeout: 10000, and all 788 Testing Library
waits in the web unit project ran on asyncUtilTimeout: 1000. None of those
numbers was chosen here, and all of them were sized for an idle machine rather
than for three or four concurrent worktree sessions each running a full
local:gate on eight cores. #2292, #2278, #2250, #1942 and #1742 were each that
class, diagnosed one site at a time.
This is the complement of #1596, not a retreat from it: every fix that issue
produced stands, and no race is hidden here. What changes is that a correct,
deterministic test is no longer cut off by a budget nobody picked.
- vitest.shared.mts exports frozen TIMEOUTS / INTEGRATION_TIMEOUTS; all six
projects spread one of them, clients/launcher importing the shared module for
the first time to do so.
- Both web projects configure Testing Library's asyncUtilTimeout. The storybook
one needs its own setup file, deliberately outside .storybook/ and free of
setProjectAnnotations so the automatic preview-annotation provisioning
(#1898) is untouched.
- 28 per-suite restatements of the integration project's own budget are gone,
so raising the shared value now moves every site under it. The handful that
genuinely differ — cross-process lock contention, a full OAuth round trip —
keep their budget as a named constant that says why.
- AppRenderer's waitFor no longer asks for exactly its enclosing budget, which
it could never win (#2292's shape); cli's e2e child timer drops below the
project budget so the bound carrying the diagnostic is the one that fires.
- The transition auto-settle's slack term is named RAF_SLACK_MS and raised;
the term derived from HEADER_ANIM_MS is untouched. The two answer to
different things and must not be scaled as one number.
- The 19 Playwright budgets scattered across three web smokes become four
named constants in scripts/lib/browser-timeouts.mjs.
scripts/verify-test-timeouts.mjs resolves each project through Vitest itself
and asserts the budgets, the asyncUtilTimeout sites, and that retry stays
unset. It was observed red against the pre-change config: unit/tui/launcher at
testTimeout 5000, five projects at hookTimeout 10000, both asyncUtilTimeout
sites unmet.
Closes#2323
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsmUAZ6aLUAFqenjJKSSRp
Signed-off-by: cliffhall <cliff@futurescale.com>
The root `ci` script was never what runs in GitHub CI, and the name worked
against us three ways: it collides with the built-in `npm ci` (which does
something else entirely and is one keystroke away), it invites the
cross-engine passes into a workflow by looking like the thing a workflow
should call, and it blurs "GitHub CI" and "the local gate" in every doc
that names it.
- Rename `ci` → `local:gate` and `ci:storybook` → `local:storybook`, with
no back-compat alias: `npm run ci` now fails with npm's missing-script
error, which prints the available scripts.
- Add `scripts/lib/workflow-gate.mjs` + its `test:scripts` suite, which
runs over the repo's real `.github/workflows/**` and fails on an
invocation of a `local:*` script, of a non-Chromium engine pass (derived
from SUPPORTED_BROWSERS), or of `smoke:web:engine`, or on `SMOKE_BROWSER`
set to anything but a literal `chromium`. It does not forbid
`npm run smoke`, `smoke:web:chromium` or `smoke:tui` — those belong in CI
and are there today. Comments are not scanned, so the workflow can still
explain itself. This is the durable half; the rest is prose.
- Document the two-tier model once, in the root README ("Two tiers: GitHub
CI and the local gate"), and point the other docs at it rather than
restating the split.
- Sweep the ~40 stale references across README, AGENTS.md,
.github/copilot-instructions.md, the client READMEs, the roadmap doc and
the script headers. AGENTS.md and copilot-instructions.md change
together, per the mirror rule.
No gate changes what it actually runs.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
The web client had exactly one connected-flow smoke and it was the Apps
tab. `smoke:web:browser` proves the bundle boots and paints but never
connects; `smoke:web:app` / `smoke:web:elicit` connect but only drive the
Apps surface. So the three highest-traffic tabs were unexercised through
the prod bundle end to end — everything covering them (unit tests,
Storybook, the integration project) mocks or bypasses the prod server, so
a regression living only in the built bundle or in the prod Hono
backend's wiring of a tab passed all of them.
`scripts/smoke-web-tabs.mjs` drives all three against one server on one
browser launch (~5s added wall-clock): Tools runs `list_items` and
asserts its `structuredContent` section rendered (#1908), Resources reads
`events` with the template list asserted separately — `resources/list`
and `resources/templates/list` are different calls and either can fail
alone — and Prompts fetches the argument-less `simple_prompt`.
Every assertion is on a `data-*` attribute, never on copy, mirroring the
`data-app-status` precedent: each screen now carries a readiness contract
(`data-tool-count` / `data-call-status` and its Resources / Prompts
equivalents), tabulated in the web README and pinned by screen unit tests
so a rename fails loudly there rather than as an opaque timeout here.
The deep-link connect step is extracted to `lib/deep-link-connect.mjs`
rather than copied a third time. That matters because `parseDeepLink`
ignores a link it cannot validate rather than reporting one, so a drifted
copy fails as an opaque connect timeout in whichever smoke was not
updated. `smoke:web:elicit`'s hand-rolled copy is retired onto it.
Chromium-only and deliberately not in `ENGINE_SMOKES`: the engine tiers
exist for the MCP Apps sandbox's genuinely divergent primitives, and
these tabs are ordinary React and Mantine. Network, Protocol, Logs and
Tasks remain uncovered — a stated gap, not an oversight.
Closes#2148
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UL9retfvAXRgvi6EWk4SY4
Signed-off-by: cliffhall <cliff@futurescale.com>
smoke-web-elicitation.mjs told the reader to run
`SMOKE_BROWSER=webkit npm run smoke:web:app` -- the App smoke, not the
elicitation smoke the file documents. Following it exercises a different
path entirely, and it does not fail while doing so: a real smoke runs and
passes, so nothing signals that you never tested what you were reading
about.
This is a direct cost of my own decision two rounds ago to make the two
App headers word-for-word identical so they could not drift. The shared
PROSE should be identical; the example command is the one line that must
not be, and copy-paste does not distinguish them. Both headers now name
their own smoke, and both point at npm run smoke:web:webkit for all three.
Guarded, since the class is invisible at runtime by construction:
run-engine-smokes.test.mjs now asserts no smoke header tells the reader to
run a sibling's command, and that every `npm run` it does name is a real
script. Both mutation-verified -- restoring the exact bug fails the first,
and naming a nonexistent script fails the second.
Addresses the suppressed comment on Copilot review 5031065042 of #2133.
Signed-off-by: cliffhall <cliff@futurescale.com>
The three headers still said the non-Chromium engines were on demand and
that 'nothing runs it for you'. Both became false when Firefox moved into
npm run ci; I updated the README and AGENTS.md for that change and did not
carry it into the files themselves.
Each header now names the three tiers separately rather than collapsing
them into a CI/not-CI binary: GitHub CI is Chromium only, the local
pre-push gate is Chromium and Firefox, and WebKit is on demand.
That distinction is the whole point of the arrangement, so a header that
blurs it is worse than one that is merely out of date.
Addresses Copilot review 5026731741 on #2133.
Signed-off-by: cliffhall <cliff@futurescale.com>
The job was trialled and did not earn its place. It was genuinely cheap —
~2 minutes, running in parallel with the 15-minute build job, so zero
added wall-clock — but across a dozen runs Firefox never once disagreed
with Chromium. It was paying runner time and carrying a real flake
surface (playwright install --with-deps runs apt-get update, which fails
whenever a third-party repo in the runner image breaks; that is what the
one red run was) to re-confirm a result already in hand.
Everything that made cross-engine checking possible stays: SMOKE_BROWSER,
the shared headless-browser helper, the installer, and the tests. So
"SMOKE_BROWSER=firefox npm run smoke:web:engine" is there for the moment
it is worth running — before touching the sandbox, the CSP builder or the
proxy page — without taxing every push to re-derive the same answer.
The workflow is now byte-identical to v2/main again; the publish jobs go
back to needing only build.
Docs say what is true rather than what was intended, and AGENTS.md
records the reasoning so the job is not re-added on the general argument
that cross-engine coverage is good — that argument was already accepted,
and the on-demand path is what serves it. Re-add it on evidence: a real
cross-engine regression a gate would have caught.
Also swept the now-stale "matrix" language out of the smokes and the
resolver, including the empty-SMOKE_BROWSER rationale, which no longer
leans on a matrix key that does not exist.
Signed-off-by: cliffhall <cliff@futurescale.com>
#2132 was closed without the investigation going anywhere, so every
'pending #2132' pointer in the tree was about to send a reader to a
closed issue whose own diagnosis had already been retracted.
The docs now say what is actually known, and nothing more: the two App
smokes fail under Playwright's WebKit for unidentified reasons; it does
not reproduce in real Safari; an isolated repro of the suspected
mechanism did not reproduce it under Playwright's WebKit either; and
chasing it further was judged not worth the effort. A reader hitting it
locally is told to treat it as unexplained rather than as a defect.
The matrix decision is unchanged and never depended on the cause — the
engine fails, so it is not gated.
No behavior change.
Signed-off-by: cliffhall <cliff@futurescale.com>
An isolated repro of the mechanism I described — a server writing three
SSE frames then going silent with the connection held open, read via
fetch + getReader — delivers every frame in all three engines, WebKit
included. So 'the SSE stream's last message is not delivered' is not a
supported description of the failure.
The keep-alive experiment that appeared to confirm it was one
observation, and the failure is not even stable in shape: two consecutive
webkit runs failed at different stages (ready wait vs connect wait),
which a deterministic buffering bug would not do.
What is unchanged: the App smokes really do fail under Playwright's
WebKit, which is why the engine stays out of the matrix; that reason
never depended on the mechanism. The parseSSE cross-chunk defect also
stands, being read off the code rather than inferred from an experiment.
No behavior change.
Signed-off-by: cliffhall <cliff@futurescale.com>
#2132 was filed asserting that a Safari user opening an MCP App gets a
permanently 'loading' widget. That was never tested. A manual check in
real Safari renders the App normally, so the claim is unsupported and is
removed from all six places it reached.
The failure under Playwright's WebKit is real and still keeps that engine
out of the CI matrix. What is not established is that any shipping
browser behaves this way — and Playwright ships its own WebKit build
whose networking stack is not Safari's, which is exactly the layer the
failure lives in.
headless-browser.mjs already carried the caveat that a green WebKit run
is not a Safari guarantee. The same sentence forbids the inverse
inference, which is the one that was made; it now says so explicitly, and
AGENTS.md carries the rule: reproduce in the real browser before claiming
user impact in one.
No behavior change.
Signed-off-by: cliffhall <cliff@futurescale.com>
The three smoke headers and one AGENTS.md bullet still said CI runs all
three engines. That was written before pointing the smokes at WebKit
found #2132; the matrix was then narrowed to Firefox and these were not
brought along, so they claimed a gate that does not exist. Each now says
what CI actually enforces — Chromium in the build job's `npm run smoke`,
Firefox in the Sandbox smokes matrix — and where WebKit stands.
smoke-web-browser.mjs additionally notes that IT passes in WebKit; the
two App smokes are the ones that do not, and a reader of this file should
not infer otherwise from a bare '#2132' reference.
The fail-fast: false rationale no longer cites a WebKit-vs-Firefox
example it cannot currently produce, and says why the flag is still right
for a one-entry matrix.
Addresses Copilot review on #2133.
Signed-off-by: cliffhall <cliff@futurescale.com>
Every browser-driven check in this repo launched Chromium and nothing else.
That is fine for most of the web client, whose behavior is React and Mantine.
It is not fine for the MCP Apps sandbox, which is built out of the primitives
that genuinely diverge between engines: a CSP <meta> injected as the first
<head> child of a srcdoc document, a nested sandboxed iframe, a
Permissions-Policy allow attribute, and postMessage origin discipline across
those two frames. Nothing else covers that surface — sandbox-csp.test.ts
asserts which policy STRING is built (environment-independent by
construction), and no Storybook story reaches the sandbox at all, since all
three App stories use a data: placeholder iframe and a mock bridge.
The engine is now a parameter:
SMOKE_BROWSER=webkit npm run smoke:web:app
SMOKE_BROWSER=firefox npm run smoke:web:engine # all three smokes
Unset means chromium, so `npm run ci` and `npm run smoke` are unchanged. An
unrecognized value is an error rather than a fallback — falling back would
report a green Chromium run under a job labelled "webkit", claiming coverage
that never ran — and a launch failure names the engine that failed and its own
`playwright install` invocation.
Selection, launching, and the fatal-vs-benign page-diagnostics split now live
in one shared scripts/lib/headless-browser.mjs. The three smokes had each
hand-rolled the diagnostics split and two had their own copy of loadChromium;
pack:verify goes through the same helper, pinned to Chromium explicitly since
it is a packaging check.
The npm scripts drop their `cd clients/web && npx playwright install chromium`
prefix for scripts/install-smoke-browser.mjs. The engine is a variable now, and
a ${VAR:-default} expansion does not expand under Windows' cmd.exe; npx is also
a .cmd shim there that a shell-free spawn cannot start, so this resolves the
Playwright CLI through resolveNodeBin — the #1939 problem solved the #1939 way.
CI gains a `Sandbox smokes (<engine>)` matrix job rather than matrixing `build`,
which also runs validate, the coverage gate, two verify gates and Storybook —
all engine-independent. Both publish jobs now need it.
WebKit is supported by the tooling but deliberately NOT in the CI matrix yet:
pointing the smokes at it immediately found #2132, a pre-existing Safari
incompatibility in the web client's SSE transport that strands the last message
on /api/mcp/events, so an MCP App never leaves "loading". That is a product bug
affecting real Safari users, with its own diagnosis and a verified fix, and it
is not a matrix change. Adding `webkit` here is a one-word diff once it lands.
An engine is either green in CI or absent from it; a continue-on-error job would
report coverage nobody is held to.
Signed-off-by: cliffhall <cliff@futurescale.com>
Round 4. Two leftovers from moving the build into the shared helper, both
invisible here because `scripts/` is prettier-gated but not ESLint-gated,
so an unused binding fails nothing:
- smoke-web-elicitation.mjs imported `existsSync` solely for the local
early return that is gone.
- pack-and-verify.mjs kept `composableServer` and its `composableServerPath`
import for the same reason; the helper now asserts that entry via
`requires: ["composable"]`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H
Signed-off-by: cliffhall <cliff@futurescale.com>
Five `scripts/*.mjs` consumers each carried their own `ensureTestServer()`
that returned early when `test-servers/build` was already on disk. Presence
is not freshness: once that directory exists — which it does on any machine
that has run the smokes once — an edit to `test-servers/src` was never
picked up again, and the smoke went on driving the previously-built fixture.
The failure is silent and misattributed. It does not read as "your fixture
is stale"; it reads as a product bug in whatever the smoke is checking.
`scripts/lib/ensure-test-servers.mjs` is now the one place that build is
produced for a script consumer, and it builds unconditionally — memoized
once per process per repo root, so pack-and-verify's two asks stay one tsc
pass while a fresh run always rebuilds. Callers name the entries they
depend on and get one actionable failure message, replacing five that had
drifted. pack-and-verify also stops shelling out to `npx tsc`, which is
what made it the one script unrunnable on Windows (#1939).
`test-servers/tsconfig.json` gains `incremental` so the up-to-date case is
cheap: 3.0s cold, 1.6s incremental here.
Unconditional emit is still not a clean — `--noCheck` leaves the stale .js
of a deleted src file behind — which is unchanged from before and recorded
in the helper's header and AGENTS.md.
Closes#2111
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H
Signed-off-by: cliffhall <cliff@futurescale.com>
Eight findings from the Copilot review, all accepted.
Ownership across connections was the substantive one, in three parts. A
mid-session transport close reaches teardown only through
`clearAndAnnouncePendingPeerRequests`, whose emptiness check looked at the
native queues alone — so a lone app elicitation survived a dropped connection
with its modal still open; the check now counts the active set too. The web
bridge factory resolves its client at call time, so an entry queued by a
replaced `InspectorClient` could rebuild against the *next* one and answer
through a different server; the host now drops every entry synchronously on
the swap (`AppElicitationController.failAll`) rather than racing core's
awaited teardown. And the request id was only monotonic within one client, so
each replacement client's first request was `app-elicitation-1` — settling
that id could resolve the wrong server's request. It now carries a
per-instance prefix.
Result validation now parses the whole value with `ElicitResultSchema` before
the requested-schema check, instead of inspecting `action` by hand: a
`{ action: "decline", content: { x: {} } }` used to pass, though the standard
result permits only primitives and string arrays in `content`.
The dialog's accessible name now includes the request id and the prompt — two
concurrent elicitations can be for the same app URI, and the previous name
made those indistinguishable, which is what the comment beside it claimed to
prevent.
`observeAppCapabilities` takes a minimal structural `onmessage` contract
(generic over the message type, since a handler typed for a narrower message
is not assignable to one typed for `unknown`), removing the double cast at the
call site.
`smoke:web:elicit` now fails on an uncaught page error after a successful
drive, and captures the async half of that class off the console channel, as
its two sibling smokes do — it could otherwise print OK over a broken bundle.
Adds the tests each fix needs, including the two the review asked for
directly: that `advertiseElicitation` reaches the `AppBridge` constructor, and
that it is absent by default.
Signed-off-by: cliffhall <cliff@futurescale.com>
Let a server hand a form `elicitation/create` to an MCP App and return the
app's ordinary `ElicitResult`, with the native elicitation form as the
fallback on every failure.
No second extension, no custom method, and no custom result shape: the only
new wire surface is a nested `elicitation` flag on the existing
`io.modelcontextprotocol/ui` extension on each side, plus
`_meta.ui.resourceUri` on the request. App rendering is selected only when
all four gates hold — client `elicitation.form`, client MCP Apps MIME type,
the nested `elicitation` setting on BOTH peers, and a valid absolute `ui://`
URI on the request.
Only the web client advertises the nested client-side setting, and only
because it has a sandbox renderer: supplying
`InspectorClientOptions.appElicitation` is what opts a client in, so CLI and
TUI keep advertising the MIME type without ever claiming they can resolve an
elicitation through an app.
Routing lives in the one funnel both entry points already use
(`enqueuePendingElicitation`), so the inbound handler and the MRTR driver
cannot diverge. Ownership is request-scoped — a per-request id keys the
renderer, iframe and bridge — so two concurrent elicitations can never
resolve through each other's bridges. An explicit `decline`/`cancel` is a
completed elicitation and goes back to the server; everything else (absent or
malformed metadata, resource/sandbox/bridge failure, an app with no
elicitation capability, a timeout, an invalid result or one that fails the
requested schema) falls back.
The Inspector speaks the ext-apps#733 / SEP-3118 wire protocol but cannot yet
consume its helpers — the released `@modelcontextprotocol/ext-apps` (1.7.5)
predates that PR. `core/mcp/appElicitation.ts` and
`AppRenderer/requestAppElicitation.ts` mirror it exactly and are marked for
deletion once a release containing it ships. One consequence needed its own
seam: ext-apps 1.7.5 parses the view's `ui/initialize` through a schema that
strips `elicitation`, so an app that correctly advertises it would look like
one that did not, silently turning every negotiated elicitation into a
fallback. `AppRenderer/appCapabilities.ts` records the raw frame instead, and
prefers the bridge's own value once it carries the key.
`AppRenderer` now takes an `AppRenderSource` union rather than a `Tool`, so
the same renderer and bridge factory serve an App tool and an elicitation
without either faking the other's shape.
Adds the public fixture (`app_choose_option` + a self-contained
`ui://demo/choose-option.html` app covering accept/decline/cancel), its two
showcase configs, and `smoke:web:elicit`, which drives the negotiated path
end to end in headless Chromium and then the same tool against a server that
never advertised the capability, asserting the native form takes it.
Closes#1854
Signed-off-by: cliffhall <cliff@futurescale.com>
