The job was trialled and did not earn its place. It was genuinely cheap —
~2 minutes, running in parallel with the 15-minute build job, so zero
added wall-clock — but across a dozen runs Firefox never once disagreed
with Chromium. It was paying runner time and carrying a real flake
surface (playwright install --with-deps runs apt-get update, which fails
whenever a third-party repo in the runner image breaks; that is what the
one red run was) to re-confirm a result already in hand.
Everything that made cross-engine checking possible stays: SMOKE_BROWSER,
the shared headless-browser helper, the installer, and the tests. So
"SMOKE_BROWSER=firefox npm run smoke:web:engine" is there for the moment
it is worth running — before touching the sandbox, the CSP builder or the
proxy page — without taxing every push to re-derive the same answer.
The workflow is now byte-identical to v2/main again; the publish jobs go
back to needing only build.
Docs say what is true rather than what was intended, and AGENTS.md
records the reasoning so the job is not re-added on the general argument
that cross-engine coverage is good — that argument was already accepted,
and the on-demand path is what serves it. Re-add it on evidence: a real
cross-engine regression a gate would have caught.
Also swept the now-stale "matrix" language out of the smokes and the
resolver, including the empty-SMOKE_BROWSER rationale, which no longer
leans on a matrix key that does not exist.
Signed-off-by: cliffhall <cliff@futurescale.com>
Every browser-driven check in this repo launched Chromium and nothing else.
That is fine for most of the web client, whose behavior is React and Mantine.
It is not fine for the MCP Apps sandbox, which is built out of the primitives
that genuinely diverge between engines: a CSP <meta> injected as the first
<head> child of a srcdoc document, a nested sandboxed iframe, a
Permissions-Policy allow attribute, and postMessage origin discipline across
those two frames. Nothing else covers that surface — sandbox-csp.test.ts
asserts which policy STRING is built (environment-independent by
construction), and no Storybook story reaches the sandbox at all, since all
three App stories use a data: placeholder iframe and a mock bridge.
The engine is now a parameter:
SMOKE_BROWSER=webkit npm run smoke:web:app
SMOKE_BROWSER=firefox npm run smoke:web:engine # all three smokes
Unset means chromium, so `npm run ci` and `npm run smoke` are unchanged. An
unrecognized value is an error rather than a fallback — falling back would
report a green Chromium run under a job labelled "webkit", claiming coverage
that never ran — and a launch failure names the engine that failed and its own
`playwright install` invocation.
Selection, launching, and the fatal-vs-benign page-diagnostics split now live
in one shared scripts/lib/headless-browser.mjs. The three smokes had each
hand-rolled the diagnostics split and two had their own copy of loadChromium;
pack:verify goes through the same helper, pinned to Chromium explicitly since
it is a packaging check.
The npm scripts drop their `cd clients/web && npx playwright install chromium`
prefix for scripts/install-smoke-browser.mjs. The engine is a variable now, and
a ${VAR:-default} expansion does not expand under Windows' cmd.exe; npx is also
a .cmd shim there that a shell-free spawn cannot start, so this resolves the
Playwright CLI through resolveNodeBin — the #1939 problem solved the #1939 way.
CI gains a `Sandbox smokes (<engine>)` matrix job rather than matrixing `build`,
which also runs validate, the coverage gate, two verify gates and Storybook —
all engine-independent. Both publish jobs now need it.
WebKit is supported by the tooling but deliberately NOT in the CI matrix yet:
pointing the smokes at it immediately found #2132, a pre-existing Safari
incompatibility in the web client's SSE transport that strands the last message
on /api/mcp/events, so an MCP App never leaves "loading". That is a product bug
affecting real Safari users, with its own diagnosis and a verified fix, and it
is not a matrix change. Adding `webkit` here is a one-word diff once it lands.
An engine is either green in CI or absent from it; a continue-on-error job would
report coverage nobody is held to.
Signed-off-by: cliffhall <cliff@futurescale.com>
Round 4. Two leftovers from moving the build into the shared helper, both
invisible here because `scripts/` is prettier-gated but not ESLint-gated,
so an unused binding fails nothing:
- smoke-web-elicitation.mjs imported `existsSync` solely for the local
early return that is gone.
- pack-and-verify.mjs kept `composableServer` and its `composableServerPath`
import for the same reason; the helper now asserts that entry via
`requires: ["composable"]`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H
Signed-off-by: cliffhall <cliff@futurescale.com>
Five `scripts/*.mjs` consumers each carried their own `ensureTestServer()`
that returned early when `test-servers/build` was already on disk. Presence
is not freshness: once that directory exists — which it does on any machine
that has run the smokes once — an edit to `test-servers/src` was never
picked up again, and the smoke went on driving the previously-built fixture.
The failure is silent and misattributed. It does not read as "your fixture
is stale"; it reads as a product bug in whatever the smoke is checking.
`scripts/lib/ensure-test-servers.mjs` is now the one place that build is
produced for a script consumer, and it builds unconditionally — memoized
once per process per repo root, so pack-and-verify's two asks stay one tsc
pass while a fresh run always rebuilds. Callers name the entries they
depend on and get one actionable failure message, replacing five that had
drifted. pack-and-verify also stops shelling out to `npx tsc`, which is
what made it the one script unrunnable on Windows (#1939).
`test-servers/tsconfig.json` gains `incremental` so the up-to-date case is
cheap: 3.0s cold, 1.6s incremental here.
Unconditional emit is still not a clean — `--noCheck` leaves the stale .js
of a deleted src file behind — which is unchanged from before and recorded
in the helper's header and AGENTS.md.
Closes#2111
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H
Signed-off-by: cliffhall <cliff@futurescale.com>
Nothing covered the intersection #1859 fell through. `smoke:web:app` drives
the App path but against the repo build tree; `pack:verify` runs against the
installed tarball but only asserted the sandbox proxy page *exists*. So a
packaging regression that ships the file present-but-unreachable — a rename
with a stale reader, a wrong position relative to clients/web/build, a future
"files"/.npmignore edit — passes both.
Extract the drive-the-App-flow half of the smoke into
scripts/lib/mcp-app-flow.mjs and call it from both, rather than copying it:
the deep-link shape is the part that rots silently, and a drifted link fails
as an opaque timeout rather than a mismatch. pack:verify then drives
connect -> open app -> data-app-status="ready" in headless Chromium against
the installed `--web` boot it already starts.
The client under test comes from the install; the App test server stays a
repo fixture.
Also isolates pack:verify's `--web` catalog (the #1977 treatment the smokes
already have). Untidy while the step only did `GET /`; the App deep link
persists a server row, which would make it destructive and non-deterministic
against the developer's real ~/.mcp-inspector/mcp.json.
Unit tests cover what neither consumer can check itself — both CSRF gates and
the appArgs encoding on the deep link, and driveAppFlow's failure branches
against a stand-in page, all dead code from a happy-path smoke's view.
Verified: pack:verify passes end to end against the real tarball, and fails
at the new step alone when sandbox-controller.ts is pointed at a renamed
proxy page (the tarball and on-disk assertions both still pass — the exact
gap this closes).
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 3, all three real:
- `require.resolve.paths()` also appends Node's GLOBAL FOLDERS
($HOME/.node_modules, $HOME/.node_libraries, $PREFIX/lib/node) and any
NODE_PATH entries, so round 2's lookup could satisfy a missing repo
install from a GLOBALLY installed typescript/vite — measuring the
programs with the wrong compiler instead of producing the actionable
"run npm install" this helper promises. Filter to the node_modules
directories on fromDir's own ancestor chain.
- The quoting predicate missed characters cmd.exe still parses, notably
parentheses (`C:\Temp(1)`), and quoting cannot suppress `%VAR%` at all
— expansion happens before quotes are processed. Extracted to
lib/win-shell-args.mjs with a broad predicate and a hard error on `%`
rather than a silently different path. `platform` is a parameter so the
Windows behavior is exercised on the POSIX machines that run the suite.
- AGENTS.md claimed pack-and-verify spawns via `process.execPath`; it does
not, and cannot — its children are `npm` and the installed `.bin` shim.
State the exception and why.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01387r3MotGk52hrkfjzHFsN
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 2 (suppressed comments), all three real:
- `require.resolve("<pkg>/package.json")` is governed by the package's
`exports`, and Node keeps no special case for `./package.json` — so a
package declaring an `exports` map without it threw
ERR_PACKAGE_PATH_NOT_EXPORTED even though its bin was installed and
spawnable, a false "not installed" pointing at `npm install` for a
package already on disk. Walk `resolve.paths()` instead, which never
consults `exports`. The docblock claiming the opposite is corrected.
- `shell: true` (needed at all on Windows to start the `.cmd` shims) makes
Node hand `cmd.exe` one space-joined string, so an argument holding a
path with a space splits in two. Every generated path in
pack-and-verify lives under `tmpdir()`, which sits beneath the user
profile. Quote them via `shellArgs()` at all four spawn sites, the
command path included — the hazard predates this PR, so fix the class.
- Document the new module: `scripts/lib/resolve-node-bin.mjs` in AGENTS.md's
scripts/ tree, its tests in the README's `test:scripts` row.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01387r3MotGk52hrkfjzHFsN
Signed-off-by: cliffhall <cliff@futurescale.com>
On Windows npx/npm are .cmd shims a shell-free execFileSync/spawnSync
cannot start (ENOENT since the CVE-2024-27980 hardening), so
verify:typecheck-coverage died at validate's second step — doubly
silently, echoing "(no diagnostic captured)" per project and then
reporting all 918 tracked files as getting no tsc pass — and
verify:build-gate, the smoke test-server bootstraps, and pack:verify's
npm pack failed the same way.
Add scripts/lib/resolve-node-bin.mjs: resolves the JS entry behind a
package's bin from <pkg>/package.json (deep resolution is blocked by
Vite 8's exports map) and spawns it via process.execPath — the same
walk npx --no-install did, cross-platform and shell-free. Switch the
six npx call sites to it; the npm pack call gets the existing
shell-on-win32 idiom instead (npm has no in-tree package to resolve).
An unresolvable tsc is now a hard "cannot measure" error with
actionable stderr, pinned by a new main() regression test.
Closes#1939
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`clients/web/static/sandbox_proxy.html` is a committed source file, not a
build artifact, and it was never listed in the root package.json "files"
allowlist. `sandbox-controller.ts` reads it at runtime as
`<runner dir>/../static/sandbox_proxy.html`, so it resolved fine in-repo but
was absent from every published tarball — the read threw and the controller
served its "Sandbox not loaded" fallback page, breaking the Apps tab for
anyone running `npx @modelcontextprotocol/inspector`.
v1 shipped the equivalent as `server/static` in both the root and the server
workspace "files" lists; that entry has no counterpart in v2.
Add `clients/web/static` to the allowlist, and assert it in pack:verify twice
over — once in the tarball packlist and once on disk after install, since what
the runtime needs is the path *relative to* clients/web/build rather than mere
presence in the tarball.
No .npmignore change is needed: clients/web/.gitignore does not list `static`,
so the nested-gitignore packlist hazard that hid `build/` does not apply here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAR2Nr9kXbrywFNUVoTe9F
* fix(scripts): await the pack:verify web child's exit before removing its work dir
`pack-and-verify.mjs` had the same kill-then-remove shape that made
`smoke:tui` flake with ENOTEMPTY in #1801: `verifyWeb()`'s
`finally { stop() }` only *delivered* SIGTERM, then the main flow
`rmSync`'d the work dir immediately, and `fail()` did the same back to
back.
Applies the #1814 treatment, factored into a shared
`scripts/lib/child-cleanup.mjs` so the two scripts can't drift:
- `stopChild()` — SIGTERM, await the child's `exit`/`close`, escalate to
SIGKILL after a grace period, and ultimately proceed with a warning so
a wedged child can never hang the script. `verifyWeb()`'s `stop()` now
goes through it and is awaited, so the child is gone before the caller
removes `work`.
- `removeSafe()` — `rmSync` that warns instead of throwing. Used on both
success-path removals (a leftover temp dir must never fail a run that
passed) and in `fail()`, where an ENOTEMPTY could otherwise bury the
real diagnostic under an rmSync stack and skip the "tarball retained
at ..." hint. `fail()` stays synchronous — it is called from sync
contexts and ends in `process.exit()` — so it best-effort signals the
child; `removeSafe()` is what makes that safe.
`smoke-tui.mjs` adopts `removeSafe()` for its `cleanup()`. Its `done()`
keeps its bespoke wait (documented inline): that one must keep waiting
for `close` *after* `exit` on a shorter re-armed deadline so the quoted
output is complete, which `stopChild()`'s first-of-exit/close contract
deliberately does not do.
Adds `scripts/lib/child-cleanup.test.mjs` (10 cases, `node --test` via
`npm run test:scripts`) covering the SIGTERM/SIGKILL escalation, the
`close`-only spawn-failure path, the never-hang give-up, late events
after giving up, and that `removeSafe` warns rather than throws.
`npm run ci` passes, and `npm run pack:verify` (not in the gate — needs
network) was run end to end and passes with no cleanup warnings.
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(scripts): normalize stopChild's graceMs and correct removeSafe's JSDoc
Copilot review round 1 on #1895.
- `stopChild()` trusted `graceMs` as given, but call sites build it as
`Number(process.env.X ?? 5000)` — a typo yields `NaN`, which
`setTimeout` treats as `0`. That would fire SIGTERM and SIGKILL in the
same tick and print "did not exit within NaNms", collapsing the whole
escalation silently. New exported `normalizeGraceMs()` falls back to
`DEFAULT_EXIT_GRACE_MS` for anything non-finite or non-positive, with
two tests: the table of bad values, and an end-to-end one asserting a
`NaN` grace period still leaves only SIGTERM sent a turn later.
- `removeSafe()`'s `@returns` said "true if the path was removed", which
is inaccurate for a missing path (`force: true` makes it a no-op
success). Reworded to say what it actually reports.
`npm run ci` passes.
Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(web): format-gate the ungated first-party TS/JS (#1778)
Prettier only covered core/ and each client's src/, leaving the web Node
backend (clients/web/server/**), web top-level configs (vite.config.ts,
tsup.runner.config.ts, eslint.config.js, ...), and the root build/verify
tooling (scripts/**/*.mjs) ungated — so their formatting was never checked
by validate/CI. eslint already reached these via `eslint .`, so this is a
prettier-only gap.
- clients/web format/format:check now cover src, server, and top-level
*.{ts,js}, not just src.
- New root format:scripts / format:check:scripts for scripts/**/*.mjs,
folded into the root format and into validate:core.
- Normalized the newly-covered files with prettier --write (pure
formatting; no runtime change).
Closes#1778
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5
* chore: future-proof the scripts format glob (#1778 review)
Widen format:scripts / format:check:scripts from "scripts/**/*.mjs" to
"scripts/**/*.{mjs,js,cjs,ts}" so a future non-.mjs file under scripts/
is auto-gated, matching the future-proof glob style clients/web now uses.
A brace glob is a single pattern, so prettier does not error on the
unmatched branches. Addresses observation #2 from the PR #1788 review.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5
* docs: sync AGENTS.md scripts glob after the widening (#1788 review r2)
d86d8f7 widened the scripts glob to {mjs,js,cjs,ts} and updated the
validate:core bullet, but the format bullet at AGENTS.md:217 still said
scripts/**/*.mjs. Bring it in line so the two mentions agree.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-review found the same process.exit()-skips-finally pattern in verifyWeb: a
failure during the --web phase goes through fail() → process.exit(1), skipping
its `finally { stop() }`, so the spawned web server was orphaned — left on the
port, where a later run could fetch a stale false-200 and mask a regression.
Track the live child in a module `webChild` and have fail() SIGTERM it before
exiting, alongside the workDir cleanup; stop() clears it on the success path.
Verified: a forced web-phase failure exits 1, retains the tarball, and leaves
no listener on the port (child killed, not orphaned).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrihGWcrM9JGRyu41nzZYw
Re-review caught that the previous fix was inert: fail() calls process.exit(1),
which skips finally blocks — so `cleanupTarball = false` never took effect and,
worse, the temp work dir (a full node_modules from the real install) leaked on
every failed run.
Move cleanup into fail() itself — the single failure-exit point, called from
everywhere including deep inside the run. On failure it removes the heavy work
dir and leaves the packed tarball in place for post-mortem inspection; the
success path removes both inline before the OK log. Drop the dead
`cleanupTarball` flag entirely.
Verified both paths: success removes work dir + tarball (exit 0); a forced
failure removes the work dir, retains the tarball, and exits 1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrihGWcrM9JGRyu41nzZYw
Review follow-up: `cleanupTarball` was always true (set true at init and again
in the catch), so the finally guard was dead logic. Make it meaningful — retain
the packed tarball on failure so a packaging problem can be inspected, and log
where it is; the happy path still removes it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrihGWcrM9JGRyu41nzZYw
The Inspector ships as one package with one version, but four client
package.jsons each carried their own `version` and were read at runtime
(cli.ts, tui App.tsx), and core hardcoded a third, stale value (0.20.0) that
the TUI and web actually reported to servers. Consolidate onto the root
package.json as the single source of truth.
- Add core/node/version.ts: shared, Node-only readInspectorVersion() that walks
up to the root package.json (always shipped in the tarball). 100% covered by a
new integration test.
- CLI and TUI read the version via the shared reader instead of their own
manifest; TUI now also passes a real clientIdentity (was falling through to
the stale core default).
- Drop the `version` field from all four client package.jsons; the root keeps
the only version. TUI header name/description become local display constants.
- core default client identity: replace the stale 0.20.0 hardcode with a neutral
0.0.0 placeholder (real clients pass clientIdentity; web will source its
version from GET /api/config, see #1639).
- Stop shipping clients/cli/package.json and clients/tui/package.json (no longer
read at runtime) — supersedes the earlier "ship them" fix; update pack:verify
and the root README packaging invariants accordingly.
- Add core/node/** to the web coverage include so the new module is gated.
- Add the shared-code architecture diagram to the root README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrihGWcrM9JGRyu41nzZYw
Prepare the single `@modelcontextprotocol/inspector` tarball for publishing
and add a script that verifies the real published package end to end.
- Add `scripts/pack-and-verify.mjs` (`npm run pack:verify`): build, `npm pack`,
install the tarball into a clean consumer, and drive the installed
`mcp-inspector` bin (web/cli/tui) — catching "works in --dev, breaks under
npx" packaging/path failures the in-repo smokes miss. Local/release tool
(install needs network), not wired into the fast validate/ci loop.
- Disable source maps in the cli/tui/web-runner tsup configs (~halves the
unpacked payload; runtime doesn't need them).
- Add `clients/web/.npmignore` so `clients/web/build` (prod web-server runner)
ships — npm's packlist was honoring the nested `.gitignore` `build/` entry
over the root "files" allowlist, dropping the runner while `dist/` slipped
through.
- Ship `clients/cli/package.json` and `clients/tui/package.json`: both bundles
read their own package.json at runtime (CLI client identity, TUI header), so
the installed CLI crashed on connect and the installed TUI crashed on launch
without them.
Document the packaging invariants and pack:verify in the launcher README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrihGWcrM9JGRyu41nzZYw
