| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cliffhall
|
4af38c719c |
fix(scripts): don't arm a survival timer after the verdict is settled (#2147)
Copilot review round 4. `exit` can be observed before the child's last output drains — `done()` already keeps reading until `close` for exactly that reason — so `onData` can run after the verdict is settled. Recording `markerAt` there is still right: the message is a thunk rendered at `finish()`, and a marker still sitting in the pipe when `exit` fired was painted all the same. Arming a survival timer is not. It could not change the result (`done` is a no-op once settled), but it holds the event loop open for the whole window, which under `node --test` is a suite lingering seconds for nothing. That implies something the comment did not say, and it had already bitten this PR: if `markerAt` can be set after the exit was observed, any wording derived from observation ORDER is unstable. The old message read "rendered at Nms and then exited Mms later", where M was `since() - markerAt` evaluated at finish — not the paint-to-exit gap at all, and incoherent whenever the marker surfaced during the drain. The test asserted on /survival window/, which the losing side of that race could drop. It could have flaked, and the flake would have looked like the bug this PR fixes. So the message now makes no ordering claim and reports both times as the observations they are, with `exitAt` captured in the exit handler rather than recomputed later. Assertions moved to the race-stable phrase, plus a test that runs the paint-then-die stub 12x asserting verdict AND wording are identical every time — one pass proves nothing about a race. The ordering could not be forced on darwin, so the guard is reasoned rather than reproduced, and the reply says so. Two attempts and what they actually showed: a 4MB payload with the marker last never gets there, because `process.exit()` discards pending stdout writes (the "before rendering" branch is correctly taken); marker on stderr with a graceful exitCode was observed first in all 30 runs, none lingering near the window. `done()` exists precisely because exit and close are separate events, so treating their order as guaranteed was worth removing anyway. Also (suppressed comment): the README's "Two more do the same" clause now lists four suites, not two, after this PR added render-smoke and pty. Signed-off-by: Cliff Hall <cliff@futurescale.com> Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
747f59892f |
fix(scripts): disarm the first-paint deadline once the marker is seen (#2147)
Copilot review round 3.
`done()` is first-caller-wins, and the render deadline stayed armed after
first paint. So a marker landing near `timeoutMs` let that timer fire
*inside* the survival window and settle the run as
child did not render "MCP Servers" within 15000ms
followed by an output tail that visibly contains "MCP Servers". A TUI that
is merely slow fails intermittently, with a diagnostic naming the one
thing that demonstrably did happen — the same wrong-attribution class this
PR exists to remove, and one that reads as flake.
First paint has happened; its deadline no longer has a question to ask, so
`onData` clears it at the moment the marker is observed.
Also hoists `renderTimer`'s declaration beside `surviveTimer`'s. It was a
`const` below both closures that clear it, safe only because `onData` and
`done` run from events rather than the synchronous body. That already held
for `done`; this fix adds a second reader, so the ordering is now stated
rather than relied upon.
The test paints at 250ms with timeoutMs 400 and surviveMs 400, so the
render deadline expires while the survival window is open — the exact
shape. Verified it fails against the module with the clearTimeout removed
and passes with it: a wider timeoutMs would have been green either way,
which is why the numbers are tight.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
|
||
|
cliffhall
|
b7a73e7da3 |
fix(scripts): address Copilot review round 2 on #2147
Two findings, both of which quietly undo the PR.
1. `SMOKE_TUI_SURVIVE_MS` was unvalidated. `??` guards only nullish, so
an env var set to the empty string reaches `Number("")` -> 0 and a typo
reaches NaN; `setTimeout` fires immediately on both. The survival check
would then resolve before the child could possibly have died, printing
"still running 0ms later" and asserting nothing — the exact first-paint
false green this PR removes, restored by an empty environment variable,
on a green run.
Fixed at the module rather than only at the env-parse site, so every
caller is covered: `normalizeMs` (same shape as child-cleanup.mjs's
`normalizeGraceMs`, matching an existing policy rather than inventing a
second one) applied to all four durations before anything reads them,
which also stops the messages interpolating "NaNms". Defaults moved to
an exported DEFAULTS so the smoke and the module cannot drift.
`surviveMs: 0` is deliberately NOT honored as "skip the wait": no
caller wants that, and it is indistinguishable from the typo.
2. (suppressed comment) `spawnSync` REPORTS ENOENT in `result.error` — it
does not throw — so `probeScriptVersion`'s try/catch read a missing
binary as "ran, printed nothing". On linux that empty output fell
through to the util-linux guess, and the smoke then hard-failed
spawning a binary that does not exist instead of taking the documented
skip.
`probeScriptVersion` now returns `{ available, output }`, and
`resolvePtyWrapper` returns `{ ok: true, flavor, wrap }` or
`{ ok: false, reason }` — the two unavailability causes deserve
different skip messages, since "no script(1) on PATH" and "nobody has
verified this platform's invocation" send the reader elsewhere. A
non-zero exit is explicitly not unavailability: BSD `script` has no
`--version` and answers with its usage, which is the evidence the
flavor fallback reads.
Tests for both, since each finding lives entirely in a branch no happy
path reaches: the normalizeMs table (empty string, typo, undefined, -1,
Infinity, a numeric string, 0), an end-to-end run proving a
`surviveMs: Number("")` paint-then-die stub still fails, the
`result.error` ENOENT and timeout shapes, and both `resolvePtyWrapper`
failure reasons.
Verified: `SMOKE_TUI_SURVIVE_MS="" npm run smoke:tui` reports "still
running 2000ms later"; with PATH blanked, resolvePtyWrapper returns
`{ok: false, reason: "no `script(1)` on PATH"}` rather than guessing.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
|
||
|
cliffhall
|
bd7297da88 |
fix(scripts): address Copilot review round 1 on #2147
- pty.mjs: narrow the BSD argv set to {darwin, freebsd}. Grouping every
BSD together was wrong — NetBSD's `script` takes the command through
`-c`, so the argv form fails at `script` startup, which this smoke would
then report as a TUI crash. Rather than add an unverified `-c` branch
for netbsd/openbsd, they fall through to `null` and the smoke skips with
the reason: a guessed invocation turns a clean skip into a false failure
while looking like support.
- pty.test.mjs: "supported unless Windows" would fail `test:scripts` on
aix/sunos/netbsd/openbsd, where a null wrapper is correct and the smoke
skips by design. State the policy once (SUPPORTED_PLATFORMS) and branch
on it, keeping the real-probe teeth that catch a `script` missing from
PATH. Pin netbsd/openbsd/aix/sunos to null in scriptFlavorFor's tests
so "unsupported" reads as the default, not a win32 special case.
- render-smoke.mjs: the @returns contract claimed a close guarantee the
code does not give — both drain give-up branches resolve after only a
warning, so output can be truncated and the child may still hold the
work dir. Document the bounded fallback and name the required cleanup
shape (removeSafe, not rmSync), since a caller "tightening" cleanup on
the old wording would reintroduce #1801 exactly where it doesn't hold.
- AGENTS.md, clients/launcher/README.md: drop the pre-#2111 claim that the
smokes build test-servers "on demand if it is missing". ensureTestServers
rebuilds unconditionally, once per process — presence is not freshness.
- .github/copilot-instructions.md: mirror the survival rule, per the
same-PR mirroring requirement. One bullet, generalized past this fix:
a smoke must assert the thing survives, not that it started, and the
started-then-died branch belongs in a scripts/lib test against a stub.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
|
||
|
cliffhall
|
b6baa30d7b |
fix(scripts): make smoke:tui assert the TUI survives its first frame (#2147)
`smoke:tui` spawned the launcher with `stdio: ["ignore", …]`, waited for
"MCP Servers" in the output, printed OK, and SIGTERMed. It never looked at
the child again.
The child was already dead. Ink mounts `useInput`, `useInput` needs raw
mode, and raw mode is a property of the file descriptor — with stdin on
/dev/null the TUI painted one frame and exited 1 about 40ms later with
"Raw mode is not supported on the current process.stdin". Measured here:
first frame at 396ms, exit 1 at 433ms. The smoke won that race and
reported success, so what it asserted was *first paint*, not survival —
contradicting its own header ("boots and renders without crashing").
This is not environment-specific: `stdio: ["ignore", …]` makes the child's
stdin /dev/null whether or not the developer's terminal is a TTY, so this
smoke has never verified a running TUI on any machine. And the race cut
both ways — an `exit` processed first failed the run, which reads as flake
rather than as the standing defect.
Two pieces:
- `scripts/lib/pty.mjs` gives the child a real terminal via `script(1)`,
no dependency. The three flavors are not interchangeable (BSD takes
argv; util-linux takes one shell string plus `-e`; busybox that string
without `-e`), so the invocation is built and unit-tested rather than
written inline. A platform with no `script(1)` skips loudly instead of
running without one, which would be a guaranteed failure rather than a
weaker check.
- `scripts/lib/render-smoke.mjs` is the fix: the child must still be
running `surviveMs` (default 2s) after first paint. Without that, a
future harness change reintroduces the same false green and nothing
notices — which is why it lives in a module `test:scripts` drives
against a stub that paints the marker and immediately exits. Today's
harness passes that stub; this one fails it, with a diagnostic that
distinguishes "died after painting" from "died before painting".
The smoke additionally fails if the raw-mode error appears in a passing
run, so a future Ink that degrades to a warning instead of throwing can't
ship a TUI with no keyboard input past this gate.
Verified end to end: with a `script` shim that allocates no pty, the smoke
now fails with "it painted one frame, it did not run"; with the real one
the TUI is still running 2s after its first frame and never logs the
raw-mode error.
The `process.env.CI` skip stays, but its comment is corrected: the PTY
removes the technical blocker it used to cite, so the smoke is local-only
by decision now, not by capability. Whether it joins GitHub CI is out of
scope here (#2146 is keeping the other local-only smokes out on purpose).
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
|
||
|
cliffhall
|
93bc7df360 |
fix(smoke): assert the requested engine is the one launched; fix the remedy
Three findings, all from Copilot's suppressed block. 1. The tests never checked that loadBrowser launches the engine it was asked for. The failure-path stand-in rejects identically for every engine, so loadBrowser(root, "firefox") could have called chromium.launch() and still produced a correctly Firefox-labelled error. The smokes could not catch it either: the Firefox gate runs after Chromium is already installed, so a mis-dispatched launch would SUCCEED and report Firefox coverage that never happened. Added a success-path test with distinct per-engine spies asserting the requested engine is invoked, that nothing else is, and that its handle is what comes back. Mutation-verified: hard-coding playwright.chromium fails it. 2. The launch-failure message told the reader to run `npx playwright install --with-deps <engine>`, which is wrong now that the npm scripts run from the repo root rather than clients/web -- I introduced that when I removed the `cd`. Playwright is pinned in clients/web, so a bare root-level npx can fetch a different version and install a browser revision the pinned one still cannot launch. It now points at `npm run smoke:web:<engine>` first, and scopes the raw command to clients/web with the reason. That message named smoke:web:webkit, which did not exist -- only the chromium and firefox scripts did. Added it, and a test asserting every supported engine has one, since a remedy nobody can run is worse than no remedy. Mutation-verified. 3. AGENTS.md said npm run ci "mirrors" the workflow and then, two sentences later, that it is "a superset, not a mirror". It now leads with strict superset, which is the topology this PR is documenting. The README and AGENTS.md remediation wording is corrected the same way as the runtime message, so a reader gets the same advice wherever they find it. Addresses Copilot review 5030668906 on #2133. Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
8a7ade7b08 |
ci: drop the Firefox smoke job; keep the engines as an on-demand tool
The job was trialled and did not earn its place. It was genuinely cheap — ~2 minutes, running in parallel with the 15-minute build job, so zero added wall-clock — but across a dozen runs Firefox never once disagreed with Chromium. It was paying runner time and carrying a real flake surface (playwright install --with-deps runs apt-get update, which fails whenever a third-party repo in the runner image breaks; that is what the one red run was) to re-confirm a result already in hand. Everything that made cross-engine checking possible stays: SMOKE_BROWSER, the shared headless-browser helper, the installer, and the tests. So "SMOKE_BROWSER=firefox npm run smoke:web:engine" is there for the moment it is worth running — before touching the sandbox, the CSP builder or the proxy page — without taxing every push to re-derive the same answer. The workflow is now byte-identical to v2/main again; the publish jobs go back to needing only build. Docs say what is true rather than what was intended, and AGENTS.md records the reasoning so the job is not re-added on the general argument that cross-engine coverage is good — that argument was already accepted, and the on-demand path is what serves it. Re-add it on evidence: a real cross-engine regression a gate would have caught. Also swept the now-stale "matrix" language out of the smokes and the resolver, including the empty-SMOKE_BROWSER rationale, which no longer leans on a matrix key that does not exist. Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
fb9e146043 |
docs(smoke): drop the #2132 references now that the issue is closed
#2132 was closed without the investigation going anywhere, so every 'pending #2132' pointer in the tree was about to send a reader to a closed issue whose own diagnosis had already been retracted. The docs now say what is actually known, and nothing more: the two App smokes fail under Playwright's WebKit for unidentified reasons; it does not reproduce in real Safari; an isolated repro of the suspected mechanism did not reproduce it under Playwright's WebKit either; and chasing it further was judged not worth the effort. A reader hitting it locally is told to treat it as unexplained rather than as a defect. The matrix decision is unchanged and never depended on the cause — the engine fails, so it is not gated. No behavior change. Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
32cb4c63a7 |
fix(smoke): reject an empty SMOKE_BROWSER; test the installer's argument logic
Two findings from Copilot's suppressed block.
1. resolveBrowserName treated a present-but-empty SMOKE_BROWSER as unset
and returned chromium. The test that covered it even documented WHY
that is dangerous — GitHub Actions renders an undefined matrix key as
"" rather than omitting the variable — and then asserted the unsafe
behavior. A misspelled matrix key would run chromium inside a job named
for another engine and report green: the same false-coverage outcome
the unrecognized-value branch exists to prevent, reached by a typo
instead of a bad value.
Only an absent variable now selects the default. Empty throws, naming
the likely cause and the actual remedy ('unset it', which is not the
same as setting it to nothing). The workflow always renders a real
value, so nothing in CI is affected.
2. install-smoke-browser.mjs had no tests. Its I/O half runs on every
smoke and fails loudly; its argument half does not — every caller
passes nothing or the literal 'chromium', so the precedence rule is
asserted by nothing at runtime. It matters: pack:verify passes chromium
BECAUSE it launches chromium, and if the environment won,
SMOKE_BROWSER=webkit npm run pack:verify would install one engine and
launch another, surfacing as a misleading "Executable doesn't exist".
resolveRequestedBrowser is now exported and pure; execution moved
behind main() with the entrypoint guard verify-typecheck-coverage.mjs
uses, so importing the module for tests cannot trigger an install.
Both mutation-verified: restoring either behavior fails the tests that
claim to cover it.
Addresses the suppressed comments on Copilot's fifth review of #2133.
Signed-off-by: cliffhall <cliff@futurescale.com>
|
||
|
cliffhall
|
4ab0139de9 |
test(smoke): cover loadBrowser's failure branches; fix a stale helper comment
Two findings from Copilot's suppressed block. 1. The launch-failure message is #2086's acceptance criterion — a missing browser must name THAT engine and its own install command — and it had no test. The smokes cannot supply one: install-smoke-browser fetches the binary first, so a passing smoke says nothing about what a failing one prints, and that message is the deliverable. loadBrowser now takes an injectable loadPlaywright purely to open that seam. The test asserts per engine that the message leads with that engine, names --with-deps <that engine> and no other, and preserves the underlying cause. Verified by mutation: hard-coding 'chromium' in the message fails it. It also pins the two branches apart — an unresolvable package must say > @modelcontextprotocol/inspector@2.3.0 postinstall > node scripts/install-clients.mjs [install-clients] Installing dependencies for clients/web... up to date, audited 583 packages in 614ms 198 packages are looking for funding run `npm fund` for details found 0 vulnerabilities [install-clients] Installing dependencies for clients/cli... up to date, audited 237 packages in 496ms 69 packages are looking for funding run `npm fund` for details found 0 vulnerabilities [install-clients] Installing dependencies for clients/tui... up to date, audited 314 packages in 518ms 102 packages are looking for funding run `npm fund` for details found 0 vulnerabilities [install-clients] Installing dependencies for clients/launcher... up to date, audited 160 packages in 449ms 50 packages are looking for funding run `npm fund` for details found 0 vulnerabilities up to date, audited 281 packages in 3s 108 packages are looking for funding run `npm fund` for details found 0 vulnerabilities and must NOT say , which fetches binaries and cannot fix a missing package. The unsupported-engine test now also asserts loadPlaywright was never called, pinning the ORDER rather than only the message: validating first is what makes the error name the real mistake. 2. prod-web-server.mjs's header still described the prefix this PR removed. Corrected, keeping why deriving paths from import.meta.url is still right, and adding the elicitation smoke to the list of callers it had also outgrown. Addresses the suppressed comments on Copilot's review of #2133. Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
25eb2fb3bf |
docs(smoke): stop describing the WebKit smoke failure as a Safari bug
#2132 was filed asserting that a Safari user opening an MCP App gets a permanently 'loading' widget. That was never tested. A manual check in real Safari renders the App normally, so the claim is unsupported and is removed from all six places it reached. The failure under Playwright's WebKit is real and still keeps that engine out of the CI matrix. What is not established is that any shipping browser behaves this way — and Playwright ships its own WebKit build whose networking stack is not Safari's, which is exactly the layer the failure lives in. headless-browser.mjs already carried the caveat that a green WebKit run is not a Safari guarantee. The same sentence forbids the inverse inference, which is the one that was made; it now says so explicitly, and AGENTS.md carries the rule: reproduce in the real browser before claiming user impact in one. No behavior change. Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
ef4259a544 |
test(smoke): run the headless web smokes in Firefox and WebKit, not just Chromium (#2086)
Every browser-driven check in this repo launched Chromium and nothing else.
That is fine for most of the web client, whose behavior is React and Mantine.
It is not fine for the MCP Apps sandbox, which is built out of the primitives
that genuinely diverge between engines: a CSP <meta> injected as the first
<head> child of a srcdoc document, a nested sandboxed iframe, a
Permissions-Policy allow attribute, and postMessage origin discipline across
those two frames. Nothing else covers that surface — sandbox-csp.test.ts
asserts which policy STRING is built (environment-independent by
construction), and no Storybook story reaches the sandbox at all, since all
three App stories use a data: placeholder iframe and a mock bridge.
The engine is now a parameter:
SMOKE_BROWSER=webkit npm run smoke:web:app
SMOKE_BROWSER=firefox npm run smoke:web:engine # all three smokes
Unset means chromium, so `npm run ci` and `npm run smoke` are unchanged. An
unrecognized value is an error rather than a fallback — falling back would
report a green Chromium run under a job labelled "webkit", claiming coverage
that never ran — and a launch failure names the engine that failed and its own
`playwright install` invocation.
Selection, launching, and the fatal-vs-benign page-diagnostics split now live
in one shared scripts/lib/headless-browser.mjs. The three smokes had each
hand-rolled the diagnostics split and two had their own copy of loadChromium;
pack:verify goes through the same helper, pinned to Chromium explicitly since
it is a packaging check.
The npm scripts drop their `cd clients/web && npx playwright install chromium`
prefix for scripts/install-smoke-browser.mjs. The engine is a variable now, and
a ${VAR:-default} expansion does not expand under Windows' cmd.exe; npx is also
a .cmd shim there that a shell-free spawn cannot start, so this resolves the
Playwright CLI through resolveNodeBin — the #1939 problem solved the #1939 way.
CI gains a `Sandbox smokes (<engine>)` matrix job rather than matrixing `build`,
which also runs validate, the coverage gate, two verify gates and Storybook —
all engine-independent. Both publish jobs now need it.
WebKit is supported by the tooling but deliberately NOT in the CI matrix yet:
pointing the smokes at it immediately found #2132, a pre-existing Safari
incompatibility in the web client's SSE transport that strands the last message
on /api/mcp/events, so an MCP App never leaves "loading". That is a product bug
affecting real Safari users, with its own diagnosis and a verified fix, and it
is not a matrix change. Adding `webkit` here is a one-word diff once it lands.
An engine is either green in CI or absent from it; a continue-on-error job would
report coverage nobody is held to.
Signed-off-by: cliffhall <cliff@futurescale.com>
|
||
|
cliffhall
|
3222537e73 |
docs(scripts): correct the deleted-source caveat — it is silent, not loud
Round 3's suppressed finding, and it corrects a claim of mine that was
simply wrong. The header said a deleted source file's stale `.js` produces
"a loud failure rather than a silent one — an import of a removed module
fails at the consumer". It does not: the stale file is still there, so the
post-build existence checks pass against it and anything still importing
that module keeps silently running the old code. That is the SAME failure
class this PR exists to close, described as though it were the opposite.
Rewritten in the helper header and AGENTS.md to say the unconditional build
does not cover deletion, and to make `rm -rf test-servers/build` after a
delete or rename an instruction rather than an aside — which is now a real
clean, since the tsbuildinfo was pinned inside build/ in
|
||
|
cliffhall
|
8d193461e3 |
fix(scripts): pin the tsbuildinfo inside build/, assert on cache hits
Two findings from review round 1, both real. The default incremental cache path derives from `rootDir`, so it landed at `test-servers/tsconfig.tsbuildinfo` — beside the config, OUTSIDE `build/`. That survives the `rm -rf test-servers/build` this project documents as the remedy for a deleted source file, leaving tsc convinced its outputs are up to date and emitting nothing. Pinned to `./build/tsconfig.tsbuildinfo`, and verified that removing the output dir now forces a full re-emit. The build is memoized per repo root; the existence assertion was memoized with it. A later caller naming an entry the first did not would be handed a path tsc never emitted, then fail with exactly the opaque ERR_MODULE_NOT_FOUND the helper exists to replace. The assertion now runs on every call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
e59452ea8c |
chore(scripts): build test-servers unconditionally from one shared helper
Five `scripts/*.mjs` consumers each carried their own `ensureTestServer()` that returned early when `test-servers/build` was already on disk. Presence is not freshness: once that directory exists — which it does on any machine that has run the smokes once — an edit to `test-servers/src` was never picked up again, and the smoke went on driving the previously-built fixture. The failure is silent and misattributed. It does not read as "your fixture is stale"; it reads as a product bug in whatever the smoke is checking. `scripts/lib/ensure-test-servers.mjs` is now the one place that build is produced for a script consumer, and it builds unconditionally — memoized once per process per repo root, so pack-and-verify's two asks stay one tsc pass while a fresh run always rebuilds. Callers name the entries they depend on and get one actionable failure message, replacing five that had drifted. pack-and-verify also stops shelling out to `npx tsc`, which is what made it the one script unrunnable on Windows (#1939). `test-servers/tsconfig.json` gains `incremental` so the up-to-date case is cheap: 3.0s cold, 1.6s incremental here. Unconditional emit is still not a clean — `--noCheck` leaves the stale .js of a deleted src file behind — which is unchanged from before and recorded in the helper's header and AGENTS.md. Closes #2111 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
df938777b1 |
Merge branch 'v2/main' into v2/chore/2003-pack-verify-app-render
One conflict, scripts/smoke-web-app.mjs: #2056 added a second phase to the smoke (the same app declaring _meta.ui.domain, asserting it renders from the dedicated app origin rather than an opaque one) while this branch extracted the smoke's flow into scripts/lib/mcp-app-flow.mjs. Resolved by keeping both: phase 2 now runs through the shared helper too, which grows three knobs it needs and pack:verify does not — - expectDeepLink: phase 2 re-navigates the SAME page, so the token gate is already proven and a rejected re-navigate would fail at the connect wait regardless; - what: so a phase-2 ready timeout doesn't read as a phase-1 one; - extraDiagnostics: phase 2 appends the frame list, which is what separates "never rendered" from "rendered at the wrong origin". Each is covered by a new unit test, since neither consumer exercises a failure branch. pack:verify deliberately drives phase 1 only. The dedicated origin is served by the same runner the packaging assertions already cover and reads nothing new from disk (app-origin-controller.ts has no file reads), so a second browser launch there would buy no packaging signal. Verified on the merged tree: smoke:web:app passes both phases (dedicated origin http://127.0.0.1:6278/app-document/<id>), and pack:verify passes end to end against the real tarball. Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
17c33080f9 |
chore(ci): render an MCP App from the installed tarball in pack:verify (#2003)
Nothing covered the intersection #1859 fell through. `smoke:web:app` drives the App path but against the repo build tree; `pack:verify` runs against the installed tarball but only asserted the sandbox proxy page *exists*. So a packaging regression that ships the file present-but-unreachable — a rename with a stale reader, a wrong position relative to clients/web/build, a future "files"/.npmignore edit — passes both. Extract the drive-the-App-flow half of the smoke into scripts/lib/mcp-app-flow.mjs and call it from both, rather than copying it: the deep-link shape is the part that rots silently, and a drifted link fails as an opaque timeout rather than a mismatch. pack:verify then drives connect -> open app -> data-app-status="ready" in headless Chromium against the installed `--web` boot it already starts. The client under test comes from the install; the App test server stays a repo fixture. Also isolates pack:verify's `--web` catalog (the #1977 treatment the smokes already have). Untidy while the step only did `GET /`; the App deep link persists a server row, which would make it destructive and non-deterministic against the developer's real ~/.mcp-inspector/mcp.json. Unit tests cover what neither consumer can check itself — both CSRF gates and the appArgs encoding on the deep link, and driveAppFlow's failure branches against a stand-in page, all dead code from a happy-path smoke's view. Verified: pack:verify passes end to end against the real tarball, and fails at the new step alone when sandbox-controller.ts is pointed at a renamed proxy page (the tarball and on-disk assertions both still pass — the exact gap this closes). Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
6b4bf68b8b |
fix(scripts): observe the final polling interval before declaring a timeout
Copilot's suppressed round-3 comment is right. The attempt-counted loop read the output and then slept, so nothing ever observed the last interval: an announcement arriving at 29.9s of a 30s budget was reported as a timeout, and a spawn error or early exit landing in that window was misattributed the same way. Inherited from the original smoke, but wrong either way. Replace it with a deadline loop that re-reads state after every wait, the last one included, and clamp the final wait to the deadline so the budget is a real bound rather than one poll longer. The regression test sets pollMs === timeoutMs, which makes the distinction deterministic rather than a race: the attempt-counted loop gets exactly one check, at t≈0 before the child has finished starting, then throws; the deadline loop re-reads after the wait and sees the announcement. The child announces immediately, ~2s inside the window, so a loaded runner can't flip it. Verified both directions — the test fails against the old loop and passes against the new one — and `npm run test:scripts` was repeated to confirm it is stable under the full 116-test parallel run (an earlier margin-based version of this test was not). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
9cade95fd0 |
test(scripts): kill the probe child on every path; document the helper
Two review findings from round 2, both correct. 1. The regression suite killed its child only after the assertions. If `assert.rejects` or either assertion failed, execution never reached the kill, leaving a 60-second child attached to the runner — a suite guarding a process leak would leak one itself, precisely on the run where the regression had come back. Teardown is now registered from `onSpawn` via `t.after`, so it runs on every path out of every case. That is the helper's own ownership rule applied to the tests; killing an already-exited child is a no-op, so the early-exit cases pay nothing for it. 2. Document the new shared helper, per the maintenance rules. AGENTS.md's scripts/lib tree entry now carries announced-child.mjs and what its contract buys, and the smoke:web:app paragraph records why the spawn/readiness step moved out of the smoke (same reason teardownWebServer left prod-web-server's stop()). README's test:scripts row names the new suite alongside resolve-node-bin's. Not mirrored into .github/copilot-instructions.md: that file is a distillation of the rules a reviewer cites against a diff, and deliberately excludes the project-structure tree and tooling inventory. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
09b35489aa |
test(scripts): cover the readiness-timeout teardown path (#2000 review)
Copilot's review is right that the fix was unproven: smoke:web:app's happy path always receives the announcement, so `npm run ci` never exercised a child that stays alive *through* the readiness timeout — exactly the case that orphaned a live server holding its port. A regression there would be silent in CI and would surface only as a stray process on a later run. Extract the spawn/readiness ownership into scripts/lib/announced-child.mjs (the same move prod-web-server.mjs made for teardownWebServer, and for the same reason), and add scripts/lib/announced-child.test.mjs under `test:scripts`. The tests drive real `node -e` children rather than spies, so the assertion is that a real process is reachable and killable, not that a mock was called: the timeout case asserts the child was published before the wait, is still alive when the throw lands, and is actually stopped by kill. The early-exit and spawn-failure paths are covered alongside it, plus the both-streams scan. smoke-web-app.mjs keeps its behavior — startMcpServer() now delegates the spawn and publishes the child via `onSpawn`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
c93552139c |
fix(scripts): keep the resolver local, complete the cmd.exe quoting
Copilot review round 3, all three real: - `require.resolve.paths()` also appends Node's GLOBAL FOLDERS ($HOME/.node_modules, $HOME/.node_libraries, $PREFIX/lib/node) and any NODE_PATH entries, so round 2's lookup could satisfy a missing repo install from a GLOBALLY installed typescript/vite — measuring the programs with the wrong compiler instead of producing the actionable "run npm install" this helper promises. Filter to the node_modules directories on fromDir's own ancestor chain. - The quoting predicate missed characters cmd.exe still parses, notably parentheses (`C:\Temp(1)`), and quoting cannot suppress `%VAR%` at all — expansion happens before quotes are processed. Extracted to lib/win-shell-args.mjs with a broad predicate and a hard error on `%` rather than a silently different path. `platform` is a parameter so the Windows behavior is exercised on the POSIX machines that run the suite. - AGENTS.md claimed pack-and-verify spawns via `process.execPath`; it does not, and cannot — its children are `npm` and the installed `.bin` shim. State the exception and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01387r3MotGk52hrkfjzHFsN Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
b399bc64e1 |
fix(scripts): find the manifest without exports, quote shell args on Windows
Copilot review round 2 (suppressed comments), all three real:
- `require.resolve("<pkg>/package.json")` is governed by the package's
`exports`, and Node keeps no special case for `./package.json` — so a
package declaring an `exports` map without it threw
ERR_PACKAGE_PATH_NOT_EXPORTED even though its bin was installed and
spawnable, a false "not installed" pointing at `npm install` for a
package already on disk. Walk `resolve.paths()` instead, which never
consults `exports`. The docblock claiming the opposite is corrected.
- `shell: true` (needed at all on Windows to start the `.cmd` shims) makes
Node hand `cmd.exe` one space-joined string, so an argument holding a
path with a space splits in two. Every generated path in
pack-and-verify lives under `tmpdir()`, which sits beneath the user
profile. Quote them via `shellArgs()` at all four spawn sites, the
command path included — the hazard predates this PR, so fix the class.
- Document the new module: `scripts/lib/resolve-node-bin.mjs` in AGENTS.md's
scripts/ tree, its tests in the README's `test:scripts` row.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01387r3MotGk52hrkfjzHFsN
Signed-off-by: cliffhall <cliff@futurescale.com>
|
||
|
cliffhall
|
4b143ce043 |
fix(scripts): validate the bin name and that its file exists
Copilot review on #1997: - A string-form `bin` is npm's shorthand for ONE command, named after the package, so ignoring `binName` let a typo silently resolve the package's only executable while bypassing the documented "declares no such bin" failure. Match it against the manifest's unscoped name instead. - A declared bin whose file is absent (partial install) was returned unchecked. `process.execPath <missing>` still spawns and exits 1 with empty stdout, which `rawProjectListing` records as "no diagnostic captured" — reproducing the bogus every-file-uncovered report this helper exists to eliminate. Throw here, where the remedy is actionable. Also restores `rawProjectListing`'s docblock, which the new `tscEntry` block had orphaned above itself. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01387r3MotGk52hrkfjzHFsN Signed-off-by: cliffhall <cliff@futurescale.com> |
||
olaservo
|
7164806be3 |
Merge origin/v2/main: move the #1939 npx fix into the shared tsc-program lib
Upstream #1965 moved the tsc --listFilesOnly machinery into scripts/lib/tsc-program.mjs (shared with the new verify:dep-lockstep), still spawning `npx` — which is ENOENT on Windows. Resolve the conflict by taking upstream's factoring and porting the #1939 fix into the lib: rawProjectListing and verify-typecheck-coverage's --showConfig pass now spawn process.execPath with the cached, hard-failing tscEntry resolver (exported from tsc-program.mjs). Fixture copies in both main tests pick up the new lib import graph. verify:dep-lockstep now runs on Windows too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cliffhall
|
a9b9a917e3 |
chore(scripts): compare only the copies a program actually met
Two findings from the Copilot review of #1965, both real. 1. The candidates were flattened to package names before the version comparison, so a third install holding a different copy failed the guard even when no program loads that copy beside another — and the diagnostic named an install that never took part. `crossInstallPackages` now returns the co-occurrences themselves (name → program → install → entry paths) and `findSkew` compares per program, across only the installs that met there. 2. Folding a nested copy onto its outermost install lost the path that identifies which copy was loaded, and the version was then read from the install's top-level entry — absent or differently versioned, either way letting a real pair pass. `classifyModulePath` now also returns the `entryPath` npm keys the lockfile by, `topLevelLockVersions` becomes `lockVersionsByPath`, and each holder is priced from the entry the program resolved. A resolved copy with no lockfile entry is now a hard failure rather than a silently dropped holder. The failure output prints `in <program>` with each holder's install path beside its version, so a nested copy is distinguishable from a top-level one. Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
7f9be24bf3 |
chore(scripts): derive dep-lockstep candidates from what enters each tsc program
`verify:dep-lockstep` derived its candidate set from the packages the shared first-party sources named *directly*, so a package whose declarations reach a program only through another package's `.d.ts` was invisible to it — it could skew across installs and recreate the two-copy blow-up #1896 was about while the guard stayed green. `@modelcontextprotocol/sdk` was the live example: never written in first-party code, yet 16 of its `.d.ts` files land in clients/web's test program. Replace the derivation with the measured one: list every client tsconfig project with `tsc --listFilesOnly`, map each resolved `node_modules` file to its owning install and package, and keep the packages that reach ONE program from TWO installs. That is exactly the set that can put two structurally distinct copies of a type in front of one checker. The rest of the guard is unchanged — top-level lockfile comparison, deny-by-default, `TOLERATED_SKEW` as a name allowlist scoped to within-a-major. The listing machinery moves to `scripts/lib/tsc-program.mjs`, shared with `verify:typecheck-coverage`, so the two guards can't disagree about what a program contains. It is deliberately not cached to disk between the two processes: a stale fingerprint would make a guard measure a program that no longer exists and pass on a real miss. The second pass costs ~14s. `TOLERATED_SKEW` is now empty. `jose` and `@modelcontextprotocol/ext-apps` are root-only since #1970 and cannot skew; `react` (whose types are `@types/react`, resolved from one install) and `hono` never reach one program from two installs. Each would be a rationale for a skew that cannot occur. Closes #1965 Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
olaservo
|
e3a026802b |
fix: resolve node bins via package.json instead of spawning npx .cmd shims so the verify/smoke scripts run on Windows
On Windows npx/npm are .cmd shims a shell-free execFileSync/spawnSync cannot start (ENOENT since the CVE-2024-27980 hardening), so verify:typecheck-coverage died at validate's second step — doubly silently, echoing "(no diagnostic captured)" per project and then reporting all 918 tracked files as getting no tsc pass — and verify:build-gate, the smoke test-server bootstraps, and pack:verify's npm pack failed the same way. Add scripts/lib/resolve-node-bin.mjs: resolves the JS entry behind a package's bin from <pkg>/package.json (deep resolution is blocked by Vite 8's exports map) and spawns it via process.execPath — the same walk npx --no-install did, cross-platform and shell-free. Switch the six npx call sites to it; the npm pack call gets the existing shell-on-win32 idiom instead (npm has no in-tree package to resolve). An unresolvable tsc is now a hard "cannot measure" error with actionable stderr, pinned by a new main() regression test. Closes #1939 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cliffhall
|
678184ed3e |
fix(smoke): pin the teardown ordering, and correct a docblock the test refuted
Both round-3 review points, both mine. 1. The docblock claimed deleting `removeSafe` would leave "both the unit suite and all three smokes green" — written before the test existed and never updated, so it contradicted the test added in the same PR. Only the smokes would stay green; the unit tests are precisely what fails on it. 2. `assert.ok(hasExited(child))` after the await proved only that the child was dead by the time teardown returned. It passes just as happily if the catalog were removed *first*, or while an un-awaited `stopChild` was still pending — so it did not lock down the #1801 ordering it claimed to. Sample the directory from the child's own `exit` handler instead, registered before teardown so it runs ahead of stopChild's listener and observes the world at the instant the child dies. `removeSafe` cannot have run by then: it is sequenced after the promise stopChild resolves from that very event. Mutation-checked against both defects the old assertion missed — removing before stopping, and dropping the await. Each now fails the test; neither did before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HDdo1rNRVQnVRRdsqSrvG3 Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
3d541388db |
test(smoke): exercise the real teardown path, not just removeSafe
Copilot was right that the previous test proved only that `removeSafe` can
delete a directory — it never drove `stop()`, so deleting the cleanup call
would have left the unit suite AND all three smokes green. That is exactly the
regression these tests exist to catch, and they did not catch it.
Extract the teardown into an exported `teardownWebServer({ child, catalogDir,
label })`, which `stop()` now delegates to, and drive it against a stand-in
child process (a real `node -e setInterval` that stays alive until signalled)
rather than a mock. Three cases:
- it removes the catalog dir it was given (asserted on the dir itself, not a
spy, so the assertion cannot drift from the behavior);
- it awaits the child's exit before removing — the #1801 race in one
assertion, since losing the await is otherwise invisible;
- an already-dead child is not an error, which is the failure path a smoke
takes when it calls fail() after the launcher has crashed.
Mutation-checked: with the `removeSafe` call deleted, 2 of the 3 now fail
(previously: 0). AGENTS.md updated to match.
Signed-off-by is now on every commit (#1978 DCO); the requirement itself is
undocumented and tracked in #1979.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDdo1rNRVQnVRRdsqSrvG3
Signed-off-by: cliffhall <cliff@futurescale.com>
|
||
|
cliffhall
|
4a993fc45a |
docs(smoke): document the throwaway-catalog invariant and fix the label JSDoc
Addresses the two review comments that still applied to HEAD (the third, the stop() race, was already fixed in a8022138). - AGENTS.md still described the prod-web-server helper as shared by only smoke:web and smoke:web:browser, and said nothing about catalog isolation. It is the source of truth for these smoke mechanics, so record the invariant there: the temp catalog per run and why, that only the catalog (not HOME) is redirected, that teardown needs both halves of child-cleanup.mjs and is therefore async for every caller, and where the contract is unit-tested. - The `label` JSDoc claimed the option prefixes the temp dir. It does not — that prefix is fixed at `smoke-web-catalog-` (deliberately, so every run's dir is greppable); `label` only prefixes teardown warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HDdo1rNRVQnVRRdsqSrvG3 Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
16c96d44c3 |
fix(smoke): await the server's exit before removing its catalog, and test it
Addresses both Copilot review comments on #1978. 1. `stop()` removed the catalog dir immediately after a bare `child.kill()`, which only *delivers* SIGTERM. That is precisely the #1801 race `child-cleanup.mjs:11-17` documents, and it says both halves are needed: `stopChild()` closes the race on the normal path, `removeSafe()` makes the residual case harmless. I had used only the second. `smoke-web-app.mjs` already used `stopChild` for its MCP test-server child, so the helper was inconsistent with its own sibling. `stop()` is now async — it awaits `stopChild` and then removes — and all three callers await it (`smoke-web.mjs`'s `fail()` became async to match, so every call site had to `await fail(...)` or execution would run past it instead of exiting). 2. The isolation contract had no test. The smokes exit the process right after teardown, so a regression reintroducing the shared catalog — or dropping cleanup — would leave them all green. Split the two pure pieces out (`createTempCatalog`, `buildWebServerEnv`) and cover them in a sibling `prod-web-server.test.mjs`: a unique dir per run, no pre-created catalog file, the dir is removable, and an inherited `MCP_CATALOG_PATH` (or `MCP_AUTO_OPEN_ENABLED`) cannot beat the explicit one — the spread-then-assign ordering that guard depends on. `startProdWebServer` itself stays untested by design; it spawns a real launcher, which is the smokes' job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HDdo1rNRVQnVRRdsqSrvG3 Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
cliffhall
|
29e3ce95b8 |
chore(smoke): run the web smokes against a throwaway catalog
`startProdWebServer` spread `process.env` without setting `MCP_CATALOG_PATH`, so `smoke:web`, `smoke:web:browser`, and `smoke:web:app` all ran against the developer's real `~/.mcp-inspector/mcp.json` — unlike `smoke:cli` / `smoke:tui`, which have always driven a temp `--catalog`. That made the web smokes both destructive and non-deterministic. `smoke:web:app`'s deep link persists a `deep-link` server row, so a second run finds it already on disk, racing hydration: the deep-link effect misses it in `servers`, POSTs `addServer` anyway, and the backend answers 409. The app swallows that 409 by design (App.tsx:2842) so the smoke still passed, but it reported a spurious non-fatal console error that was really just residue from the previous run. Give the helper its own temp catalog dir and clean it up in `stop()` via `removeSafe`, so cleanup can never turn a passing smoke red. Isolation lives in the shared helper rather than per script, since every web smoke needs it. Only the catalog is redirected — other per-user state under `~/.mcp-inspector` stays shared, since isolating it means redirecting HOME wholesale, which would also move the npx and Playwright caches these smokes depend on. Verified: `smoke:web:app` twice in a row is now clean both times (previously the second run reported the 409), the real catalog's mtime is unchanged, and no temp dirs leak. CI never saw this — a fresh HOME per run made every CI run look like a first run. Closes #1977 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HDdo1rNRVQnVRRdsqSrvG3 Signed-off-by: cliffhall <cliff@futurescale.com> |
||
|
Cliff Hall
|
8801d0f8bc |
fix(scripts): await the pack:verify web child's exit before removing its work dir (#1895)
* fix(scripts): await the pack:verify web child's exit before removing its work dir `pack-and-verify.mjs` had the same kill-then-remove shape that made `smoke:tui` flake with ENOTEMPTY in #1801: `verifyWeb()`'s `finally { stop() }` only *delivered* SIGTERM, then the main flow `rmSync`'d the work dir immediately, and `fail()` did the same back to back. Applies the #1814 treatment, factored into a shared `scripts/lib/child-cleanup.mjs` so the two scripts can't drift: - `stopChild()` — SIGTERM, await the child's `exit`/`close`, escalate to SIGKILL after a grace period, and ultimately proceed with a warning so a wedged child can never hang the script. `verifyWeb()`'s `stop()` now goes through it and is awaited, so the child is gone before the caller removes `work`. - `removeSafe()` — `rmSync` that warns instead of throwing. Used on both success-path removals (a leftover temp dir must never fail a run that passed) and in `fail()`, where an ENOTEMPTY could otherwise bury the real diagnostic under an rmSync stack and skip the "tarball retained at ..." hint. `fail()` stays synchronous — it is called from sync contexts and ends in `process.exit()` — so it best-effort signals the child; `removeSafe()` is what makes that safe. `smoke-tui.mjs` adopts `removeSafe()` for its `cleanup()`. Its `done()` keeps its bespoke wait (documented inline): that one must keep waiting for `close` *after* `exit` on a shorter re-armed deadline so the quoted output is complete, which `stopChild()`'s first-of-exit/close contract deliberately does not do. Adds `scripts/lib/child-cleanup.test.mjs` (10 cases, `node --test` via `npm run test:scripts`) covering the SIGTERM/SIGKILL escalation, the `close`-only spawn-failure path, the never-hang give-up, late events after giving up, and that `removeSafe` warns rather than throws. `npm run ci` passes, and `npm run pack:verify` (not in the gate — needs network) was run end to end and passes with no cleanup warnings. Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(scripts): normalize stopChild's graceMs and correct removeSafe's JSDoc Copilot review round 1 on #1895. - `stopChild()` trusted `graceMs` as given, but call sites build it as `Number(process.env.X ?? 5000)` — a typo yields `NaN`, which `setTimeout` treats as `0`. That would fire SIGTERM and SIGKILL in the same tick and print "did not exit within NaNms", collapsing the whole escalation silently. New exported `normalizeGraceMs()` falls back to `DEFAULT_EXIT_GRACE_MS` for anything non-finite or non-positive, with two tests: the table of bad values, and an end-to-end one asserting a `NaN` grace period still leaves only SIGTERM sent a turn later. - `removeSafe()`'s `@returns` said "true if the path was removed", which is inaccurate for a missing path (`force: true` makes it a no-op success). Reworded to say what it actually reports. `npm run ci` passes. Claude-Session: https://claude.ai/code/session_01YAt8rqxysNbhYWLhoRm3fU Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Cliff Hall
|
d9e4f444a2 |
test(cli,tui): typecheck the __tests__ dirs (#1791) (#1799)
* test(cli,tui): typecheck the __tests__ dirs (#1791) The cli/tui `typecheck` scripts only checked `src` (the src-only tsconfig excludes `**/*.test.*`), leaving the `__tests__` dirs — which do the most `as` casting — with no tsc pass. Add a `tsconfig.test.json` per client that includes `__tests__` with the test-exclusions dropped and the test-only path aliases (`@modelcontextprotocol/inspector-test-server`, the `@inspector/core/*` deep paths, express/vitest) resolving what vitest resolves, then fold `tsc -p tsconfig.test.json` into each `typecheck`. Fixes the 76 errors it surfaced: - cli: a wrong `@modelcontextprotocol/inspector-core/...` module specifier (should be `@inspector/core/...`; only a type import so vitest never caught it), `r.headers` optional-access guards, and partial OAuth mocks replaced with typed `makeFakeCliOAuthClient()` / `makeFakeServerSettings()` factories instead of `as` casts. Adds `@types/express` (devDep) so the transitively aliased test-server source typechecks, mirroring clients/web. - tui: rest-param signatures on the App OAuth spies (fixes the spread-arg and narrow-return errors), a widened callback param (`iss?`), a `makeFakeServer- Settings()` factory for the tuiOAuth tests, required OAuth-metadata/client fields in AuthTab, and a loosened `entry()` message param for the deliberately-malformed wire messages HistoryTab's defensive branches render. Gate coverage only; no runtime behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test(cli,tui): address review — real spy signatures, config typecheck, nits Responds to the @claude review of #1799: - App.test.tsx: type the OAuth spies against the real InspectorClient method signatures (`vi.fn<InspectorClient["…"]>()`) and forward `Parameters<…>` tuples from the FakeClient wrappers, so `toHaveBeenCalledWith(...)` on the six OAuth-lifecycle spies stays argument-checked under the new gate rather than accepting anything via `unknown[]`. `getOAuthState` regains its wide `OAuthConnectionState | undefined` return. - Typecheck the client config files too: add `vitest.config.ts` / `tsup.config.ts` (and tui `dev.ts`) to each src `tsconfig.json` include — they were the one first-party surface still getting no tsc pass. - Trim the redundant `src/**/*` from each `tsconfig.test.json` include (tsc pulls in the src the tests import; the src-only config already checks it). - Rename the tui settings helper to `oauth-test-fakes.ts` to match cli's, and soften both factories' docstrings to stop naming a loader they don't mirror field-for-field. - Fix a stale `@modelcontextprotocol/inspector-core/...` specifier in a createRemoteTransport JSDoc example (the same wrong package name the gate caught in the cli tests, surviving in a comment). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test: address round-2 review — correct claims, tighten types, typecheck launcher Responds to the round-2 @claude review of #1799: - Correct the toHaveBeenCalledWith claim: vitest doesn't type-check `toHaveBeenCalledWith(...)` arguments against the mock's signature, so the App.test.tsx comment no longer claims it does — the typed spies buy checked implementations/return payloads (the stale-literal fix), not arg-checking. - authenticate spy now uses the real `InspectorClient["authenticate"]` signature (returns a `URL`), so it no longer diverges from production; the one mockResolvedValue is updated to a URL. - makeFakeCliOAuthClient members are typed against the real CliOAuthClient signatures (not bare vi.fn()), so a wrong-signature override is rejected. - HistoryTab: revert the `entry()` `message: unknown` widening (it un-checked the ~15 well-formed fixtures) and instead scope a named `MALFORMED` cast to the two intentionally-malformed messages. - Typecheck the launcher `__tests__` too (the same gate hole in the one client the PR hadn't touched): new clients/launcher/tsconfig.test.json + a typecheck script folded into validate. Completes the "every client's __tests__ is typechecked" invariant. - Doc the new test-typecheck + config-file typecheck in AGENTS.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test: address round-3 nits — launcher config typecheck, callTool typed, docs Responds to the round-3 @claude review of #1799 (all non-blocking): - launcher: add `vitest.config.ts` to tsconfig.test.json's include (it was in neither project — the build config's rootDir: ./src rejects it). Now typechecked. - App.test.tsx: type `callTool` against `InspectorClient["callTool"]` (+ the Parameters<…> wrapper) so its `mockResolvedValue` payloads are checked, and give `clearOAuthTokens` an `async () => {}` impl so the mock actually returns the `Promise<void>` its type promises. - AGENTS.md: correct the per-client alias-set description (cli's is widest, tui's is core+react only, launcher's has no paths) and note launcher's config file lives in its test project, not the src project. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test: address round-4 — durable typecheck-coverage guard + launcher/doc nits Responds to the round-4 @claude review of #1799 (all non-blocking): - Add `verify:typecheck-coverage` (scripts/verify-typecheck-coverage.mjs), the typecheck-coverage analog of verify:format-coverage and the durable guard for the invariant this PR establishes: for each Node client it runs the tsconfig projects its `typecheck` names with `tsc --listFiles`, unions them, and fails on any tracked .ts/.tsx that lands in no project. This closes the class the review kept surfacing by hand (launcher/__tests__ in round 3, launcher/vitest.config.ts in round 4) — a new top-level file can no longer go untypechecked silently. Wired into `npm run ci` and the GitHub workflow. - launcher: reorder `validate` to run `typecheck` before `build` (matches tui and the documented order), and make `typecheck` self-contained — it now runs both `tsconfig.json` and `tsconfig.test.json`, so running it standalone means the same thing it does in cli/tui. - makeFakeCliOAuthClient: note on the comment that the typing covers the defaults, not `overrides` (a bare vi.fn() override is Mock<(...args)=>any>). - Docs: README table + AGENTS.md updated for the new guard and the corrected launcher typecheck/validate ordering. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test: address round-5 — wire-check the guard, --listFilesOnly, into validate Responds to the round-5 @claude review of #1799: - Finding 1 (the pre-merge one): the guard now ports verify:format-coverage's reachability checks — it asserts each client's `typecheck` is reachable from its own `validate` and that the root chain runs each client's `validate`, so it can't stay green while measuring a typecheck nothing invokes. Verified: stripping `npm run typecheck` from launcher's validate now fails the guard. - Nit 3: switch `tsc --listFiles` → `tsc --listFilesOnly` (identical file list, ~4× faster: whole guard 23s → ~4s). Now cheap enough for the inner loop, so it moves into `validate` (right after verify:format-coverage) instead of a standalone ci step — matching its sibling. Removed the separate ci-chain entry and GitHub workflow step (validate covers it). - Nit 4: `npx --no-install tsc` so a missing local tsc fails fast instead of reaching for the registry (matches verify-build-gate.mjs). - Nit 5: corrected projectFiles' JSDoc to describe the filter that's actually there (drops repo-external + node_modules paths; core/ + test-servers/ stay but are harmless). - Finding 2: fixed the 2-space misindent on the new root package.json line (prettier --write). Docs (README/AGENTS) updated for the validate placement. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): address round-6 guard nits — dedupe wiring, robust diagnostics Responds to the round-6 @claude review of #1799 (all guard-ergonomics nits): - Nit 2: extract the shared reachability logic (`reachableScripts`, `rootRunsClientValidate`) into scripts/lib/npm-scripts.mjs and consume it from BOTH verify-format-coverage.mjs and verify-typecheck-coverage.mjs, so the "is this gate actually run?" wiring can't drift between them (same rationale as scripts/lib/prod-web-server.mjs). - Nit 1: typecheckProjects now harvests `-p`/`--project` from every script reachable from `typecheck` (not the single string), so a delegating `typecheck` (`npm run typecheck:src && …`) is handled instead of failing with a false "names no -p project" diagnosis. Also accepts `--project`. - Nit 3: on a tsc config error, echo the `error TS…` diagnostic (scanning both streams — tsc prints config errors to stdout) so a broken tsconfig is self-explaining rather than surfacing only as a misleading "file in no project" report. - Nit 4: refreshed the CI workflow's stale "Validate" step name/comment (it now runs the coverage guards + validate:core + per-client typecheck). - Nit 5: reworded the `.d.ts` carve-out comment to read as pre-emptive (there are none under cli/tui/launcher today). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-7 — guard .mts/.cts too; drop dead export Responds to the round-7 @claude review of #1799: - Finding 1: verify-typecheck-coverage now requires a tsc pass for tracked `.mts`/`.cts` (not just `.ts`/`.tsx`), matching verify-format-coverage's extension set — `.mts` is already the shared-config idiom here, so a client `vitest.config.ts` → `.mts` rename can no longer drop out of both the tsconfig include and the guard's required set. No-op on today's tree (no client `.mts`/`.cts`); confirmed a tracked `.mts` with a type error is now caught. Ambient `.d.{ts,mts,cts}` stay excluded. - Nit 2: drop the unused `export` on `rootReachedCommands` (internal to the lib; nothing imports it, and scripts/ isn't eslint-gated to catch dead exports). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test: round-8 — single casts in HistoryTab, uniform wrappers, doc scope Responds to the round-8 @claude review of #1799: - Finding 1: reduce the `MALFORMED` helper and `entry()`'s return cast from `as unknown as` to a single `as` (both compile clean — the malformed shapes are comparable to the JSONRPCMessage union), and delete the false "only option" sentence. Fixes the AGENTS.md avoid-double-casts violation. - Nit 3: write all seven FakeClient OAuth wrappers as `(...a: Parameters<InspectorClient["…"]>)` (the three zero-arg ones were hand-written `()` forwarders), so the block comment's "forward the same Parameters<…> tuple" is literally true and a future added parameter can't be silently dropped. - Nit 2: update the five places that still described the guard as `.ts`/`.tsx` only to `.ts`/`.tsx`/`.mts`/`.cts` (script header + trackedSourceFiles JSDoc, AGENTS.md ×2, README.md). Nits 4 (js config files out of the typecheck guard by construction) and the footnoted callTool/ancestor-tsc items left as agreed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test: round-9 — drop entry()'s harmful cast; guards vouch for each other Responds to the round-9 @claude review of #1799: - Finding 1: remove `entry()`'s `as MessageEntry` return cast in HistoryTab.test.tsx. It was unnecessary (the spread of Partial<MessageEntry> over complete defaults is directly assignable) and harmful — it silenced type errors in the helper's OWN default fields (a `direction`/`method` typo now surfaces TS2820/TS2353). MALFORMED's single cast stays (load-bearing). This leaves MALFORMED as the file's only cast, matching what the comments describe. - Nit 2: add `rootReachesScript` to scripts/lib/npm-scripts.mjs and have each coverage guard assert its SIBLING is still wired into the root `validate`. A guard can't detect being unrun itself, but the two now vouch for each other, so dropping either from `validate` is caught by the other (only deleting both slips through). Verified: removing verify:typecheck-coverage from validate now fails verify:format-coverage, and vice-versa. Nits 3 (git-index visibility) and 4 (pre-existing as-never/as-unknown casts in tests — out of scope, none added by this PR) left as agreed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test: round-10 — reject non-checking typecheck passes; pre/post hooks; docs Responds to the round-10 @claude review of #1799: - Finding 1: verify-typecheck-coverage now rejects a `-p <project>` whose own command carries `--noCheck` or `--listFilesOnly` — such a pass lists files without type-checking them, which previously satisfied both the wiring and file-coverage halves while checking nothing. (Splits each script on `&&`/`||`/`;` so a flag on one command can't taint another.) Verified a `typecheck` using `tsc … --noCheck` now fails with a clear message; `--noCheck` is a live idiom here (cli's test-servers:build), so this is a real copy-paste hazard. - Nit 2: reachableScripts (shared lib) now also follows npm's implicit `pre<name>`/`post<name>` lifecycle hooks, so moving a gate into e.g. `prevalidate` is recognized instead of producing a false "not reachable" failure (cli already uses a `pretest` hook). Verified. - Nit 3: AGENTS.md "Test placement" now states the Node clients (cli/tui/ launcher) keep all tests in `__tests__/`, enforced by verify:typecheck-coverage (a co-located src test lands in no tsconfig project). - Nit 4: noted on the factory comments that `Partial<…>` overrides admit an explicit `undefined` for a required field (exactOptionalPropertyTypes off). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * fix: round-11 — launcher build excludes tests; guard catches all noCheck vectors Responds to the round-11 @claude review of #1799: - Finding 1 (shipping bug): clients/launcher/tsconfig.json now excludes `**/*.test.{ts,tsx}` (matching cli/tui). launcher's `build` is an EMITTING tsc over src/**, and clients/launcher/build is in the root `files` allowlist, so a co-located `src/**/*.test.ts` would have been compiled into the published tarball. Excluding it also makes such a test land in no tsconfig project, so verify:typecheck-coverage catches it — making the AGENTS.md placement rule literally true for all three clients. - Finding 2: verify-typecheck-coverage now catches both remaining ways a typecheck could check nothing: (a) case-insensitive `--noCheck`/`--nocheck`/ `--listFilesOnly` on the command, and (b) `noCheck` set in the tsconfig itself (detected via `tsc --showConfig`, which surfaces it from the config or its extends chain). Verified both fail the guard. - Nits 3 & 4: restructured into an explicit gate-integrity phase (wiring + neutered + config-noCheck + no-projects) that reports under its own accurate header and exits BEFORE the file-coverage pass — so an inert gate no longer prints file-remediation advice, nor buries the real cause under an "in no tsconfig project" line for every file that gate covered. The one CIMD OAuth E2E integration test that flaked in the coverage run passes 32/32 in isolation and on re-run; unrelated to these script/config changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * fix: round-12 — exclude .mts/.cts tests from build; drop false integrity line Responds to the round-12 @claude review of #1799: - Finding 1 (tarball, again): the round-11 test exclude only covered `.ts`/`.tsx`, so a co-located `src/*.test.mts` still compiled into launcher's shipped `build/`. All three clients now exclude `**/*.test.*` (one glob, covering .ts/.tsx/.mts/.cts) — which is also exactly what AGENTS.md already claims. Verified a `src/probe.test.mts` is no longer emitted and now fails the guard. - Finding 2: the gate-integrity phase no longer prints a false "names no `-p <project>`" line when config-`noCheck` disables projects — the condition now keys off `projects.length` (harvested), not `checking.length` (post- filter), so a client whose projects were all config-disabled shows only the accurate per-project lines. - Nit 3: recorded the boundary on the phase comment — it does not detect shell-level failure suppression (`… || true`, `; exit 0`). - Nit 5: the file-coverage remediation now leads with "for a co-located test, move it to `__tests__/`" (adding it to the src `include` would reintroduce the build emit) before the general add-to-include advice. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-13 — implicit-tsconfig + derived clients; diagnostics Responds to the round-13 @claude review of #1799: - Finding 1: verify-typecheck-coverage now understands a `tsc` command with no `-p`/`-b` project flag — it resolves the implicit `./tsconfig.json` (tsc's own default), so the idiomatic `tsc --noEmit` form counts as coverage instead of producing false "in no tsconfig project" failures. Also harvests `-b`/`--build` project paths. Verified `tsc --noEmit && tsc --noEmit -p tsconfig.test.json` now reports OK. - Finding 2: CLIENTS is no longer hardcoded — it's derived from the `cd clients/<x> && npm run validate` entries in the root `validate` chain (minus an explicit web exclusion), so a new Node client is auto-required, matching the repo-wide reach of its sibling verify-format-coverage. Added a guard-would-check-nothing bail if the derivation yields an empty set. Verified a new `clients/newclient` is picked up and its uncovered file flagged. - Nit 3: the phase-2 remediation now leads with the general add-to-`include` advice and only appends the "move it to __tests__/" note when a failing path is actually a test file (was misdirecting plain files). - Nit 4: AGENTS.md no longer says the test config includes "only" __tests__ (launcher's also includes its vitest.config.ts). (Two different web integration tests each flaked once under parallel load during the coverage runs — both pass in isolation and on a clean re-run; timing- sensitive and unrelated to these script/doc-only changes.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-14 — enumerate clients from disk (fail-closed) Responds to the round-14 @claude review of #1799: - Finding 1: replace the root-chain `CLIENTS` derivation (round 13) with a disk enumeration — every `clients/<name>` whose package.json has a `typecheck` script. The chain derivation was fail-open: a root-chain edit that kept running a client but didn't match the strict regex silently dropped it from the required set (112→53 files, still green). Disk is fail-closed — a client can't be silently dropped, a new client with a typecheck is auto-required, and web (no typecheck script; builds via `tsc -b`) is excluded on its own, so the explicit exclusion is gone. The existing `rootRunsClientValidate` wiring check still verifies the root chain runs each — now LOUD when it doesn't. Verified: `cd ./clients/tui && …` now hard-fails the integrity phase instead of silently shrinking coverage; `cd clients/tui && npm run build && npm run validate` stays gated. Removed the now-unused `clientsRunByRootValidate` from the shared lib. - Nits 2 & 3: documented the two unreachable-today limitations on typecheckProjects — a solution-style `tsc -b` (empty `files` + `references`) would under-report, and the implicit-tsconfig fallback assumes no file operands. - Nit 4: refreshed the file header to mention `-b`/`--build` + the implicit `./tsconfig.json` (only the JSDoc had been updated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-15 — enroll-or-exempt enumeration; normalize `cd ./` Responds to the round-15 @claude review of #1799: - Finding 1: the disk enumeration keyed on the presence of a script named `typecheck`, so renaming it silently dropped the client (112→53, still green) — contradicting the docstring. Now every `clients/*` with a readable package.json must EITHER declare a `typecheck` (→ enrolled) OR be in an explicit EXEMPT map (`clients/web`, reason: `tsc -b`); anything else is a hard gate-integrity failure. This is fail-closed on the rename axis too, and web's skip is a stated decision rather than an accident of which scripts it declares — so web gaining a `typecheck` no longer auto-enrolls it into the solution-`-b` false failures. Verified: renaming tui's `typecheck` now hard-fails; a `tsc -b` on web stays exempt. - Nit 3: `rootRunsClientValidate` normalizes a leading `./`, so `cd ./clients/tui && npm run validate` (which genuinely runs it) is no longer a false "no longer runs" alarm. - Nit 2: README/AGENTS/script-header prose no longer names a fixed cli/tui/launcher scope — they describe the disk-discovered "every clients/* with a typecheck script (web exempt)" rule. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-16 — anchor wiring match; close enumeration axes Responds to the round-16 @claude review of #1799: - Finding 1 (only false-PASS): rootRunsClientValidate matched the client dir as a bare substring, so a prefix-sibling (`clients/tui` vs `clients/tui-next`) satisfied the wiring check for the shorter name — dropping a client from the root chain went green if a sibling was present. Now an anchored regex (escaped dir + optional leading `./` + a trailing boundary). Verified the sibling case now hard-fails. - Nit 2: a `clients/*` dir holding tracked TS but no readable package.json is now a gate-integrity failure (was treated as "not a client"), so the "fail-closed on every axis" claim holds. Verified. - Nit 3: readdirSync is wrapped, so a missing `clients/` dir yields the intended "found no clients/* dir" message rather than a raw ENOENT stack. - Nit 4: EXEMPT's reason is now surfaced in the success line ("clients/web exempt: typechecks via tsc -b …"), so the exemption is visible at runtime, not only in source. (Also moved `isRequiredSource` above nodeClients — nit 2's new tracked-TS check calls trackedSourceFiles at module-init, which referenced it before init.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-17 — share quote-aware tokenizer between guards Responds to the round-17 @claude review of #1799: - Finding 1: the two coverage guards parsed shell strings differently — verify-format-coverage had a quote-aware tokenize(); verify-typecheck-coverage used a raw regex. So a quoted `-p "tsconfig.test.json"` (a form tsc accepts) made the typecheck guard report every file that project covers as uncovered, and a quoted `cd "clients/tui"` in the root chain tripped a false wiring failure. Moved tokenize() (now single+double quote) into the shared lib, imported it into both guards, rewrote typecheckProjects to walk tokens (`-p`/`--project`/`-b`/`--build` → next token), and strip quotes before the `cd` match in rootRunsClientValidate. Verified both quoted forms now pass. - Nit 2: a stale EXEMPT key (a dir that no longer exists) was asserted in the success line unchecked. nodeClients now fails if an EXEMPT key isn't a real `clients/*` directory. Verified. - Nit 3 (build/dist filter belt-and-braces) left as a recorded boundary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-18 — match tsc by basename; strip quotes on both halves Responds to the round-18 @claude review of #1799: - Finding 1 (regression from round 17): `tokens.includes("tsc")` didn't match a path-invoked binary (`node_modules/.bin/tsc`), which the previous `\btsc\b` regex did — so such a `typecheck` was falsely reported as naming no project. Now matches on the token basename (`isTsc`), keeping round-17's quoting fix while restoring the tolerance. Verified a `node_modules/.bin/tsc` typecheck reports OK. - Nit 2: `rootRunsClientValidate` stripped quotes for the `cd` match but tested `npm run validate` against the unstripped string, so `npm run "validate"` false-failed. Both halves now test the stripped string. - Nits 3 & 4: documented on typecheckProjects — the contrived `--noCheck false` (checking on) is still treated as disabling, and the `&&`/`||`/`;` split runs before tokenizing (a quoted operator inside an arg would split mid-token, unreachable for project paths). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test: round-19 — typecheck test-servers' unimported bin; --prefix wiring form Responds to the round-19 @claude review of #1799: - Finding 1: `test-servers/src/server-composable.ts` (a `#!/usr/bin/env node` bin entry nothing imports) got no tsc pass — the barrel alias covers the other 13 test-servers/src files transitively, but not this one, and its own build runs `tsc --noCheck`, so its errors were emitted into `test-servers/build` unchecked. Added it explicitly to `clients/cli/tsconfig.test.json`'s include (zero source changes; verified a bad line in it now fails cli's typecheck). Softened the AGENTS.md clause that implied the whole test-server source is transitively covered. - Nit 2: `rootRunsClientValidate` now also recognizes the `npm --prefix <dir> run validate` invocation form (was `cd`-only). - Nit 3 (stderr spy drops the completion-callback overload): left as-is — latent, not live (no exercised path awaits a stderr drain), and adding typed overload params would reintroduce the write-overload complexity the round-1 vi.spyOn refactor removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-20 — anchor validate tail; guard shared-source class Responds to the round-20 @claude review of #1799: - Finding 1 (false PASS): the wiring matcher's tail `/\brun validate\b/` also matched `run validate:fast` (`:` is a word boundary) — a DIFFERENT script that may skip typecheck. So the root chain could run a client's `validate:fast` while the guard kept measuring its `validate` (unrun), green. Anchored the tail to a boundary (`/\brun validate(?=$|[\s&;])/`). Verified `validate:fast` now hard-fails. - Finding 2: durably close the test-servers/src class instead of point-fixing. Added a SHARED_ROOTS check — every tracked TS under `test-servers/src` + the root `vitest.shared.mts` (the `format:check:shared`/`lint:shared` surface) must land in the GLOBAL union of client projects. So a NEW unimported bin entry is caught (not just server-composable.ts, named in round 19). Verified a fresh `server-orphan-bin.ts` now fails. Zero source changes — the union already covers 15/15. - Nit 4: the integrity message and rootRunsClientValidate docstring now mention the `npm --prefix` form too. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-21 — deny-by-default non-client coverage Responds to the round-21 @claude review of #1799 (findings 1 & 2, one fix): - Replaced the `SHARED_ROOTS` allowlist with a deny-by-default non-client set: all tracked TS minus `clients/*` (per-client pass) minus a reasoned `core/` exemption (covered by web's `tsc -b`). This closes BOTH round-21 findings at once: - Finding 1 (stale allowlist fail-open): a rename of `test-servers/src` no longer silently drops it — its files reappear in the required "other" set and hard-fail (verified: `git mv test-servers/src test-servers/source` now fails instead of going 127→113 green). The subtractive `core/` exemption is fail-closed on staleness for the same reason. - Finding 2 (allowlist vs deny-by-default): a new top-level TS location is now auto-required (verified `tools/probe.ts` is flagged), matching the repo-wide reach of the sibling verify-format-coverage. - Nit 3: added the shared-source clause to the README table row (AGENTS got it last round). Header comment updated to describe the non-client coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-22 — narrow core/ exemption to what web's tsc -b covers Responds to the round-22 @claude review of #1799: - Finding 1: the `core/` exemption was whole-tree, but web's `tsc -b` only covers `core/**/*.ts` MINUS co-located `*.test.ts(x)` (its app project's include). So a `core/*.tsx`/`*.mts`/`*.cts` or a co-located `core` test got no tsc pass from anything. Narrowed the exemption to exactly that shape (`isExemptCoreFile`): a core `.ts` that isn't a co-located test. So a core `.tsx`/`.mts`/co-located test is now required in some project. Zero-fix today (0 such files); verified `core/mcp/probe-widget.tsx` and a co-located core `.test.ts` are now flagged. - Nit 2: `EXEMPT_ROOTS` is now a Map (dir → reason) and its reason is surfaced in the success line alongside the client exemption. - Nit 3: the non-client miss message is now the general "in no client's tsconfig project"; the `test-servers` remediation clause is conditional on an actual `test-servers/` path (tracked via `nonClientMisses`, not string-sniffing the composed message). - Nit 4: success line says "(N clients + non-client)". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-23 — core exemption mirrors both web projects Responds to the round-23 @claude review of #1799: - Finding 1: round 22's `isExemptCoreFile` mirrored only web's app project, but web's `tsc -b` also builds `tsconfig.test.json`, which includes `core/**/__tests__/**/*.{ts,tsx}`. So a `core/**/__tests__/*.test.ts` (a live layout — `core/mcp/__tests__/fakeInspectorClient.ts` already sits there) was flagged though web checks it. Now mirrors the UNION: under `__tests__/` → exempt `.ts`/`.tsx`; otherwise `.ts` minus co-located `*.test.ts(x)`. Verified a core `__tests__` test now passes while a core `.tsx` outside `__tests__` stays required. - Finding 2: `EXEMPT_ROOTS` was a Map whose keys nothing read (round 22 replaced its filter with hardcoded `isExemptCoreFile`) — a phantom registry. Collapsed to a single `CORE_EXEMPT_REASON` const used in the OK line. - Nit 3: README/AGENTS scope descriptions updated (core is no longer wholesale excluded; the non-client set is deny-by-default). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-24 — enroll clients/web via tsconfig references Responds to the round-24 @claude review of #1799: - Finding 1: `clients/web` was whole-tree EXEMPT, but web's `tsc -b` only reaches what its four referenced projects include — a file at web's package root (e.g. `playwright.config.ts`) got no tsc pass and passed EVERY gate, fully silently. This was the last whole-tree exemption wider than the gate it defers to. Now web is ENROLLED: a client with no `typecheck` script but a `tsconfig.json` `references` array is measured through those reference projects (`tsc -p <ref> --listFilesOnly`), and is wired iff its `validate` runs `tsc -b` (`validateRunsTscBuild`). EXEMPT is now the empty escape hatch. Verified: a web-root `playwright.config.ts` with a type error is now flagged, and dropping `tsc -b` from web's build hard-fails the integrity phase. Zero-fix on the tree — the union of web's 4 projects covers all 625 tracked web files. - Nit 2: assert `core/` exists (the OK line unconditionally states its exemption) — parity with the EXEMPT stale-key check. - Nit 3: a `core/` miss now gets core-specific remediation (widen web's `tsconfig.app.json` include, not a cli/tui project). - Nit 4: README/AGENTS scope descriptions corrected — web is enrolled (not exempt), and the exact core-exempt shape (app `*.ts` minus co-located tests + test `__tests__/**/*.{ts,tsx}`). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-25 — drop redundant core exemption; guard tsc -b --noCheck Responds to the round-25 @claude review of #1799: - Finding 1: enrolling web (round 24) made the hand-maintained `core/` exemption both redundant and fail-open — web's measured `tsc -b` projects already cover core, and the mirror could go stale (narrowing web's include let an unimported core error pass). Deleted `isExemptCoreFile`, `CORE_EXEMPT_REASON`, the core presence assertion, and the exemptNote core clause. `core/` is now required like everything else and covered by measurement (127→894 files; a `core/*.tsx` web doesn't reach is now caught by measurement, not a hand-model). Kept the core-specific remediation clause. - Finding 2: the reference (`tsc -b`) path skipped the non-inertness check — a `tsc -b --noCheck` build gated nothing while the guard stayed green. Turned `validateRunsTscBuild` into `tscBuildStatus` (ok/neutered/none); a neutered `tsc -b` now hard-fails, matching the typecheck-script path. Factored the disabling-flag predicate (`isDisablingFlag`) so both paths share it. - Nit 3: `clientTsconfigReferences` now strips block comments too (every other tsconfig in the repo uses `/* */`). - Nit 5: README/AGENTS wiring clause covers web's `tsc -b` (not just the `typecheck` script), and core is described as measured, not exempt. Nit 4 resolves itself: EXEMPT is empty and the exemptNote/ternary now collapse cleanly. Nit 6 (tsc -b bound to a specific tsconfig) left as recorded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-26 — expand solution configs wherever harvested Responds to the round-26 @claude review of #1799: - Finding 1: the reference (`tsc -b`) path was selected only by "no typecheck script", so adding `typecheck: "tsc -b"` to web (the natural uniformity move) flipped it to the typecheck-script path, where the harvested solution config lists nothing → 665 false failures. Fixed at the measurement layer: projectFiles now expands a solution config (`{"files":[], "references":[…]}` — lists nothing under --listFilesOnly) to its references and measures those recursively, so a `tsc -b` project is measured no matter which path harvests it. Generalized the references reader (tsconfigReferences(path)). Verified web with `typecheck: "tsc -b"` now passes. Dropped the stale "web … is out of scope" JSDoc clause. - Nit 2: a `core/**/__tests__/**` miss now points at `tsconfig.test.json` (its actual owner), not `tsconfig.app.json`. - Nit 4: AGENTS.md's one-line summary now mentions the non-client (`core/`, `test-servers/src`) coverage, matching the detailed bullet. Nit 3 (EXEMPT empty, dormant stale-key check) left as the intended escape hatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-27 — non-inertness follows references on both paths Responds to the round-27 @claude review of #1799: - Finding 1: round 26 made COVERAGE follow a solution config's references, but the non-inertness (`noCheck`) check on the typecheck-script path still only inspected the harvested solution (empty compilerOptions) — so a `noCheck` in a REFERENCED project was invisible there (caught only on the reference path). Factored a shared `resolveLeafProjects` (a solution → its real leaf projects, cached rawProjectFiles) used by BOTH coverage (`projectFiles`) and a shared `checkingLeaves` disable-check, so a `noCheck` in a referenced project is now caught on either path. Verified: `typecheck: "tsc -b"` + `noCheck` in tsconfig.app.json now hard-fails (was green). Caching also cut the guard from ~20s to ~11s. - Nit 2: directory-form references (`{ "path": "./packages/a" }`) now resolve to `<dir>/tsconfig.json`, so a nested-solution dir reference is expanded. - Nit 3: reworded the empty-listFiles comment (it's "solution config OR errored config", both of which the reference expansion handles). - Nit 4: script header notes a `tsc -b` solution is reduced to its references on either enrollment path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-28 — dedup shared leaf in checkingLeaves Responds to the round-28 @claude review of #1799 (which found no new gate hole): - Finding 1: `checkingLeaves` dedup'd the `checking` array but ran projectDisablesChecking + integrity.push on every visit, so a leaf reachable from two harvested projects produced a duplicate `noCheck` line (and a redundant `tsc --showConfig`). Now tracks visited leaves in a Set and skips an already-seen leaf entirely. Verified a duplicate harvested project yields one line, not two. - Nit 3 (record-completeness): noted in AGENTS that the one tier the guard structurally can't see — a per-file `// @ts-nocheck` — is owned by `@typescript-eslint/ban-ts-comment` across lint:core/lint:shared/each client. Nit 2 (the ~11s perf claim isn't reproducible on a CI-class runner) and nit 4 (hybrid-solution / composite boundaries, both tsc-rejected → unreachable) left as recorded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test(scripts): unit-test the typecheck-coverage guard's parsers Responds to the round-29 @claude review of #1799 (no new gate hole; the finding is that the guard itself has no tests): - Made the guard's pure helpers importable: exported `typecheckProjects`, `tscBuildStatus`, `tsconfigReferences`, `clientTsconfigReferences`, `resolveLeafProjects`, `isTsc`, `isDisablingFlag`, `isRequiredSource`, and moved the module-init + phase execution behind an exported `main()` run only when the file is executed directly (importing it for tests no longer runs the guard). Behavior when run directly is unchanged (894 files). - Added table-driven tests (node's built-in `node --test`; the root has no vitest harness by design) for the shared lib (`tokenize`, `reachableScripts` incl. pre/post hooks, `rootRunsClientValidate` across cd/--prefix/quoted/ prefix-sibling/`validate:fast`, `rootReachesScript`) and the guard's parsers (`isRequiredSource`, `isTsc`, `isDisablingFlag`, `typecheckProjects`, `tscBuildStatus`) — one case per rule a review round found (r7/r10/r13/r15/ r16/r17/r18/r19/r20/r25). 12 tests. Wired `test:scripts` into `validate`. - Nit 2: a `scripts/*.ts` miss now gets a scripts-specific remediation (keep it `.mjs`, or give `scripts/` its own tsconfig project). - Documented the guard's test suite in AGENTS.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * test(scripts): make test:scripts self-guarding; fill parser test gaps Responds to the round-30 @claude review of #1799 (mutation testing killed 5/5; findings are about the new test gate + suite completeness): - Finding 1: `test:scripts` lacked the two properties the guard enforces for every tsc pass. verify-typecheck-coverage now asserts, in phase 1, that `test:scripts` is reachable from the root `validate` (via the already-imported rootReachesScript) AND that its tracked `scripts/**/*.{test,spec}.*` file set is non-empty and every file is matched by the command's glob (a new exported `globToRegExp`). So dropping it from validate, or renaming a test to `*.spec.mjs`/`*.test.mts` (which node --test silently skips), now hard-fails. Verified both. - Nit 2: extracted a pure `parseTsconfigReferences(raw)` from tsconfigReferences and table-tested the JSONC tolerance (block/line comments, trailing comma, no references, malformed, no path) — the r17/r26 rules had no cover. - Nit 3: added `-b`/`--build`/bare-`tsc -b` and explicit `--project` cases to typecheckProjects's test so its title matches its table. - Nit 4: rewrote the rootRunsClientValidate test to put `cmd` in a script reached from validate, and added the reachability cases (an orphan cd-validate must NOT count; validate:tui indirection must) — the r5 property. - Nit 5: added test:scripts to AGENTS's validate ordering + the README table. 15 guard tests now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * chore(scripts): round-31 — glob harvest follows delegation; brace/? globs Round-31 review findings, both false failures in the new `test:scripts` self-guard: 1. The glob half read `rootPkg.scripts["test:scripts"]` literally while the wiring half two lines above used `reachableScripts`, so a delegating `test:scripts` (`npm run test:scripts:lib && npm run test:scripts:guard`) — which genuinely runs all 15 tests — reported every tracked test file as "not matched by the glob", advising a rename the files already satisfy. Now harvested across every script reachable FROM `test:scripts`, the same fix round 6 applied to the `typecheck` harvest; `pretest:scripts` counts for free. 2. `globToRegExp` escaped `{`/`}` as literals and left `?` as a regex quantifier, both divergent from `node --test`'s glob. `*.{test,spec}.mjs` is the natural widening here (the guard's own file scan already treats a `.spec.` file as a test) and false-failed; `?` silently meant "optional previous char" and threw outright when segment-leading. Adds `?` → `[^/]` and a brace-alternation pass, with an unbalanced brace falling back to a literal. Nit 3: `resolveLeafProjects`'s two path-resolution rules (the r26 directory-form reference, and refs resolving against the referring config's dir) extracted as `projectConfigFile`/`refToProject` so they are one-row tables like the other parsers, rather than untestable behind a `tsc` spawn. Nit 4: AGENTS.md + README document that the guard now enforces `test:scripts` on three axes (wired, non-empty, glob-covered). Verified: probes C and D exit 0; probe B (rename to `*.spec.mjs`) still caught; `test:scripts` 18/18; `verify:typecheck-coverage` OK 894 files; `verify:format-coverage` OK 919; `npm run ci` green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vgJ6ZYj9VLhfSjgua48zw * chore(scripts): round-32 — measure globs with node's own matcher Round-32 review; both findings point at deleting code. 1. `globToRegExp` hand-modeled `node --test`'s glob and was still one form short — character classes (`*[.]test.mjs`, `*.test.[mc]js`) hit the escape set and became literals, so the guard false-failed on files node genuinely runs. Replaced the whole translator (and its brace balance pre-scan) with node's `path.matchesGlob` — 22.5.0+, and the repo floor is >=22.19.0. Same *measure, don't model* move round 25 made when it deleted `isExemptCoreFile` in favor of measuring web's projects; being exact for every form node accepts is the point, since node's matcher is the thing being modeled. 2. The round-31 glob harvest was inline in `main()`, so mutation-reverting it to the literal single-script read left the suite green — the commit's headline fix was its one untested change. Extracted as `testScriptGlobs`, with rows for the direct form, the delegating form (both child globs, not `npm`/`run`/`<name>`), a `pretest:scripts` hook, and an unreachable script contributing nothing. Nit 3: `resolveLeafProjects`' JSDoc moved back onto it (the extraction left it stranded above `projectConfigFile`, which had two stacked doc blocks). Nit 4a: a `test:scripts` naming no path/glob harvested nothing, and `.some()` over an empty list blamed every test file with unfollowable advice. Now one explicit message naming the real problem. Verified: probes for the direct, delegating, brace, `[.]` and `[mc]` forms all exit 0 (the last two were false failures before); probe B (rename to `*.spec.mjs`) still caught; bare `node --test` reports the new single message; `test:scripts` 18/18; `verify:typecheck-coverage` OK 894; `npm run ci` green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vgJ6ZYj9VLhfSjgua48zw * chore(scripts): round-33 — scope the glob harvest to `node --test` segments 1. False pass. `testScriptGlobs` kept every non-flag token from every reachable script, so a glob belonging to a NEIGHBOURING command was attributed to the test runner — and since `.some()` needs only one glob to match, a broader one suppressed a real miss rather than adding an inert pattern. A `pretest:scripts` of `prettier --check "scripts/**/*.mjs"` (reachable via the pre-hook round 31 added) was enough: the probe-B rename to `*.spec.mjs` then passed while `node --test` silently ran 12 of 18 tests. Now scoped per command segment to the ones invoking `--test`, exactly as `typecheckProjects` gates on `tokens.some(isTsc)`. The JSDoc paragraph claiming a surplus token can't suppress a miss is corrected — it was false. 2. The gate-integrity block is extracted as `testScriptProblems(scripts, testFiles)`, pure and outside `main()`. The nit-4a branch added last round was the third consecutive commit whose newest branch sat outside the test seam and mutation-survived; extracting the block ends the pattern rather than pinning one more condition from the outside. Verified. The finding-1 probe (prettier pre-hook + `*.spec.mjs` rename) now exits 1 where it exited 0; the hook alone still exits 0. Mutation testing over the whole helper — empty-harvest branch to `if (false)`, segment gate removed, wiring axis removed, non-empty axis removed, segment split removed, `reachableScripts` to `Object.keys`, `matchesTestGlob` to equality — kills 7/7, including the branch that survived the last three rounds. `test:scripts` 21/21 · `verify:typecheck-coverage` OK 894 · `verify:format-coverage` OK 919 · `npm run ci` green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vgJ6ZYj9VLhfSjgua48zw * chore(scripts): round-34 — harvest only positional args; normalize `./` Both findings are the axis round 33 fixed, one level in. 1. False pass. The segment gate stopped a neighbouring COMMAND's glob, but within a kept segment the filter was still "any non-flag token", so a glob-valued flag's VALUE was harvested as a positional arg. `--test-coverage-include "scripts/**/*.mjs"` is broader than the test glob and matches a renamed `*.spec.mjs`, so it vouched for the very file the runner skips — and it isn't hypothetical: `scripts/` sits outside the ≥90 coverage gate, so that flag is the obvious next thing added here. Tokens preceded by a value-taking `--test-*` flag are now dropped. 2. False failure. `./scripts/**/*.test.mjs` — which `node --test` accepts, and the common npm-script idiom — matched nothing, because `git ls-files` emits no `./`. Both files were then blamed for a rename that never happened, the unfollowable-remediation shape fixed for the bare-runner case last round. A leading `./` is normalized off, as `rootRunsClientValidate` already does. Nit 4: the gate-integrity footer's typecheck-wiring advice is now conditioned on at least one issue being about the typecheck wiring — a `test:scripts` failure carries its own per-line remediation, and appending `--noCheck`/`tsc -b` advice under a renamed-test-file message sends the reader after the wrong thing. Verified. The coverage-include probe with the rename now exits 1 where it exited 0, and exits 0 without the rename; the `./` probe exits 0 where it exited 1; a `test:scripts`-only failure no longer prints the typecheck footer. Mutation sweep 9/9 killed, including both new rules. `test:scripts` 22/22 · `verify:typecheck-coverage` OK 894 · `verify:format-coverage` OK 919 · `npm run ci` green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vgJ6ZYj9VLhfSjgua48zw * chore(scripts): round-35 — fourth axis: flags that narrow the run 1. False pass on a new axis. The three axes ask whether the tests are run, whether there are any, and whether the runner is told about each — none asks what it then DOES with them. `--test-shard 1/2` keeps every glob intact and executes 6 of 22 tests, exiting 0, with the guard green; `--test-only` runs 2, as does a non-matching `--test-name-pattern`. This is `--noCheck` for the test runner, one domain over. Adds `NARROWING_TEST_FLAGS` and a fourth axis in `testScriptProblems`. Three of those flags were already in `VALUE_TAKING_TEST_FLAGS` — the guard knew them well enough to drop their values but not that they shrink the run. Both `--flag value` and `--flag=value` report as the bare flag name. 2. The footer conditioning added last round was itself inside `main()` — the pattern the same commit's comment documented, recurring in the commit that documented it. Extracted as `integrityAdvice(problems, nonTypecheck)`. Doing that fixes nit 3 in the same move: the old condition keyed off `testScriptIssues` specifically, so it covered one of three non-typecheck categories. A sibling-vouch failure ("the root validate no longer runs verify:format-coverage") and client-enrollment problems still got `--noCheck`/`tsc -b` advice about a typecheck pass they have nothing to do with. All three sets are now excluded. Verified. `--test-shard 1/2`, `--test-only` and `--test-name-pattern zzz` each now exit 1 naming the flag, while node runs 6/2/2 of 22. A sibling-vouch failure no longer prints the typecheck footer. Mutation sweep 12/12 killed, including all three new rules and the branch this commit replaces. `test:scripts` 24/24 · `verify:typecheck-coverage` OK 894 · `verify:format-coverage` OK 919 · `npm run ci` green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vgJ6ZYj9VLhfSjgua48zw * chore(scripts): round-36 — discover test files by content, not name 1. False pass on the axis underneath the other four. The required test-file set was built with the same name pattern the `test:scripts` glob uses, so a rename escaping BOTH was invisible: the file left `testFiles` entirely, so nothing asked whether the runner was told about it, and the survivor satisfied the non-empty axis. A dot→hyphen slip (`npm-scripts.test.mjs` → `npm-scripts-test.mjs`) dropped 6 of 24 tests with every gate green — and the file that stops running is the one pinning the r16 prefix-sibling and r20 tail anchors, the series' two other false passes. Round 30 fixed "renamed to a form the glob misses" by broadening the name pattern; a fifth name pattern only moves the boundary again. `node --test` decides what a test is by whether it registers tests, so `isScriptsTestFile` reads that instead — the same measure-don't-model move as r25 and r32. It also covers what no name pattern can: a NEW test file added under a name the glob misses, never run and never noticed. Nit 2: `enrollmentProblems` no longer suppresses the footer. "declares no `typecheck` script …" IS about the typecheck pass and the footer's first clause is its canonical fix; excluding it lost the advice for the r15 rename scenario. The criterion is "is this about the typecheck pass at all", not "does it carry per-line advice" — several typecheck problems carry their own and still want the footer. JSDoc corrected accordingly. Nit 3: the complementary-shard boundary (`1/2` + `2/2` across two reachable scripts genuinely runs everything, and is flagged anyway) recorded on the helper, alongside the guard's other documented boundaries. Nit 4: the three `--test-coverage-{lines,branches,functions}` thresholds added to VALUE_TAKING_TEST_FLAGS. Their values are numeric and so inert rather than suppressing, but model completeness is cheaper than re-deriving that later. Verified. The hyphen rename now exits 1 naming the file; the `.spec.mjs` rename is still caught; an enrollment failure prints the footer again; the guard is unchanged at 894 files OK (exactly the 2 tracked test files reference `node:test`, none of the other 12 `scripts/` files do). Mutation sweep 14/14. `test:scripts` 25/25 · `verify:format-coverage` OK 919 · `npm run ci` green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vgJ6ZYj9VLhfSjgua48zw * chore(scripts): round-37 — register-not-mention; guard the discovery read Both findings were introduced by round 36's content predicate. 1. False failure. The predicate matched a bare `node:test` mention, so a shared helper importing the runner's `mock` API — registering no tests — was enrolled as a required test file and told to rename itself to `*.test.mjs`, which would only make node run an empty file. `scripts/lib/` is already the repo's shared-helper location, so this is the plausible next file there. Tightened to node's actual criterion: an import/require OF `node:test` AND a test registration. 2. Regression. `readFileSync` was passed as an argument, so it ran for every tracked `scripts/` file before the predicate's extension gate could reject it — a file in the index but not the worktree (`rm` without `git rm`) took the guard down with a raw ENOENT stack, aborting phase 1 before any real finding printed. Discovery did no reads at all before round 36. Extension is now gated first and the read wrapped: an unreadable tracked file warns and is skipped, the round-6 precedent where a failed `tsc` echoes its diagnostic instead of dying. Verified: the mock-helper probe exits 0; the tracked-but-deleted probe exits 0 with a one-line skip warning instead of a stack; the dot→hyphen rename is still caught. Mutation sweep over the predicate 3/3 (import form, registration, extension). `test:scripts` 25/25 · `verify:typecheck-coverage` OK 894 · `verify:format-coverage` OK 919 · `npm run ci` green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vgJ6ZYj9VLhfSjgua48zw * chore(scripts): round-37 nit 3 — one definition of the JS-extension check Finding 2's fix hoisted the extension gate into `main()` (it must run before the read), leaving a second copy of the regex inside the predicate that reads as protection but can never fire there. Extracted as `isJsFile` and used in both positions, so the two can't drift. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vgJ6ZYj9VLhfSjgua48zw --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Cliff Hall
|
742e82515a |
ci: add headless-browser boot smoke for the web client (#1615) (#1768)
Some checks failed
Run install, format, lint, build, and test on every push / build (push) Has been cancelled
Run install, format, lint, build, and test on every push / publish (push) Has been cancelled
Run install, format, lint, build, and test on every push / publish-github-container-registry (push) Has been cancelled
* ci: add headless-browser boot smoke for the web client (#1615) smoke:web only asserts GET / serves the SPA HTML with the injected token; it never runs the React app, so a Node built-in reaching the browser bundle (which crashes the app to a blank page at runtime, e.g. #1612) slips through. Add smoke:web:browser, which boots the same prod --web server and runs the bundle in headless Chromium (Playwright, already a clients/web devDependency), asserting the app renders its first frame (the "Add Servers" control) with no uncaught page errors — in particular no `Module "node:*" has been externalized`. This catches "Node code reached the browser bundle" as a class, not one import at a time. Wire it into the root smoke chain (and thus npm run ci / GitHub CI), and move the Playwright install/cache steps ahead of the smoke step so the new smoke reuses the cache. Update AGENTS.md and README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * ci: address review on the headless-browser boot smoke - Don't fail on benign console.error (CDN font loads, React warnings) — the #1612 class arrives as an uncaught pageerror. console errors are now diagnostics, hard-failing only when they carry the externalization signature. - Broaden the externalization regex to non-prefixed built-ins ("fs" as well as "node:fs") and label the signature on both the pageerror and console channels. - Rename the port override to SMOKE_WEB_BROWSER_PORT so it doesn't collide with smoke:web's SMOKE_WEB_PORT. - Add a settle window (networkidle + 500ms) so a late-evaluating lazy chunk that throws is caught, and race the render against launcher death so a mid-load crash reports the real cause instead of a render timeout. - Print an actionable hint when Playwright/chromium can't launch (missing --with-deps system libs). - Extract the shared spawn/readiness boilerplate into scripts/lib/prod-web-server.mjs so smoke-web and smoke-web-browser can't drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * ci: fix playwright resolution in the browser smoke (CI failure) The bare `import("playwright")` resolved relative to the script's own directory (scripts/), not the cwd — so `cd clients/web` in the npm script never made it resolvable. It only passed locally because an ancestor node_modules happened to carry playwright; CI has none, so it failed with "Cannot find package 'playwright'". Resolve it explicitly from clients/web via createRequire, independent of cwd and ancestor dirs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * ci: address second review on the prod-web-server helper - waitForReady now records the last non-ok status and includes it in the timeout message, so a server that boots but persistently answers 500 (broken dist / injection failure) reports "server did not start … (last response: HTTP 500)" instead of a bare timeout that points at the wrong thing. Keeps polling on non-ok (a warming server may answer 503). - Add a child 'error' handler so a spawn-level failure surfaces as the smoke's FAILED line instead of an uncaught raw stack; readiness and the child-exit race both report it. - Drop the unused isExited/exitCode/child exports from the helper API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * ci: reframe browser smoke around uncaught pageerror; fix stop() edge Third-review findings. 1. The `Module "…" has been externalized` string can't fire against a Vite 8 *prod* bundle — verified end-to-end against the installed vite@8.0.0: a node: import reaching the browser graph is a BUILD-time warning, and the shipped stub is a silent `module.exports = {}`. The real #1612 manifestation is the first *call* into that empty stub throwing a TypeError at module init (confirmed: injecting a used `node:fs` import fails the smoke with "readFileSync is not a function"; an unused one ships {} and is harmless). So the signature machinery was dead against the bundle it runs on — dropped it. The smoke now asserts exactly what it verifies: the prod bundle boots and paints with no uncaught page error. Reworded the script header, the AGENTS.md bullet, and the README row to match. Console errors stay diagnostic-only. 2. The helper's child 'error' handler set `exited = true`, but Node emits 'error' for a failed kill/send too (child still alive) — so `stop()` (guarded on `exited`) would skip the SIGTERM and orphan the launcher on the port. Track the error separately (`childError`); `bootFailed()` = exited || childError, while `stop()` still SIGTERMs a live child. Docs: fixed the stale AGENTS.md claim that `cd clients/web` makes `import("playwright")` resolve (createRequire does, cwd-independently; the cd is only for `npx playwright install`), and noted the shared scripts/lib/prod-web-server.mjs helper on both smoke bullets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * ci: also hard-fail the async half of the crash class (fourth review) page.on("pageerror") fires for uncaught SYNCHRONOUS exceptions only. The async twin of the #1612 shape — the same empty-stub TypeError reached through an await/.then(), or a failed dynamic import (this app lazy-loads chunks) — lands on the console channel as `Uncaught (in promise) …` / `Failed to fetch dynamically imported module`, which were diagnostic-only, so the smoke would have passed. Hard-fail console errors matching /^Uncaught\b|Failed to fetch dynamically imported module/; every other console error stays a printed non-fatal note. The prefixes are unambiguous (a font/CDN miss reads "Failed to load resource: net::ERR_…"; React warnings never start with "Uncaught"), so this can't reintroduce the flake the earlier round fixed. Verified: an unhandled rejection at module init now fails the smoke where it previously wouldn't. Also prod-scope the AGENTS.md note that the "externalized" string isn't a runtime message (under `npm run dev` Vite's stub is a Proxy that console.warns it), and document both the sync and async failure signals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * docs: reword smoke success line + README to two-channel wording (fifth review) After the async hard-failure signature, the assertion spans both the pageerror (sync) and console (unhandled rejection / failed dynamic import) channels. The OK console line and the README smoke row still said "page error"; match the two-channel wording already in the AGENTS.md bullet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 * ci: correct the playwright-package-resolution hint (sixth review) The catch around requireFromWeb("playwright") advised `playwright install --with-deps chromium`, but that installs browser binaries, not the npm package — and the npm script already ran `playwright install` a step earlier. A failure there means the package isn't resolvable from clients/web, so the actionable fix is `npm install` at the repo root (the postinstall cascade installs clients/web's devDependencies). The chromium.launch() catch keeps its --with-deps advice (correct for the missing-system-libraries case). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XNdjEPKLG637X8YmhDiEk5 --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4af38c719c | ||
|
|
747f59892f | ||
|
|
b7a73e7da3 | ||
|
|
bd7297da88 | ||
|
|
b6baa30d7b | ||
|
|
93bc7df360 | ||
|
|
8a7ade7b08 | ||
|
|
fb9e146043 | ||
|
|
32cb4c63a7 | ||
|
|
4ab0139de9 | ||
|
|
25eb2fb3bf | ||
|
|
ef4259a544 | ||
|
|
3222537e73 | ||
|
|
8d193461e3 | ||
|
|
e59452ea8c | ||
|
|
df938777b1 | ||
|
|
17c33080f9 | ||
|
|
6b4bf68b8b | ||
|
|
9cade95fd0 | ||
|
|
09b35489aa | ||
|
|
c93552139c | ||
|
|
b399bc64e1 | ||
|
|
4b143ce043 | ||
|
|
7164806be3 | ||
|
|
a9b9a917e3 | ||
|
|
7f9be24bf3 | ||
|
|
e3a026802b | ||
|
|
678184ed3e | ||
|
|
3d541388db | ||
|
|
4a993fc45a | ||
|
|
16c96d44c3 | ||
|
|
29e3ce95b8 | ||
|
|
8801d0f8bc |
||
|
|
d9e4f444a2 |
||
|
|
742e82515a |