The case reproduces an overlap — the render deadline expiring while the
survival window is still open — and it had to buy that overlap with a
startup allowance. `paintAt` is measured from the child's first JS tick
while `renderTimer` is armed in the parent at spawn, so a plain
`setTimeout(paint, N)` lands at `N + startup` and the deadline has to
cover whatever startup turns out to be. #2177 widened that allowance
(150ms to 950ms), which moves the threshold rather than removing it: the
case still fails, as "did not render", on any machine loaded enough to
blow the new number.
The child is now told the wall-clock instant to paint at and sleeps for
whatever is left of it on arrival, so startup is absorbed by the sleep
instead of added to it and the paint lands at ~300ms from spawn however
slow the boot was. `timeoutMs: 600` / `surviveMs: 900` then satisfies
the right-hand side of the overlap for ANY paint time including 0, so a
boot slow enough to swallow the anchor entirely still reproduces the
shape rather than degrading into a different case. The one startup
assumption left is that boot completes before the deadline at all, an
order of magnitude over the 46-59ms measured idle here.
Passing is no longer taken as evidence on its own: the verdict's
first-paint time is read back and the overlap asserted, so a paint
landing early enough to put the deadline outside the survival window
fails loudly instead of passing while testing nothing.
Verified by mutation — removing the `clearTimeout(renderTimer)` this
case guards fails it at 607ms, i.e. the timer still fires inside the
survival window. Verified under load — 8/8 with 16 busy loops pinning
all 8 cores. Runtime drops from ~1.75s to ~1.2s.
Closes#2180
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot round 1. The comment blamed `script(1)` PTY allocation, but this
test's `stub()` spawns `process.execPath` directly — the PTY wrapper
lives in `smoke-tui.mjs`, the caller — so the fixture never pays that
cost and the 66-114ms figure (measured through the wrapper) was not
this path's.
Re-measured without it: Node boot alone is 46-59ms idle, which fits the
old 150ms budget comfortably. That is *why* it read as a flake rather
than a bug — what actually exhausted the budget was scheduling delay
under a full gate run, on both the child's boot and the parent's timer.
The fix is unchanged; only the reasoning was wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev
Signed-off-by: cliffhall <cliff@futurescale.com>
The test gave itself a 150ms budget that had to cover `script(1)` PTY
allocation plus Node boot: `paintAt` is measured from the child's first
JS tick, while `renderTimer` is armed in the parent at spawn. Measured
idle, that startup is 66-114ms — ~36ms of slack at best — and under the
load of a full `local:gate` run it goes over, so the render timer fires
before the paint and the run is reported as "did not render", which is
precisely the misdiagnosis this test exists to catch. It failed three
gate runs in a row while passing standalone every time.
The constants are widened rather than the assertion changed: the shape
under test is the overlap (paint < timeoutMs < paint + surviveMs), and
1200/1500 keeps it with real margin on both sides. Verified by mutation
— removing the `clearTimeout(renderTimer)` this test guards makes it
fail at 1204ms, i.e. the timer still fires inside the survival window.
Closes#2177
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 5 — four findings, all in the suppressed block.
1+4. The regression stubs did `console.log(MARKER); process.exit(1)`.
Writing to a PIPE is asynchronous and `process.exit()` tears the
process down without draining, so the marker can be lost outright —
and then the stub that pins the paint-then-die path fails as "exited
before rendering", the neighbouring branch and the wrong diagnostic
for the thing it guards.
Worth recording that I met this mechanism last round and did not join
the dots: probing the drain race, a 4MB write followed by
`process.exit()` reached the parent with the marker missing
entirely. That was noted as a reason the repro failed, while the same
construct stayed in the stubs. Both now set `process.exitCode` and
let the loop drain — still exiting immediately after the write — with
a comment, since `process.exit()` is the more obvious spelling and
will read as a simplification to the next person. Same fix for the
`console.error("boom")` stub, whose assertion had the same exposure.
2. DEFAULTS claimed "every one is a hard floor as well as a default",
which contradicts `normalizeMs` (any finite positive value passes)
and the suite two files over, which passes 400ms against a 2000ms
default. It rejects the shape a bad env var produces, not smallness.
3. The raw-mode forbidden-output check was unreachable from any test:
the real PTY happy path only exercises the pattern NOT matching, so
a regression breaking it would stay green forever — an
acceptance-criterion guard with nothing guarding it, which is the
recursive form of this PR's own complaint.
Now a `runRenderSmoke` option (`forbidOutput`), so it runs inside the
tested state machine and against the fully drained output.
`smoke:tui` passes its pattern in and drops the post-hoc check. It
can only turn a pass into a failure, never rewrite an existing
failure's message: a crash reason is more useful than "and it also
printed X", which is usually a symptom of that crash. Three tests —
matching, not matching, and that asymmetry.
Verified: test:scripts 213/213; `smoke:tui OK — rendered "MCP Servers" at
360ms and was still running 2000ms later`; and the no-PTY shim still
fails through the refactored path.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 4.
`exit` can be observed before the child's last output drains — `done()`
already keeps reading until `close` for exactly that reason — so `onData`
can run after the verdict is settled. Recording `markerAt` there is still
right: the message is a thunk rendered at `finish()`, and a marker still
sitting in the pipe when `exit` fired was painted all the same. Arming a
survival timer is not. It could not change the result (`done` is a no-op
once settled), but it holds the event loop open for the whole window,
which under `node --test` is a suite lingering seconds for nothing.
That implies something the comment did not say, and it had already bitten
this PR: if `markerAt` can be set after the exit was observed, any wording
derived from observation ORDER is unstable. The old message read
"rendered at Nms and then exited Mms later", where M was
`since() - markerAt` evaluated at finish — not the paint-to-exit gap at
all, and incoherent whenever the marker surfaced during the drain. The
test asserted on /survival window/, which the losing side of that race
could drop. It could have flaked, and the flake would have looked like
the bug this PR fixes.
So the message now makes no ordering claim and reports both times as the
observations they are, with `exitAt` captured in the exit handler rather
than recomputed later. Assertions moved to the race-stable phrase, plus a
test that runs the paint-then-die stub 12x asserting verdict AND wording
are identical every time — one pass proves nothing about a race.
The ordering could not be forced on darwin, so the guard is reasoned
rather than reproduced, and the reply says so. Two attempts and what they
actually showed: a 4MB payload with the marker last never gets there,
because `process.exit()` discards pending stdout writes (the "before
rendering" branch is correctly taken); marker on stderr with a graceful
exitCode was observed first in all 30 runs, none lingering near the
window. `done()` exists precisely because exit and close are separate
events, so treating their order as guaranteed was worth removing anyway.
Also (suppressed comment): the README's "Two more do the same" clause now
lists four suites, not two, after this PR added render-smoke and pty.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Copilot review round 3.
`done()` is first-caller-wins, and the render deadline stayed armed after
first paint. So a marker landing near `timeoutMs` let that timer fire
*inside* the survival window and settle the run as
child did not render "MCP Servers" within 15000ms
followed by an output tail that visibly contains "MCP Servers". A TUI that
is merely slow fails intermittently, with a diagnostic naming the one
thing that demonstrably did happen — the same wrong-attribution class this
PR exists to remove, and one that reads as flake.
First paint has happened; its deadline no longer has a question to ask, so
`onData` clears it at the moment the marker is observed.
Also hoists `renderTimer`'s declaration beside `surviveTimer`'s. It was a
`const` below both closures that clear it, safe only because `onData` and
`done` run from events rather than the synchronous body. That already held
for `done`; this fix adds a second reader, so the ordering is now stated
rather than relied upon.
The test paints at 250ms with timeoutMs 400 and surviveMs 400, so the
render deadline expires while the survival window is open — the exact
shape. Verified it fails against the module with the clearTimeout removed
and passes with it: a wider timeoutMs would have been green either way,
which is why the numbers are tight.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
Two findings, both of which quietly undo the PR.
1. `SMOKE_TUI_SURVIVE_MS` was unvalidated. `??` guards only nullish, so
an env var set to the empty string reaches `Number("")` -> 0 and a typo
reaches NaN; `setTimeout` fires immediately on both. The survival check
would then resolve before the child could possibly have died, printing
"still running 0ms later" and asserting nothing — the exact first-paint
false green this PR removes, restored by an empty environment variable,
on a green run.
Fixed at the module rather than only at the env-parse site, so every
caller is covered: `normalizeMs` (same shape as child-cleanup.mjs's
`normalizeGraceMs`, matching an existing policy rather than inventing a
second one) applied to all four durations before anything reads them,
which also stops the messages interpolating "NaNms". Defaults moved to
an exported DEFAULTS so the smoke and the module cannot drift.
`surviveMs: 0` is deliberately NOT honored as "skip the wait": no
caller wants that, and it is indistinguishable from the typo.
2. (suppressed comment) `spawnSync` REPORTS ENOENT in `result.error` — it
does not throw — so `probeScriptVersion`'s try/catch read a missing
binary as "ran, printed nothing". On linux that empty output fell
through to the util-linux guess, and the smoke then hard-failed
spawning a binary that does not exist instead of taking the documented
skip.
`probeScriptVersion` now returns `{ available, output }`, and
`resolvePtyWrapper` returns `{ ok: true, flavor, wrap }` or
`{ ok: false, reason }` — the two unavailability causes deserve
different skip messages, since "no script(1) on PATH" and "nobody has
verified this platform's invocation" send the reader elsewhere. A
non-zero exit is explicitly not unavailability: BSD `script` has no
`--version` and answers with its usage, which is the evidence the
flavor fallback reads.
Tests for both, since each finding lives entirely in a branch no happy
path reaches: the normalizeMs table (empty string, typo, undefined, -1,
Infinity, a numeric string, 0), an end-to-end run proving a
`surviveMs: Number("")` paint-then-die stub still fails, the
`result.error` ENOENT and timeout shapes, and both `resolvePtyWrapper`
failure reasons.
Verified: `SMOKE_TUI_SURVIVE_MS="" npm run smoke:tui` reports "still
running 2000ms later"; with PATH blanked, resolvePtyWrapper returns
`{ok: false, reason: "no `script(1)` on PATH"}` rather than guessing.
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
`smoke:tui` spawned the launcher with `stdio: ["ignore", …]`, waited for
"MCP Servers" in the output, printed OK, and SIGTERMed. It never looked at
the child again.
The child was already dead. Ink mounts `useInput`, `useInput` needs raw
mode, and raw mode is a property of the file descriptor — with stdin on
/dev/null the TUI painted one frame and exited 1 about 40ms later with
"Raw mode is not supported on the current process.stdin". Measured here:
first frame at 396ms, exit 1 at 433ms. The smoke won that race and
reported success, so what it asserted was *first paint*, not survival —
contradicting its own header ("boots and renders without crashing").
This is not environment-specific: `stdio: ["ignore", …]` makes the child's
stdin /dev/null whether or not the developer's terminal is a TTY, so this
smoke has never verified a running TUI on any machine. And the race cut
both ways — an `exit` processed first failed the run, which reads as flake
rather than as the standing defect.
Two pieces:
- `scripts/lib/pty.mjs` gives the child a real terminal via `script(1)`,
no dependency. The three flavors are not interchangeable (BSD takes
argv; util-linux takes one shell string plus `-e`; busybox that string
without `-e`), so the invocation is built and unit-tested rather than
written inline. A platform with no `script(1)` skips loudly instead of
running without one, which would be a guaranteed failure rather than a
weaker check.
- `scripts/lib/render-smoke.mjs` is the fix: the child must still be
running `surviveMs` (default 2s) after first paint. Without that, a
future harness change reintroduces the same false green and nothing
notices — which is why it lives in a module `test:scripts` drives
against a stub that paints the marker and immediately exits. Today's
harness passes that stub; this one fails it, with a diagnostic that
distinguishes "died after painting" from "died before painting".
The smoke additionally fails if the raw-mode error appears in a passing
run, so a future Ink that degrades to a warning instead of throwing can't
ship a TUI with no keyboard input past this gate.
Verified end to end: with a `script` shim that allocates no pty, the smoke
now fails with "it painted one frame, it did not run"; with the real one
the TUI is still running 2s after its first frame and never logs the
raw-mode error.
The `process.env.CI` skip stays, but its comment is corrected: the PTY
removes the technical blocker it used to cite, so the smoke is local-only
by decision now, not by capability. Whether it joins GitHub CI is out of
scope here (#2146 is keeping the other local-only smokes out on purpose).
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
cliffhall