`smoke:tui` spawned the launcher with `stdio: ["ignore", …]`, waited for
"MCP Servers" in the output, printed OK, and SIGTERMed. It never looked at
the child again.
The child was already dead. Ink mounts `useInput`, `useInput` needs raw
mode, and raw mode is a property of the file descriptor — with stdin on
/dev/null the TUI painted one frame and exited 1 about 40ms later with
"Raw mode is not supported on the current process.stdin". Measured here:
first frame at 396ms, exit 1 at 433ms. The smoke won that race and
reported success, so what it asserted was *first paint*, not survival —
contradicting its own header ("boots and renders without crashing").
This is not environment-specific: `stdio: ["ignore", …]` makes the child's
stdin /dev/null whether or not the developer's terminal is a TTY, so this
smoke has never verified a running TUI on any machine. And the race cut
both ways — an `exit` processed first failed the run, which reads as flake
rather than as the standing defect.
Two pieces:
- `scripts/lib/pty.mjs` gives the child a real terminal via `script(1)`,
no dependency. The three flavors are not interchangeable (BSD takes
argv; util-linux takes one shell string plus `-e`; busybox that string
without `-e`), so the invocation is built and unit-tested rather than
written inline. A platform with no `script(1)` skips loudly instead of
running without one, which would be a guaranteed failure rather than a
weaker check.
- `scripts/lib/render-smoke.mjs` is the fix: the child must still be
running `surviveMs` (default 2s) after first paint. Without that, a
future harness change reintroduces the same false green and nothing
notices — which is why it lives in a module `test:scripts` drives
against a stub that paints the marker and immediately exits. Today's
harness passes that stub; this one fails it, with a diagnostic that
distinguishes "died after painting" from "died before painting".
The smoke additionally fails if the raw-mode error appears in a passing
run, so a future Ink that degrades to a warning instead of throwing can't
ship a TUI with no keyboard input past this gate.
Verified end to end: with a `script` shim that allocates no pty, the smoke
now fails with "it painted one frame, it did not run"; with the real one
the TUI is still running 2s after its first frame and never logs the
raw-mode error.
The `process.env.CI` skip stays, but its comment is corrected: the PTY
removes the technical blocker it used to cite, so the smoke is local-only
by decision now, not by capability. Whether it joins GitHub CI is out of
scope here (#2146 is keeping the other local-only smokes out on purpose).
Signed-off-by: Cliff Hall <cliff@futurescale.com>
Signed-off-by: cliffhall <cliff@futurescale.com>
172 lines
6.8 KiB
JavaScript
172 lines
6.8 KiB
JavaScript
#!/usr/bin/env node
|
|
/**
|
|
* Boot smoke test for the prod TUI launcher path (#1347).
|
|
*
|
|
* `npm run smoke:launcher` only checks `--tui --help`; it never actually starts
|
|
* the Ink app, so the launcher → TUI → core path and `--catalog` loading went
|
|
* unverified by the launcher smokes. This script launches the built launcher in
|
|
* `--tui` mode against a temp `--catalog`, waits for the app to render its first
|
|
* frame (the "MCP Servers" panel), waits again to confirm it is *still running*,
|
|
* then sends SIGTERM and exits.
|
|
*
|
|
* It asserts the TUI boots, renders, and keeps running — not full interaction
|
|
* (driving an Ink UI deterministically is flaky, so this is intentionally a
|
|
* shallow check). Exits non-zero on a crash before or after first paint, or on
|
|
* a render timeout.
|
|
*
|
|
* ## Why it runs under a pseudoterminal (#2147)
|
|
*
|
|
* It used to spawn the TUI with its stdin on `/dev/null` and settle OK the
|
|
* moment the marker appeared. Ink mounts `useInput`, `useInput` needs raw mode,
|
|
* and raw mode is a property of the file descriptor — so the TUI painted one
|
|
* frame and died ~40ms later with "Raw mode is not supported on the current
|
|
* process.stdin". The smoke won that race and reported OK, on every machine,
|
|
* TTY or not: what it actually asserted was *first paint*, not survival.
|
|
*
|
|
* Two changes, and the second is the fix: `scripts/lib/pty.mjs` gives the child
|
|
* a real terminal via `script(1)`, and `scripts/lib/render-smoke.mjs` requires
|
|
* it to outlive its first frame. Without the survival assertion a future
|
|
* harness change reintroduces exactly this false green and nothing notices —
|
|
* which is why that assertion lives in a tested module rather than here.
|
|
*
|
|
* Expects `clients/launcher/build` and `clients/tui/build` to be built first
|
|
* (the validate / CI ordering guarantees this). The bundled stdio test server
|
|
* (`test-servers/build`) is rebuilt on every run — see
|
|
* `scripts/lib/ensure-test-servers.mjs` for why presence isn't freshness.
|
|
*/
|
|
|
|
import { mkdtempSync, existsSync, writeFileSync } from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import { join, resolve } from "node:path";
|
|
import { removeSafe } from "./lib/child-cleanup.mjs";
|
|
import { ensureTestServers } from "./lib/ensure-test-servers.mjs";
|
|
import { resolvePtyWrapper } from "./lib/pty.mjs";
|
|
import { runRenderSmoke } from "./lib/render-smoke.mjs";
|
|
|
|
const repoRoot = resolve(import.meta.dirname, "..");
|
|
const launcher = join(repoRoot, "clients", "launcher", "build", "index.js");
|
|
const RENDER_MARKER = "MCP Servers";
|
|
const TIMEOUT_MS = Number(process.env.SMOKE_TUI_TIMEOUT_MS ?? 15000);
|
|
// How long past first paint the TUI must still be running. Generous relative to
|
|
// the failure it catches (the raw-mode crash landed ~40ms after the marker) —
|
|
// the cost of a longer window is only wall-clock on a passing run.
|
|
const SURVIVE_MS = Number(process.env.SMOKE_TUI_SURVIVE_MS ?? 2000);
|
|
const EXIT_GRACE_MS = Number(process.env.SMOKE_TUI_EXIT_GRACE_MS ?? 5000);
|
|
// Ink's own error text when it is asked for raw mode on a non-terminal fd.
|
|
const RAW_MODE_ERROR = /Raw mode is not supported/;
|
|
|
|
function fail(message) {
|
|
console.error(`smoke:tui FAILED — ${message}`);
|
|
process.exit(1);
|
|
}
|
|
|
|
function skip(message) {
|
|
console.log(`smoke:tui SKIPPED — ${message}`);
|
|
process.exit(0);
|
|
}
|
|
|
|
// Kept out of GitHub CI by decision, not by capability. The PTY above removes
|
|
// the technical blocker this skip used to cite (a headless runner has no TTY),
|
|
// but whether this smoke joins CI is a separate call for the maintainers, and
|
|
// #2146 is deliberately keeping the other local-only smokes out. Making it
|
|
// *valid* stands on its own: it is a local-only gate, which is exactly where a
|
|
// false green is least likely to be caught by anything else.
|
|
if (process.env.CI) {
|
|
skip(
|
|
"local-only by decision (see the header); the TUI is built and unit-tested in CI",
|
|
);
|
|
}
|
|
|
|
if (!existsSync(launcher)) {
|
|
fail(`launcher build not found at ${launcher} — run \`npm run build\` first`);
|
|
}
|
|
|
|
// No `script(1)` (Windows). Running without a PTY is not a weaker check, it is
|
|
// a guaranteed failure — so say why rather than report a crash as a defect.
|
|
// `node-pty` is the portable answer if this ever needs to run there.
|
|
const pty = resolvePtyWrapper();
|
|
if (!pty) {
|
|
skip(
|
|
`no pseudoterminal available on ${process.platform} — the Ink TUI needs raw mode, ` +
|
|
`which requires a real terminal fd (see scripts/lib/pty.mjs)`,
|
|
);
|
|
}
|
|
|
|
// Rebuilt on every run — presence is not freshness (#2111).
|
|
let testServer;
|
|
try {
|
|
[testServer] = ensureTestServers({
|
|
repoRoot,
|
|
label: "smoke:tui",
|
|
requires: ["stdio"],
|
|
});
|
|
} catch (e) {
|
|
fail(e.message);
|
|
}
|
|
|
|
const work = mkdtempSync(join(tmpdir(), "smoke-tui-"));
|
|
const catalogPath = join(work, "catalog.json");
|
|
writeFileSync(
|
|
catalogPath,
|
|
JSON.stringify({
|
|
mcpServers: {
|
|
test: { type: "stdio", command: process.execPath, args: [testServer] },
|
|
},
|
|
}),
|
|
);
|
|
|
|
const { command, args } = pty.wrap({
|
|
command: process.execPath,
|
|
args: [launcher, "--tui", "--catalog", catalogPath],
|
|
});
|
|
|
|
let result;
|
|
try {
|
|
result = await runRenderSmoke({
|
|
command,
|
|
args,
|
|
marker: RENDER_MARKER,
|
|
timeoutMs: TIMEOUT_MS,
|
|
surviveMs: SURVIVE_MS,
|
|
exitGraceMs: EXIT_GRACE_MS,
|
|
spawnOptions: {
|
|
cwd: repoRoot,
|
|
// Redirect HOME so the TUI's storage never touches the real
|
|
// ~/.mcp-inspector. Pin MCP_OAUTH_CALLBACK_URL="" (empty reads as unset)
|
|
// so an ambient non-loopback value can't crash the TUI before render via
|
|
// the loopback callback guard — same class smoke-cli.mjs's
|
|
// SMOKE_BASE_ENV neutralizes.
|
|
env: {
|
|
...process.env,
|
|
MCP_OAUTH_CALLBACK_URL: "",
|
|
HOME: work,
|
|
USERPROFILE: work,
|
|
},
|
|
},
|
|
warn: (m) => console.warn(m.replace(/^render-smoke/, "smoke:tui")),
|
|
});
|
|
} finally {
|
|
// Never turn a passing smoke into a failure over a leftover temp dir; the OS
|
|
// reclaims tmpdir anyway. runRenderSmoke resolves only after the child's
|
|
// `close`, so on the normal path the #1801 ENOTEMPTY race is already closed
|
|
// and a warning here should not fire — its give-up branches resolve without
|
|
// that guarantee, so it is expected rather than anomalous on those.
|
|
removeSafe(work, { label: "smoke:tui" });
|
|
}
|
|
|
|
// Belt and braces for the acceptance criterion this fix is measured against.
|
|
// Survival already implies it today, since the raw-mode throw is fatal — but
|
|
// if Ink ever degrades to a warning instead, a TUI with no keyboard input would
|
|
// otherwise pass silently, and this smoke is the only thing watching.
|
|
if (result.code === 0 && RAW_MODE_ERROR.test(result.output)) {
|
|
fail(
|
|
`TUI rendered and survived, but still logged the raw-mode error — the ` +
|
|
`pseudoterminal did not take effect (flavor: ${pty.flavor})`,
|
|
);
|
|
}
|
|
|
|
if (result.code === 0) {
|
|
console.log(`smoke:tui OK — ${result.message}`);
|
|
process.exit(0);
|
|
}
|
|
fail(result.message);
|