* test(cli,tui): add in-process source coverage gates (#1484) CLI and TUI source had no coverage gate: CLI tests spawned the built binary out-of-process (coverage never saw clients/cli/src) and the TUI had only two smoke tests. Regressions there were invisible to CI. CLI: - Rewrite __tests__/helpers/cli-runner.ts to invoke runCli() in-process (capture process.std*.write, map thrown errors to an exit code), preserving the CliResult shape so all existing tests run unchanged and clients/cli/src is now measured under v8 instrumentation. - cli.ts: program.exitOverride() so a commander usage error throws (catchable in-process) instead of tearing down the worker; --help / --version still exit cleanly. Drop the dead, never-imported main(). - Add a thin out-of-process e2e.test.ts that spawns the built binary for the shebang / process.exit paths the in-process runner can't reach. - Add @vitest/coverage-v8, a per-file gate (90/85/80/50) excluding only src/index.ts (binary bootstrap), and pool: 'forks' for capture isolation. cli.ts lands at ~98% lines / 100% functions. TUI: - Add @vitest/coverage-v8 and an interim per-file gate over the non-React logic (logger.ts, components/tabsConfig.ts, utils/*); the Ink components, App.tsx, and hooks/ are excluded with a documented follow-up (#1501). Add tests covering the included modules (100%). Wiring + docs: - CI runs each client's test:coverage; root test:cli / test:tui (used by validate) and root test:coverage now enforce the gates. - gitignore clients/**/coverage; update AGENTS.md and client READMEs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): clarify in-process runner footguns from PR #1502 review Review notes on the in-process CLI test runner (no behavior change): - Warn in runCli's docblock that --help/--version must NOT go through the in-process runner: exitOverride only re-throws non-zero exits, so exitCode 0 falls through to process.exit(0) and would kill the vitest worker. Those belong in the out-of-process e2e.test.ts layer. - Document that a timeout orphans the uncancellable invokeCli promise (its stdio subprocess stays alive; later writes hit the restored stdout) — a known, acceptable trade-off on the hang-only failure path. - Fix a stale reference to the removed main() (errors now route through index.ts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): fix remaining stale main() reference in vitest exclusion comment PR #1502 re-review caught a second stale `main()` mention (the first was fixed in 52c23e41). src/index.ts inlines the bootstrap as runCli(process.argv).then(...).catch(handleError) — there is no main(). Comment-only; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: compose root test:coverage from per-client scripts Replace the brittle inline cd-chain with a composition of the existing test:cli / test:tui scripts plus a new matching test:web. Behavior is unchanged (web → cli → tui coverage gates) but there's a single source of truth for each client's coverage invocation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cliff Hall
GitHub
parent
e72d4307f3
commit
388ea5a7bd
1 changed files with 1304 additions and 197 deletions
|
|
@ -1,10 +1,4 @@
|
|||
import { spawn } from "child_process";
|
||||
import { resolve } from "path";
|
||||
import { fileURLToPath } from "url";
|
||||
import { dirname } from "path";
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const CLI_PATH = resolve(__dirname, "../../build/index.js");
|
||||
import { runCli as invokeCli } from "../../src/cli.js";
|
||||
|
||||
export interface CliResult {
|
||||
exitCode: number | null;
|
||||
|
|
@ -15,83 +9,142 @@ export interface CliResult {
|
|||
|
||||
export interface CliOptions {
|
||||
timeout?: number;
|
||||
/**
|
||||
* Accepted for source-compatibility with the previous out-of-process runner.
|
||||
* The in-process runner shares the test worker's working directory, so it is
|
||||
* ignored — no current test sets it. The out-of-process E2E layer
|
||||
* (`e2e.test.ts`) still honors a real cwd.
|
||||
*/
|
||||
cwd?: string;
|
||||
env?: Record<string, string>;
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
type WriteArgs = [
|
||||
chunk: unknown,
|
||||
encoding?: unknown,
|
||||
callback?: (() => void) | undefined,
|
||||
];
|
||||
|
||||
/**
|
||||
* Run the CLI with given arguments and capture output
|
||||
* Build a `process.std{out,err}.write` replacement that appends every chunk to
|
||||
* a buffer and still honors the optional write callback. `awaitableLog` (the
|
||||
* CLI's stdout writer) passes its resolve handler as that callback, so it MUST
|
||||
* be invoked or `runCli` would hang forever waiting for the flush.
|
||||
*/
|
||||
function captureWrite(append: (text: string) => void) {
|
||||
return (...args: WriteArgs): boolean => {
|
||||
const [chunk, encoding, callback] = args;
|
||||
append(typeof chunk === "string" ? chunk : String(chunk));
|
||||
const cb = typeof encoding === "function" ? encoding : callback;
|
||||
if (typeof cb === "function") cb();
|
||||
return true;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the CLI **in-process** by importing and invoking `runCli` directly, so
|
||||
* its source (`clients/cli/src/**`) is measured under vitest's coverage
|
||||
* instrumentation. The previous implementation spawned `build/index.js` as a
|
||||
* subprocess, which left CLI source invisible to coverage (#1484).
|
||||
*
|
||||
* The returned shape matches the old out-of-process runner exactly
|
||||
* (`exitCode` / `stdout` / `stderr` / `output`) so every existing test and the
|
||||
* shared assertion helpers keep working unchanged:
|
||||
* - stdout/stderr are captured by temporarily patching `process.std*.write`
|
||||
* (and `console.error`/`console.warn`, which commander and error paths use).
|
||||
* - A thrown error (bad args, failed connect, unsupported method) maps to
|
||||
* `exitCode: 1` with the message appended to stderr — mirroring how the real
|
||||
* binary (`index.ts`) routes errors through `handleError` → `process.exit(1)`.
|
||||
* - `options.env` is applied to `process.env` for the duration of the call and
|
||||
* fully restored afterward.
|
||||
*
|
||||
* The real binary, its shebang, and actual `process.exit` codes are covered by
|
||||
* the thin out-of-process layer in `e2e.test.ts` and `scripts/smoke-cli.mjs`.
|
||||
*
|
||||
* DO NOT drive `--help` / `--version` through this runner. `cli.ts` sets
|
||||
* `program.exitOverride((err) => { if (err.exitCode !== 0) throw err })`, so for
|
||||
* those (exitCode 0) commander does NOT throw — it falls through to its normal
|
||||
* `process.exit(0)`, which run in-process would terminate the vitest worker.
|
||||
* Cover `--help` / `--version` in the out-of-process `e2e.test.ts` layer instead.
|
||||
*/
|
||||
export async function runCli(
|
||||
args: string[],
|
||||
options: CliOptions = {},
|
||||
): Promise<CliResult> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn("node", [CLI_PATH, ...args], {
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
cwd: options.cwd,
|
||||
env: { ...process.env, ...options.env },
|
||||
signal: options.signal,
|
||||
// Own process group on Unix so timeout can kill CLI + stdio grandchildren.
|
||||
detached: process.platform !== "win32",
|
||||
});
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
let resolved = false;
|
||||
const originalStdoutWrite = process.stdout.write;
|
||||
const originalStderrWrite = process.stderr.write;
|
||||
const originalConsoleError = console.error;
|
||||
const originalConsoleWarn = console.warn;
|
||||
|
||||
// Default timeout of 10 seconds (less than vitest's 15s)
|
||||
const timeoutMs = options.timeout ?? 10000;
|
||||
const timeout = setTimeout(() => {
|
||||
if (!resolved) {
|
||||
resolved = true;
|
||||
// Kill the process and all its children
|
||||
try {
|
||||
if (process.platform !== "win32" && child.pid != null) {
|
||||
process.kill(-child.pid, "SIGTERM");
|
||||
} else {
|
||||
child.kill("SIGTERM");
|
||||
}
|
||||
} catch {
|
||||
// Process might already be dead, try direct kill
|
||||
try {
|
||||
child.kill("SIGKILL");
|
||||
} catch {
|
||||
// Process is definitely dead
|
||||
}
|
||||
}
|
||||
reject(new Error(`CLI command timed out after ${timeoutMs}ms`));
|
||||
}
|
||||
}, timeoutMs);
|
||||
// Snapshot and apply env overrides; `undefined` records keys that did not
|
||||
// exist so they can be deleted (not set to the string "undefined") on restore.
|
||||
const envBackup: Record<string, string | undefined> = {};
|
||||
if (options.env) {
|
||||
for (const [key, value] of Object.entries(options.env)) {
|
||||
envBackup[key] = process.env[key];
|
||||
process.env[key] = value;
|
||||
}
|
||||
}
|
||||
|
||||
child.stdout.on("data", (data) => {
|
||||
stdout += data.toString();
|
||||
});
|
||||
process.stdout.write = captureWrite((text) => {
|
||||
stdout += text;
|
||||
}) as typeof process.stdout.write;
|
||||
process.stderr.write = captureWrite((text) => {
|
||||
stderr += text;
|
||||
}) as typeof process.stderr.write;
|
||||
console.error = (...parts: unknown[]) => {
|
||||
stderr += parts.map(String).join(" ") + "\n";
|
||||
};
|
||||
console.warn = (...parts: unknown[]) => {
|
||||
stderr += parts.map(String).join(" ") + "\n";
|
||||
};
|
||||
|
||||
child.stderr.on("data", (data) => {
|
||||
stderr += data.toString();
|
||||
});
|
||||
// `runCli` reads `process.argv.slice(2)`, so prepend two placeholder entries
|
||||
// ([node, script]) to mirror how the binary is launched.
|
||||
const argv = ["node", "inspector-cli", ...args];
|
||||
|
||||
child.on("close", (code) => {
|
||||
if (!resolved) {
|
||||
resolved = true;
|
||||
clearTimeout(timeout);
|
||||
resolve({
|
||||
exitCode: code,
|
||||
stdout,
|
||||
stderr,
|
||||
output: stdout + stderr,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
child.on("error", (error) => {
|
||||
if (!resolved) {
|
||||
resolved = true;
|
||||
clearTimeout(timeout);
|
||||
reject(error);
|
||||
}
|
||||
});
|
||||
// Safety net for a genuinely hung CLI. Note: when the timeout wins the race
|
||||
// below, `invokeCli` is NOT cancellable in-process — it keeps running (its
|
||||
// stdio MCP subprocess stays alive) and any later writes land on the
|
||||
// *restored* real stdout, since `finally` has already unpatched it. The old
|
||||
// out-of-process runner could SIGTERM the child group; this one structurally
|
||||
// cannot. Acceptable because it only fires on the failure path (a hang),
|
||||
// which no passing test should reach.
|
||||
const timeoutMs = options.timeout ?? 10000;
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
const timeout = new Promise<never>((_, reject) => {
|
||||
timer = setTimeout(
|
||||
() => reject(new Error(`CLI command timed out after ${timeoutMs}ms`)),
|
||||
timeoutMs,
|
||||
);
|
||||
});
|
||||
|
||||
let exitCode = 0;
|
||||
try {
|
||||
await Promise.race([invokeCli(argv), timeout]);
|
||||
} catch (error) {
|
||||
exitCode = 1;
|
||||
stderr += (error instanceof Error ? error.message : String(error)) + "\n";
|
||||
} finally {
|
||||
if (timer) clearTimeout(timer);
|
||||
process.stdout.write = originalStdoutWrite;
|
||||
process.stderr.write = originalStderrWrite;
|
||||
console.error = originalConsoleError;
|
||||
console.warn = originalConsoleWarn;
|
||||
for (const [key, value] of Object.entries(envBackup)) {
|
||||
if (value === undefined) delete process.env[key];
|
||||
else process.env[key] = value;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
exitCode,
|
||||
stdout,
|
||||
stderr,
|
||||
output: stdout + stderr,
|
||||
};
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue