inspector/clients/cli/__tests__/e2e.test.ts
Cliff Hall 388ea5a7bd
test(cli,tui): add in-process source coverage gates (#1484) (#1502)
* test(cli,tui): add in-process source coverage gates (#1484)

CLI and TUI source had no coverage gate: CLI tests spawned the built
binary out-of-process (coverage never saw clients/cli/src) and the TUI
had only two smoke tests. Regressions there were invisible to CI.

CLI:
- Rewrite __tests__/helpers/cli-runner.ts to invoke runCli() in-process
  (capture process.std*.write, map thrown errors to an exit code),
  preserving the CliResult shape so all existing tests run unchanged and
  clients/cli/src is now measured under v8 instrumentation.
- cli.ts: program.exitOverride() so a commander usage error throws
  (catchable in-process) instead of tearing down the worker; --help /
  --version still exit cleanly. Drop the dead, never-imported main().
- Add a thin out-of-process e2e.test.ts that spawns the built binary for
  the shebang / process.exit paths the in-process runner can't reach.
- Add @vitest/coverage-v8, a per-file gate (90/85/80/50) excluding only
  src/index.ts (binary bootstrap), and pool: 'forks' for capture
  isolation. cli.ts lands at ~98% lines / 100% functions.

TUI:
- Add @vitest/coverage-v8 and an interim per-file gate over the
  non-React logic (logger.ts, components/tabsConfig.ts, utils/*); the
  Ink components, App.tsx, and hooks/ are excluded with a documented
  follow-up (#1501). Add tests covering the included modules (100%).

Wiring + docs:
- CI runs each client's test:coverage; root test:cli / test:tui (used by
  validate) and root test:coverage now enforce the gates.
- gitignore clients/**/coverage; update AGENTS.md and client READMEs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(cli): clarify in-process runner footguns from PR #1502 review

Review notes on the in-process CLI test runner (no behavior change):
- Warn in runCli's docblock that --help/--version must NOT go through the
  in-process runner: exitOverride only re-throws non-zero exits, so exitCode
  0 falls through to process.exit(0) and would kill the vitest worker. Those
  belong in the out-of-process e2e.test.ts layer.
- Document that a timeout orphans the uncancellable invokeCli promise (its
  stdio subprocess stays alive; later writes hit the restored stdout) — a
  known, acceptable trade-off on the hang-only failure path.
- Fix a stale reference to the removed main() (errors now route through
  index.ts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(cli): fix remaining stale main() reference in vitest exclusion comment

PR #1502 re-review caught a second stale `main()` mention (the first was
fixed in 52c23e41). src/index.ts inlines the bootstrap as
runCli(process.argv).then(...).catch(handleError) — there is no main().
Comment-only; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: compose root test:coverage from per-client scripts

Replace the brittle inline cd-chain with a composition of the existing
test:cli / test:tui scripts plus a new matching test:web. Behavior is
unchanged (web → cli → tui coverage gates) but there's a single source of
truth for each client's coverage invocation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 15:05:58 -04:00

70 lines
2.4 KiB
TypeScript

import { describe, it, expect } from "vitest";
import { spawn } from "node:child_process";
import { resolve, dirname } from "node:path";
import { fileURLToPath } from "node:url";
import { getTestMcpServerCommand } from "@modelcontextprotocol/inspector-test-server";
const here = dirname(fileURLToPath(import.meta.url));
const BIN = resolve(here, "../build/index.js");
interface SpawnResult {
exitCode: number | null;
stdout: string;
stderr: string;
}
/**
* Spawn the **built** CLI binary as a real subprocess. This is the deliberately
* thin out-of-process layer that the in-process suite (cli-runner.ts) cannot
* reach: the shebang, `index.ts`'s `isMain` bootstrap, and the actual
* `process.exit` codes. Functional behavior is covered in-process under the
* coverage gate; this only asserts the binary boots and exits correctly. The
* binary is built by the `pretest` / `test:coverage` scripts before tests run.
*/
function spawnCli(args: string[]): Promise<SpawnResult> {
return new Promise((resolvePromise, reject) => {
const child = spawn("node", [BIN, ...args], {
stdio: ["pipe", "pipe", "pipe"],
detached: process.platform !== "win32",
});
let stdout = "";
let stderr = "";
const timer = setTimeout(() => {
if (process.platform !== "win32" && child.pid != null) {
process.kill(-child.pid, "SIGTERM");
} else {
child.kill("SIGTERM");
}
reject(new Error("E2E CLI timed out"));
}, 15000);
child.stdout.on("data", (d) => (stdout += d.toString()));
child.stderr.on("data", (d) => (stderr += d.toString()));
child.on("error", (err) => {
clearTimeout(timer);
reject(err);
});
child.on("close", (exitCode) => {
clearTimeout(timer);
resolvePromise({ exitCode, stdout, stderr });
});
});
}
describe("CLI binary (out-of-process E2E)", () => {
const { command, args } = getTestMcpServerCommand();
it("exits 0 and prints tools JSON on a successful run", async () => {
const result = await spawnCli([command, ...args, "--method", "tools/list"]);
expect(result.exitCode).toBe(0);
const json = JSON.parse(result.stdout);
expect(Array.isArray(json.tools)).toBe(true);
});
it("exits non-zero when required --method is missing", async () => {
const result = await spawnCli([command, ...args]);
expect(result.exitCode).not.toBe(0);
expect(result.stderr).toContain("Method is required");
});
});