inspector/clients/cli/__tests__
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Cliff Hall 388ea5a7bd
test(cli,tui): add in-process source coverage gates (#1484) (#1502)
* test(cli,tui): add in-process source coverage gates (#1484)

CLI and TUI source had no coverage gate: CLI tests spawned the built
binary out-of-process (coverage never saw clients/cli/src) and the TUI
had only two smoke tests. Regressions there were invisible to CI.

CLI:
- Rewrite __tests__/helpers/cli-runner.ts to invoke runCli() in-process
  (capture process.std*.write, map thrown errors to an exit code),
  preserving the CliResult shape so all existing tests run unchanged and
  clients/cli/src is now measured under v8 instrumentation.
- cli.ts: program.exitOverride() so a commander usage error throws
  (catchable in-process) instead of tearing down the worker; --help /
  --version still exit cleanly. Drop the dead, never-imported main().
- Add a thin out-of-process e2e.test.ts that spawns the built binary for
  the shebang / process.exit paths the in-process runner can't reach.
- Add @vitest/coverage-v8, a per-file gate (90/85/80/50) excluding only
  src/index.ts (binary bootstrap), and pool: 'forks' for capture
  isolation. cli.ts lands at ~98% lines / 100% functions.

TUI:
- Add @vitest/coverage-v8 and an interim per-file gate over the
  non-React logic (logger.ts, components/tabsConfig.ts, utils/*); the
  Ink components, App.tsx, and hooks/ are excluded with a documented
  follow-up (#1501). Add tests covering the included modules (100%).

Wiring + docs:
- CI runs each client's test:coverage; root test:cli / test:tui (used by
  validate) and root test:coverage now enforce the gates.
- gitignore clients/**/coverage; update AGENTS.md and client READMEs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(cli): clarify in-process runner footguns from PR #1502 review

Review notes on the in-process CLI test runner (no behavior change):
- Warn in runCli's docblock that --help/--version must NOT go through the
  in-process runner: exitOverride only re-throws non-zero exits, so exitCode
  0 falls through to process.exit(0) and would kill the vitest worker. Those
  belong in the out-of-process e2e.test.ts layer.
- Document that a timeout orphans the uncancellable invokeCli promise (its
  stdio subprocess stays alive; later writes hit the restored stdout) — a
  known, acceptable trade-off on the hang-only failure path.
- Fix a stale reference to the removed main() (errors now route through
  index.ts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(cli): fix remaining stale main() reference in vitest exclusion comment

PR #1502 re-review caught a second stale `main()` mention (the first was
fixed in 52c23e41). src/index.ts inlines the bootstrap as
runCli(process.argv).then(...).catch(handleError) — there is no main().
Comment-only; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: compose root test:coverage from per-client scripts

Replace the brittle inline cd-chain with a composition of the existing
test:cli / test:tui scripts plus a new matching test:web. Behavior is
unchanged (web → cli → tui coverage gates) but there's a single source of
truth for each client's coverage invocation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 15:05:58 -04:00
..
helpers test(cli,tui): add in-process source coverage gates (#1484) (#1502) 2026-06-20 15:05:58 -04:00
cli.test.ts feat(cli): lift config-file headers/timeouts/OAuth into serverSettings (#1482) (#1500) 2026-06-20 11:33:21 -04:00
e2e.test.ts test(cli,tui): add in-process source coverage gates (#1484) (#1502) 2026-06-20 15:05:58 -04:00
error-handler.test.ts test(cli,tui): add in-process source coverage gates (#1484) (#1502) 2026-06-20 15:05:58 -04:00
headers.test.ts feat: port CLI, TUI, and launcher from v1.5 into v2 (#1452) 2026-06-14 20:27:49 -04:00
metadata.test.ts feat: port CLI, TUI, and launcher from v1.5 into v2 (#1452) 2026-06-14 20:27:49 -04:00
methods.test.ts test(cli,tui): add in-process source coverage gates (#1484) (#1502) 2026-06-20 15:05:58 -04:00
README.md test(cli,tui): add in-process source coverage gates (#1484) (#1502) 2026-06-20 15:05:58 -04:00
tools.test.ts feat: port CLI, TUI, and launcher from v1.5 into v2 (#1452) 2026-06-14 20:27:49 -04:00

CLI Tests

Running Tests

# Run all tests (pretest builds test-servers + the CLI binary)
npm test

# Run all tests under the per-file coverage gate
npm run test:coverage

# Run in watch mode (useful for test file changes)
npm run test:watch

# Run specific test file
npm run test:cli          # cli.test.ts
npm run test:cli-tools   # tools.test.ts
npm run test:cli-headers # headers.test.ts
npm run test:cli-metadata # metadata.test.ts

How the CLI is exercised

Most tests run the CLI in-process: helpers/cli-runner.ts imports runCli() from ../src/cli.ts and invokes it directly, capturing process.stdout/stderr and mapping thrown errors to an exit code. This is what lets vitest's v8 coverage instrument clients/cli/src (#1484). Because commander uses .exitOverride(), a parse/usage error throws instead of calling process.exit and tearing down the worker.

e2e.test.ts is the deliberately thin out-of-process layer: it spawns the built build/index.js as a real subprocess to cover the shebang, the index.ts isMain bootstrap, and the actual process.exit codes — the only parts the in-process runner can't reach. (scripts/smoke-cli.mjs at the repo root provides a further end-to-end check of the binary.)

Test Files

  • cli.test.ts - Basic CLI functionality: CLI mode, environment variables, config files, resources, prompts, logging, transport types
  • tools.test.ts - Tool-related tests: Tool discovery, JSON argument parsing, error handling, prompts
  • headers.test.ts - Header parsing and validation
  • metadata.test.ts - Metadata functionality: General metadata, tool-specific metadata, parsing, merging, validation
  • methods.test.ts - Method/option-validation paths not covered elsewhere (resource templates, missing/invalid options, the -- target separator)
  • error-handler.test.ts - The binary's handleError error sink, exercised in-process with process.exit stubbed
  • e2e.test.ts - Out-of-process spawn of the built binary (exit codes + boot)

Helpers

The helpers/ directory contains shared utilities:

  • cli-runner.ts - Invokes runCli() in-process and captures stdout/stderr + exit code
  • assertions.ts - Custom assertion helpers for CLI output validation
  • fixtures.ts - Test config/catalog file generators and temporary directory management

HTTP/SSE test servers (for transport tests) come from the @modelcontextprotocol/inspector-test-server package (createTestServerHttp), and the bundled stdio server is launched via getTestMcpServerCommand().

Notes

  • Test files run in separate forked processes (pool: 'forks'), so the in-process runner's process.std*.write patching never overlaps across files; tests within a file run sequentially
  • Config files use crypto.randomUUID() for uniqueness
  • HTTP/SSE servers use dynamic port allocation to avoid conflicts
  • Coverage is enforced per-file (lines ≥ 90, statements ≥ 85, functions ≥ 80, branches ≥ 50). src/index.ts (the binary bootstrap) is excluded because it only runs in the spawned binary, which coverage can't instrument
  • All tests use the built-in MCP test servers — no external/registry dependencies