388ea5a7bd* test(cli,tui): add in-process source coverage gates (#1484) CLI and TUI source had no coverage gate: CLI tests spawned the built binary out-of-process (coverage never saw clients/cli/src) and the TUI had only two smoke tests. Regressions there were invisible to CI. CLI: - Rewrite __tests__/helpers/cli-runner.ts to invoke runCli() in-process (capture process.std*.write, map thrown errors to an exit code), preserving the CliResult shape so all existing tests run unchanged and clients/cli/src is now measured under v8 instrumentation. - cli.ts: program.exitOverride() so a commander usage error throws (catchable in-process) instead of tearing down the worker; --help / --version still exit cleanly. Drop the dead, never-imported main(). - Add a thin out-of-process e2e.test.ts that spawns the built binary for the shebang / process.exit paths the in-process runner can't reach. - Add @vitest/coverage-v8, a per-file gate (90/85/80/50) excluding only src/index.ts (binary bootstrap), and pool: 'forks' for capture isolation. cli.ts lands at ~98% lines / 100% functions. TUI: - Add @vitest/coverage-v8 and an interim per-file gate over the non-React logic (logger.ts, components/tabsConfig.ts, utils/*); the Ink components, App.tsx, and hooks/ are excluded with a documented follow-up (#1501). Add tests covering the included modules (100%). Wiring + docs: - CI runs each client's test:coverage; root test:cli / test:tui (used by validate) and root test:coverage now enforce the gates. - gitignore clients/**/coverage; update AGENTS.md and client READMEs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): clarify in-process runner footguns from PR #1502 review Review notes on the in-process CLI test runner (no behavior change): - Warn in runCli's docblock that --help/--version must NOT go through the in-process runner: exitOverride only re-throws non-zero exits, so exitCode 0 falls through to process.exit(0) and would kill the vitest worker. Those belong in the out-of-process e2e.test.ts layer. - Document that a timeout orphans the uncancellable invokeCli promise (its stdio subprocess stays alive; later writes hit the restored stdout) — a known, acceptable trade-off on the hang-only failure path. - Fix a stale reference to the removed main() (errors now route through index.ts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): fix remaining stale main() reference in vitest exclusion comment PR #1502 re-review caught a second stale `main()` mention (the first was fixed in 52c23e41). src/index.ts inlines the bootstrap as runCli(process.argv).then(...).catch(handleError) — there is no main(). Comment-only; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: compose root test:coverage from per-client scripts Replace the brittle inline cd-chain with a composition of the existing test:cli / test:tui scripts plus a new matching test:web. Behavior is unchanged (web → cli → tui coverage gates) but there's a single source of truth for each client's coverage invocation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
69 lines
3.1 KiB
Markdown
69 lines
3.1 KiB
Markdown
# CLI Tests
|
|
|
|
## Running Tests
|
|
|
|
```bash
|
|
# Run all tests (pretest builds test-servers + the CLI binary)
|
|
npm test
|
|
|
|
# Run all tests under the per-file coverage gate
|
|
npm run test:coverage
|
|
|
|
# Run in watch mode (useful for test file changes)
|
|
npm run test:watch
|
|
|
|
# Run specific test file
|
|
npm run test:cli # cli.test.ts
|
|
npm run test:cli-tools # tools.test.ts
|
|
npm run test:cli-headers # headers.test.ts
|
|
npm run test:cli-metadata # metadata.test.ts
|
|
```
|
|
|
|
## How the CLI is exercised
|
|
|
|
Most tests run the CLI **in-process**: `helpers/cli-runner.ts` imports `runCli()`
|
|
from `../src/cli.ts` and invokes it directly, capturing `process.stdout`/`stderr`
|
|
and mapping thrown errors to an exit code. This is what lets vitest's v8 coverage
|
|
instrument `clients/cli/src` (#1484). Because `commander` uses `.exitOverride()`,
|
|
a parse/usage error throws instead of calling `process.exit` and tearing down the
|
|
worker.
|
|
|
|
`e2e.test.ts` is the deliberately thin **out-of-process** layer: it spawns the
|
|
**built** `build/index.js` as a real subprocess to cover the shebang, the
|
|
`index.ts` `isMain` bootstrap, and the actual `process.exit` codes — the only
|
|
parts the in-process runner can't reach. (`scripts/smoke-cli.mjs` at the repo
|
|
root provides a further end-to-end check of the binary.)
|
|
|
|
## Test Files
|
|
|
|
- `cli.test.ts` - Basic CLI functionality: CLI mode, environment variables, config files, resources, prompts, logging, transport types
|
|
- `tools.test.ts` - Tool-related tests: Tool discovery, JSON argument parsing, error handling, prompts
|
|
- `headers.test.ts` - Header parsing and validation
|
|
- `metadata.test.ts` - Metadata functionality: General metadata, tool-specific metadata, parsing, merging, validation
|
|
- `methods.test.ts` - Method/option-validation paths not covered elsewhere (resource templates, missing/invalid options, the `--` target separator)
|
|
- `error-handler.test.ts` - The binary's `handleError` error sink, exercised in-process with `process.exit` stubbed
|
|
- `e2e.test.ts` - Out-of-process spawn of the built binary (exit codes + boot)
|
|
|
|
## Helpers
|
|
|
|
The `helpers/` directory contains shared utilities:
|
|
|
|
- `cli-runner.ts` - Invokes `runCli()` in-process and captures stdout/stderr + exit code
|
|
- `assertions.ts` - Custom assertion helpers for CLI output validation
|
|
- `fixtures.ts` - Test config/catalog file generators and temporary directory management
|
|
|
|
HTTP/SSE test servers (for transport tests) come from the
|
|
`@modelcontextprotocol/inspector-test-server` package (`createTestServerHttp`),
|
|
and the bundled stdio server is launched via `getTestMcpServerCommand()`.
|
|
|
|
## Notes
|
|
|
|
- Test files run in separate forked processes (`pool: 'forks'`), so the in-process
|
|
runner's `process.std*.write` patching never overlaps across files; tests within
|
|
a file run sequentially
|
|
- Config files use `crypto.randomUUID()` for uniqueness
|
|
- HTTP/SSE servers use dynamic port allocation to avoid conflicts
|
|
- Coverage is enforced per-file (lines ≥ 90, statements ≥ 85, functions ≥ 80,
|
|
branches ≥ 50). `src/index.ts` (the binary bootstrap) is excluded because it
|
|
only runs in the spawned binary, which coverage can't instrument
|
|
- All tests use the built-in MCP test servers — no external/registry dependencies
|