* test(cli,tui): add in-process source coverage gates (#1484) CLI and TUI source had no coverage gate: CLI tests spawned the built binary out-of-process (coverage never saw clients/cli/src) and the TUI had only two smoke tests. Regressions there were invisible to CI. CLI: - Rewrite __tests__/helpers/cli-runner.ts to invoke runCli() in-process (capture process.std*.write, map thrown errors to an exit code), preserving the CliResult shape so all existing tests run unchanged and clients/cli/src is now measured under v8 instrumentation. - cli.ts: program.exitOverride() so a commander usage error throws (catchable in-process) instead of tearing down the worker; --help / --version still exit cleanly. Drop the dead, never-imported main(). - Add a thin out-of-process e2e.test.ts that spawns the built binary for the shebang / process.exit paths the in-process runner can't reach. - Add @vitest/coverage-v8, a per-file gate (90/85/80/50) excluding only src/index.ts (binary bootstrap), and pool: 'forks' for capture isolation. cli.ts lands at ~98% lines / 100% functions. TUI: - Add @vitest/coverage-v8 and an interim per-file gate over the non-React logic (logger.ts, components/tabsConfig.ts, utils/*); the Ink components, App.tsx, and hooks/ are excluded with a documented follow-up (#1501). Add tests covering the included modules (100%). Wiring + docs: - CI runs each client's test:coverage; root test:cli / test:tui (used by validate) and root test:coverage now enforce the gates. - gitignore clients/**/coverage; update AGENTS.md and client READMEs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): clarify in-process runner footguns from PR #1502 review Review notes on the in-process CLI test runner (no behavior change): - Warn in runCli's docblock that --help/--version must NOT go through the in-process runner: exitOverride only re-throws non-zero exits, so exitCode 0 falls through to process.exit(0) and would kill the vitest worker. Those belong in the out-of-process e2e.test.ts layer. - Document that a timeout orphans the uncancellable invokeCli promise (its stdio subprocess stays alive; later writes hit the restored stdout) — a known, acceptable trade-off on the hang-only failure path. - Fix a stale reference to the removed main() (errors now route through index.ts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): fix remaining stale main() reference in vitest exclusion comment PR #1502 re-review caught a second stale `main()` mention (the first was fixed in 52c23e41). src/index.ts inlines the bootstrap as runCli(process.argv).then(...).catch(handleError) — there is no main(). Comment-only; no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: compose root test:coverage from per-client scripts Replace the brittle inline cd-chain with a composition of the existing test:cli / test:tui scripts plus a new matching test:web. Behavior is unchanged (web → cli → tui coverage gates) but there's a single source of truth for each client's coverage invocation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
e72d4307f3
commit
388ea5a7bd
1 changed files with 1304 additions and 197 deletions
|
|
@ -121,7 +121,10 @@ All work should be driven by items on the project board.
|
|||
- Run CLI tests with `npm run test` from `clients/cli/` (builds test-servers + CLI bin first via `pretest`)
|
||||
- Run TUI tests with `npm run test` from `clients/tui/`
|
||||
- From repo root, `npm run test` runs web unit tests, then CLI and TUI tests
|
||||
- Run `npm run test:coverage` to verify the per-file gate: lines ≥ 90, statements ≥ 85, functions ≥ 80, branches ≥ 50 (CI enforces this gate). Branches is intentionally relaxed because Mantine portal/media-query branches are not exercisable under happy-dom; new business-logic branches should still be covered.
|
||||
- Run `npm run test:coverage` to verify the per-file gate: lines ≥ 90, statements ≥ 85, functions ≥ 80, branches ≥ 50 (CI enforces this gate). Branches is intentionally relaxed because Mantine portal/media-query branches are not exercisable under happy-dom; new business-logic branches should still be covered. From the repo root, `npm run test:coverage` runs the gate for web **and** CLI **and** TUI; each client also exposes its own `test:coverage`.
|
||||
- The **same per-file gate** is enforced for the CLI and TUI (#1484), not just web:
|
||||
- **CLI** (`clients/cli`): tests run **in-process** by importing `runCli()` (see `__tests__/helpers/cli-runner.ts`) so `clients/cli/src` is measured under v8 instrumentation. A thin out-of-process layer (`__tests__/e2e.test.ts` + `scripts/smoke-cli.mjs`) still spawns the built binary for the shebang/`process.exit` paths; `src/index.ts` (binary bootstrap) is the only coverage exclusion. `commander` uses `.exitOverride()` so a parse error throws instead of tearing down the test worker.
|
||||
- **TUI** (`clients/tui`): the gate covers the **non-React logic** only — `logger.ts`, `components/tabsConfig.ts`, and `utils/*` (server resolution lives in `core/` and is measured by the web suite). The Ink components, `App.tsx`, and `hooks/` are an **interim exclusion** in `clients/tui/vitest.config.ts` pending the renderer-based follow-up (#1501). When adding new **non-React** logic under `clients/tui/src`, it falls under the gate automatically — add tests for it.
|
||||
- Run `npm run test:integration` (also from `clients/web/`) for the v1.5-ported InspectorClient + transport + auth integration suite. It runs under a separate `integration` vitest project in node env (no happy-dom) with 30s timeouts. The script builds `test-servers/` first via `tsc -p ../../test-servers --noCheck` so the stdio MCP test server can be spawned as a real subprocess. CI runs it as its own step after unit tests.
|
||||
- Test files live alongside the source as `<Name>.test.tsx` (or `.test.ts` for non-React modules). v1.5-ported integration tests live under `clients/web/src/test/integration/`, mirroring the `core/` source layout (`mcp/`, `mcp/node/`, `mcp/remote/`, `auth/`, `auth/node/`, `storage/`). Any test file under that folder is automatically picked up by the `integration` vitest project (node env, 30s timeouts) via the folder glob in `vite.config.ts` — placement is the manifest, there is no enumeration to keep in sync. Tests outside the folder run in the `unit` project (happy-dom). When adding a new test for, e.g., `core/mcp/remote/foo.ts`, put it at `src/test/integration/mcp/remote/foo.test.ts`.
|
||||
- Use `renderWithMantine` from `src/test/renderWithMantine.tsx` to render components — it wraps in `MantineProvider` with the project theme
|
||||
|
|
@ -134,7 +137,7 @@ All work should be driven by items on the project board.
|
|||
|
||||
### Lint-fixed, Formatted code
|
||||
- ALWAYS do `npm run format` before committing — it auto-fixes any Prettier issues. `validate` runs `format:check` (the non-fixing variant) and will fail in CI on any unformatted file, so always run the auto-fixer first rather than letting `format:check` catch it.
|
||||
- ALWAYS do `npm run validate` before pushing any changes — from the repo root it mirrors what CI enforces across every client: web `format:check` + `lint` + `build` + `test:coverage` (`validate:web`), then the CLI and TUI test suites (`test:cli` / `test:tui`), then a build of CLI + TUI + launcher, the launcher `--help` smoke (`smoke:launcher`), the prod CLI and TUI end-to-end smokes (`smoke:cli` / `smoke:tui`), and the prod web end-to-end smoke (`smoke:web`) — together `validate:launcher`. The sub-scripts (`validate:web`, `validate:launcher`, `smoke:launcher`, `smoke:cli`, `smoke:tui`, `smoke:web`) can be run individually when iterating on one client. Storybook is the only CI step left out (see below).
|
||||
- ALWAYS do `npm run validate` before pushing any changes — from the repo root it mirrors what CI enforces across every client: web `format:check` + `lint` + `build` + `test:coverage` (`validate:web`), then the CLI and TUI suites **with their coverage gates** (`test:cli` / `test:tui`, which now run each client's `test:coverage`), then a build of CLI + TUI + launcher, the launcher `--help` smoke (`smoke:launcher`), the prod CLI and TUI end-to-end smokes (`smoke:cli` / `smoke:tui`), and the prod web end-to-end smoke (`smoke:web`) — together `validate:launcher`. The sub-scripts (`validate:web`, `validate:launcher`, `smoke:launcher`, `smoke:cli`, `smoke:tui`, `smoke:web`) can be run individually when iterating on one client. Storybook is the only CI step left out (see below).
|
||||
- `smoke:web` (`scripts/smoke-web.mjs`) starts `mcp-inspector --web` (prod, no `--dev`) against the built `clients/web/dist` and asserts `GET /` serves the SPA (HTTP 200) with the injected `__INSPECTOR_API_TOKEN__`. Prod `--web` serves from `clients/web/dist`, which ships in the published package but is absent in a fresh checkout — the runner builds it on demand (`build:client` = `vite build`) on first launch, or exits with an actionable error if that build can't run (see `clients/web/server/ensure-web-build.ts` and the launcher README). `--dev` runs Vite directly and never needs `dist`.
|
||||
- `smoke:cli` (`scripts/smoke-cli.mjs`) drives `mcp-inspector --cli` through the built launcher against the bundled stdio test server via a temp `--catalog`: it asserts `tools/list` returns the server's tools (real connect over stdio), the default writable catalog is seeded empty on first run, a missing read-only `--config` errors without seeding, and `--catalog` + `--config` is rejected. `smoke:tui` (`scripts/smoke-tui.mjs`) launches `mcp-inspector --tui --catalog <temp>` and asserts the Ink app renders its first frame (the "MCP Servers" panel) within a timeout, then SIGTERMs it — a shallow boot/render check, not full interaction. Both build `test-servers/build` on demand if it's missing.
|
||||
- Also run `npm run test:storybook` before pushing — it executes every story's `play` function in headless Chromium via `@vitest/browser-playwright` (~10s). CI runs this as a separate step after the unit/lint/build checks; failures block merge. It is kept out of `validate` because it needs the Playwright browser binary and is much slower than the unit suite.
|
||||
|
|
|
|||
Loading…
Reference in a new issue