CLI: catalog listing, auth flags, and OAuth browser UX (#1782)
* feat(cli): catalog listing, auth flags, and OAuth browser UX Add servers/list and servers/show, --relogin / --stored-auth-only, browser OAuth navigation with OSC 8 links, and extract method handlers under clients/cli/src/handlers. Tracks #1781. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): address #1782 review — relogin clear, stored-auth-only, docs Clear both raw and normalized OAuth keys on --relogin; check the shared store before bailing under --stored-auth-only mid-session; restore the tests README link target; document servers/show redaction limits; keep mcpi groundwork with TODO(#1432) markers. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): gate OAuth browser open and harden #1782 review follow-ups Arm auto-open only for CLI-owned interactive OAuth, honor MCP_AUTO_OPEN_ENABLED, move --relogin clear after validation, skip keychain on servers/list, and cover secret redaction after rehydrate. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): silence disarmed OAuth URLs and round-3 review nits Suppress authorize URL print when navigation is disarmed or --stored-auth-only; document MCP_AUTO_OPEN_ENABLED; redact Cookie/auth headers; keep servers/show unit tests off the OS keychain. Co-authored-by: Cursor <cursoragent@cursor.com> * test(cli): cover withArmedAutoOpen for the ≥90 branch gate CI failed on cliOAuth.ts branch coverage after the auto-open arming helper landed; exercise the armed path through runCliInteractiveOAuth. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): mid-RPC unauthorized recovery and clear round-4 nits Mirror connect's UnauthorizedError path in withCliAuthRecoveryRetry, let MCP_AUTO_OPEN_ENABLED=true force open off-TTY, mock open-url in the in-process runner, reject --relogin on catalog methods, restore the __tests__ README inventory, and narrow OAuth helpers to structural client types. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): reconnect after mid-RPC OAuth and finish round-5 nits Reconnect before retrying after unauthorized recovery, guard non-OAuth configs, decouple forceAutoOpen, move open-url mock to vitest setupFiles, and reject --relogin with stored-auth short-circuits. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): fail fast on non-TTY interactive OAuth and round-6 nits Skip interactive OAuth without a TTY unless MCP_AUTO_OPEN_ENABLED=true; document --stored-auth-only for CI; redact settings.metadata; reject --relogin for stdio; clarify AuthRecoveryRequired reconnect comments. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): admit interactive OAuth when stdin is a TTY (round-7) Gate on stdin||stderr so `2>&1 | tee` still works; fold confirm/auto-open/ TTY into CliOAuthConnectOptions; rename authRequiredFailure; document MCP_AUTO_OPEN_ENABLED=true as non-TTY admit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): fail fast on step-up without stdin (round-8) Decline [y/N] on readline EOF/close, require a stdin TTY for the default step-up confirmer (force-open admit is not enough), document stderr-only browser auto-open vs stdin||stderr flow admit, and drop the alias wrapper. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): allow piped step-up answers; bound silent pipes (round-9) Drop the hard stdin-TTY gate that broke `echo y | …`. Keep EOF decline and add a short non-TTY timeout so an open pipe that never writes cannot hang. Clarify isTTY as admit-only; extract the readline mock factory. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): keep partial piped step-up answers (round-10) Capture readline `line` before close so `printf y` without a newline is not treated as decline; timeout is auth_required with a distinct message; document the 5s bound and post-confirm callback wait; pin the TTY no-timeout arm. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(cli): tighten step-up timeout messaging and TTY test (round-11) Make the TTY no-timeout assertion use a settled flag (the Promise.race check was vacuous), say "timed out" instead of "declined" on the pipe bound, and document that piped answers need a newline or stdin close. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Bob Dickinson
GitHub
parent
a7b2fdb2ef
commit
33e88eac9f
1 changed files with 4041 additions and 520 deletions
|
|
@ -1,73 +1,65 @@
|
|||
# CLI Tests
|
||||
|
||||
## Running Tests
|
||||
Tests live under `__tests__/` and run via Vitest.
|
||||
|
||||
- Most tests import `runCli()` **in-process** (see `helpers/cli-runner.ts`) so
|
||||
`clients/cli/src` is measured under the coverage gate. Suite-wide
|
||||
`helpers/mock-open-url.ts` (vitest `setupFiles`) mocks `open-url` so an armed
|
||||
interactive OAuth path cannot launch a real browser.
|
||||
- `e2e.test.ts` (and root `scripts/smoke-cli.mjs`) spawn the built binary for
|
||||
shebang / `process.exit` paths — `pretest` builds `test-servers` + the CLI
|
||||
bundle first.
|
||||
|
||||
## Scripts (from `clients/cli/`)
|
||||
|
||||
```bash
|
||||
# Run all tests (pretest builds test-servers + the CLI binary)
|
||||
npm test
|
||||
|
||||
# Run all tests under the per-file coverage gate
|
||||
npm run test:coverage
|
||||
|
||||
# Run in watch mode (useful for test file changes)
|
||||
npm run test:watch
|
||||
|
||||
# Run specific test file
|
||||
npm run test:cli # cli.test.ts
|
||||
npm run test:cli-tools # tools.test.ts
|
||||
npm run test:cli-headers # headers.test.ts
|
||||
npm run test:cli-metadata # metadata.test.ts
|
||||
npx vitest run oauth-interactive.test.ts cliOAuth.test.ts # OAuth interactive smoke parity
|
||||
npm test # pretest build + all tests
|
||||
npm run test:cli # subset: cli.test.ts
|
||||
npm run test:cli-tools # subset: tools.test.ts
|
||||
npm run test:cli-headers # subset: headers.test.ts
|
||||
npm run test:cli-metadata # subset: metadata.test.ts
|
||||
npm run test:coverage # build + ≥90 per-file coverage gate
|
||||
npm run validate # format:check && lint && typecheck && test
|
||||
```
|
||||
|
||||
## How the CLI is exercised
|
||||
## Test files
|
||||
|
||||
Most tests run the CLI **in-process**: `helpers/cli-runner.ts` imports `runCli()`
|
||||
from `../src/cli.ts` and invokes it directly, capturing `process.stdout`/`stderr`
|
||||
and mapping thrown errors to an exit code. This is what lets vitest's v8 coverage
|
||||
instrument `clients/cli/src` (#1484). Because `commander` uses `.exitOverride()`,
|
||||
a parse/usage error throws instead of calling `process.exit` and tearing down the
|
||||
worker.
|
||||
|
||||
`e2e.test.ts` is the deliberately thin **out-of-process** layer: it spawns the
|
||||
**built** `build/index.js` as a real subprocess to cover the shebang, the
|
||||
`index.ts` `isMain` bootstrap, and the actual `process.exit` codes — the only
|
||||
parts the in-process runner can't reach. (`scripts/smoke-cli.mjs` at the repo
|
||||
root provides a further end-to-end check of the binary.)
|
||||
|
||||
## Test Files
|
||||
|
||||
- `cli.test.ts` - Basic CLI functionality: CLI mode, environment variables, config files, resources, prompts, logging, transport types
|
||||
- `tools.test.ts` - Tool-related tests: Tool discovery, JSON argument parsing, error handling, prompts
|
||||
- `headers.test.ts` - Header parsing and validation
|
||||
- `metadata.test.ts` - Metadata functionality: General metadata, tool-specific metadata, parsing, merging, validation
|
||||
- `methods.test.ts` - Method/option-validation paths not covered elsewhere (resource templates, missing/invalid options, the `--` target separator)
|
||||
- `error-handler.test.ts` - The binary's `handleError` error sink, exercised in-process with `process.exit` stubbed
|
||||
- `oauth-runner.test.ts` - OAuth flag wiring (`--client-config`, `--callback-url`, CIMD overrides)
|
||||
- `cliOAuth.test.ts` - Unit tests for `cliOAuth.ts` (step-up confirm, helper wiring, retry)
|
||||
- `oauth-interactive.test.ts` - **Integration** smoke parity for CLI interactive OAuth: connect-time callback server + step-up **y/N** against composable `TestServerHttp` (auto-completes authorize URL programmatically; not a subprocess binary e2e)
|
||||
- `e2e.test.ts` - Out-of-process spawn of the built binary (exit codes + boot; no OAuth)
|
||||
| File | Focus |
|
||||
| --------------------------------------- | ------------------------------------------- |
|
||||
| `cli.test.ts` | Core connect / method / config matrix |
|
||||
| `tools.test.ts` | `tools/call` argument coercion |
|
||||
| `headers.test.ts` | `--header` merging |
|
||||
| `metadata.test.ts` | `--metadata` / `--tool-metadata` |
|
||||
| `methods.test.ts` | Method allow-list / rejection |
|
||||
| `app-info.test.ts` | `--app-info` probe paths |
|
||||
| `format-json.test.ts` | `--format json` envelopes |
|
||||
| `format-output.test.ts` | Text/json writers |
|
||||
| `emit-result.test.ts` | Result emission helpers |
|
||||
| `method-types.test.ts` | `ONE_SHOT_METHODS` / guards |
|
||||
| `run-method.test.ts` | Handler dispatch against a real test server |
|
||||
| `run-method-mocks.test.ts` | Handler edge cases with mocks |
|
||||
| `servers-list.test.ts` | `servers/list` / `servers/show` + redaction |
|
||||
| `cliOAuth.test.ts` | Connect / mid-RPC OAuth recovery |
|
||||
| `cli-oauth-navigation.test.ts` | OSC 8, arm/disarm, `MCP_AUTO_OPEN_ENABLED` |
|
||||
| `open-url.test.ts` | `open` package wrapper |
|
||||
| `oauth-runner.test.ts` | Runner client-config / CIMD flags |
|
||||
| `oauth-interactive.test.ts` | Loopback callback OAuth (in-process) |
|
||||
| `stored-auth.test.ts` | `--use-stored-auth` / handoff / wait |
|
||||
| `clear-stored-auth-for-relogin.test.ts` | Dual-key `--relogin` clear |
|
||||
| `programmatic-ergonomics.test.ts` | Flag conflicts, timeouts, ergonomics |
|
||||
| `error-handler.test.ts` | Exit codes / error shaping |
|
||||
| `style.test.ts` | ANSI / OSC 8 helpers |
|
||||
| `e2e.test.ts` | Out-of-process binary smoke |
|
||||
| `helpers/assertions.test.ts` | Assertion helpers |
|
||||
|
||||
## Helpers
|
||||
|
||||
The `helpers/` directory contains shared utilities:
|
||||
| Helper | Role |
|
||||
| -------------------------- | --------------------------------------------------------- |
|
||||
| `helpers/cli-runner.ts` | In-process `runCli` with stdout/stderr capture |
|
||||
| `helpers/mock-open-url.ts` | Suite-wide `open-url` mock (vitest `setupFiles`) |
|
||||
| `helpers/assertions.ts` | `expectCliSuccess` / `expectCliFailure` / output matchers |
|
||||
| `helpers/fixtures.ts` | Temp config / client.json factories |
|
||||
|
||||
- `cli-runner.ts` - Invokes `runCli()` in-process and captures stdout/stderr + exit code
|
||||
- `assertions.ts` - Custom assertion helpers for CLI output validation
|
||||
- `fixtures.ts` - Test config/catalog file generators and temporary directory management
|
||||
|
||||
HTTP/SSE test servers (for transport tests) come from the
|
||||
`@modelcontextprotocol/inspector-test-server` package (`createTestServerHttp`),
|
||||
and the bundled stdio server is launched via `getTestMcpServerCommand()`.
|
||||
|
||||
## Notes
|
||||
|
||||
- Test files run in separate forked processes (`pool: 'forks'`), so the in-process
|
||||
runner's `process.std*.write` patching never overlaps across files; tests within
|
||||
a file run sequentially
|
||||
- Config files use `crypto.randomUUID()` for uniqueness
|
||||
- HTTP/SSE servers use dynamic port allocation to avoid conflicts
|
||||
- Coverage is enforced per-file (lines ≥ 90, statements ≥ 85, functions ≥ 80,
|
||||
branches ≥ 50). `src/index.ts` (the binary bootstrap) is excluded because it
|
||||
only runs in the spawned binary, which coverage can't instrument
|
||||
- All tests use the built-in MCP test servers — no external/registry dependencies
|
||||
OAuth interactive tests stub or drive the loopback callback in-process rather
|
||||
than requiring a real browser.
|
||||
|
|
|
|||
Loading…
Reference in a new issue