Four distinct issues across 2 inline and 3 suppressed comments. All were
verified against the CLI's own code and behavior before being accepted;
one was diagnosed correctly but explained wrongly, and is documented as
it actually behaves rather than as the comment stated.
1. MCP_STORAGE_DIR alone is not isolation. MCP_INSPECTOR_OAUTH_STATE_PATH
is checked first, so an inherited value silently wins and a smoke run
still reaches the real token file — the guide documented that
precedence and then ignored it one paragraph later. Both variables are
now exported, in precedence order, in §6 and in the §8 script, with a
--list-stored-auth check that prints the path actually resolved.
Verified: with a poisoned MCP_INSPECTOR_OAUTH_STATE_PATH in the
environment, the two-line export wins.
2. `--format json` can carry a third top-level key. `emitResult` sets
`schemaFindings` whenever --strict is passed to tools/list and there
is at least one finding, so "that is the whole contract" was wrong.
Replaced with a table of all three keys and a note to parse by key.
3. --strict findings under --format json. The actionable half is right —
a JSON pipeline should read `schemaFindings` from stdout. The stated
mechanism is not: `writeSchemaLintReport` runs unconditionally, so
--format json --strict emits the full human report on stderr AND the
structured copy on stdout. Documented both streams as measured, with
a jq gate over the per-tool grouping. The `?` operators in that filter
are load-bearing: the key is absent on a clean run, and without them
jq errors instead of passing. Both arms verified.
4. The ErrorEnvelope snippet returned 0. `cmd || { … }` makes the brace
group's status the list's, so a successful jq turned a failed run
green — the same failure class the guide warns about twice elsewhere,
which makes shipping it worse. Now captures, reports, and exits with
the status; verified propagating exit 4.
The §8 script was re-run end to end against a live HTTP test server with
the added export and still passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KsKC3HniVF9dCZtDVbYp5B
Signed-off-by: cliffhall <cliff@futurescale.com>
Three findings, all real.
1. The secret-scan snippet swallowed the CLI's exit code. `out=$(…)`
with no status capture means a non-matching `grep` returns 1, the
`if` is simply not taken, and the step exits 0 even though the tool
call failed — the scan silently becomes the only assertion. Under
`set -e` the opposite happens: the script dies at the assignment and
never scans the error output, which is exactly where a leaked
credential is most likely to appear. Now captures with `|| status=$?`,
scans, then propagates, with a callout explaining both failure modes.
2. `@2` is a major-version range, not a pin, and it was written directly
under a sentence claiming it pinned the version. Both occurrences now
use an exact published release (2.5.0, the current `latest`), and the
prerequisites note says outright that a range is not a pin. `--yes`
added to the prerequisite form so an unattended job cannot hang on
the first-run install prompt.
3. `--stored-auth-only` was described as failing when the store has no
token. It is a no-op against a server that never challenges — the
guide's own verification run passed with an empty store against a
non-auth server. Reworded to say exit 3 happens only when a challenge
cannot be satisfied, and added the way to actually assert that
authentication happened: one run with an isolated empty
MCP_STORAGE_DIR that requires exit 3.
Both corrected shell snippets were executed: secret+failure exits 1,
failure without a secret propagates exit 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KsKC3HniVF9dCZtDVbYp5B
Signed-off-by: cliffhall <cliff@futurescale.com>
Adds `docs/cli-smoke-testing.md` — a copyable connect → list → call →
assert workflow for testing an arbitrary MCP server from a shell or a CI
job, plus index rows in the root README and a pointer from the CLI
README.
The gap this fills is a user-facing one. `clients/cli/README.md` is a
flag reference and `scripts/smoke-cli.mjs` is an internal end-to-end
check of the launcher → CLI path; neither tells someone how to compose
those flags into a check they run on every commit against their own
server.
Covers, in order: connecting over stdio/HTTP/SSE (including the `--`
separator, which splits the opposite way under `--cli`); `--format json`
and its `{result[, appInfo]}` envelope; asserting a tool exists with
`jq -e`; one representative `tools/call` and the `--tool-arg` vs
`--tool-args-json` coercion difference; the exit-code map and how to
branch on it; keeping OAuth non-interactive with `--stored-auth-only`
and an isolated `MCP_STORAGE_DIR`; and negative assertions — refusals,
a secret-shape scan of captured output, and `--strict` schema
portability. Ends with a complete script and a GitHub Actions job.
Every command and output shape in the guide was verified against the
bundled composable test servers over both stdio and HTTP rather than
transcribed from the reference, which corrected three claims that would
otherwise have shipped wrong:
- `--tool-arg zip=012` does not send `12`. `012` is not valid JSON, so
it is sent as the literal string. The real trap is `zip=10001`, which
does coerce to a number and is then rejected by a string schema.
- `if ! cmd; then status=$?` yields `0`, because `!` inverts the
pipeline's status — every failure class would look identical. The
guide uses `|| status=$?` and says why.
- The sample script does not report a failure class on its own; that is
the opt-in `case` block from §5.
The §8 script was run end to end against a live HTTP test server, and
exits 0/3/4/5/6 were each reproduced against a real server.
Closes#1886
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KsKC3HniVF9dCZtDVbYp5B
Signed-off-by: cliffhall <cliff@futurescale.com>
