Commit graph inspector/clients/cli/__tests__/schema-lint-report.test.ts
Author SHA1 Message Date
cliffhall
a3d864ed6e fix: rename the strict exit code, cover the config path, refresh the docs
Copilot review round 3.

- Rename the `--strict` failure from "invalid" to "unportable"
  (`EXIT_CODES.SCHEMA_UNPORTABLE`, envelope `schema_unportable`, and the
  message text). The module argues at length that these schemas ARE
  valid JSON Schema and are merely refused by some clients, so a
  machine-readable code of `schema_invalid` told automated callers the
  opposite. Done now, while nothing depends on it.

- Add a fixture-path integration case. Every other test built a
  `ServerConfig` by hand, so `resolveConfig`'s `rawToolSchemas`
  forwarding was uncovered and `unportable-schemas-http.json` was loaded
  by nothing — the shipped fixture could have lost its overrides with
  the suite still green. It now goes through loadConfig/resolveConfig
  and asserts the exact per-tool verdict the README claims.

- Refresh the CLI README's sample output, which still showed the
  pre-round-2 wording and fixture, and so dropped the contract warning
  that rewording existed to add. Captured from a real run rather than
  written by hand, plus a paragraph making the suggestion pattern
  explicit.

- Drop the self-referential "[Try it](#unportable-tool-schemas)" link in
  the root README, which pointed back at its own section.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UL9retfvAXRgvi6EWk4SY4
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-25 02:17:58 -04:00
cliffhall
c389565783 fix: address Copilot review on the schema portability lint
- Scope `non-object-root` to `inputSchema`. The pinned SDK types
  `inputSchema` with `type: literal("object")` but `outputSchema` as a
  bare `looseObject`, so a `{"type":"string"}` output is conforming; the
  rule was claiming a requirement MCP does not make, at error severity.

- Decide `untyped-schema` from an allowlist of constraining keywords
  rather than a denylist of annotations. JSON Schema ignores keywords it
  does not recognize, so `{"vendorHint": true}` accepts every value and a
  denylist passed it as constrained. `$defs`-only schemas now flag too.

- Walk `dependentSchemas` and draft-07 `dependencies`, with a test
  pinning that the property-name-array form is not a schema position.

- Suggest `anyOf` branches for an array-form `type`, never "drop it from
  `required`" — accepting `null` and permitting absence are independent
  contracts, so the old suggestion silently changed the schema's meaning.

- Reject `--strict` alongside `--app-info`. That path returns NDJSON from
  `runMethod` and never reaches the lint, so the flag was a no-op gate
  that could never fail.

- Colour the web sidebar flag from the `--inspector-*` severity tokens,
  matching the detail panel and avoiding the default yellow that failed
  the a11y check.

- Add a live integration test for `rawToolSchemas`. It caught that an
  `outputSchema` override on a preset returning no structured content
  makes every call to that tool fail client-side validation — the
  showcase config did exactly that, so the flagged tool could not be
  run. Moved the bare `true` onto `get_temp`, documented the caveat, and
  pinned both halves.

- Drop the double cast from the lint test fixture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UL9retfvAXRgvi6EWk4SY4
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-25 01:41:12 -04:00
cliffhall
2f9e0901c1 feat: flag unportable tool schemas in all three clients
Closes #1005

A tool schema can be perfectly legal JSON Schema and still be rejected by
the client the server is meant to run against — Go's `jsonschema` emits a
bare `true` for `interface{}`, which the SDK accepts and Claude Code
refuses with an opaque "Invalid input". The Inspector passed it through
silently.

Adds `core/json/schemaLint.ts`, a portability lint over a tool's
inputSchema/outputSchema, and surfaces one shared verdict everywhere:
the CLI's `--strict` report (exit 6 on an error-severity finding), the
TUI's tool list marker + detail pane, and the web Tools sidebar flag +
detail section.

Deliberately not a JSON Schema validator: a census of 617 public servers
found zero that fail the SDK's own parse, so a conformance check would
report nothing on real servers. Each rule is instead a construct that is
legal JSON Schema and known to be refused or mishandled by a shipping
client — a bare boolean in a schema position, an array-form `type`, a
remote `$ref`, a schema with no validation keyword, a non-object root.

`test-servers/configs/unportable-schemas-http.json` exercises every rule,
via a new `rawToolSchemas` override — the Zod-built presets cannot emit
any of these shapes, which is the same reason a real server only hits
this when its schemas come from another generator.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UL9retfvAXRgvi6EWk4SY4
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-25 01:18:53 -04:00