feat: flag unportable tool schemas in all three clients
Closes #1005 A tool schema can be perfectly legal JSON Schema and still be rejected by the client the server is meant to run against — Go's `jsonschema` emits a bare `true` for `interface{}`, which the SDK accepts and Claude Code refuses with an opaque "Invalid input". The Inspector passed it through silently. Adds `core/json/schemaLint.ts`, a portability lint over a tool's inputSchema/outputSchema, and surfaces one shared verdict everywhere: the CLI's `--strict` report (exit 6 on an error-severity finding), the TUI's tool list marker + detail pane, and the web Tools sidebar flag + detail section. Deliberately not a JSON Schema validator: a census of 617 public servers found zero that fail the SDK's own parse, so a conformance check would report nothing on real servers. Each rule is instead a construct that is legal JSON Schema and known to be refused or mishandled by a shipping client — a bare boolean in a schema position, an array-form `type`, a remote `$ref`, a schema with no validation keyword, a non-object root. `test-servers/configs/unportable-schemas-http.json` exercises every rule, via a new `rawToolSchemas` override — the Zod-built presets cannot emit any of these shapes, which is the same reason a real server only hits this when its schemas come from another generator. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UL9retfvAXRgvi6EWk4SY4 Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall
parent
2b129087a3
commit
2f9e0901c1
1 changed files with 1979 additions and 13 deletions
47
README.md
47
README.md
|
|
@ -34,8 +34,9 @@ inspector/
|
|||
│ │ # plus per-server secret storage — the keychain/file/memory SecretStore
|
||||
│ │ # implementations, the selection policy, and the descriptor the banner and UI report
|
||||
│ ├── client/ # Install-level client config (`client.json`): browser-safe parse/validate + Node load/save, remote backend, secrets
|
||||
│ ├── json/ # JSON + parameter/argument conversion utilities, and the nullable-union
|
||||
│ │ # schema collapse shared by the web and TUI form builders
|
||||
│ ├── json/ # JSON + parameter/argument conversion utilities, the nullable-union
|
||||
│ │ # schema collapse shared by the web and TUI form builders, and the
|
||||
│ │ # tool-schema portability lint all three clients report from
|
||||
│ ├── logging/ # Silent pino logger singleton
|
||||
│ ├── mcp/ # InspectorClient runtime, state stores, transports, config import,
|
||||
│ │ # and the RFC 6570 URI-template helpers the web form and TUI expand through
|
||||
|
|
@ -152,6 +153,7 @@ Each config below is a ready-made server for exercising one feature by hand. Loa
|
|||
| `structured-output-http.json` | Tools tab: a result's `structuredContent` section | [#1908](https://github.com/modelcontextprotocol/inspector/issues/1908) |
|
||||
| `duplicate-tool-names-http.json` | A `tools/list` that repeats a tool name | [#1957](https://github.com/modelcontextprotocol/inspector/issues/1957) |
|
||||
| `nullable-fields-http.json` | Tools tab: nullable (`anyOf` + `null`) arguments | [#1928](https://github.com/modelcontextprotocol/inspector/issues/1928) |
|
||||
| `unportable-schemas-http.json` **(legacy era)** | Tool schemas a real client rejects, flagged in all three clients | [#1005](https://github.com/modelcontextprotocol/inspector/issues/1005) |
|
||||
| `rfc6570-templates-http.json` | Resources tab: RFC 6570 resource-template expansion | [#1919](https://github.com/modelcontextprotocol/inspector/issues/1919) |
|
||||
| `advertised-extensions-http.json` | Tool registration gated on advertised extensions | [#1739](https://github.com/modelcontextprotocol/inspector/issues/1739) |
|
||||
| `oauth-custom-resource-metadata-http.json` **(legacy era)** | OAuth discovery driven by the challenge's `resource_metadata` | [#2071](https://github.com/modelcontextprotocol/inspector/issues/2071) |
|
||||
|
|
@ -286,6 +288,47 @@ Open the Tools tab and select `record_shipment`: `direction` must render as a **
|
|||
|
||||
The **TUI** had the same gap and is worth checking against the same server (`--tui`, then test `record_shipment`): `direction` is a select, `quantity` an integer field, `express` a boolean. Both clients now share one collapse step — `normalizeNullableUnion` in [`core/json/nullableUnion.ts`](./core/json/nullableUnion.ts) — precisely so they cannot drift on which schemas they can render.
|
||||
|
||||
#### Unportable tool schemas
|
||||
|
||||
`unportable-schemas-http.json` serves three tools, two of whose advertised
|
||||
schemas carry constructs that are legal JSON Schema and are refused or
|
||||
mishandled by real MCP clients: `echo` declares `outputSchema.properties.data`
|
||||
as a bare `true` (what Go's `jsonschema` package emits for `interface{}` —
|
||||
the case reported in [#1005](https://github.com/modelcontextprotocol/inspector/issues/1005)),
|
||||
an array-form `"type": ["null","boolean"]`, and an `opts` property that
|
||||
constrains nothing; `add` points a property at a remote `$ref`. `get_temp` is
|
||||
left clean, so a flagged tool sits next to an unflagged one. Plain
|
||||
streamable-HTTP — connect with the **default (legacy)** protocol era.
|
||||
|
||||
The Inspector is where a server author looks first, so a construct that will
|
||||
fail downstream is named here rather than passed through silently. All three
|
||||
clients report the same verdict from
|
||||
[`core/json/schemaLint.ts`](./core/json/schemaLint.ts), each with the room it
|
||||
has:
|
||||
|
||||
```bash
|
||||
mcp-inspector --cli http://127.0.0.1:6603/mcp --method tools/list --strict # exits 6
|
||||
```
|
||||
|
||||
- **CLI** — `--strict` prints the full report (path, issue, suggested fix) on
|
||||
stderr and exits `6` on an error-severity finding; without it, one summary
|
||||
line. See [Schema portability](./clients/cli/README.md#schema-portability---strict).
|
||||
- **TUI** — the tools list marks `echo` with a red `!` and `add` with a yellow
|
||||
`?`; the detail pane lists each finding under **Schema Portability**.
|
||||
- **Web** — the Tools sidebar row carries the same flag as a hover-labelled
|
||||
icon, and selecting the tool shows a **Schema portability** section above the
|
||||
argument form.
|
||||
|
||||
This is deliberately **not** a JSON Schema validator. A census of 617 public
|
||||
servers (14,804 tool schemas) reported on that issue found **0** that fail the
|
||||
SDK's own `ListToolsResultSchema.safeParse`, so a conformance check would
|
||||
report nothing on essentially every real server. What bites is the narrower
|
||||
subset each consumer accepts, and each rule here is a construct known to be
|
||||
refused or degraded by a shipping client. The schemas are supplied through the
|
||||
test server's `rawToolSchemas` override, because the Zod-built presets cannot
|
||||
express any of them — which is the same reason a real server hits this only
|
||||
when its schemas come from another generator.
|
||||
|
||||
#### RFC 6570 resource templates
|
||||
|
||||
`rfc6570-templates-http.json` serves two resource templates straight out of [#1919](https://github.com/modelcontextprotocol/inspector/issues/1919) — `events_by_topic` (`foobar://events/{topic}`) and `events_by_query` (`foobar://events{?topic}`) — each echoing the URI it was matched against, plus a plain `foobar://events` resource (see below). Plain streamable-HTTP; connect with the **default (legacy)** protocol era.
|
||||
|
|
|
|||
Loading…
Reference in a new issue