fix: drop the unreachable root rule, correct the always-false suggestion
Copilot review round 2.
- Remove `non-object-root` entirely. It could never fire: the SDK types
`inputSchema` with `type: literal("object")`, so a tool with any other
root fails `ListToolsResultSchema` and `salvageListItems` drops it
before any client sees it. Verified against a live server advertising
`inputSchema: {type: "array"}` — `listAllTools()` returns []. A rule
that cannot fire is worse than none, because the docs then claim a
check the tool does not perform; the condition is already reported
through the malformed-items surface. Pinned by an integration test so
a change in that behavior is a signal rather than a silent gap.
- Fix the bare-`false` suggestion. `properties: {a: false}` forbids the
property; deleting the entry *permits* it with any value under the
default `additionalProperties`, so the advice inverted the schema it
replaced. Now recommends `{"not": {}}` and says why deletion differs.
- Exempt `not`-parented schemas from `untyped-schema`, so the `{"not":
{}}` above does not trip this module's own rule — under negation an
always-accepting subschema means always-reject, making "accepts any
value" the exact opposite of what it does. Scoped to that position,
not inherited by its subtree.
- Say plainly, in the `true` and untyped suggestions, that they narrow
the schema deliberately rather than implying an equivalent rewrite.
`true` has no portable object-form equivalent — `{}` is the very shape
this lint flags.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UL9retfvAXRgvi6EWk4SY4
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
c389565783
commit
cac97c142c
5 changed files with 136 additions and 78 deletions
25
AGENTS.md
25
AGENTS.md
|
|
@ -162,15 +162,22 @@ v2/main/
|
||||||
│ │ # constructs that are legal JSON Schema and are
|
│ │ # constructs that are legal JSON Schema and are
|
||||||
│ │ # refused or mishandled by real MCP clients (a bare
|
│ │ # refused or mishandled by real MCP clients (a bare
|
||||||
│ │ # `true` in `properties`, an array-form `type`, a
|
│ │ # `true` in `properties`, an array-form `type`, a
|
||||||
│ │ # remote `$ref`, a schema carrying no validation
|
│ │ # remote `$ref`, a schema carrying no constraining
|
||||||
│ │ # keyword, a non-object root). Deliberately NOT a
|
│ │ # keyword). Deliberately NOT a validator: a census of
|
||||||
│ │ # validator: a census of 617 public servers found 0
|
│ │ # 617 public servers found 0 that fail the SDK's own
|
||||||
│ │ # that fail the SDK's own parse, so a conformance
|
│ │ # parse, so a conformance check reports nothing on
|
||||||
│ │ # check reports nothing on real servers. ONE verdict
|
│ │ # real servers. Equally deliberately it has NO
|
||||||
│ │ # for all three clients — the CLI's `--strict` report
|
│ │ # "inputSchema must be an object" rule, even though
|
||||||
│ │ # (exit 6), the TUI's tool detail pane, the web Tools
|
│ │ # MCP requires that: the SDK types inputSchema with
|
||||||
│ │ # tab — so they cannot disagree about what is
|
│ │ # `type: literal("object")`, so such a tool fails
|
||||||
│ │ # portable; the report TEXT is shared too — #1005)
|
│ │ # ListToolsResultSchema and salvageListItems drops it
|
||||||
|
│ │ # before any client sees it — a rule that cannot fire
|
||||||
|
│ │ # is worse than none, because the docs then claim a
|
||||||
|
│ │ # check the tool does not perform. ONE verdict for all
|
||||||
|
│ │ # three clients — the CLI's `--strict` report (exit 6),
|
||||||
|
│ │ # the TUI's tool detail pane, the web Tools tab — so
|
||||||
|
│ │ # they cannot disagree about what is portable; the
|
||||||
|
│ │ # report TEXT is shared too — #1005)
|
||||||
│ ├── logging/ # Silent pino logger singleton
|
│ ├── logging/ # Silent pino logger singleton
|
||||||
│ ├── mcp/ # InspectorClient runtime + state stores
|
│ ├── mcp/ # InspectorClient runtime + state stores
|
||||||
│ │ # (uriTemplate.ts: RFC 6570 parse/classify/expand.
|
│ │ # (uriTemplate.ts: RFC 6570 parse/classify/expand.
|
||||||
|
|
|
||||||
|
|
@ -174,11 +174,18 @@ Error: tool "info"
|
||||||
```
|
```
|
||||||
|
|
||||||
Severity decides the exit code, not the report: **errors** are constructs a
|
Severity decides the exit code, not the report: **errors** are constructs a
|
||||||
shipping MCP client refuses outright (a bare `true` in a `properties` map, a
|
shipping MCP client refuses outright (a bare `true` or `false` where a schema
|
||||||
non-object schema root), **warnings** are ones handled unevenly (an array-form
|
object belongs), **warnings** are ones handled unevenly (an array-form `type`,
|
||||||
`type`, a remote `$ref`, a schema carrying no validation keyword at all). Only
|
a remote `$ref`, a schema carrying no constraining keyword at all). Only errors
|
||||||
errors fail the run — a `--strict` that failed on warnings would be unusable as
|
fail the run — a `--strict` that failed on warnings would be unusable as a CI
|
||||||
a CI gate against servers that are in fact fine.
|
gate against servers that are in fact fine.
|
||||||
|
|
||||||
|
There is deliberately **no "inputSchema must be an object" check**, even though
|
||||||
|
MCP requires one. The SDK types `inputSchema` with `type: literal("object")`,
|
||||||
|
so a tool whose input root is anything else fails `ListToolsResultSchema` and
|
||||||
|
is dropped from the list before the lint could see it — it is reported through
|
||||||
|
the malformed-items path instead. A rule that cannot fire would only make this
|
||||||
|
documentation claim a check the CLI does not perform.
|
||||||
|
|
||||||
The report goes to stderr, so the result on stdout stays parseable. Under
|
The report goes to stderr, so the result on stdout stays parseable. Under
|
||||||
`--format json` the findings are folded into the same envelope instead
|
`--format json` the findings are folded into the same envelope instead
|
||||||
|
|
|
||||||
|
|
@ -131,14 +131,48 @@ describe("lintToolSchemas — boolean-schema", () => {
|
||||||
expect(findings[0]!.suggestion).toContain("additionalProperties");
|
expect(findings[0]!.suggestion).toContain("additionalProperties");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("flags a bare false, with a different suggestion", () => {
|
it("suggests the always-false object form for a bare false", () => {
|
||||||
|
// `properties: {a: false}` forbids the property. Deleting the entry would
|
||||||
|
// *permit* it with any value under the default `additionalProperties`, so
|
||||||
|
// the suggestion has to preserve the constraint rather than drop it.
|
||||||
const findings = lintToolSchemas(
|
const findings = lintToolSchemas(
|
||||||
tool({
|
tool({
|
||||||
inputSchema: { type: "object", properties: { data: false } },
|
inputSchema: { type: "object", properties: { data: false } },
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
expect(rules(findings)).toEqual(["boolean-schema"]);
|
expect(rules(findings)).toEqual(["boolean-schema"]);
|
||||||
expect(findings[0]!.suggestion).toContain("Remove the property");
|
expect(findings[0]!.suggestion).toContain('{"not": {}}');
|
||||||
|
expect(findings[0]!.suggestion).toContain("different contract");
|
||||||
|
expect(findings[0]!.suggestion).not.toMatch(/^Remove the property/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not flag the `{"not": {}}` form it recommends', () => {
|
||||||
|
// The suggestion above would be self-defeating if this module reported its
|
||||||
|
// own recommended replacement.
|
||||||
|
expect(
|
||||||
|
lintToolSchemas(
|
||||||
|
tool({
|
||||||
|
inputSchema: { type: "object", properties: { data: { not: {} } } },
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("still flags an unconstrained schema elsewhere under a not branch", () => {
|
||||||
|
// The exemption is scoped to the `not` position itself, not inherited by
|
||||||
|
// everything beneath it.
|
||||||
|
const findings = lintToolSchemas(
|
||||||
|
tool({
|
||||||
|
inputSchema: {
|
||||||
|
type: "object",
|
||||||
|
properties: { a: { not: { properties: { b: {} } } } },
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(rules(findings)).toEqual(["untyped-schema"]);
|
||||||
|
expect(paths(findings)).toEqual([
|
||||||
|
"inputSchema.properties.a.not.properties.b",
|
||||||
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it.each([
|
it.each([
|
||||||
|
|
@ -274,9 +308,7 @@ describe("lintToolSchemas — type-union", () => {
|
||||||
expect(findings[0]!.suggestion).toContain("anyOf");
|
expect(findings[0]!.suggestion).toContain("anyOf");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("does not raise non-object-root for an array-typed root", () => {
|
it("raises only the union rule for an array-typed root", () => {
|
||||||
// The root rule reads a *string* `type`; an array root is the union rule's
|
|
||||||
// business, and reporting both would name one defect twice.
|
|
||||||
const findings = lintToolSchemas(
|
const findings = lintToolSchemas(
|
||||||
tool({ inputSchema: { type: ["object", "null"] } }),
|
tool({ inputSchema: { type: ["object", "null"] } }),
|
||||||
);
|
);
|
||||||
|
|
@ -354,38 +386,27 @@ describe("lintToolSchemas — untyped-schema", () => {
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("lintToolSchemas — non-object-root", () => {
|
describe("lintToolSchemas — no root-type rule", () => {
|
||||||
it("flags a non-object inputSchema root", () => {
|
// There is deliberately no "inputSchema must be an object" rule: the SDK
|
||||||
const findings = lintToolSchemas(
|
// types `inputSchema` with `type: literal("object")`, so such a tool fails
|
||||||
tool({
|
// `ListToolsResultSchema` and `salvageListItems` drops it before any client
|
||||||
inputSchema: { type: "array", items: { type: "string" } },
|
// sees it. `raw-tool-schemas.test.ts` pins that live behaviour; these cases
|
||||||
}),
|
// pin that the module stays quiet rather than reporting a check nothing can
|
||||||
);
|
// reach.
|
||||||
expect(rules(findings)).toEqual(["non-object-root"]);
|
it.each([
|
||||||
expect(findings[0]!.severity).toBe("error");
|
["a non-object inputSchema root", { inputSchema: { type: "array" } }],
|
||||||
expect(findings[0]!.issue).toContain("inputSchema");
|
["a non-object outputSchema root", { outputSchema: { type: "string" } }],
|
||||||
});
|
[
|
||||||
|
"a nested non-object schema",
|
||||||
it("does NOT flag a non-object outputSchema root", () => {
|
{
|
||||||
// The SDK's `Tool` schema types `inputSchema` with `type: literal("object")`
|
|
||||||
// but `outputSchema` as a plain loose object with no `type` constraint, so
|
|
||||||
// a string-typed output is conforming. Flagging it would report an error
|
|
||||||
// MCP does not require and would fail `--strict` on a valid server.
|
|
||||||
expect(lintToolSchemas(tool({ outputSchema: { type: "string" } }))).toEqual(
|
|
||||||
[],
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("does not flag a nested non-object schema", () => {
|
|
||||||
const findings = lintToolSchemas(
|
|
||||||
tool({
|
|
||||||
inputSchema: {
|
inputSchema: {
|
||||||
type: "object",
|
type: "object",
|
||||||
properties: { a: { type: "array", items: { type: "string" } } },
|
properties: { a: { type: "array", items: { type: "string" } } },
|
||||||
},
|
},
|
||||||
}),
|
},
|
||||||
);
|
],
|
||||||
expect(findings).toEqual([]);
|
])("reports nothing for %s", (_label, overrides) => {
|
||||||
|
expect(lintToolSchemas(tool(overrides))).toEqual([]);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -110,6 +110,24 @@ describe("rawToolSchemas override (#1005)", () => {
|
||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("drops a tool whose inputSchema root is not an object, before any client sees it", async () => {
|
||||||
|
// This is why `schemaLint` has no "inputSchema must be an object" rule.
|
||||||
|
// The SDK types `inputSchema` with `type: literal("object")`, so such a
|
||||||
|
// tool fails `ListToolsResultSchema` and `salvageListItems` removes it —
|
||||||
|
// no client can pass it to the lint, and a rule for it could never fire.
|
||||||
|
// If this ever starts returning the tool, that rule becomes worth adding
|
||||||
|
// back and this test is the signal.
|
||||||
|
const started = await start({
|
||||||
|
rawToolSchemas: {
|
||||||
|
echo: { inputSchema: { type: "array", items: { type: "string" } } },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const connected = await connect(started.url);
|
||||||
|
|
||||||
|
const { tools } = await connected.listAllTools();
|
||||||
|
expect(tools.map((t) => t.name)).toEqual(["get_weather"]);
|
||||||
|
});
|
||||||
|
|
||||||
it("leaves an unnamed tool's real schema alone", async () => {
|
it("leaves an unnamed tool's real schema alone", async () => {
|
||||||
const started = await start({
|
const started = await start({
|
||||||
rawToolSchemas: { echo: { outputSchema: { type: "object" } } },
|
rawToolSchemas: { echo: { outputSchema: { type: "object" } } },
|
||||||
|
|
|
||||||
|
|
@ -40,7 +40,6 @@ export type SchemaLintRule =
|
||||||
| "boolean-schema"
|
| "boolean-schema"
|
||||||
| "type-union"
|
| "type-union"
|
||||||
| "untyped-schema"
|
| "untyped-schema"
|
||||||
| "non-object-root"
|
|
||||||
| "remote-ref";
|
| "remote-ref";
|
||||||
|
|
||||||
/** One problem found at one location in one schema. */
|
/** One problem found at one location in one schema. */
|
||||||
|
|
@ -275,8 +274,16 @@ function walk(
|
||||||
path,
|
path,
|
||||||
`Bare \`${String(node)}\` used where a schema object is expected.`,
|
`Bare \`${String(node)}\` used where a schema object is expected.`,
|
||||||
node
|
node
|
||||||
? 'Spell the "anything" schema out — `{"type": "object", "additionalProperties": true}` for a free-form object, or the concrete type if you know it.'
|
? // `true` accepts any JSON value, and no portable object form says
|
||||||
: "Remove the property (or the keyword) rather than schema-ing it away with `false`.",
|
// exactly that — `{}` is the same shape this lint flags as untyped.
|
||||||
|
// So the advice is to narrow, and it says so rather than implying an
|
||||||
|
// equivalent rewrite.
|
||||||
|
'Declare what the value actually is — e.g. `{"type": "object", "additionalProperties": true}` for a free-form object. That narrows the schema deliberately; `true` accepts any JSON value at all.'
|
||||||
|
: // `false` in a property position means the property is FORBIDDEN.
|
||||||
|
// Deleting the entry is not the same thing: with `additionalProperties`
|
||||||
|
// absent (default `true`) the property becomes allowed with any value,
|
||||||
|
// so the suggestion has to preserve the always-false constraint.
|
||||||
|
'Use the object form of the always-false schema — `{"not": {}}`. (Deleting the entry is a different contract: under the default `additionalProperties` the property would then be allowed with any value, not forbidden.)',
|
||||||
);
|
);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
@ -286,7 +293,7 @@ function walk(
|
||||||
// is the right place to reject it, so this lint stays quiet and stops here.
|
// is the right place to reject it, so this lint stays quiet and stops here.
|
||||||
if (!isRecord(node)) return;
|
if (!isRecord(node)) return;
|
||||||
|
|
||||||
lintNode(node, path, ctx);
|
lintNode(node, path, keyword, ctx);
|
||||||
|
|
||||||
for (const key of SUBSCHEMA_MAP_KEYWORDS) {
|
for (const key of SUBSCHEMA_MAP_KEYWORDS) {
|
||||||
const map = node[key];
|
const map = node[key];
|
||||||
|
|
@ -319,7 +326,12 @@ function walk(
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Rules that apply to a single schema object, ignoring its children. */
|
/** Rules that apply to a single schema object, ignoring its children. */
|
||||||
function lintNode(node: SchemaRecord, path: string, ctx: WalkContext): void {
|
function lintNode(
|
||||||
|
node: SchemaRecord,
|
||||||
|
path: string,
|
||||||
|
keyword: string | undefined,
|
||||||
|
ctx: WalkContext,
|
||||||
|
): void {
|
||||||
const type = node.type;
|
const type = node.type;
|
||||||
|
|
||||||
if (Array.isArray(type)) {
|
if (Array.isArray(type)) {
|
||||||
|
|
@ -361,14 +373,19 @@ function lintNode(node: SchemaRecord, path: string, ctx: WalkContext): void {
|
||||||
const constrains = Object.keys(node).some((key) =>
|
const constrains = Object.keys(node).some((key) =>
|
||||||
CONSTRAINING_KEYWORDS.has(key),
|
CONSTRAINING_KEYWORDS.has(key),
|
||||||
);
|
);
|
||||||
if (!constrains) {
|
// Under `not`, an always-accepting subschema is the idiomatic spelling of
|
||||||
|
// always-*reject* — `{"not": {}}` is the object form of `false`, and the one
|
||||||
|
// this module's own `boolean-schema` suggestion recommends. Reporting it as
|
||||||
|
// "accepts any value" would be the exact opposite of what it does, so the
|
||||||
|
// rule stops at that position rather than contradicting itself.
|
||||||
|
if (!constrains && keyword !== "not") {
|
||||||
add(
|
add(
|
||||||
ctx,
|
ctx,
|
||||||
"untyped-schema",
|
"untyped-schema",
|
||||||
"warning",
|
"warning",
|
||||||
path,
|
path,
|
||||||
"Schema carries no validation keyword at all, so it accepts any value — the object-literal spelling of a bare `true`.",
|
"Schema carries no validation keyword at all, so it accepts any value — the object-literal spelling of a bare `true`.",
|
||||||
'Give it an explicit `type`. For a genuinely free-form value use `{"type": "object", "additionalProperties": true}`.',
|
'Declare what the value actually is — e.g. `{"type": "object", "additionalProperties": true}` for a free-form object. That narrows the schema deliberately, which is the point: as written it constrains nothing.',
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -376,37 +393,25 @@ function lintNode(node: SchemaRecord, path: string, ctx: WalkContext): void {
|
||||||
/**
|
/**
|
||||||
* Lint one of a tool's schemas.
|
* Lint one of a tool's schemas.
|
||||||
*
|
*
|
||||||
* `inputSchema` gets one extra rule at the root: MCP requires it to describe
|
* There is deliberately **no root-level "inputSchema must be an object" rule**,
|
||||||
* an object, and the SDK enforces that — its `Tool` schema types `inputSchema`
|
* even though MCP does require that. The SDK's `ToolSchema` types `inputSchema`
|
||||||
* with `type: literal("object")`, so a non-object root is refused by the
|
* with `type: literal("object")`, so a tool whose input root is anything else
|
||||||
* parser rather than merely degraded.
|
* fails `ListToolsResultSchema.safeParse`; `salvageListItems` then drops it
|
||||||
|
* before `ManagedToolsState` hands the list to anyone. Verified against a live
|
||||||
|
* server advertising `inputSchema: {type: "array"}` — `listAllTools()` returns
|
||||||
|
* an empty array, so no client can ever pass such a tool to this module.
|
||||||
*
|
*
|
||||||
* That rule is **scoped to `inputSchema` deliberately**. The same SDK types
|
* A rule that cannot fire is worse than no rule: it makes the documentation
|
||||||
* `outputSchema` as a plain loose object with no `type` constraint at all, so
|
* claim a check the tool does not perform. The condition is already reported
|
||||||
* a tool whose output is `{"type": "string"}` is conforming; flagging it would
|
* through the salvage path's malformed-items surface, which is where a dropped
|
||||||
* report an error MCP does not require and would fail `--strict` on a valid
|
* tool legitimately belongs.
|
||||||
* server.
|
|
||||||
*/
|
*/
|
||||||
function lintSchema(
|
function lintSchema(
|
||||||
schema: unknown,
|
schema: unknown,
|
||||||
kind: SchemaKind,
|
kind: SchemaKind,
|
||||||
findings: SchemaFinding[],
|
findings: SchemaFinding[],
|
||||||
): void {
|
): void {
|
||||||
const ctx: WalkContext = { schema: kind, findings };
|
walk(schema, "", undefined, 0, { schema: kind, findings });
|
||||||
if (kind === "inputSchema" && isRecord(schema)) {
|
|
||||||
const type = schema.type;
|
|
||||||
if (typeof type === "string" && type !== "object") {
|
|
||||||
add(
|
|
||||||
ctx,
|
|
||||||
"non-object-root",
|
|
||||||
"error",
|
|
||||||
"",
|
|
||||||
`Root \`type\` is "${type}". MCP requires a tool's inputSchema to describe an object.`,
|
|
||||||
'Wrap the value in an object schema — `{"type": "object", "properties": {…}}`.',
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
walk(schema, "", undefined, 0, ctx);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue