ruby-sdk/test/mcp/tool/output_schema_test.rb
Koichi ITO 9d4dcbf1a4 Speed Up Tool::Schema Validation by 5x to 100x
## Motivation and Context

`MCP::Tool::Schema` previously used the pure-Ruby `json-schema` gem for
both construction-time metaschema validation and runtime argument /
result validation. For deep schemas this cost ~32-100ms per
construction (see issue #364).

`json_schemer` (https://rubygems.org/gems/json_schemer, v2.5.0) is
an actively maintained alternative that is much faster on this workload.
Switching reduces cold-start warming pressure in horizontally scaled
multi-tenant deployments (the scenario raised in #364) and brings
runtime data validation down to sub-millisecond per call once
the per-instance schemer is memoized.

The implementation preserves the legacy behavior of the `json-schema`
based code. Metaschema is pinned to draft-04 (`meta_schema:` option);
the `$schema` dialect URI is still emitted in `to_h` as 2020-12
but runtime validation continues to use draft-04, exactly as before.
Moving the runtime validator itself to 2020-12 is out of scope for
this PR and warrants its own discussion. `format` keywords are not enforced
(`format: false` option), matching what `json-schema`'s draft-04 path did.
Malformed schemas (e.g. an invalid `pattern` regex) continue to
surface as `ArgumentError, "Invalid JSON Schema: ..."`;
the `RegexpError` `json_schemer` raises during construction is wrapped.
`Symbol` values in argument data are coerced to strings inside
the internal `stringify` helper so they validate against `type: "string"`
the same way they did before.

The public `Schema#schema` accessor is dropped. It was a refactor
artifact from #198 (the consolidation of `InputSchema`'s
`properties` / `required` readers into a single hash) with no callers
in `lib/`; tests that needed the merged hash now read it through
`to_h`. The accessor's only remaining role would have been to expose
a mutation path that could desynchronize the memoized `@schemer`,
which this change removes.

Runtime dependency delta: drops `json-schema` (and `addressable`),
adds `json_schemer`, `hana`, `regexp_parser`, and `simpleidn`. Each added
gem is a single-purpose library tied to JSON Schema spec compliance.

Closes #364.

## How Has This Been Tested?

Side-by-side benchmark on representative schemas (simple, with `$ref` /
`additionalProperties`, nested, depth 20, depth 40):
construction-time metaschema validation is 4.7x to ~100x faster;
runtime data validation with the memoized schemer is sub-millisecond
across all sizes.

Four existing tests had their stub targets or message assertions updated
to track the new validator while preserving the original test intent.
The "unexpected errors bubble up" tests in `input_schema_test.rb`
and `output_schema_test.rb` now stub `JSONSchemer::Schema#validate`
instead of `JSON::Validator.fully_validate`. The cache tests in
`schema_test.rb` now stub `JSONSchemer::Schema#validate_schema`.
The two "detailed error message" tests in `tool_test.rb` switched from
asserting `json-schema`'s exact wording to format-agnostic substrings
(`"properties/count/minimum"`, `"number"`). The "required arguments are
converted to strings" test in `input_schema_test.rb` now reads the result
through `to_h[:required]` instead of the removed `schema` accessor.

Regression tests added for the legacy-behavior preservations above:
`format` keyword is not enforced, invalid `pattern` raises `ArgumentError`
(not `RegexpError`), and `Symbol` values validate against `type: "string"`.

## Breaking Changes

`Schema#schema` is no longer a public method. The accessor had no callers
outside the test suite, but anyone reading the internal representation
directly (rather than through `to_h`) will need to switch.
`Schema` / `InputSchema` / `OutputSchema` otherwise keep
their constructor signatures, `to_h` output, `ValidationError` /
`ArgumentError` raise points, and the `$schema` dialect emission.
The validator error message wording inside `ArgumentError` and
`ValidationError` now comes from `json_schemer` and still includes
the JSON pointer path and a description of the mismatch.
2026-06-01 21:47:30 +09:00

240 lines
8.1 KiB
Ruby

# frozen_string_literal: true
require "test_helper"
module MCP
class Tool
class OutputSchemaTest < ActiveSupport::TestCase
test "to_h returns a hash representation of the output schema" do
output_schema = OutputSchema.new(properties: { result: { type: "string" } }, required: ["result"])
assert_equal(
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
type: "object",
properties: { result: { type: "string" } },
required: ["result"],
},
output_schema.to_h,
)
end
test "to_h preserves user-supplied $schema dialect" do
output_schema = OutputSchema.new(
"$schema": "https://json-schema.org/draft/2019-09/schema",
properties: { result: { type: "string" } },
)
assert_equal "https://json-schema.org/draft/2019-09/schema", output_schema.to_h[:"$schema"]
end
test "validate_result works when user supplies a 2020-12 $schema" do
output_schema = OutputSchema.new(
"$schema": "https://json-schema.org/draft/2020-12/schema",
properties: { result: { type: "string" } },
required: ["result"],
)
assert_nothing_raised do
output_schema.validate_result({ result: "success" })
end
assert_raises(OutputSchema::ValidationError) do
output_schema.validate_result({ result: 123 })
end
end
test "validate_result validates result against the schema" do
output_schema = OutputSchema.new(properties: { result: { type: "string" } }, required: ["result"])
assert_nothing_raised do
output_schema.validate_result({ result: "success" })
end
end
test "validate_result validates result with additional properties against the schema when additionalProperties set to nil (default)" do
output_schema = OutputSchema.new(properties: { result: { type: "string" } }, required: ["result"])
assert_nothing_raised do
output_schema.validate_result({ result: "success", extra: 123 })
end
end
test "validate_result validates result with additional properties against the schema when additionalProperties set to true" do
output_schema = OutputSchema.new(properties: { result: { type: "string" } }, required: ["result"], additionalProperties: true)
assert_nothing_raised do
output_schema.validate_result({ result: "success", extra: 123 })
end
end
test "validate_result raises error with additional properties when additionalProperties set to false)" do
output_schema = OutputSchema.new(properties: { result: { type: "string" } }, required: ["result"], additionalProperties: false)
assert_raises(OutputSchema::ValidationError) do
output_schema.validate_result({ result: "success", extra: 123 })
end
end
test "validate_result raises error for invalid result" do
output_schema = OutputSchema.new(properties: { result: { type: "string" } }, required: ["result"])
assert_raises(OutputSchema::ValidationError) do
output_schema.validate_result({ result: 123 })
end
end
test "validate_result raises error for missing required field" do
output_schema = OutputSchema.new(properties: { result: { type: "string" } }, required: ["result"])
assert_raises(OutputSchema::ValidationError) do
output_schema.validate_result({})
end
end
test "valid schema initialization" do
schema = OutputSchema.new(properties: { foo: { type: "string" } }, required: ["foo"])
assert_equal(
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
type: "object",
properties: { foo: { type: "string" } },
required: ["foo"],
},
schema.to_h,
)
end
test "invalid schema raises argument error" do
assert_raises(ArgumentError) do
OutputSchema.new(properties: { foo: { type: "invalid_type" } }, required: ["foo"])
end
end
test "schema without required arguments is valid" do
assert_nothing_raised do
OutputSchema.new(properties: { foo: { type: "string" } })
end
end
test "unexpected errors bubble up from validate_result" do
schema = OutputSchema.new(properties: { foo: { type: "string" } }, required: ["foo"])
JSONSchemer::Schema.any_instance.stubs(:validate).raises("unexpected error")
assert_raises(RuntimeError) do
schema.validate_result(foo: "bar")
end
end
test "accepts schemas with $ref references" do
schema = OutputSchema.new(
properties: {
foo: { type: "string" },
},
definitions: {
bar: { type: "string" },
},
required: ["foo"],
)
assert_includes schema.to_h.keys, :definitions
end
test "accepts schemas with $ref string key and includes $ref in to_h" do
schema = OutputSchema.new({
"properties" => {
"foo" => { "$ref" => "#/definitions/bar" },
},
"definitions" => {
"bar" => { "type" => "string" },
},
})
assert_equal "#/definitions/bar", schema.to_h[:properties][:foo][:$ref]
end
test "accepts schemas with $ref symbol key and includes $ref in to_h" do
schema = OutputSchema.new({
properties: {
foo: { :$ref => "#/definitions/bar" },
},
definitions: {
bar: { type: "string" },
},
})
assert_equal "#/definitions/bar", schema.to_h[:properties][:foo][:$ref]
end
test "== compares two output schemas with the same properties and required fields" do
schema1 = OutputSchema.new(properties: { foo: { type: "string" } }, required: ["foo"])
schema2 = OutputSchema.new(properties: { foo: { type: "string" } }, required: ["foo"])
assert_equal schema1, schema2
schema3 = OutputSchema.new(properties: { bar: { type: "string" } }, required: ["bar"])
refute_equal schema1, schema3
schema4 = OutputSchema.new(properties: { foo: { type: "string" } }, required: ["bar"])
refute_equal schema1, schema4
schema5 = OutputSchema.new(properties: { bar: { type: "string" } }, required: ["foo"])
refute_equal schema1, schema5
end
test "empty schema is valid" do
schema = OutputSchema.new
assert_equal(
{ "$schema": "https://json-schema.org/draft/2020-12/schema", type: "object" },
schema.to_h,
)
end
test "validates complex nested schemas" do
schema = OutputSchema.new(
properties: {
data: {
type: "object",
properties: {
items: { type: "array", items: { type: "string" } },
count: { type: "integer", minimum: 0 },
},
required: ["items"],
},
},
required: ["data"],
)
valid_result = {
data: {
items: ["item1", "item2"],
count: 2,
},
}
assert_nothing_raised do
schema.validate_result(valid_result)
end
invalid_result = {
data: {
items: [123, 456], # Should be strings
count: 2,
},
}
assert_raises(OutputSchema::ValidationError) do
schema.validate_result(invalid_result)
end
end
test "allow to declare array schemas" do
schema = OutputSchema.new({
type: "array",
items: {
properties: { foo: { type: "string" } },
required: ["foo"],
},
})
assert_equal(
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
type: "array",
items: {
properties: { foo: { type: "string" } },
required: ["foo"],
},
},
schema.to_h,
)
end
end
end
end