This PR add cycle detection where it return error instead of waiting for
stack overflow.
This is a fix for https://github.com/modelcontextprotocol/go-sdk/issues/77.
The root cause of the issue was the type being processed is deeply
nested and has cycle. Hence, the heap memory growth is faster than the
stack growth, it went OOM before stack overflow panic can kick in.
Support additional (extra) keywords beyond those specified in the
`Schema` type. The current behavior is additional keywords are dropped.
To follow up:
- There are similar util functions in `mcp/util.go` which don't seem to
be used. Maybe we should move it to `internal/util` instead of
`jsonchema/util.go` package?
- The method `[Un]MarshalJSON` is only on `pointer receiver`, is this
intended? (that means it will only work if we [un]marshal pointer to the
`Schema` instance)
- The fix is using the [un]marshalStructWithMap from `mcp/util.go` and
there are requirements for the function to work properly but these are
not strictly checked e.g. need `json:"-"` tag for the Extra field.
Fixes#69.
Add Wrapf, a function that helps with error wrapping.
Change-Id: I102695f75e4d217e09f828f4723cbfcbbe83f887
Reviewed-on: https://go-review.googlesource.com/c/tools/+/679177
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Do a thorough job of parsing json struct tags.
Move the code to a place where both mcp and jsonschema can use it.
Change-Id: I6c86308834706b12a1e22cd96ae120590f49d3f4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/678757
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
A tool now validates its input with its InputSchema.
Change schema inference to allow explicit nulls for fields of
pointer type.
We assume the schema has no external references. For example, the
following schema cannot be handled:
{
"$ref": "https://example.com/other.json"
}
Schemas with internal references, like to a "$defs", are fine.
Change-Id: I6ee7c18c2c5cb609df0b22a66da986f7ea64bbe4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/670676
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
This CL handles only the limited case of defaults on properties.
This is just a start: I need to think more about defaults on other
schema keywords, and I need to set defaults recursively.
As part of this work, I realized that the Schema.Default field is
more usable when represented as a json.RawMessage instead of a *any.
Change-Id: Iedcedb405b52607a312dbd895fa61e7f7f94c6bc
Reviewed-on: https://go-review.googlesource.com/c/tools/+/676938
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
Schemas must form a tree. This is important for Schema.Resolve, and
also for modifying parts of the schema before resolution, as with
mcp SchemaOptions.
Remove the `seen` argument from inference, creating all schemas
afresh.
Change-Id: Ief8295e8470f3f575ef51cc2ba697901250bbe6f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/676937
Auto-Submit: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Hongxiang Jiang <hxjiang@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Optionally validate a schema's defaults when resolving it.
We simply walk the tree of schemas, validating each locally. This can't
handle dynamic references, which require maintaing the stack from the
root. It seems unlikely that a default would require one, however.
Since there may be other options to Resolve, use an options struct
to future-proof it.
Also, fix a bug: you can't resolve a schema twice.
Change-Id: I374c582d9cc49d66d4ed56233d1070b8b3263bdf
Reviewed-on: https://go-review.googlesource.com/c/tools/+/676395
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Redo iteration. It was too subtle.
Write a separate recursive check for whether the schemas form a tree and
have any invalid nils.
While doing that, assign paths to schemas for better error messages,
even as we check structure. This also lets us detect re-occurring
schemas without a separate map.
Everything else, including the iterators, can now assume valid structure
and populated paths.
Change-Id: I90ec93d79d1d95c92258ad7afc94ef068fdf123f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/676376
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
Add a wrapf function to simplify defers that wrap errors.
Change-Id: If4bedec7d3f56f609d3ca4620aebe7637a59f6bf
Reviewed-on: https://go-review.googlesource.com/c/tools/+/676475
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Validation errors now more clearly indicate the path from the root
schema.
- Enhance schema iteration to yield the path, and generalize it by using
reflection.
- Store each schema's path inside the schema.
- Remove the path arg from validate: we don't need it anymore.
- Improve the test for the iterators and add one for validation
errors.
Change-Id: Ief55133ef4ae2eac0736208a3b9f78ed69933b31
Reviewed-on: https://go-review.googlesource.com/c/tools/+/676375
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
Meta is a struct that contains a map[string]any, and a ProgressToken
field.
Each Params and Result type has a Meta as a value field.
Meta marshals and unmarshals specially, so the ProgressToken field
becomes part of the map. The marshalling code is generalized in
anticipation of every Params and Result growing a map of additional
properties.
Change-Id: I6d2f34270d33501c96437f24f91ef3e51d20d955
Reviewed-on: https://go-review.googlesource.com/c/tools/+/674956
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Validating a struct is ambiguous, because a zero field could be
considered missing or present.
Interpret a zero optional struct field in whichever way results in
success.
Change-Id: I6a9474e6f6558d2a8522bc5ba0451967367e467d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/675956
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Change-Id: I8066b22cb87e09fb160de3b8ff2d8fdde6ec3dcf
Reviewed-on: https://go-review.googlesource.com/c/tools/+/675815
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Document the package.
Change-Id: I770c14d6339b75ea6827280c23171c3584dc1071
Reviewed-on: https://go-review.googlesource.com/c/tools/+/674977
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add additional fields to Schema which are mentioned in the spec but
do not affect validation.
Change-Id: I89488261a00c207d01cd7fc59d782f43692fe528
Reviewed-on: https://go-review.googlesource.com/c/tools/+/674976
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
A dynamic reference is like a dynamic variable in a programming
language, with two twists:
- There must be a dynamic anchor in lexical scope for the ref to
resolve. (Think of it as declaring the dynamic variable.)
- The ref resolves to the schema highest on the stack, not lowest.
That feels odd, but it turns out to work for the intended purpose,
which is to provide extension points for additional features.
There is some distracting code motion and refactoring in this CL:
resolveRefs was moved down in the file, and resolution of a single
ref was split off.
Change-Id: I17da6052e0782fc540d14b8a41b06a2645923872
Reviewed-on: https://go-review.googlesource.com/c/tools/+/674155
Auto-Submit: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Use pointers for the protocol's struct types, to future-proof against
them getting large.
Change-Id: I55b204080a41c8140539697ba264e502e912e613
Reviewed-on: https://go-review.googlesource.com/c/tools/+/672575
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When we check the root schema during resolution, first make sure
its descendants form a tree.
Change-Id: Icc430e7df463aa84b961c28616be567f98960430
Reviewed-on: https://go-review.googlesource.com/c/tools/+/670776
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Support remote references.
Most of the machinery was in place; we just had to define the
right loader, copy files from the test repo, and download
the meta-schema files from json-schema.org.
Change-Id: Ib588200ef0fa483641b9b53f9904f53b39dc4b82
Reviewed-on: https://go-review.googlesource.com/c/tools/+/670795
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Implement the $ref keyword, except for remote references (those
pointing outside the root schema).
Change-Id: Ia5ca88464e62e9c613970d8d9fcd1512ab288934
Reviewed-on: https://go-review.googlesource.com/c/tools/+/670775
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
s.every now visits the entire subtree of schemas at s, not only the
immediate children of s.
This re-introduces a tower: now of closures instead of iterators.
We could remove it having everyChild call every recursively instead
of returning, just passing f down all the way unwrapped.
But this is going to complicate the recursive function inside
resolveURIs, which needs to pass parent context down. It would have
the same problem that ast.Inspect has, and we'd have to adopt the same
solution, maintaining a stack explicitly.
I feel it's cleaner to keep the recursion simple at the cost of building
several closures. To do otherwise would be premature optimization.
Change-Id: Ic453cdaac482da8147dafc34b0117d25458d1c9d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/670675
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Implement enough of the JSON Pointer spec to suffice for JSON Schema.
For details, see the comment at the top of json_pointer.go.
Change-Id: Ic297f1a1a08903b4b3c6666fe57ac1f45c96c0f7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/670535
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The instance passed to Resolved.Validate can be a struct, or can
contain structs. A struct is treated like the JSON object that it
marshals to.
Also support pointers to any depth. That is, if an instance is a **int,
then Validate will treat it like an int (or nil). That won't happen if
the instance is unmarshaled into an `any`, but may if it's unmarshaled
into a struct.
Change-Id: I39f9af58028bd779887495754519615cd5dfb6c8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/670395
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Assign a base URI to every schema.
See the comment on resolveURIs for details.
Change-Id: I2332d825c486766832503cabdd0bf9d819171d53
Reviewed-on: https://go-review.googlesource.com/c/tools/+/669997
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
The jsonschema package is part of of the API of the mcp package,
so it should not be internal.
Change-Id: I9520797837329fc2cda09c2f46ec4d48d031b9d3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/669996
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
