Commit graph conformance/src/scenarios/server/json-schema-2020-12.ts
Author SHA1 Message Date
Felix Weinberger
3f9048a019
feat: add specVersion classification to conformance scenarios (#147)
* feat: add specVersions classification to conformance scenarios

Each scenario declares which spec versions it applies to as a list.
Scenarios that carry forward (e.g. initialize) list all applicable versions
['2025-06-18', '2025-11-25']. Scenarios removed from newer specs (e.g.
backcompat auth) only list their original version ['2025-03-26'].

- specVersions list on Scenario and ClientScenario interfaces
- --spec-version CLI filter uses simple .includes()
- Tier-check conformance matrix (Server / Client: Core / Client: Auth)
  with per-version columns and unique All* count
- 7 unit tests for specVersions helpers
- Updated tier-audit skill docs with matrix format

* fix: fix console output template for tier audit skill

The template had an orphan table header (Check | Value | T2 | T1)
with no rows above the conformance matrix, causing an empty table
to render. The scorecard rows below the matrix also lacked their
own header.

Fix: two self-contained tables with clear labels — 'Conformance:'
for the per-version matrix, 'Scorecard:' for the check rows.

* fix: align skill console template with tier-check script output

Add asterisk footnote ('unique scenarios — a scenario may apply to
multiple spec versions') and rename 'Scorecard' to 'Repository Health'
to match the labels used by the tier-check CLI output.

* fix: add specVersions to CrossAppAccessCompleteFlowScenario

Added in 83c446d on main after this branch diverged.
2026-02-13 18:08:15 +00:00
Felix Weinberger
f087e44bdb
Add JSON Schema 2020-12 conformance test (SEP-1613) (#45)
Add conformance test to validate that MCP servers correctly preserve
JSON Schema 2020-12 keywords ($schema, $defs, additionalProperties)
in tool definitions.

Changes:
- Add JsonSchema2020_12Scenario in src/scenarios/server/json-schema-2020-12.ts
- Add json_schema_2020_12_tool to everything-server with tools/list override
  to return raw JSON Schema instead of Zod-converted schema
- Register scenario in pendingClientScenariosList (requires SDK PR #1135)

The test is added to pending because the client SDK currently strips
the JSON Schema 2020-12 keywords. Once PR #1135 is merged into the SDK,
this scenario can be moved to the active list.
2025-11-20 18:20:06 +00:00