conformance/examples
Repository files (latest commit first)
Filename Latest commit message Latest commit date
AKS 542381b7dc
feat(tools-list): check deterministic tools/list ordering on the 2026-07-28 wire (#492)
* feat(tools-list): check deterministic tools/list ordering on the 2026-07-28 wire

The 2026-07-28 revision adds to server/tools.mdx: "Servers SHOULD return
tools in a deterministic order (i.e., the same ordering across requests
when the underlying set of tools has not changed)." Nothing in the suite
exercised it.

Add a tools-list-deterministic-order check to the existing tools-list
scenario. From the 2026-07-28 wire on, the scenario takes two more
consecutive tools/list snapshots and compares the three name sequences.
Same multiset of names, different order is WARNING (SHOULD). A set that
changed between probes is reported as untestable rather than as a
violation, because the spec scopes the SHOULD to an unchanged set. Fewer
than two tools on every probe is INFO. The check carries
source.introducedIn '2026-07-28', written as a literal since that revision
is published and dated, so runners drop it on earlier wires.

Prove it fails: examples/servers/typescript/tools-list-rotated-order.ts
advertises the same four tools and rotates the list by one position per
call; a negative.test.ts case asserts WARNING while tools-list itself
still passes. Unit tests cover the helper that builds the check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjeC3sCvfHRpdEPz2x41B2

* tools-list: inline the deterministic-order check id and introducedIn literal

Check ids and introducedIn revisions are written inline everywhere else in
src/scenarios (tests pin the literal slug on purpose, since SDK baselines
reference it), so drop the two exported constants.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com>
Co-authored-by: Paul Carleton <paulc@anthropic.com>
2026-09-06 20:13:29 +00:00
..
clients/typescript feat(client/auth): check the resource parameter matches the PRM-published identifier (#488) 2026-09-06 20:41:25 +01:00
servers/typescript feat(tools-list): check deterministic tools/list ordering on the 2026-07-28 wire (#492) 2026-09-06 20:13:29 +00:00
authorization-server-settings.example.json feat: add JSON-based settings file for authorization-server tests (supersedes #228) (#360) 2026-06-24 15:07:32 +01:00