Commit graph

50 commits

Author SHA1 Message Date
Paul Carleton
eed3b222a1
Typecheck and lint example client
Don't do server yet, since it has its own package.json which breaks
checking imports.
2025-11-12 14:38:07 +00:00
Felix Weinberger
e19918d968
Add conformance tests for SEP-1330 elicitation enums (#22)
* Add SEP-1330 conformance test for elicitation enum schemas

Adds server-side conformance test that validates servers properly request
elicitation with SEP-1330 enum schema improvements:
- Untitled single-select enum (type: string, enum: [...])
- Titled single-select enum (oneOf with const/title)
- Legacy titled enum (enumNames for backward compatibility)
- Untitled multi-select enum (type: array, items.enum)
- Titled multi-select enum (items.anyOf with const/title)

Test expects server to implement `test_elicitation_sep1330_enums` tool.

* Add reference server implementation for SEP-1330 conformance test

Adds test_elicitation_sep1330_enums tool to everything-server that
requests elicitation with all 5 enum schema variants. Also updates
SERVER_REQUIREMENTS.md with full specification.

* Fix prettier formatting

* only test active tests (#24)

* refactor maps to pull names from the test

* bring names back

* separate active from pending

* only test active

---------

Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com>
2025-11-11 19:34:47 +00:00
Paul Carleton
73af777bc2
Add interactive mode with verbose checking (#23)
* add back interactive mode, todo: print instructions for running w/ command and fix optional error message on command

* add verbose checking
2025-11-11 17:49:36 +00:00
Paul Carleton
091a95d059
0.1.5 v0.1.5 2025-11-11 17:31:54 +00:00
Paul Carleton
0509702849
fix repo url again 2025-11-11 17:31:49 +00:00
Paul Carleton
bf9ce3abdc
0.1.4 v0.1.4 2025-11-11 17:08:00 +00:00
Paul Carleton
207eaed73e
fix url in package.json 2025-11-11 17:07:45 +00:00
Paul Carleton
a359b25b74
0.1.3 v0.1.3 2025-11-11 16:52:40 +00:00
Paul Carleton
113dfaa1c4
go back to token auth 2025-11-11 16:52:32 +00:00
Paul Carleton
b164f5cdc8
0.1.2 v0.1.2 2025-11-11 16:32:47 +00:00
Paul Carleton
569e86704c
bump npm version in release flow 2025-11-11 16:32:24 +00:00
Paul Carleton
4b30a171e7
0.1.1 v0.1.1 2025-11-11 16:25:25 +00:00
Paul Carleton
595bf4dec8
bump beta version for release publishing (#21)
* use trusted publishing

* 0.1.1-beta.0
2025-11-11 14:59:00 +00:00
Paul Carleton
1e41ead8ac
use trusted publishing (#20) 0.1.0-alpha.1 2025-11-11 14:52:27 +00:00
Paul Carleton
f95f487ae8
[scenario] Add test for broken oauth client (#12)
* starting on broken auth client

* switch to middleware approach

* add broken example

* fix failure testing

* subset matching
2025-11-11 14:17:49 +00:00
Felix Weinberger
f7bc687819
Add client-side SEP-1034 elicitation defaults conformance test (#19)
* feat: add client-side SEP-1034 conformance test for elicitation defaults

Adds a conformance test that validates client SDKs properly apply
default values for omitted fields in elicitation responses per SEP-1034.

The test spins up a server that sends elicitation requests with defaults,
and validates that the client returns content with those defaults applied.

* feat: use elicitInput() for SEP-1034 default validation

Updates client-side test to validate server-side default application
using elicitInput() helper, which applies defaults via validation layer.
This aligns with the TypeScript SDK's SEP-1034 implementation approach.

The test now validates:
- Server uses elicitInput() with schemas containing defaults
- Client returns empty/partial content
- Server automatically applies defaults during validation
- Defaults are correctly merged for all primitive types

* refactor: use server.request() instead of elicitInput() for client-side defaults test

Ensures conformance test validates true client-side default application
rather than server-side validation behavior.

* feat: add applyDefaults capability flag for opt-in default application

Enables conformance test to opt into client-side elicitation defaults
by setting the applyDefaults capability flag.
2025-11-11 14:05:11 +00:00
Felix Weinberger
fa7a0bc6c7
fix: run npm i (#13) 2025-11-11 13:22:27 +00:00
Paul Carleton
768f155f2c
fix package-lock.json 2025-11-11 13:12:59 +00:00
Paul Carleton
8df6de59cd
fix conformance 0.1.0 2025-11-11 12:11:55 +00:00
Paul Carleton
76e9226b4b
Add publish step (#16)
* feat: tsdown

* fix: formatting

* automate version updates in cli

* fix: docs

* scenarios

* fix: commander deps

* Add publish step

* add publish step

* add tsgo

* delete old scenario

* tidy package and license

* bugs link

* combo build/test

* combo, and release type

* prepack

---------

Co-authored-by: Matt Carey <mcarey@cloudflare.com>
2025-11-11 12:05:12 +00:00
Matt
7f1515c475
tsdown + cli with commander (#17)
* feat: tsdown

* fix: formatting

* automate version updates in cli

* fix: docs

* scenarios

* fix: commander deps
2025-11-11 11:51:55 +00:00
Felix Weinberger
18f48e5dea
Add test_elicitation_sep1034_defaults to SERVER_REQUIREMENTS.md (#15)
Addresses comment from PR #14 to document the new tool that tests
SEP-1034 elicitation with default values for all primitive types.

This ensures SDK implementers know to include this tool in their
conformance test servers.
2025-11-10 18:02:54 +00:00
Felix Weinberger
0e8f13ba1e
feat: add SEP-1034 conformance test for elicitation defaults (#14)
Add comprehensive conformance test for SEP-1034 which extends MCP
elicitation to support default values across all primitive types.

Changes:
- Add test_elicitation_sep1034_defaults tool to reference server
- Create ElicitationDefaultsScenario with 5 granular checks:
  * String schema default value
  * Integer schema default value
  * Number schema default value
  * Enum schema default value (validates it's a valid enum member)
  * Boolean schema default value (regression test)
- Register scenario in index.ts

The test validates that servers properly include default values in
elicitation requestedSchema per SEP-1034 specification.

Test results: All 5 checks pass against TypeScript reference server
2025-11-10 16:04:09 +00:00
Paul Carleton
69f02a4557
[scenario] OAuth basic & PRM variation (#11)
* readme for auth tests

* wip prm test

* add example helper for local oauth w/o browser

* working through dangling timeout issue

* add better test function

* pretty printing

* add mcp method

* some more logging tweaks

* refactor logger into 1 class

* use common request logger

* fix base scenario

* some renaming

* refactor into helpers

* refactor to support second test

* rm readme

* add comment about www-authenticate
2025-11-10 11:51:30 +00:00
Paolo Ricciuti
78d6bec42a
docs: update server requirements (#10)
* docs: update server requirements

* Update SERVER_REQUIREMENTS.md

---------

Co-authored-by: Paul Carleton <paulcarletonjr@gmail.com>
2025-11-09 14:35:41 +00:00
Paul Carleton
bf658c0378
Add test that ensures example server passes conformance (#9)
* add unit test for good server

* fix outdated ts-expect

* lint

* changed action to use node 22, updated package.json version to be >1.0
2025-11-04 15:43:52 +00:00
Paul Carleton
b377251b15
adjust prettier to use more defaults (#8)
* simplify prettier to use more defaults

* apply new prettier
2025-11-04 15:34:27 +00:00
Paul Carleton
05df535e40
add permissions to test github action (#5)
* Potential fix for code scanning alert no. 1: Workflow does not contain permissions

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* lint

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-04 15:29:22 +00:00
Inna Harper
a692dd7bba
[compliance tests] initial server scenarios (#4)
* spec for servers

* initial version of everything server and tests - tets are very simple

* some clean up

* lint etc.

* fix package lock agian

---------

Co-authored-by: Paul Carleton <paulc@anthropic.com>
2025-11-04 15:08:20 +00:00
Paul Carleton
44a2732b53
Add lefthook for linting and testing hooks, and optional pre-commit hook via local (#7)
* unbreak package lock

* lint

* add lefthook for prepush test and lint hook

* attempt at exiting 0 if we made a change in lint

* fix lint

* add fail text

* pre-commit test

* adjust autofix

* add lefthook example

* fix comment
2025-11-04 14:50:17 +00:00
Paul Carleton
e751676b04
unbreak package lock (#6)
* unbreak package lock

* lint
2025-11-04 14:10:03 +00:00
Paul Carleton
3b9da98c6a
initial runner for checking server conformance (#3)
* create some checks

* basic initialize working

* rename
2025-11-03 16:43:43 +00:00
Paul Carleton
90ab89ffe1
Update README.md 2025-11-03 14:28:36 +00:00
Paul Carleton
2644074125
move scenarios to be client based 2025-10-31 18:01:46 +00:00
Paul Carleton
40f16c265d
Merge pull request #2 from modelcontextprotocol/pcarleton/tools_call_test
Add conformance test for tools/call
2025-10-31 17:57:42 +00:00
Paul Carleton
3d7343abd8
Merge pull request #1 from modelcontextprotocol/pcarleton-runner1
Adjust test and allow running in interactive mode
2025-10-31 17:54:48 +00:00
Paul Carleton
b78f6a54df
Adjust test and allow running in interactive mode 2025-10-31 17:53:52 +00:00
Paul Carleton
8bce5d194b
fix status 2025-10-31 17:07:42 +00:00
Paul Carleton
cff9669f8a
Add tools_call scenario and test 2025-10-30 13:26:33 +00:00
Paul Carleton
1fa165df3d
add tools-call to index 2025-10-30 13:26:29 +00:00
Paul Carleton
da695eb56c
fix runner 2025-10-30 13:26:21 +00:00
Paul Carleton
f32b01c57d
apply linting 2025-10-30 13:09:16 +00:00
Paul Carleton
ae9451f5e0
add linting 2025-10-30 13:08:11 +00:00
Paul Carleton
2c1cea291f
readme 2025-10-29 14:56:27 +00:00
Paul Carleton
fec2c90778
first passing test 2025-10-29 14:44:48 +00:00
Paul Carleton
98bf5aaf87
wip first crack at scenario and runner 2025-10-29 14:38:11 +00:00
Paul Carleton
fb3482530d
add very simple test client 2025-10-29 14:03:36 +00:00
Paul Carleton
b7709c07ed
add test for check function 2025-10-29 13:55:31 +00:00
Paul Carleton
0ad7be85d4
initialize check 2025-10-29 13:53:48 +00:00
Paul Carleton
32fd95dc3f
mvp conformance runner 2025-10-29 13:40:36 +00:00