feat: add --expected-failures baseline and composite GitHub Action (#113)

Add support for SDKs that don't yet pass all conformance tests to still
run conformance in CI without failing, by specifying a YAML baseline file
of known/expected failures.

CLI changes:
- Add --expected-failures <path> flag to both server and client commands
- Exit 0 if only expected failures occur
- Exit 1 if unexpected failures occur (regression)
- Exit 1 if an expected failure now passes (stale baseline, must update)

GitHub Action (action.yml):
- Composite action that builds from source and runs conformance tests
- SDKs reference it as: uses: modelcontextprotocol/conformance@<ref>
- Inputs: mode, url, command, expected-failures, suite, scenario, etc.

New files:
- src/expected-failures.ts: YAML loading, baseline evaluation, reporting
- src/expected-failures.test.ts: 16 unit tests
- action.yml: composite GitHub Action

Closes #99
This commit is contained in:
Max Isbey 2026-01-22 10:41:04 +00:00 • committed by GitHub
commit 83ff334e0e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -50,6 +50,7 @@
"eventsource-parser": "^3.0.6",
"express": "^5.1.0",
"jose": "^6.1.2",
"yaml": "^2.8.2",
"zod": "^3.25.76"
}
}