modelcontextprotocol/php-sdk
Watch
1
0
Fork
You've already forked php-sdk
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

[CI] Matrix conformance jobs over both spec revisions (#458)

Browse source 
* [CI] Matrix the conformance jobs over both spec revisions

Runs server/client conformance against both 2025-11-25 and 2026-07-28,
each against its own endpoint and baseline. PR pipeline pins an exact
conformance package version; conformance-weekly tracks the moving
target on both dist-tags.

* [CI] Run the draft conformance suite from the alpha package

The conformance repo now ships 2026-07-28 scenarios on its alpha
dist-tag, so the draft targets no longer need a local checkout.
Pinned to the same version the pipeline runs.

* [CI] Baseline the failures --suite all newly exposes

The old workflow ran without --suite all/--spec-version, so several
pre-existing gaps never surfaced in CI. Now that both revisions run
the full suite:

- server: json-schema-2020-12 and resources-templates-read are
  fixture bugs shared by both revisions; 2026-07-28 also needs
  resources/subscribe and resources/unsubscribe declared removed
  under the stateless wire, which the server doesn't yet do
- client: the example client doesn't call add_numbers or echo back
  a tool's inputSchema; against 2026-07-28 it doesn't speak the
  lifecycle or OAuth at all yet, so every such scenario fails

Verified locally: all four conformance jobs (server/client x
2025-11-25/2026-07-28) pass against these baselines.
This commit is contained in:
Christopher Hertel 2026-08-18 23:30:57 +02:00 • committed by GitHub
commit 7ffc739fe7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 161 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

29
Makefile
View file

@ -1,10 +1,10 @@
.PHONY: deps-stable deps-low cs phpstan tests unit-tests integration-tests inspector-tests coverage ci ci-stable ci-lowest conformance-tests conformance-server conformance-client conformance-draft conformance-draft-server conformance-draft-client check-conformance-repo docs
.PHONY: deps-stable deps-low cs phpstan tests unit-tests integration-tests inspector-tests coverage ci ci-stable ci-lowest conformance-tests conformance-server conformance-client conformance-draft conformance-draft-server conformance-draft-client docs
# The published conformance release carries no 2026-07-28 scenarios, so the
# draft targets run a local checkout instead.
# Override with `make conformance-draft CONFORMANCE_REPO=/path/to/conformance`.
CONFORMANCE_REPO ?= $(CURDIR)/../conformance
CONFORMANCE_DRAFT = node $(CONFORMANCE_REPO)/dist/index.js
# The 2026-07-28 scenarios ship on the `alpha` dist-tag; `latest` (0.1.x) has
# none of them. Pinned to the same version CI runs (see
# .github/workflows/pipeline.yaml), so a local pass means a green pipeline.
CONFORMANCE_VERSION ?= 0.2.0-alpha.11
CONFORMANCE = npx --yes @modelcontextprotocol/conformance@$(CONFORMANCE_VERSION)
deps-stable:
composer update --prefer-stable
@ -47,29 +47,20 @@ conformance-client:
php tests/Conformance/score.php client
# --- 2026-07-28 (SEP-2575 stateless lifecycle) ------------------------------
# Local-checkout only until upstream publishes the draft scenarios.
conformance-draft: conformance-draft-server conformance-draft-client
check-conformance-repo:
@test -f $(CONFORMANCE_REPO)/dist/index.js || { \
echo "No conformance build at $(CONFORMANCE_REPO)/dist/index.js."; \
echo "Clone modelcontextprotocol/conformance and run 'npm install && npm run build' there,"; \
echo "or point CONFORMANCE_REPO at an existing checkout."; \
exit 1; \
}
conformance-draft-server: check-conformance-repo
conformance-draft-server:
docker compose -f tests/Conformance/Fixtures/docker-compose.yml up -d
@echo "Waiting for server to start..."
@sleep 5
rm -rf tests/Conformance/results-2026-07-28
cd tests/Conformance && $(CONFORMANCE_DRAFT) server --url http://localhost:8000/stateless --suite all --spec-version 2026-07-28 --expected-failures conformance-baseline-2026-07-28.yml --output-dir results-2026-07-28 || true
cd tests/Conformance && $(CONFORMANCE) server --url http://localhost:8000/stateless --suite all --spec-version 2026-07-28 --expected-failures conformance-baseline-2026-07-28.yml --output-dir results-2026-07-28 || true
docker compose -f tests/Conformance/Fixtures/docker-compose.yml down
conformance-draft-client: check-conformance-repo
conformance-draft-client:
rm -rf tests/Conformance/results-2026-07-28
cd tests/Conformance && $(CONFORMANCE_DRAFT) client --command "php $(CURDIR)/tests/Conformance/client.php" --suite all --spec-version 2026-07-28 --expected-failures conformance-baseline-2026-07-28.yml --output-dir results-2026-07-28 || true
cd tests/Conformance && $(CONFORMANCE) client --command "php $(CURDIR)/tests/Conformance/client.php" --suite all --spec-version 2026-07-28 --expected-failures conformance-baseline-2026-07-28.yml --output-dir results-2026-07-28 || true
coverage:
XDEBUG_MODE=coverage vendor/bin/phpunit --testsuite=unit --coverage-html=coverage