80 lines
2.7 KiB
Markdown
80 lines
2.7 KiB
Markdown
# MCP Java SDK Conformance Test Validation Results
|
|
|
|
## Summary
|
|
|
|
**Server Tests:** 37/40 passed (92.5%)
|
|
**Client Tests:** 3/4 scenarios passed (9/10 checks passed)
|
|
|
|
## Server Test Results
|
|
|
|
### Passing (37/40)
|
|
|
|
- **Lifecycle & Utilities (4/4):** initialize, ping, logging-set-level, completion-complete
|
|
- **Tools (11/11):** All scenarios including progress notifications ✨
|
|
- **Elicitation (10/10):** SEP-1034 defaults (5 checks), SEP-1330 enums (5 checks)
|
|
- **Resources (4/6):** list, read-text, read-binary, templates-read
|
|
- **Prompts (4/4):** list, simple, with-args, embedded-resource, with-image
|
|
- **SSE Transport (2/2):** Multiple streams
|
|
- **Security (2/2):** Localhost validation passes, DNS rebinding protection
|
|
|
|
### Failing (3/40)
|
|
|
|
1. **resources-subscribe** - Not implemented in SDK
|
|
2. **resources-unsubscribe** - Not implemented in SDK
|
|
|
|
## Client Test Results
|
|
|
|
### Passing (3/4 scenarios, 9/10 checks)
|
|
|
|
- **initialize (1/1):** Protocol negotiation, clientInfo, capabilities
|
|
- **tools_call (1/1):** Tool discovery and invocation
|
|
- **elicitation-sep1034-client-defaults (5/5):** Default values for string, integer, number, enum, boolean
|
|
|
|
### Partially Passing (1/4 scenarios, 1/2 checks)
|
|
|
|
- **sse-retry (1/2 + 1 warning):**
|
|
- ✅ Reconnects after stream closure
|
|
- ❌ Does not respect retry timing
|
|
- ⚠️ Does not send Last-Event-ID header (SHOULD requirement)
|
|
|
|
**Issue:** Client treats `retry:` SSE field as invalid instead of parsing it for reconnection timing.
|
|
|
|
## Known Limitations
|
|
|
|
1. **Resource Subscriptions:** SDK doesn't implement `resources/subscribe` and `resources/unsubscribe` handlers
|
|
2. **Client SSE Retry:** Client doesn't parse or respect the `retry:` field, reconnects immediately, and doesn't send Last-Event-ID header
|
|
|
|
## Running Tests
|
|
|
|
### Server
|
|
```bash
|
|
# Start server
|
|
cd conformance-tests/server-servlet
|
|
../../mvnw compile exec:java -Dexec.mainClass="io.modelcontextprotocol.conformance.server.ConformanceServlet"
|
|
|
|
# Run tests (in another terminal)
|
|
npx @modelcontextprotocol/conformance server --url http://localhost:8080/mcp --suite active
|
|
```
|
|
|
|
### Client
|
|
```bash
|
|
# Build
|
|
cd conformance-tests/client-jdk-http-client
|
|
../../mvnw clean package -DskipTests
|
|
|
|
# Run all scenarios
|
|
for scenario in initialize tools_call elicitation-sep1034-client-defaults sse-retry; do
|
|
npx @modelcontextprotocol/conformance client \
|
|
--command "java -jar target/client-jdk-http-client-0.18.0-SNAPSHOT.jar" \
|
|
--scenario $scenario
|
|
done
|
|
```
|
|
|
|
## Recommendations
|
|
|
|
### High Priority
|
|
1. Fix client SSE retry field handling in `HttpClientStreamableHttpTransport`
|
|
2. Implement resource subscription handlers in `McpStatelessAsyncServer`
|
|
|
|
### Medium Priority
|
|
3. Add Host/Origin validation in `HttpServletStreamableServerTransportProvider` for DNS rebinding protection
|