java-sdk/conformance-tests/VALIDATION_RESULTS.md
Christian Tzolov 3ab829285d Bump version from 0.18.0-SNAPSHOT to 1.0.0-SNAPSHOT
Update project version across all POM files, documentation, and CI
workflow to prepare for the 1.0.x release

Signed-off-by: Christian Tzolov <christian.tzolov@broadcom.com>
2026-02-18 12:45:20 +01:00

2.7 KiB

MCP Java SDK Conformance Test Validation Results

Summary

Server Tests: 37/40 passed (92.5%)
Client Tests: 3/4 scenarios passed (9/10 checks passed)

Server Test Results

Passing (37/40)

  • Lifecycle & Utilities (4/4): initialize, ping, logging-set-level, completion-complete
  • Tools (11/11): All scenarios including progress notifications ✨
  • Elicitation (10/10): SEP-1034 defaults (5 checks), SEP-1330 enums (5 checks)
  • Resources (4/6): list, read-text, read-binary, templates-read
  • Prompts (4/4): list, simple, with-args, embedded-resource, with-image
  • SSE Transport (2/2): Multiple streams
  • Security (2/2): Localhost validation passes, DNS rebinding protection

Failing (3/40)

  1. resources-subscribe - Not implemented in SDK
  2. resources-unsubscribe - Not implemented in SDK

Client Test Results

Passing (3/4 scenarios, 9/10 checks)

  • initialize (1/1): Protocol negotiation, clientInfo, capabilities
  • tools_call (1/1): Tool discovery and invocation
  • elicitation-sep1034-client-defaults (5/5): Default values for string, integer, number, enum, boolean

Partially Passing (1/4 scenarios, 1/2 checks)

  • sse-retry (1/2 + 1 warning):
    • ✅ Reconnects after stream closure
    • ❌ Does not respect retry timing
    • ⚠️ Does not send Last-Event-ID header (SHOULD requirement)

Issue: Client treats retry: SSE field as invalid instead of parsing it for reconnection timing.

Known Limitations

  1. Resource Subscriptions: SDK doesn't implement resources/subscribe and resources/unsubscribe handlers
  2. Client SSE Retry: Client doesn't parse or respect the retry: field, reconnects immediately, and doesn't send Last-Event-ID header

Running Tests

Server

# Start server
cd conformance-tests/server-servlet
../../mvnw compile exec:java -Dexec.mainClass="io.modelcontextprotocol.conformance.server.ConformanceServlet"

# Run tests (in another terminal)
npx @modelcontextprotocol/conformance server --url http://localhost:8080/mcp --suite active

Client

# Build
cd conformance-tests/client-jdk-http-client
../../mvnw clean package -DskipTests

# Run all scenarios
for scenario in initialize tools_call elicitation-sep1034-client-defaults sse-retry; do
  npx @modelcontextprotocol/conformance client \
    --command "java -jar target/client-jdk-http-client-1.0.0-SNAPSHOT.jar" \
    --scenario $scenario
done

Recommendations

High Priority

  1. Fix client SSE retry field handling in HttpClientStreamableHttpTransport
  2. Implement resource subscription handlers in McpStatelessAsyncServer

Medium Priority

  1. Add Host/Origin validation in HttpServletStreamableServerTransportProvider for DNS rebinding protection