conformance-tests: upgrade to MCP-security 0.1.11, implement CIMD

Signed-off-by: Daniel Garnier-Moiroux <git@garnier.wf>
This commit is contained in:
Daniel Garnier-Moiroux 2026-05-20 19:29:44 +02:00 • committed by Daniel Garnier-Moiroux
commit fc330a5885
7 changed files with 94 additions and 67 deletions

View file

@ -5,7 +5,7 @@
**Server Tests (active suite):** 44/44 passed (31 scenarios, 100%) **Server Tests (active suite):** 44/44 passed (31 scenarios, 100%)
**Server Tests (spec 2025-11-25):** 4/4 passed — SEP-1613 `json-schema-2020-12` scenario ✨ **Server Tests (spec 2025-11-25):** 4/4 passed — SEP-1613 `json-schema-2020-12` scenario ✨
**Client Tests:** 3/4 scenarios passed (9/10 checks passed) **Client Tests:** 3/4 scenarios passed (9/10 checks passed)
**Auth Tests:** 14/15 scenarios fully passing (196 passed, 0 failed, 1 warning, 93.3% scenarios, 99.5% checks) **Auth Tests:** 15/15 scenarios fully passing (195 passed, 0 failed, 0 warnings, 100% scenarios, 100% checks)
## Server Test Results ## Server Test Results
@ -46,16 +46,17 @@
## Auth Test Results (Spring HTTP Client) ## Auth Test Results (Spring HTTP Client)
**Status: 196 passed, 0 failed, 1 warning across 15 scenarios** **Status: 195 passed, 0 failed, 0 warnings across 15 scenarios**
Uses the `client-spring-http-client` module with Spring Security OAuth2 and the [mcp-client-security](https://github.com/springaicommunity/mcp-client-security) library. Uses the `client-spring-http-client` module with Spring Security OAuth2 and the [mcp-client-security](https://github.com/springaicommunity/mcp-client-security) library.
### Fully Passing (14/15 scenarios) ### Fully Passing (15/15 scenarios)
- **auth/metadata-default (13/13):** Default metadata discovery - **auth/metadata-default (13/13):** Default metadata discovery
- **auth/metadata-var1 (13/13):** Metadata discovery variant 1 - **auth/metadata-var1 (13/13):** Metadata discovery variant 1
- **auth/metadata-var2 (13/13):** Metadata discovery variant 2 - **auth/metadata-var2 (13/13):** Metadata discovery variant 2
- **auth/metadata-var3 (13/13):** Metadata discovery variant 3 - **auth/metadata-var3 (13/13):** Metadata discovery variant 3
- **auth/basic-cimd (12/12):** Basic Client-Initiated Metadata Discovery
- **auth/scope-from-www-authenticate (14/14):** Scope extraction from WWW-Authenticate header - **auth/scope-from-www-authenticate (14/14):** Scope extraction from WWW-Authenticate header
- **auth/scope-from-scopes-supported (14/14):** Scope extraction from scopes_supported - **auth/scope-from-scopes-supported (14/14):** Scope extraction from scopes_supported
- **auth/scope-omitted-when-undefined (14/14):** Scope omitted when not defined - **auth/scope-omitted-when-undefined (14/14):** Scope omitted when not defined
@ -67,14 +68,9 @@ Uses the `client-spring-http-client` module with Spring Security OAuth2 and the
- **auth/resource-mismatch (2/2):** Resource mismatch handling - **auth/resource-mismatch (2/2):** Resource mismatch handling
- **auth/pre-registration (6/6):** Pre-registered client credentials flow - **auth/pre-registration (6/6):** Pre-registered client credentials flow
### Partially Passing (1/15 scenarios)
- **auth/basic-cimd (13/13 + 1 warning):** Basic Client-Initiated Metadata Discovery — all checks pass, minor warning
## Known Limitations ## Known Limitations
1. **Client SSE Retry:** Client doesn't parse or respect the `retry:` field, reconnects immediately, and doesn't send Last-Event-ID header 1. **Client SSE Retry:** Client doesn't parse or respect the `retry:` field, reconnects immediately, and doesn't send Last-Event-ID header
2. **Auth Basic CIMD:** Minor conformance warning in the basic Client-Initiated Metadata Discovery flow
## Running Tests ## Running Tests
@ -132,4 +128,3 @@ npx @modelcontextprotocol/conformance@0.1.15 client \
### High Priority ### High Priority
1. Fix client SSE retry field handling in `HttpClientStreamableHttpTransport` 1. Fix client SSE retry field handling in `HttpClientStreamableHttpTransport`
2. Implement CIMD

View file

@ -14,23 +14,24 @@ Test with @modelcontextprotocol/conformance@0.1.15.
## Conformance Test Results ## Conformance Test Results
**Status: 178 passed, 1 failed, 1 warning across 14 scenarios** **Status: 195 passed, 0 failed, 0 warnings across 15 scenarios**
| Scenario | Result | Details | | Scenario | Result | Details |
|---|---|---| |---|---|---|
| auth/metadata-default | ✅ Pass | 12/12 | | auth/metadata-default | ✅ Pass | 13/13 |
| auth/metadata-var1 | ✅ Pass | 12/12 | | auth/metadata-var1 | ✅ Pass | 13/13 |
| auth/metadata-var2 | ✅ Pass | 12/12 | | auth/metadata-var2 | ✅ Pass | 13/13 |
| auth/metadata-var3 | ✅ Pass | 12/12 | | auth/metadata-var3 | ✅ Pass | 13/13 |
| auth/basic-cimd | ⚠️ Warning | 12/12 passed, 1 warning | | auth/basic-cimd | ✅ Pass | 12/12 |
| auth/scope-from-www-authenticate | ✅ Pass | 13/13 | | auth/scope-from-www-authenticate | ✅ Pass | 14/14 |
| auth/scope-from-scopes-supported | ✅ Pass | 13/13 | | auth/scope-from-scopes-supported | ✅ Pass | 14/14 |
| auth/scope-omitted-when-undefined | ✅ Pass | 13/13 | | auth/scope-omitted-when-undefined | ✅ Pass | 14/14 |
| auth/scope-step-up | ✅ Pass | 12/12 | | auth/scope-step-up | ✅ Pass | 16/16 |
| auth/scope-retry-limit | ✅ Pass | 11/11 | | auth/scope-retry-limit | ✅ Pass | 11/11 |
| auth/token-endpoint-auth-basic | ✅ Pass | 17/17 | | auth/token-endpoint-auth-basic | ✅ Pass | 18/18 |
| auth/token-endpoint-auth-post | ✅ Pass | 17/17 | | auth/token-endpoint-auth-post | ✅ Pass | 18/18 |
| auth/token-endpoint-auth-none | ✅ Pass | 17/17 | | auth/token-endpoint-auth-none | ✅ Pass | 18/18 |
| auth/resource-mismatch | ✅ Pass | 2/2 |
| auth/pre-registration | ✅ Pass | 6/6 | | auth/pre-registration | ✅ Pass | 6/6 |
See [VALIDATION_RESULTS.md](../VALIDATION_RESULTS.md) for the full project validation results. See [VALIDATION_RESULTS.md](../VALIDATION_RESULTS.md) for the full project validation results.
@ -113,8 +114,7 @@ java -jar conformance-tests/client-spring-http-client/target/client-spring-http-
## Known Issues ## Known Issues
1. **auth/scope-step-up** (1 failure) — The client does not fully handle scope step-up challenges where the server requests additional scopes after initial authorization. Currently, there are no known issues in the auth suite implementation.
2. **auth/basic-cimd** (1 warning) — Minor conformance warning in the basic Client-Initiated Metadata Discovery flow.
## References ## References

View file

@ -23,8 +23,8 @@
<properties> <properties>
<java.version>17</java.version> <java.version>17</java.version>
<spring-boot.version>4.0.5</spring-boot.version> <spring-boot.version>4.0.5</spring-boot.version>
<spring-ai.version>2.0.0-M4</spring-ai.version> <spring-ai.version>2.0.0-M6</spring-ai.version>
<spring-ai-mcp-security.version>0.1.5</spring-ai-mcp-security.version> <spring-ai-mcp-security.version>0.1.11</spring-ai-mcp-security.version>
<maven.deploy.skip>true</maven.deploy.skip> <maven.deploy.skip>true</maven.deploy.skip>
</properties> </properties>

View file

@ -8,17 +8,23 @@ import java.util.Optional;
import io.modelcontextprotocol.conformance.client.scenario.Scenario; import io.modelcontextprotocol.conformance.client.scenario.Scenario;
import org.springaicommunity.mcp.security.client.sync.oauth2.metadata.McpMetadataDiscoveryService; import org.springaicommunity.mcp.security.client.sync.oauth2.metadata.McpMetadataDiscoveryService;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DefaultMcpOAuth2ClientManager; import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DefaultMcpOAuth2DcrClientManager;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DynamicClientRegistrationService; import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DynamicClientRegistrationService;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.InMemoryMcpClientRegistrationRepository; import org.springaicommunity.mcp.security.client.sync.oauth2.registration.InMemoryMcpClientRegistrationRepository;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository; import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager; import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2DcrClientManager;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.cimd.DefaultMcpOAuth2CimdClientManager;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.cimd.McpOAuth2CimdClientManager;
import org.springaicommunity.mcp.security.common.url.DefaultUrlValidator;
import org.springframework.boot.ApplicationArguments; import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner; import org.springframework.boot.ApplicationRunner;
import org.springframework.boot.SpringApplication; import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
import org.springframework.security.oauth2.client.OAuth2AuthorizedClientManager;
import org.springframework.security.oauth2.client.web.DefaultOAuth2AuthorizedClientManager;
import org.springframework.security.oauth2.client.web.OAuth2AuthorizedClientRepository;
/** /**
* MCP Conformance Test Client - Spring HTTP Client Implementation. * MCP Conformance Test Client - Spring HTTP Client Implementation.
@ -42,13 +48,15 @@ public class ConformanceSpringClientApplication {
public static final String REGISTRATION_ID = "default_registration"; public static final String REGISTRATION_ID = "default_registration";
private final DefaultUrlValidator URL_VALIDATOR = new DefaultUrlValidator(true);
public static void main(String[] args) { public static void main(String[] args) {
SpringApplication.run(ConformanceSpringClientApplication.class, args); SpringApplication.run(ConformanceSpringClientApplication.class, args);
} }
@Bean @Bean
McpMetadataDiscoveryService discovery() { McpMetadataDiscoveryService discovery() {
return new McpMetadataDiscoveryService(); return new McpMetadataDiscoveryService(URL_VALIDATOR);
} }
@Bean @Bean
@ -57,10 +65,24 @@ public class ConformanceSpringClientApplication {
} }
@Bean @Bean
McpOAuth2ClientManager mcpOAuth2ClientManager(McpClientRegistrationRepository mcpClientRegistrationRepository, McpOAuth2DcrClientManager mcpOAuth2ClientManager(McpClientRegistrationRepository mcpClientRegistrationRepository,
McpMetadataDiscoveryService mcpMetadataDiscoveryService) { McpMetadataDiscoveryService mcpMetadataDiscoveryService) {
return new DefaultMcpOAuth2ClientManager(mcpClientRegistrationRepository, return new DefaultMcpOAuth2DcrClientManager(mcpClientRegistrationRepository,
new DynamicClientRegistrationService(), mcpMetadataDiscoveryService); new DynamicClientRegistrationService(URL_VALIDATOR), mcpMetadataDiscoveryService, URL_VALIDATOR);
}
@Bean
McpOAuth2CimdClientManager mcpOAuth2CimdClientManager(McpMetadataDiscoveryService mcpMetadataDiscoveryService,
McpClientRegistrationRepository mcpClientRegistrationRepository) {
return new DefaultMcpOAuth2CimdClientManager(mcpMetadataDiscoveryService, mcpClientRegistrationRepository,
URL_VALIDATOR);
}
@Bean
OAuth2AuthorizedClientManager oAuth2AuthorizedClientManager(
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository,
McpClientRegistrationRepository clientRegistrationRepository) {
return new DefaultOAuth2AuthorizedClientManager(clientRegistrationRepository, oAuth2AuthorizedClientRepository);
} }
@Bean @Bean

View file

@ -4,38 +4,65 @@
package io.modelcontextprotocol.conformance.client.configuration; package io.modelcontextprotocol.conformance.client.configuration;
import io.modelcontextprotocol.conformance.client.ConformanceSpringClientApplication; import io.modelcontextprotocol.client.transport.HttpClientStreamableHttpTransport;
import io.modelcontextprotocol.conformance.client.scenario.DefaultScenario; import io.modelcontextprotocol.conformance.client.scenario.DefaultScenario;
import org.springaicommunity.mcp.security.client.sync.config.McpClientOAuth2Configurer; import org.springaicommunity.mcp.security.client.sync.config.McpClientOAuth2Configurer;
import org.springaicommunity.mcp.security.client.sync.oauth2.http.client.OAuth2CimdHttpClientTransportCustomizer;
import org.springaicommunity.mcp.security.client.sync.oauth2.http.client.OAuth2DcrHttpClientTransportCustomizer;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository; import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager; import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2DcrClientManager;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.cimd.DefaultMcpOAuth2CimdClientManager;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.cimd.McpOAuth2CimdClientManager;
import org.springframework.ai.mcp.customizer.McpClientCustomizer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression; import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
import org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext; import org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext;
import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.annotation.web.builders.HttpSecurity; import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.oauth2.client.web.OAuth2AuthorizedClientRepository; import org.springframework.security.oauth2.client.OAuth2AuthorizedClientManager;
import org.springframework.security.oauth2.client.registration.ClientRegistration;
import org.springframework.security.web.SecurityFilterChain; import org.springframework.security.web.SecurityFilterChain;
@Configuration @Configuration
@ConditionalOnExpression("#{environment['MCP_CONFORMANCE_SCENARIO'] != 'auth/pre-registration'}") @ConditionalOnExpression("#{environment['MCP_CONFORMANCE_SCENARIO'] != 'auth/pre-registration'}")
public class DefaultConfiguration { public class DefaultConfiguration {
private final String TEST_CLIENT_ID_URL = "https://conformance-test.local/client-metadata.json";
@Bean @Bean
DefaultScenario defaultScenario(McpClientRegistrationRepository clientRegistrationRepository, DefaultScenario defaultScenario(ServletWebServerApplicationContext serverCtx,
ServletWebServerApplicationContext serverCtx, McpClientCustomizer<HttpClientStreamableHttpTransport.Builder> transportCustomizer) {
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository, return new DefaultScenario(serverCtx, transportCustomizer);
McpOAuth2ClientManager mcpOAuth2ClientManager) {
return new DefaultScenario(clientRegistrationRepository, serverCtx, oAuth2AuthorizedClientRepository,
mcpOAuth2ClientManager);
} }
@Bean @Bean
SecurityFilterChain securityFilterChain(HttpSecurity http, ConformanceSpringClientApplication.ServerUrl serverUrl) { McpClientCustomizer<HttpClientStreamableHttpTransport.Builder> transportCustomizer(
OAuth2AuthorizedClientManager oAuth2AuthorizedClientManager,
McpClientRegistrationRepository clientRegistrationRepository,
McpOAuth2DcrClientManager mcpOAuth2ClientManager, McpOAuth2CimdClientManager mcpOAuth2CimdClientManager,
@Value("${mcp.conformance.scenario}") String scenario) {
if (scenario.equals("auth/basic-cimd")) {
if (mcpOAuth2CimdClientManager instanceof DefaultMcpOAuth2CimdClientManager mgr) {
// Hardcode the client_id
mgr.setClientRegistrationCustomizer(
cr -> ClientRegistration.withClientRegistration(cr).clientId(TEST_CLIENT_ID_URL).build());
}
return new OAuth2CimdHttpClientTransportCustomizer(oAuth2AuthorizedClientManager,
clientRegistrationRepository, mcpOAuth2CimdClientManager);
}
else {
return new OAuth2DcrHttpClientTransportCustomizer(oAuth2AuthorizedClientManager,
clientRegistrationRepository, mcpOAuth2ClientManager);
}
}
@Bean
SecurityFilterChain securityFilterChain(HttpSecurity http) {
return http.authorizeHttpRequests(authz -> authz.anyRequest().permitAll()) return http.authorizeHttpRequests(authz -> authz.anyRequest().permitAll())
.with(new McpClientOAuth2Configurer(), Customizer.withDefaults()) .with(new McpClientOAuth2Configurer(), mcp -> mcp.cimd(true))
.build(); .build();
} }

View file

@ -17,14 +17,10 @@ import org.jspecify.annotations.NonNull;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import org.springaicommunity.mcp.security.client.sync.AuthenticationMcpTransportContextProvider; import org.springaicommunity.mcp.security.client.sync.AuthenticationMcpTransportContextProvider;
import org.springaicommunity.mcp.security.client.sync.oauth2.http.client.OAuth2HttpClientTransportCustomizer;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager;
import org.springframework.ai.mcp.customizer.McpClientCustomizer;
import org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext; import org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext;
import org.springframework.http.client.JdkClientHttpRequestFactory; import org.springframework.http.client.JdkClientHttpRequestFactory;
import org.springframework.security.oauth2.client.web.DefaultOAuth2AuthorizedClientManager;
import org.springframework.security.oauth2.client.web.OAuth2AuthorizedClientRepository;
import org.springframework.web.client.RestClient; import org.springframework.web.client.RestClient;
import org.springframework.web.util.UriComponentsBuilder; import org.springframework.web.util.UriComponentsBuilder;
@ -34,23 +30,14 @@ public class DefaultScenario implements Scenario {
private final ServletWebServerApplicationContext serverCtx; private final ServletWebServerApplicationContext serverCtx;
private final DefaultOAuth2AuthorizedClientManager authorizedClientManager; private final McpClientCustomizer<HttpClientStreamableHttpTransport.Builder> transportCustomizer;
private final McpClientRegistrationRepository clientRegistrationRepository;
private final McpOAuth2ClientManager mcpOAuth2ClientManager;
private McpSyncClient client; private McpSyncClient client;
public DefaultScenario(McpClientRegistrationRepository clientRegistrationRepository, public DefaultScenario(ServletWebServerApplicationContext serverCtx,
ServletWebServerApplicationContext serverCtx, McpClientCustomizer<HttpClientStreamableHttpTransport.Builder> transportCustomizer) {
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository,
McpOAuth2ClientManager mcpOAuth2ClientManager) {
this.serverCtx = serverCtx; this.serverCtx = serverCtx;
this.clientRegistrationRepository = clientRegistrationRepository; this.transportCustomizer = transportCustomizer;
this.mcpOAuth2ClientManager = mcpOAuth2ClientManager;
this.authorizedClientManager = new DefaultOAuth2AuthorizedClientManager(clientRegistrationRepository,
oAuth2AuthorizedClientRepository);
} }
@Override @Override
@ -59,12 +46,10 @@ public class DefaultScenario implements Scenario {
var testServerUrl = "http://localhost:" + serverCtx.getWebServer().getPort(); var testServerUrl = "http://localhost:" + serverCtx.getWebServer().getPort();
var testClient = buildTestClient(testServerUrl); var testClient = buildTestClient(testServerUrl);
var customizer = new OAuth2HttpClientTransportCustomizer(authorizedClientManager, clientRegistrationRepository,
mcpOAuth2ClientManager);
var baseUri = UriComponentsBuilder.fromUriString(serverUrl).replacePath(null).toUriString(); var baseUri = UriComponentsBuilder.fromUriString(serverUrl).replacePath(null).toUriString();
var path = UriComponentsBuilder.fromUriString(serverUrl).build().getPath(); var path = UriComponentsBuilder.fromUriString(serverUrl).build().getPath();
var transportBuilder = HttpClientStreamableHttpTransport.builder(baseUri).endpoint(path); var transportBuilder = HttpClientStreamableHttpTransport.builder(baseUri).endpoint(path);
customizer.customize("default-transport", transportBuilder); transportCustomizer.customize("default-transport", transportBuilder);
HttpClientStreamableHttpTransport transport = transportBuilder.build(); HttpClientStreamableHttpTransport transport = transportBuilder.build();
this.client = McpClient.sync(transport) this.client = McpClient.sync(transport)

View file

@ -7,5 +7,3 @@ client:
# - Client does not parse or respect retry: field timing # - Client does not parse or respect retry: field timing
# - Client does not send Last-Event-ID header # - Client does not send Last-Event-ID header
- sse-retry - sse-retry
# CIMD not implemented yet
- auth/basic-cimd