conformance-tests: upgrade to MCP-security 0.1.11, implement CIMD
Signed-off-by: Daniel Garnier-Moiroux <git@garnier.wf>
This commit is contained in:
Daniel Garnier-Moiroux
Daniel Garnier-Moiroux
parent
d81430a532
commit
fc330a5885
7 changed files with 94 additions and 67 deletions
|
|
@ -5,7 +5,7 @@
|
|||
**Server Tests (active suite):** 44/44 passed (31 scenarios, 100%)
|
||||
**Server Tests (spec 2025-11-25):** 4/4 passed — SEP-1613 `json-schema-2020-12` scenario ✨
|
||||
**Client Tests:** 3/4 scenarios passed (9/10 checks passed)
|
||||
**Auth Tests:** 14/15 scenarios fully passing (196 passed, 0 failed, 1 warning, 93.3% scenarios, 99.5% checks)
|
||||
**Auth Tests:** 15/15 scenarios fully passing (195 passed, 0 failed, 0 warnings, 100% scenarios, 100% checks)
|
||||
|
||||
## Server Test Results
|
||||
|
||||
|
|
@ -46,16 +46,17 @@
|
|||
|
||||
## Auth Test Results (Spring HTTP Client)
|
||||
|
||||
**Status: 196 passed, 0 failed, 1 warning across 15 scenarios**
|
||||
**Status: 195 passed, 0 failed, 0 warnings across 15 scenarios**
|
||||
|
||||
Uses the `client-spring-http-client` module with Spring Security OAuth2 and the [mcp-client-security](https://github.com/springaicommunity/mcp-client-security) library.
|
||||
|
||||
### Fully Passing (14/15 scenarios)
|
||||
### Fully Passing (15/15 scenarios)
|
||||
|
||||
- **auth/metadata-default (13/13):** Default metadata discovery
|
||||
- **auth/metadata-var1 (13/13):** Metadata discovery variant 1
|
||||
- **auth/metadata-var2 (13/13):** Metadata discovery variant 2
|
||||
- **auth/metadata-var3 (13/13):** Metadata discovery variant 3
|
||||
- **auth/basic-cimd (12/12):** Basic Client-Initiated Metadata Discovery
|
||||
- **auth/scope-from-www-authenticate (14/14):** Scope extraction from WWW-Authenticate header
|
||||
- **auth/scope-from-scopes-supported (14/14):** Scope extraction from scopes_supported
|
||||
- **auth/scope-omitted-when-undefined (14/14):** Scope omitted when not defined
|
||||
|
|
@ -67,14 +68,9 @@ Uses the `client-spring-http-client` module with Spring Security OAuth2 and the
|
|||
- **auth/resource-mismatch (2/2):** Resource mismatch handling
|
||||
- **auth/pre-registration (6/6):** Pre-registered client credentials flow
|
||||
|
||||
### Partially Passing (1/15 scenarios)
|
||||
|
||||
- **auth/basic-cimd (13/13 + 1 warning):** Basic Client-Initiated Metadata Discovery — all checks pass, minor warning
|
||||
|
||||
## Known Limitations
|
||||
|
||||
1. **Client SSE Retry:** Client doesn't parse or respect the `retry:` field, reconnects immediately, and doesn't send Last-Event-ID header
|
||||
2. **Auth Basic CIMD:** Minor conformance warning in the basic Client-Initiated Metadata Discovery flow
|
||||
|
||||
## Running Tests
|
||||
|
||||
|
|
@ -132,4 +128,3 @@ npx @modelcontextprotocol/conformance@0.1.15 client \
|
|||
|
||||
### High Priority
|
||||
1. Fix client SSE retry field handling in `HttpClientStreamableHttpTransport`
|
||||
2. Implement CIMD
|
||||
|
|
|
|||
|
|
@ -14,23 +14,24 @@ Test with @modelcontextprotocol/conformance@0.1.15.
|
|||
|
||||
## Conformance Test Results
|
||||
|
||||
**Status: 178 passed, 1 failed, 1 warning across 14 scenarios**
|
||||
**Status: 195 passed, 0 failed, 0 warnings across 15 scenarios**
|
||||
|
||||
| Scenario | Result | Details |
|
||||
|---|---|---|
|
||||
| auth/metadata-default | ✅ Pass | 12/12 |
|
||||
| auth/metadata-var1 | ✅ Pass | 12/12 |
|
||||
| auth/metadata-var2 | ✅ Pass | 12/12 |
|
||||
| auth/metadata-var3 | ✅ Pass | 12/12 |
|
||||
| auth/basic-cimd | ⚠️ Warning | 12/12 passed, 1 warning |
|
||||
| auth/scope-from-www-authenticate | ✅ Pass | 13/13 |
|
||||
| auth/scope-from-scopes-supported | ✅ Pass | 13/13 |
|
||||
| auth/scope-omitted-when-undefined | ✅ Pass | 13/13 |
|
||||
| auth/scope-step-up | ✅ Pass | 12/12 |
|
||||
| auth/metadata-default | ✅ Pass | 13/13 |
|
||||
| auth/metadata-var1 | ✅ Pass | 13/13 |
|
||||
| auth/metadata-var2 | ✅ Pass | 13/13 |
|
||||
| auth/metadata-var3 | ✅ Pass | 13/13 |
|
||||
| auth/basic-cimd | ✅ Pass | 12/12 |
|
||||
| auth/scope-from-www-authenticate | ✅ Pass | 14/14 |
|
||||
| auth/scope-from-scopes-supported | ✅ Pass | 14/14 |
|
||||
| auth/scope-omitted-when-undefined | ✅ Pass | 14/14 |
|
||||
| auth/scope-step-up | ✅ Pass | 16/16 |
|
||||
| auth/scope-retry-limit | ✅ Pass | 11/11 |
|
||||
| auth/token-endpoint-auth-basic | ✅ Pass | 17/17 |
|
||||
| auth/token-endpoint-auth-post | ✅ Pass | 17/17 |
|
||||
| auth/token-endpoint-auth-none | ✅ Pass | 17/17 |
|
||||
| auth/token-endpoint-auth-basic | ✅ Pass | 18/18 |
|
||||
| auth/token-endpoint-auth-post | ✅ Pass | 18/18 |
|
||||
| auth/token-endpoint-auth-none | ✅ Pass | 18/18 |
|
||||
| auth/resource-mismatch | ✅ Pass | 2/2 |
|
||||
| auth/pre-registration | ✅ Pass | 6/6 |
|
||||
|
||||
See [VALIDATION_RESULTS.md](../VALIDATION_RESULTS.md) for the full project validation results.
|
||||
|
|
@ -113,8 +114,7 @@ java -jar conformance-tests/client-spring-http-client/target/client-spring-http-
|
|||
|
||||
## Known Issues
|
||||
|
||||
1. **auth/scope-step-up** (1 failure) — The client does not fully handle scope step-up challenges where the server requests additional scopes after initial authorization.
|
||||
2. **auth/basic-cimd** (1 warning) — Minor conformance warning in the basic Client-Initiated Metadata Discovery flow.
|
||||
Currently, there are no known issues in the auth suite implementation.
|
||||
|
||||
## References
|
||||
|
||||
|
|
|
|||
|
|
@ -23,8 +23,8 @@
|
|||
<properties>
|
||||
<java.version>17</java.version>
|
||||
<spring-boot.version>4.0.5</spring-boot.version>
|
||||
<spring-ai.version>2.0.0-M4</spring-ai.version>
|
||||
<spring-ai-mcp-security.version>0.1.5</spring-ai-mcp-security.version>
|
||||
<spring-ai.version>2.0.0-M6</spring-ai.version>
|
||||
<spring-ai-mcp-security.version>0.1.11</spring-ai-mcp-security.version>
|
||||
<maven.deploy.skip>true</maven.deploy.skip>
|
||||
</properties>
|
||||
|
||||
|
|
|
|||
|
|
@ -8,17 +8,23 @@ import java.util.Optional;
|
|||
|
||||
import io.modelcontextprotocol.conformance.client.scenario.Scenario;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.metadata.McpMetadataDiscoveryService;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DefaultMcpOAuth2ClientManager;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DefaultMcpOAuth2DcrClientManager;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.DynamicClientRegistrationService;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.InMemoryMcpClientRegistrationRepository;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2DcrClientManager;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.cimd.DefaultMcpOAuth2CimdClientManager;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.cimd.McpOAuth2CimdClientManager;
|
||||
import org.springaicommunity.mcp.security.common.url.DefaultUrlValidator;
|
||||
|
||||
import org.springframework.boot.ApplicationArguments;
|
||||
import org.springframework.boot.ApplicationRunner;
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.security.oauth2.client.OAuth2AuthorizedClientManager;
|
||||
import org.springframework.security.oauth2.client.web.DefaultOAuth2AuthorizedClientManager;
|
||||
import org.springframework.security.oauth2.client.web.OAuth2AuthorizedClientRepository;
|
||||
|
||||
/**
|
||||
* MCP Conformance Test Client - Spring HTTP Client Implementation.
|
||||
|
|
@ -42,13 +48,15 @@ public class ConformanceSpringClientApplication {
|
|||
|
||||
public static final String REGISTRATION_ID = "default_registration";
|
||||
|
||||
private final DefaultUrlValidator URL_VALIDATOR = new DefaultUrlValidator(true);
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(ConformanceSpringClientApplication.class, args);
|
||||
}
|
||||
|
||||
@Bean
|
||||
McpMetadataDiscoveryService discovery() {
|
||||
return new McpMetadataDiscoveryService();
|
||||
return new McpMetadataDiscoveryService(URL_VALIDATOR);
|
||||
}
|
||||
|
||||
@Bean
|
||||
|
|
@ -57,10 +65,24 @@ public class ConformanceSpringClientApplication {
|
|||
}
|
||||
|
||||
@Bean
|
||||
McpOAuth2ClientManager mcpOAuth2ClientManager(McpClientRegistrationRepository mcpClientRegistrationRepository,
|
||||
McpOAuth2DcrClientManager mcpOAuth2ClientManager(McpClientRegistrationRepository mcpClientRegistrationRepository,
|
||||
McpMetadataDiscoveryService mcpMetadataDiscoveryService) {
|
||||
return new DefaultMcpOAuth2ClientManager(mcpClientRegistrationRepository,
|
||||
new DynamicClientRegistrationService(), mcpMetadataDiscoveryService);
|
||||
return new DefaultMcpOAuth2DcrClientManager(mcpClientRegistrationRepository,
|
||||
new DynamicClientRegistrationService(URL_VALIDATOR), mcpMetadataDiscoveryService, URL_VALIDATOR);
|
||||
}
|
||||
|
||||
@Bean
|
||||
McpOAuth2CimdClientManager mcpOAuth2CimdClientManager(McpMetadataDiscoveryService mcpMetadataDiscoveryService,
|
||||
McpClientRegistrationRepository mcpClientRegistrationRepository) {
|
||||
return new DefaultMcpOAuth2CimdClientManager(mcpMetadataDiscoveryService, mcpClientRegistrationRepository,
|
||||
URL_VALIDATOR);
|
||||
}
|
||||
|
||||
@Bean
|
||||
OAuth2AuthorizedClientManager oAuth2AuthorizedClientManager(
|
||||
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository,
|
||||
McpClientRegistrationRepository clientRegistrationRepository) {
|
||||
return new DefaultOAuth2AuthorizedClientManager(clientRegistrationRepository, oAuth2AuthorizedClientRepository);
|
||||
}
|
||||
|
||||
@Bean
|
||||
|
|
|
|||
|
|
@ -4,38 +4,65 @@
|
|||
|
||||
package io.modelcontextprotocol.conformance.client.configuration;
|
||||
|
||||
import io.modelcontextprotocol.conformance.client.ConformanceSpringClientApplication;
|
||||
import io.modelcontextprotocol.client.transport.HttpClientStreamableHttpTransport;
|
||||
import io.modelcontextprotocol.conformance.client.scenario.DefaultScenario;
|
||||
import org.springaicommunity.mcp.security.client.sync.config.McpClientOAuth2Configurer;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.http.client.OAuth2CimdHttpClientTransportCustomizer;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.http.client.OAuth2DcrHttpClientTransportCustomizer;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2DcrClientManager;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.cimd.DefaultMcpOAuth2CimdClientManager;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.cimd.McpOAuth2CimdClientManager;
|
||||
|
||||
import org.springframework.ai.mcp.customizer.McpClientCustomizer;
|
||||
import org.springframework.beans.factory.annotation.Value;
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
|
||||
import org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
import org.springframework.security.config.Customizer;
|
||||
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
|
||||
import org.springframework.security.oauth2.client.web.OAuth2AuthorizedClientRepository;
|
||||
import org.springframework.security.oauth2.client.OAuth2AuthorizedClientManager;
|
||||
import org.springframework.security.oauth2.client.registration.ClientRegistration;
|
||||
import org.springframework.security.web.SecurityFilterChain;
|
||||
|
||||
@Configuration
|
||||
@ConditionalOnExpression("#{environment['MCP_CONFORMANCE_SCENARIO'] != 'auth/pre-registration'}")
|
||||
public class DefaultConfiguration {
|
||||
|
||||
private final String TEST_CLIENT_ID_URL = "https://conformance-test.local/client-metadata.json";
|
||||
|
||||
@Bean
|
||||
DefaultScenario defaultScenario(McpClientRegistrationRepository clientRegistrationRepository,
|
||||
ServletWebServerApplicationContext serverCtx,
|
||||
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository,
|
||||
McpOAuth2ClientManager mcpOAuth2ClientManager) {
|
||||
return new DefaultScenario(clientRegistrationRepository, serverCtx, oAuth2AuthorizedClientRepository,
|
||||
mcpOAuth2ClientManager);
|
||||
DefaultScenario defaultScenario(ServletWebServerApplicationContext serverCtx,
|
||||
McpClientCustomizer<HttpClientStreamableHttpTransport.Builder> transportCustomizer) {
|
||||
return new DefaultScenario(serverCtx, transportCustomizer);
|
||||
}
|
||||
|
||||
@Bean
|
||||
SecurityFilterChain securityFilterChain(HttpSecurity http, ConformanceSpringClientApplication.ServerUrl serverUrl) {
|
||||
McpClientCustomizer<HttpClientStreamableHttpTransport.Builder> transportCustomizer(
|
||||
OAuth2AuthorizedClientManager oAuth2AuthorizedClientManager,
|
||||
McpClientRegistrationRepository clientRegistrationRepository,
|
||||
McpOAuth2DcrClientManager mcpOAuth2ClientManager, McpOAuth2CimdClientManager mcpOAuth2CimdClientManager,
|
||||
@Value("${mcp.conformance.scenario}") String scenario) {
|
||||
if (scenario.equals("auth/basic-cimd")) {
|
||||
if (mcpOAuth2CimdClientManager instanceof DefaultMcpOAuth2CimdClientManager mgr) {
|
||||
// Hardcode the client_id
|
||||
mgr.setClientRegistrationCustomizer(
|
||||
cr -> ClientRegistration.withClientRegistration(cr).clientId(TEST_CLIENT_ID_URL).build());
|
||||
}
|
||||
return new OAuth2CimdHttpClientTransportCustomizer(oAuth2AuthorizedClientManager,
|
||||
clientRegistrationRepository, mcpOAuth2CimdClientManager);
|
||||
|
||||
}
|
||||
else {
|
||||
return new OAuth2DcrHttpClientTransportCustomizer(oAuth2AuthorizedClientManager,
|
||||
clientRegistrationRepository, mcpOAuth2ClientManager);
|
||||
}
|
||||
}
|
||||
|
||||
@Bean
|
||||
SecurityFilterChain securityFilterChain(HttpSecurity http) {
|
||||
return http.authorizeHttpRequests(authz -> authz.anyRequest().permitAll())
|
||||
.with(new McpClientOAuth2Configurer(), Customizer.withDefaults())
|
||||
.with(new McpClientOAuth2Configurer(), mcp -> mcp.cimd(true))
|
||||
.build();
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -17,14 +17,10 @@ import org.jspecify.annotations.NonNull;
|
|||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springaicommunity.mcp.security.client.sync.AuthenticationMcpTransportContextProvider;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.http.client.OAuth2HttpClientTransportCustomizer;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpClientRegistrationRepository;
|
||||
import org.springaicommunity.mcp.security.client.sync.oauth2.registration.McpOAuth2ClientManager;
|
||||
|
||||
import org.springframework.ai.mcp.customizer.McpClientCustomizer;
|
||||
import org.springframework.boot.web.server.servlet.context.ServletWebServerApplicationContext;
|
||||
import org.springframework.http.client.JdkClientHttpRequestFactory;
|
||||
import org.springframework.security.oauth2.client.web.DefaultOAuth2AuthorizedClientManager;
|
||||
import org.springframework.security.oauth2.client.web.OAuth2AuthorizedClientRepository;
|
||||
import org.springframework.web.client.RestClient;
|
||||
import org.springframework.web.util.UriComponentsBuilder;
|
||||
|
||||
|
|
@ -34,23 +30,14 @@ public class DefaultScenario implements Scenario {
|
|||
|
||||
private final ServletWebServerApplicationContext serverCtx;
|
||||
|
||||
private final DefaultOAuth2AuthorizedClientManager authorizedClientManager;
|
||||
|
||||
private final McpClientRegistrationRepository clientRegistrationRepository;
|
||||
|
||||
private final McpOAuth2ClientManager mcpOAuth2ClientManager;
|
||||
private final McpClientCustomizer<HttpClientStreamableHttpTransport.Builder> transportCustomizer;
|
||||
|
||||
private McpSyncClient client;
|
||||
|
||||
public DefaultScenario(McpClientRegistrationRepository clientRegistrationRepository,
|
||||
ServletWebServerApplicationContext serverCtx,
|
||||
OAuth2AuthorizedClientRepository oAuth2AuthorizedClientRepository,
|
||||
McpOAuth2ClientManager mcpOAuth2ClientManager) {
|
||||
public DefaultScenario(ServletWebServerApplicationContext serverCtx,
|
||||
McpClientCustomizer<HttpClientStreamableHttpTransport.Builder> transportCustomizer) {
|
||||
this.serverCtx = serverCtx;
|
||||
this.clientRegistrationRepository = clientRegistrationRepository;
|
||||
this.mcpOAuth2ClientManager = mcpOAuth2ClientManager;
|
||||
this.authorizedClientManager = new DefaultOAuth2AuthorizedClientManager(clientRegistrationRepository,
|
||||
oAuth2AuthorizedClientRepository);
|
||||
this.transportCustomizer = transportCustomizer;
|
||||
}
|
||||
|
||||
@Override
|
||||
|
|
@ -59,12 +46,10 @@ public class DefaultScenario implements Scenario {
|
|||
var testServerUrl = "http://localhost:" + serverCtx.getWebServer().getPort();
|
||||
var testClient = buildTestClient(testServerUrl);
|
||||
|
||||
var customizer = new OAuth2HttpClientTransportCustomizer(authorizedClientManager, clientRegistrationRepository,
|
||||
mcpOAuth2ClientManager);
|
||||
var baseUri = UriComponentsBuilder.fromUriString(serverUrl).replacePath(null).toUriString();
|
||||
var path = UriComponentsBuilder.fromUriString(serverUrl).build().getPath();
|
||||
var transportBuilder = HttpClientStreamableHttpTransport.builder(baseUri).endpoint(path);
|
||||
customizer.customize("default-transport", transportBuilder);
|
||||
transportCustomizer.customize("default-transport", transportBuilder);
|
||||
HttpClientStreamableHttpTransport transport = transportBuilder.build();
|
||||
|
||||
this.client = McpClient.sync(transport)
|
||||
|
|
|
|||
|
|
@ -7,5 +7,3 @@ client:
|
|||
# - Client does not parse or respect retry: field timing
|
||||
# - Client does not send Last-Event-ID header
|
||||
- sse-retry
|
||||
# CIMD not implemented yet
|
||||
- auth/basic-cimd
|
||||
|
|
|
|||
Loading…
Reference in a new issue