[compliance tests] initial server scenarios (#4)

* spec for servers

* initial version of everything server and tests - tets are very simple

* some clean up

* lint etc.

* fix package lock agian

---------

Co-authored-by: Paul Carleton <paulc@anthropic.com>
This commit is contained in:
Inna Harper 2025-11-04 15:08:20 +00:00 • committed by GitHub
commit a692dd7bba
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 5678 additions and 78 deletions

View file

@ -0,0 +1,173 @@
# MCP Conformance Test Server
A reference implementation of an MCP server that implements all features required for conformance testing.
## Features
This server implements:
### Tools
- `test_simple_text` - Returns simple text content
- `test_image_content` - Returns image content (base64 PNG)
- `test_audio_content` - Returns audio content (base64 WAV)
- `test_embedded_resource` - Returns embedded resource
- `test_multiple_content_types` - Returns mixed content types
- `test_tool_with_logging` - Emits log messages during execution
- `test_tool_with_progress` - Reports progress notifications
- `test_error_handling` - Returns error response
- `test_sampling` - Requests LLM completion from client
- `test_elicitation` - Requests user input from client
- `test_dynamic_tool` - Dynamically added/removed tool
### Resources
- `test://static-text` - Static text resource
- `test://static-binary` - Static binary resource (image)
- `test://template/{id}/data` - Resource template with parameter
- `test://watched-resource` - Subscribable resource with updates
### Prompts
- `test_simple_prompt` - Simple prompt without arguments
- `test_prompt_with_arguments(arg1, arg2)` - Parameterized prompt
- `test_prompt_with_embedded_resource(resourceUri)` - Prompt with embedded resource
- `test_prompt_with_image` - Prompt with image content
### Other Capabilities
- Logging at all levels (debug, info, notice, warning, error, critical, alert, emergency)
- Completion support for prompt and resource arguments
- List changed notifications for tools, resources, and prompts
- Resource subscription and update notifications
## Installation
```bash
npm install
```
## Running the Server
```bash
npm start
```
The server will start on `http://localhost:3000` (or the port specified in `PORT` environment variable).
## Endpoints
### MCP Endpoint
- `POST /mcp` - Main MCP protocol endpoint
### Health Check
- `GET /health` - Server health check
## Automatic Behaviors
The server automatically demonstrates dynamic capabilities:
- **Dynamic Tool** - `test_dynamic_tool` is automatically added 2 seconds after server starts
- **Dynamic Resource** - `test://dynamic-resource` is automatically added 2 seconds after server starts
- **Dynamic Prompt** - `test_dynamic_prompt` is automatically added 2 seconds after server starts
- **Resource Updates** - `test://watched-resource` automatically updates every 3 seconds with new content
These behaviors allow testing of MCP notifications without requiring manual triggers.
## Example Usage
### Starting the Server
```bash
npm start
```
### Testing with MCP Inspector
```bash
npx @modelcontextprotocol/inspector http://localhost:3000/mcp
```
### Testing with curl
#### Initialize
```bash
curl -X POST http://localhost:3000/mcp \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": {
"protocolVersion": "2025-06-18",
"capabilities": {},
"clientInfo": {
"name": "test-client",
"version": "1.0.0"
}
}
}'
```
#### List Tools
```bash
curl -X POST http://localhost:3000/mcp \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/list"
}'
```
#### Call Tool
```bash
curl -X POST http://localhost:3000/mcp \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 3,
"method": "tools/call",
"params": {
"name": "test_simple_text",
"arguments": {}
}
}'
```
## Conformance Testing
This server implements the MCP Server Conformance Requirements specified in `../../../SERVER_REQUIREMENTS.md`. All tools, resources, and prompts use standardized naming conventions for consistent testing across SDK implementations.
To run conformance tests against this server:
```bash
cd ../../../
npm run test:server -- --server-url http://localhost:3000/mcp --all
```
## Implementation Notes
- All tool, resource, and prompt names follow the standardized naming conventions (`test_*` for tools/prompts, `test://` for resources)
- Names are descriptive of the feature being tested (e.g., `test_image_content`, `test_tool_with_progress`)
- The server uses the TypeScript MCP SDK (`@modelcontextprotocol/sdk`) high-level API
- Uses `registerTool()`, `registerResource()`, and `registerPrompt()` methods
- Transport is Streamable HTTP (Express) for web-based testing compatibility
- Promise rejections from notifications are caught and handled gracefully
## For SDK Implementers
If you're implementing MCP in another language/SDK:
1. **Read the Requirements**: See `../../../SERVER_REQUIREMENTS.md` for complete specifications
2. **Use This as Reference**: This TypeScript implementation demonstrates all required features
3. **Follow Naming Conventions**: Use exact tool/resource/prompt names specified in requirements
4. **Implement Automatic Behaviors**: Dynamic registration after 2s, resource updates every 3s
5. **Handle Notifications Carefully**: Catch/ignore errors when no client is connected
**Goal**: All SDK example servers provide the same interface, enabling a single test suite to verify conformance across all implementations.

View file

@ -0,0 +1,748 @@
#!/usr/bin/env node
/**
* MCP Everything Server - Conformance Test Server
*
* Server implementing all MCP features for conformance testing based on Conformnace Server Specification.
* Should be using registerTool(), registerResource(), and registerPrompt().
* we use tool() instead of registerTool() as there is a bug with logging in registerTool().
*/
import { McpServer, ResourceTemplate } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { z } from 'zod';
import express from 'express';
import cors from 'cors';
import { randomUUID } from 'crypto';
// Server state
const resourceSubscriptions = new Set<string>();
const watchedResourceContent = 'Watched resource content';
// Session management
const transports: { [sessionId: string]: StreamableHTTPServerTransport } = {};
const servers: { [sessionId: string]: McpServer } = {};
// Sample base64 encoded 1x1 red PNG pixel for testing
const TEST_IMAGE_BASE64 = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8DwHwAFBQIAX8jx0gAAAABJRU5ErkJggg==';
// Sample base64 encoded minimal WAV file for testing
const TEST_AUDIO_BASE64 = 'UklGRiYAAABXQVZFZm10IBAAAAABAAEAQB8AAAB9AAACABAAZGF0YQIAAAA=';
// Function to create a new MCP server instance (one per session)
function createMcpServer() {
const mcpServer = new McpServer(
{
name: 'mcp-conformance-test-server',
version: '1.0.0'
},
{
capabilities: {
tools: {
listChanged: true
},
resources: {
subscribe: true,
listChanged: true
},
prompts: {
listChanged: true
},
logging: {},
completions: {}
}
}
);
// Helper to send log messages using the underlying server
function sendLog(
level: 'debug' | 'info' | 'notice' | 'warning' | 'error' | 'critical' | 'alert' | 'emergency',
message: string,
data?: any
) {
mcpServer.server
.notification({
method: 'notifications/message',
params: {
level,
logger: 'conformance-test-server',
data: data || message
}
})
.catch(() => {
// Ignore error if no client is connected
});
}
// ===== TOOLS =====
// Simple text tool
mcpServer.tool('test_simple_text', 'Tests simple text content response', {}, async () => {
return {
content: [{ type: 'text', text: 'This is a simple text response for testing.' }]
};
});
// Image content tool
mcpServer.registerTool(
'test_image_content',
{
description: 'Tests image content response'
},
async () => {
return {
content: [{ type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' }]
};
}
);
// Audio content tool
mcpServer.registerTool(
'test_audio_content',
{
description: 'Tests audio content response'
},
async () => {
return {
content: [{ type: 'audio', data: TEST_AUDIO_BASE64, mimeType: 'audio/wav' }]
};
}
);
// Embedded resource tool
mcpServer.registerTool(
'test_embedded_resource',
{
description: 'Tests embedded resource content response'
},
async () => {
return {
content: [
{
type: 'resource',
resource: {
uri: 'test://embedded-resource',
mimeType: 'text/plain',
text: 'This is an embedded resource content.'
}
}
]
};
}
);
// Multiple content types tool
mcpServer.registerTool(
'test_multiple_content_types',
{
description: 'Tests response with multiple content types (text, image, resource)'
},
async () => {
return {
content: [
{ type: 'text', text: 'Multiple content types test:' },
{ type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' },
{
type: 'resource',
resource: {
uri: 'test://mixed-content-resource',
mimeType: 'application/json',
text: JSON.stringify({ test: 'data', value: 123 })
}
}
]
};
}
);
// Tool with logging - registerTool with empty inputSchema to get (args, extra) signature
mcpServer.registerTool(
'test_tool_with_logging',
{
description: 'Tests tool that emits log messages during execution',
inputSchema: {} // Empty schema so callback gets (args, extra) instead of just (extra)
},
async (_args, { sendNotification }) => {
await sendNotification({
method: 'notifications/message',
params: {
level: 'info',
data: 'Tool execution started'
}
});
await new Promise(resolve => setTimeout(resolve, 50));
await sendNotification({
method: 'notifications/message',
params: {
level: 'info',
data: 'Tool processing data'
}
});
await new Promise(resolve => setTimeout(resolve, 50));
await sendNotification({
method: 'notifications/message',
params: {
level: 'info',
data: 'Tool execution completed'
}
});
return {
content: [{ type: 'text', text: 'Tool with logging executed successfully' }]
};
}
);
// Tool with progress - registerTool with empty inputSchema to get (args, extra) signature
mcpServer.registerTool(
'test_tool_with_progress',
{
description: 'Tests tool that reports progress notifications',
inputSchema: {} // Empty schema so callback gets (args, extra) instead of just (extra)
},
async (_args, { sendNotification, _meta }) => {
const progressToken = _meta?.progressToken ?? 0;
console.log('???? Progress token:', progressToken);
await sendNotification({
method: 'notifications/progress',
params: {
progressToken,
progress: 0,
total: 100,
message: `Completed step ${0} of ${100}`
}
});
await new Promise(resolve => setTimeout(resolve, 50));
await sendNotification({
method: 'notifications/progress',
params: {
progressToken,
progress: 50,
total: 100,
message: `Completed step ${50} of ${100}`
}
});
await new Promise(resolve => setTimeout(resolve, 50));
await sendNotification({
method: 'notifications/progress',
params: {
progressToken,
progress: 100,
total: 100,
message: `Completed step ${100} of ${100}`
}
});
return {
content: [{ type: 'text', text: String(progressToken) }]
};
}
);
// Error handling tool
mcpServer.registerTool(
'test_error_handling',
{
description: 'Tests error response handling'
},
async () => {
throw new Error('This tool intentionally returns an error for testing');
}
);
// Sampling tool - requests LLM completion from client
mcpServer.registerTool(
'test_sampling',
{
description: 'Tests server-initiated sampling (LLM completion request)',
inputSchema: {
prompt: z.string().describe('The prompt to send to the LLM')
}
},
async (args: { prompt: string }) => {
try {
// Request sampling from client
const result = await mcpServer.server.request(
{
method: 'sampling/createMessage',
params: {
messages: [
{
role: 'user',
content: {
type: 'text',
text: args.prompt
}
}
],
maxTokens: 100
}
},
z.object({ method: z.literal('sampling/createMessage') }).passthrough() as any
);
const samplingResult = result as any;
const modelResponse = samplingResult.content?.text || samplingResult.message?.content?.text || 'No response';
return {
content: [
{
type: 'text',
text: `LLM response: ${modelResponse}`
}
]
};
} catch (error: any) {
return {
content: [
{
type: 'text',
text: `Sampling not supported or error: ${error.message}`
}
]
};
}
}
);
// Elicitation tool - requests user input from client
mcpServer.registerTool(
'test_elicitation',
{
description: 'Tests server-initiated elicitation (user input request)',
inputSchema: {
message: z.string().describe('The message to show the user')
}
},
async (args: { message: string }) => {
try {
// Request user input from client
const result = await mcpServer.server.request(
{
method: 'elicitation/create',
params: {
message: args.message,
requestedSchema: {
type: 'object',
properties: {
response: {
type: 'string',
description: "User's response"
}
},
required: ['response']
}
}
},
z.object({ method: z.literal('elicitation/create') }).passthrough() as any
);
const elicitResult = result as any;
return {
content: [
{
type: 'text',
text: `User response: action=${elicitResult.action}, content=${JSON.stringify(elicitResult.content || {})}`
}
]
};
} catch (error: any) {
return {
content: [
{
type: 'text',
text: `Elicitation not supported or error: ${error.message}`
}
]
};
}
}
);
// Dynamic tool (registered later via timer)
// ===== RESOURCES =====
// Static text resource
mcpServer.registerResource(
'static-text',
'test://static-text',
{
title: 'Static Text Resource',
description: 'A static text resource for testing',
mimeType: 'text/plain'
},
async () => {
return {
contents: [
{
uri: 'test://static-text',
mimeType: 'text/plain',
text: 'This is the content of the static text resource.'
}
]
};
}
);
// Static binary resource
mcpServer.registerResource(
'static-binary',
'test://static-binary',
{
title: 'Static Binary Resource',
description: 'A static binary resource (image) for testing',
mimeType: 'image/png'
},
async () => {
return {
contents: [
{
uri: 'test://static-binary',
mimeType: 'image/png',
blob: TEST_IMAGE_BASE64
}
]
};
}
);
// Resource template
mcpServer.registerResource(
'template',
new ResourceTemplate('test://template/{id}/data', {
list: undefined
}),
{
title: 'Resource Template',
description: 'A resource template with parameter substitution',
mimeType: 'application/json'
},
async (uri, variables) => {
const id = variables.id;
return {
contents: [
{
uri: uri.toString(),
mimeType: 'application/json',
text: JSON.stringify({
id,
templateTest: true,
data: `Data for ID: ${id}`
})
}
]
};
}
);
// Watched resource
mcpServer.registerResource(
'watched-resource',
'test://watched-resource',
{
title: 'Watched Resource',
description: 'A resource that auto-updates every 3 seconds',
mimeType: 'text/plain'
},
async () => {
return {
contents: [
{
uri: 'test://watched-resource',
mimeType: 'text/plain',
text: watchedResourceContent
}
]
};
}
);
// Subscribe/Unsubscribe handlers
mcpServer.server.setRequestHandler(z.object({ method: z.literal('resources/subscribe') }).passthrough(), async (request: any) => {
const uri = request.params.uri;
resourceSubscriptions.add(uri);
sendLog('info', `Subscribed to resource: ${uri}`);
return {};
});
mcpServer.server.setRequestHandler(z.object({ method: z.literal('resources/unsubscribe') }).passthrough(), async (request: any) => {
const uri = request.params.uri;
resourceSubscriptions.delete(uri);
sendLog('info', `Unsubscribed from resource: ${uri}`);
return {};
});
// ===== PROMPTS =====
// Simple prompt
mcpServer.registerPrompt(
'test_simple_prompt',
{
title: 'Simple Test Prompt',
description: 'A simple prompt without arguments'
},
async () => {
return {
messages: [
{
role: 'user',
content: { type: 'text', text: 'This is a simple prompt for testing.' }
}
]
};
}
);
// Prompt with arguments
mcpServer.registerPrompt(
'test_prompt_with_arguments',
{
title: 'Prompt With Arguments',
description: 'A prompt with required arguments',
argsSchema: {
arg1: z.string().describe('First test argument'),
arg2: z.string().describe('Second test argument')
}
},
async args => {
const { arg1, arg2 } = args;
return {
messages: [
{
role: 'user',
content: {
type: 'text',
text: `Prompt with arguments: arg1='${arg1}', arg2='${arg2}'`
}
}
]
};
}
);
// Prompt with embedded resource
mcpServer.registerPrompt(
'test_prompt_with_embedded_resource',
{
title: 'Prompt With Embedded Resource',
description: 'A prompt that includes an embedded resource',
argsSchema: {
resourceUri: z.string().describe('URI of the resource to embed')
}
},
async args => {
const uri = args.resourceUri;
return {
messages: [
{
role: 'user',
content: {
type: 'resource',
resource: {
uri,
mimeType: 'text/plain',
text: 'Embedded resource content for testing.'
}
}
},
{
role: 'user',
content: { type: 'text', text: 'Please process the embedded resource above.' }
}
]
};
}
);
// Prompt with image
mcpServer.registerPrompt(
'test_prompt_with_image',
{
title: 'Prompt With Image',
description: 'A prompt that includes image content'
},
async () => {
return {
messages: [
{
role: 'user',
content: { type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' }
},
{
role: 'user',
content: { type: 'text', text: 'Please analyze the image above.' }
}
]
};
}
);
// ===== LOGGING =====
mcpServer.server.setRequestHandler(z.object({ method: z.literal('logging/setLevel') }).passthrough(), async (request: any) => {
const level = request.params.level;
sendLog('info', `Log level set to: ${level}`);
return {};
});
// ===== COMPLETION =====
mcpServer.server.setRequestHandler(z.object({ method: z.literal('completion/complete') }).passthrough(), async (_request: any) => {
// Basic completion support - returns empty array for conformance
// Real implementations would provide contextual suggestions
return {
completion: {
values: [],
total: 0,
hasMore: false
}
};
});
return mcpServer;
}
// Helper to check if request is an initialize request
function isInitializeRequest(body: any): boolean {
return body?.method === 'initialize';
}
// ===== EXPRESS APP =====
const app = express();
app.use(express.json());
// Configure CORS to expose Mcp-Session-Id header for browser-based clients
app.use(
cors({
origin: '*', // Allow all origins
exposedHeaders: ['Mcp-Session-Id'],
allowedHeaders: ['Content-Type', 'mcp-session-id', 'last-event-id']
})
);
// Handle POST requests - stateful mode
app.post('/mcp', async (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined;
try {
let transport: StreamableHTTPServerTransport;
if (sessionId && transports[sessionId]) {
// Reuse existing transport for established sessions
transport = transports[sessionId];
} else if (!sessionId && isInitializeRequest(req.body)) {
// Create new transport for initialization requests
const mcpServer = createMcpServer();
transport = new StreamableHTTPServerTransport({
sessionIdGenerator: () => randomUUID(),
onsessioninitialized: newSessionId => {
transports[newSessionId] = transport;
servers[newSessionId] = mcpServer;
console.log(`Session initialized with ID: ${newSessionId}`);
}
});
transport.onclose = () => {
const sid = transport.sessionId;
if (sid && transports[sid]) {
delete transports[sid];
if (servers[sid]) {
servers[sid].close();
delete servers[sid];
}
console.log(`Session ${sid} closed`);
}
};
await mcpServer.connect(transport);
await transport.handleRequest(req, res, req.body);
return;
} else {
res.status(400).json({
jsonrpc: '2.0',
error: {
code: -32000,
message: 'Invalid or missing session ID'
},
id: null
});
return;
}
await transport.handleRequest(req, res, req.body);
} catch (error) {
console.error('Error handling MCP request:', error);
if (!res.headersSent) {
res.status(500).json({
jsonrpc: '2.0',
error: {
code: -32603,
message: 'Internal server error'
},
id: null
});
}
}
});
// Handle GET requests - SSE streams for sessions
app.get('/mcp', async (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined;
if (!sessionId || !transports[sessionId]) {
res.status(400).send('Invalid or missing session ID');
return;
}
const lastEventId = req.headers['last-event-id'] as string | undefined;
if (lastEventId) {
console.log(`Client reconnecting with Last-Event-ID: ${lastEventId}`);
} else {
console.log(`Establishing SSE stream for session ${sessionId}`);
}
try {
const transport = transports[sessionId];
await transport.handleRequest(req, res);
} catch (error) {
console.error('Error handling SSE stream:', error);
if (!res.headersSent) {
res.status(500).send('Error establishing SSE stream');
}
}
});
// Handle DELETE requests - session termination
app.delete('/mcp', async (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined;
if (!sessionId || !transports[sessionId]) {
res.status(400).send('Invalid or missing session ID');
return;
}
console.log(`Received session termination request for session ${sessionId}`);
try {
const transport = transports[sessionId];
await transport.handleRequest(req, res);
} catch (error) {
console.error('Error handling termination:', error);
if (!res.headersSent) {
res.status(500).send('Error processing session termination');
}
}
});
// Start server
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`MCP Conformance Test Server running on http://localhost:${PORT}`);
console.log(` - MCP endpoint: http://localhost:${PORT}/mcp`);
});

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,27 @@
{
"name": "mcp-conformance-test-server",
"version": "1.0.0",
"description": "Reference implementation of MCP server for conformance testing",
"type": "module",
"main": "everything-server.ts",
"scripts": {
"start": "tsx everything-server.ts",
"dev": "tsx watch everything-server.ts"
},
"keywords": [
"mcp",
"model-context-protocol",
"conformance",
"testing"
],
"dependencies": {
"@modelcontextprotocol/sdk": "^1.20.1",
"@types/cors": "^2.8.19",
"cors": "^2.8.5",
"express": "^5.1.0"
},
"devDependencies": {
"tsx": "^4.7.0",
"typescript": "^5.5.4"
}
}

View file

@ -0,0 +1,8 @@
{
"extends": "../../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "."
},
"include": ["*.ts"]
}