[compliance tests] initial server scenarios (#4)

* spec for servers

* initial version of everything server and tests - tets are very simple

* some clean up

* lint etc.

* fix package lock agian

---------

Co-authored-by: Paul Carleton <paulc@anthropic.com>
This commit is contained in:
Inna Harper 2025-11-04 15:08:20 +00:00 • committed by GitHub
commit a692dd7bba
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 5678 additions and 78 deletions

View file

@ -6,11 +6,19 @@ A framework for testing MCP (Model Context Protocol) client implementations agai
## Quick Start
### Testing Clients
```bash
npm install
npm run start -- --command "tsx examples/clients/typescript/test1.ts" --scenario initialize
```
### Testing Servers
```bash
npm run test:server -- --server-url http://localhost:3000/mcp --all
```
## Overview
The conformance test framework validates MCP client implementations by:

933
SERVER_REQUIREMENTS.md Normal file
View file

@ -0,0 +1,933 @@
# MCP Server Conformance Requirements
This document specifies the requirements for building an MCP "Everything Server" for conformance testing. SDK maintainers should implement a server meeting these requirements to enable automated conformance testing.
## Purpose
The Everything Server is a reference implementation that:
- Demonstrates all MCP server features in a single, testable server
- Uses standardized naming conventions for tools, resources, and prompts
- Enables automated conformance testing across different SDK implementations
- Serves as a working example for SDK users
## Protocol Version
<!-- TODO: need to change to the new version before the official release, probaby worth havign separate tests for new and old version? -->
**Target MCP Specification**: `2025-06-18`
## Transport
**Required Transport**: Streamable HTTP (for initial conformance testing)
## Specification Compliance
This document specifies requirements based on the MCP specification. All features and behaviors described are mandated by the MCP specification itself and enable automated conformance testing.
---
## Server Information
### Server Identity
Your server MUST provide:
```typescript
{
name: "mcp-conformance-test-server", // Or your SDK-specific name
version: "1.0.0"
}
```
### Capabilities Declaration
Your server MUST declare these capabilities during initialization:
```typescript
{
capabilities: {
tools: {
listChanged: true
},
resources: {
subscribe: true,
listChanged: true
},
prompts: {
listChanged: true
},
logging: {},
completions: {}
}
}
```
**Note**: All capabilities listed are required for conformance testing. The MCP specification also supports optional `experimental` capabilities for non-standard features, but these are not required for conformance.
---
## 1. Lifecycle Requirements
### 1.1. Initialize Handshake
**Endpoint**: `initialize`
**Requirements**:
- Accept `initialize` request with client info and capabilities
- Return server info, protocol version, and capabilities
- Protocol version MUST be `"2025-06-18"`
**Example Response**:
```json
{
"protocolVersion": "2025-06-18",
"serverInfo": {
"name": "mcp-conformance-test-server",
"version": "1.0.0"
},
"capabilities": {
/* as above */
}
}
```
### 1.2. Initialized Notification
**Notification**: `initialized`
**Requirements**:
- Accept `initialized` notification from client after handshake
- No response required (it's a notification)
- Server should be ready for requests after receiving this
---
## 2. Tools Requirements
### 2.1. List Tools
**Endpoint**: `tools/list`
**Requirements**:
- Return array of all available tools
- Each tool MUST have:
- `name` (string)
- `description` (string)
- `inputSchema` (valid JSON Schema object)
### 2.2. Call Tool
**Endpoint**: `tools/call`
**Requirements**:
- Accept tool name and arguments
- Execute tool and return result
- Result MUST have `content` array
- Support `_meta.progressToken` for progress reporting (if provided)
### 2.3. Required Tools
Implement these tools with exact names:
#### `test_simple_text`
**Arguments**: None
**Returns**: Text content
```typescript
{
content: [{ type: 'text', text: 'This is a simple text response for testing.' }];
}
```
#### `test_image_content`
**Arguments**: None
**Returns**: Image content with base64 data
```typescript
{
content: [
{
type: 'image',
data: '<base64-encoded-png>',
mimeType: 'image/png'
}
];
}
```
**Implementation Note**: Use a minimal test image (e.g., 1x1 red pixel PNG)
#### `test_audio_content`
**Arguments**: None
**Returns**: Audio content with base64 data
```typescript
{
content: [
{
type: 'audio',
data: '<base64-encoded-wav>',
mimeType: 'audio/wav'
}
];
}
```
**Implementation Note**: Use a minimal test audio file
#### `test_embedded_resource`
**Arguments**: None
**Returns**: Embedded resource content
```typescript
{
content: [
{
type: 'resource',
resource: {
uri: 'test://embedded-resource',
mimeType: 'text/plain',
text: 'This is an embedded resource content.'
}
}
];
}
```
#### `test_multiple_content_types`
**Arguments**: None
**Returns**: Multiple content items (text + image + resource)
```typescript
{
content: [
{ type: 'text', text: 'Multiple content types test:' },
{ type: 'image', data: '<base64>', mimeType: 'image/png' },
{
type: 'resource',
resource: {
uri: 'test://mixed-content-resource',
mimeType: 'application/json',
text: '{"test":"data","value":123}'
}
}
];
}
```
#### `test_tool_with_logging`
**Arguments**: None
**Behavior**: During execution, send 3 log notifications at info level:
1. "Tool execution started"
2. "Tool processing data" (after ~50ms delay)
3. "Tool execution completed" (after another ~50ms delay)
**Returns**: Text content confirming execution
**Implementation Note**: The delays are important to test that clients can receive multiple log notifications during tool execution
#### `test_tool_with_progress`
**Arguments**: None
**Behavior**: If `_meta.progressToken` is provided in request:
- Send progress notification: `0/100`
- Wait ~50ms
- Send progress notification: `50/100`
- Wait ~50ms
- Send progress notification: `100/100`
If no progress token provided, just execute with delays.
**Returns**: Text content confirming execution
**Progress Notification Format**:
```typescript
{
method: "notifications/progress",
params: {
progressToken: <from request._meta.progressToken>,
progress: 50,
total: 100
}
}
```
#### `test_error_handling`
**Arguments**: None
**Behavior**: Always throw an error
**Returns**: JSON-RPC error
```json
{
"error": {
"code": -32000,
"message": "This tool intentionally returns an error for testing"
}
}
```
#### `test_sampling`
**Arguments**:
- `prompt` (string, required) - The prompt to send to the LLM
**Behavior**: Request LLM sampling from the client using `sampling/createMessage`
**Sampling Request**:
```typescript
{
"method": "sampling/createMessage",
"params": {
"messages": [
{
"role": "user",
"content": { "type": "text", "text": "<prompt from arguments>" }
}
],
"maxTokens": 100
}
}
```
**Returns**: Text content with the LLM's response
```typescript
{
"content": [
{
"type": "text",
"text": "LLM response: <response from sampling>"
}
]
}
```
**Implementation Note**: If the client doesn't support sampling (no `sampling` capability), return an error.
#### `test_elicitation`
**Arguments**:
- `message` (string, required) - The message to show the user
**Behavior**: Request user input from the client using `elicitation/create`
**Elicitation Request**:
```typescript
{
"method": "elicitation/create",
"params": {
"message": "<message from arguments>",
"requestedSchema": {
"type": "object",
"properties": {
"response": {
"type": "string",
"description": "User's response"
}
},
"required": ["response"]
}
}
}
```
**Returns**: Text content with the user's response
```typescript
{
"content": [
{
"type": "text",
"text": "User response: <action: accept/decline/cancel, content: {...}>"
}
]
}
```
**Implementation Note**: If the client doesn't support elicitation (no `elicitation` capability), return an error.
---
## 3. Resources Requirements
### 3.1. List Resources
**Endpoint**: `resources/list`
**Requirements**:
- Return array of all available **direct resources** (not templates)
- Support optional cursor-based pagination (see Section 6.1)
- Each resource MUST have:
- `uri` (string)
- `name` (string)
- `description` (string)
- `mimeType` (string, optional)
**Note**: Resource templates are listed via separate `resources/templates/list` endpoint (see 3.1a)
### 3.1a. List Resource Templates
**Endpoint**: `resources/templates/list`
**Requirements**:
- Return array of all available **resource templates**
- Support optional cursor-based pagination (see Section 6.1)
- Each template MUST have:
- `uriTemplate` (string) - RFC 6570 URI template
- `name` (string)
- `description` (string)
- `mimeType` (string, optional)
### 3.2. Read Resource
**Endpoint**: `resources/read`
**Requirements**:
- Accept resource URI
- Return resource contents
- Response MUST have `contents` array
- Each content item MUST have `uri`, `mimeType`, and either `text` or `blob`
### 3.3. Required Resources
Implement these resources with exact URIs:
#### `test://static-text`
**Type**: Static text resource
**Metadata** (for `resources/list`):
```typescript
{
uri: "test://static-text",
name: "Static Text Resource",
description: "A static text resource for testing",
mimeType: "text/plain"
}
```
**Content** (for `resources/read`):
```typescript
{
contents: [
{
uri: 'test://static-text',
mimeType: 'text/plain',
text: 'This is the content of the static text resource.'
}
];
}
```
#### `test://static-binary`
**Type**: Static binary resource (image)
**Metadata**:
```typescript
{
uri: "test://static-binary",
name: "Static Binary Resource",
description: "A static binary resource (image) for testing",
mimeType: "image/png"
}
```
**Content**:
```typescript
{
contents: [
{
uri: 'test://static-binary',
mimeType: 'image/png',
blob: '<base64-encoded-png>'
}
];
}
```
#### `test://template/{id}/data`
**Type**: Resource template with parameter
**Metadata** (for `resources/list`):
```typescript
{
uriTemplate: "test://template/{id}/data",
name: "Resource Template",
description: "A resource template with parameter substitution",
mimeType: "application/json"
}
```
**Behavior**: When client requests `test://template/123/data`, substitute `{id}` with `123`
**Content** (for `resources/read` with `uri: "test://template/123/data"`):
```typescript
{
contents: [
{
uri: 'test://template/123/data',
mimeType: 'application/json',
text: '{"id":"123","templateTest":true,"data":"Data for ID: 123"}'
}
];
}
```
**Implementation Note**: Use RFC 6570 URI template syntax
#### `test://watched-resource`
**Type**: Subscribable resource
**Metadata**:
```typescript
{
uri: "test://watched-resource",
name: "Watched Resource",
description: "A resource that can be subscribed to",
mimeType: "text/plain"
}
```
**Content**:
```typescript
{
contents: [
{
uri: 'test://watched-resource',
mimeType: 'text/plain',
text: 'Watched resource content'
}
];
}
```
### 3.4. Resource Subscription
**Endpoint**: `resources/subscribe`
**Requirements**:
- Accept subscription request with URI
- Track subscribed URIs
- Send `notifications/resources/updated` when subscribed resources change
- Return empty object `{}`
**Example Request**:
```json
{
"method": "resources/subscribe",
"params": {
"uri": "test://watched-resource"
}
}
```
**Endpoint**: `resources/unsubscribe`
**Requirements**:
- Accept unsubscribe request with URI
- Remove URI from subscriptions
- Stop sending update notifications for that URI
- Return empty object `{}`
---
## 4. Prompts Requirements
### 4.1. List Prompts
**Endpoint**: `prompts/list`
**Requirements**:
- Return array of all available prompts
- Each prompt MUST have:
- `name` (string)
- `description` (string)
- `arguments` (array, optional) - list of required arguments
### 4.2. Get Prompt
**Endpoint**: `prompts/get`
**Requirements**:
- Accept prompt name and arguments
- Return prompt messages
- Response MUST have `messages` array
- Each message MUST have `role` and `content`
### 4.3. Required Prompts
Implement these prompts with exact names:
#### `test_simple_prompt`
**Arguments**: None
**Returns**:
```typescript
{
messages: [
{
role: 'user',
content: {
type: 'text',
text: 'This is a simple prompt for testing.'
}
}
];
}
```
#### `test_prompt_with_arguments`
**Arguments**:
- `arg1` (string, required) - First test argument
- `arg2` (string, required) - Second test argument
**Metadata** (for `prompts/list`):
```typescript
{
name: "test_prompt_with_arguments",
description: "A prompt with required arguments",
arguments: [
{ name: "arg1", description: "First test argument", required: true },
{ name: "arg2", description: "Second test argument", required: true }
]
}
```
**Returns** (with args `{arg1: "hello", arg2: "world"}`):
```typescript
{
messages: [
{
role: 'user',
content: {
type: 'text',
text: "Prompt with arguments: arg1='hello', arg2='world'"
}
}
];
}
```
#### `test_prompt_with_embedded_resource`
**Arguments**:
- `resourceUri` (string, required) - URI of the resource to embed
**Returns**:
```typescript
{
messages: [
{
role: 'user',
content: {
type: 'resource',
resource: {
uri: '<resourceUri from arguments>',
mimeType: 'text/plain',
text: 'Embedded resource content for testing.'
}
}
},
{
role: 'user',
content: {
type: 'text',
text: 'Please process the embedded resource above.'
}
}
];
}
```
#### `test_prompt_with_image`
**Arguments**: None
**Returns**:
```typescript
{
messages: [
{
role: 'user',
content: {
type: 'image',
data: '<base64-encoded-png>',
mimeType: 'image/png'
}
},
{
role: 'user',
content: {
type: 'text',
text: 'Please analyze the image above.'
}
}
];
}
```
---
## 5. Logging Requirements
### 5.1. Set Log Level
**Endpoint**: `logging/setLevel`
**Requirements**:
- Accept log level setting
- Filter subsequent log notifications based on level
- Return empty object `{}`
**Log Levels** (in order of severity):
- `debug`
- `info`
- `notice`
- `warning`
- `error`
- `critical`
- `alert`
- `emergency`
### 5.2. Log Notifications
**Notification**: `notifications/message`
**Requirements**:
- Send log notifications during operations
- Each log MUST have:
- `level` (string) - one of the log levels above
- `data` (any) - log message or structured data
- `logger` (string, optional) - logger name
**Example**:
```typescript
{
method: "notifications/message",
params: {
level: "info",
logger: "conformance-test-server",
data: "Tool execution started"
}
}
```
**Implementation Note**: When no client is connected, log notifications may fail. Handle this gracefully by catching promise rejections:
```typescript
mcpServer.server
.notification({
method: 'notifications/message',
params: { level, logger, data }
})
.catch(() => {
// Ignore error if no client is connected
});
```
---
## 6. Pagination (Utility Feature)
### 6.1. Cursor-Based Pagination
**Purpose**: Allow servers to return large result sets in manageable chunks.
**Applies To**:
- `tools/list`
- `resources/list`
- `resources/templates/list`
- `prompts/list`
**Requirements**:
**Request Format**:
```typescript
{
"method": "tools/list", // or resources/list, prompts/list, etc.
"params": {
"cursor": "optional-opaque-token" // Optional: for subsequent pages
}
}
```
**Response Format**:
```typescript
{
"tools": [ /* array of tools */ ], // or resources, prompts, etc.
"nextCursor": "optional-opaque-token" // Optional: indicates more results available
}
```
**Implementation Requirements**:
- If `cursor` parameter is provided, return results starting after that position
- If more results are available, include `nextCursor` in response
- Cursor tokens MUST be opaque strings (format is server-defined)
- Page size is determined by server (clients MUST NOT assume fixed page size)
- For conformance testing, pagination is optional (all items can be returned in single response)
**Example**:
```typescript
// First request (no cursor)
Request: { "method": "tools/list" }
Response: {
"tools": [/* first 10 tools */],
"nextCursor": "page2token"
}
// Second request (with cursor)
Request: { "method": "tools/list", "params": { "cursor": "page2token" } }
Response: {
"tools": [/* next 10 tools */],
"nextCursor": "page3token"
}
// Last page (no nextCursor)
Request: { "method": "tools/list", "params": { "cursor": "page3token" } }
Response: {
"tools": [/* remaining tools */]
// No nextCursor = end of results
}
```
---
## 7. Completion Requirements
### 6.1. Complete Request
**Endpoint**: `completion/complete`
**Requirements**:
- Accept completion requests for prompt or resource template arguments
- Provide contextual suggestions based on partial input
- Return array of completion values ranked by relevance
**Request Format**:
```typescript
{
"method": "completion/complete",
"params": {
"ref": {
"type": "ref/prompt", // or "ref/resource"
"name": "test_prompt_with_arguments" // or uri for resources
},
"argument": {
"name": "arg1",
"value": "par" // partial value to complete
}
}
}
```
**Response Format**:
```typescript
{
"completion": {
"values": ["paris", "park", "party"], // Up to 100 items
"total": 150, // Optional: total available matches
"hasMore": true // Optional: indicates more results exist
}
}
```
**Implementation Note**: For conformance testing, completion support can be minimal or return empty arrays. The capability just needs to be declared and the endpoint must respond correctly.
---
## 8. Testing Your Server
### 8.1. Starting Your Server
```bash
# Example for TypeScript
cd examples/servers/typescript
npm install
npm start
```
Server should output:
```
MCP Conformance Test Server running on http://localhost:3000
- MCP endpoint: http://localhost:3000/mcp
```
### 8.2. Running Conformance Tests
```bash
# Single test
npm run test:server -- --server-url http://localhost:3000/mcp --scenario server-initialize
# All tests
npm run test:server -- --server-url http://localhost:3000/mcp --all
```

View file

@ -0,0 +1,173 @@
# MCP Conformance Test Server
A reference implementation of an MCP server that implements all features required for conformance testing.
## Features
This server implements:
### Tools
- `test_simple_text` - Returns simple text content
- `test_image_content` - Returns image content (base64 PNG)
- `test_audio_content` - Returns audio content (base64 WAV)
- `test_embedded_resource` - Returns embedded resource
- `test_multiple_content_types` - Returns mixed content types
- `test_tool_with_logging` - Emits log messages during execution
- `test_tool_with_progress` - Reports progress notifications
- `test_error_handling` - Returns error response
- `test_sampling` - Requests LLM completion from client
- `test_elicitation` - Requests user input from client
- `test_dynamic_tool` - Dynamically added/removed tool
### Resources
- `test://static-text` - Static text resource
- `test://static-binary` - Static binary resource (image)
- `test://template/{id}/data` - Resource template with parameter
- `test://watched-resource` - Subscribable resource with updates
### Prompts
- `test_simple_prompt` - Simple prompt without arguments
- `test_prompt_with_arguments(arg1, arg2)` - Parameterized prompt
- `test_prompt_with_embedded_resource(resourceUri)` - Prompt with embedded resource
- `test_prompt_with_image` - Prompt with image content
### Other Capabilities
- Logging at all levels (debug, info, notice, warning, error, critical, alert, emergency)
- Completion support for prompt and resource arguments
- List changed notifications for tools, resources, and prompts
- Resource subscription and update notifications
## Installation
```bash
npm install
```
## Running the Server
```bash
npm start
```
The server will start on `http://localhost:3000` (or the port specified in `PORT` environment variable).
## Endpoints
### MCP Endpoint
- `POST /mcp` - Main MCP protocol endpoint
### Health Check
- `GET /health` - Server health check
## Automatic Behaviors
The server automatically demonstrates dynamic capabilities:
- **Dynamic Tool** - `test_dynamic_tool` is automatically added 2 seconds after server starts
- **Dynamic Resource** - `test://dynamic-resource` is automatically added 2 seconds after server starts
- **Dynamic Prompt** - `test_dynamic_prompt` is automatically added 2 seconds after server starts
- **Resource Updates** - `test://watched-resource` automatically updates every 3 seconds with new content
These behaviors allow testing of MCP notifications without requiring manual triggers.
## Example Usage
### Starting the Server
```bash
npm start
```
### Testing with MCP Inspector
```bash
npx @modelcontextprotocol/inspector http://localhost:3000/mcp
```
### Testing with curl
#### Initialize
```bash
curl -X POST http://localhost:3000/mcp \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": {
"protocolVersion": "2025-06-18",
"capabilities": {},
"clientInfo": {
"name": "test-client",
"version": "1.0.0"
}
}
}'
```
#### List Tools
```bash
curl -X POST http://localhost:3000/mcp \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/list"
}'
```
#### Call Tool
```bash
curl -X POST http://localhost:3000/mcp \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 3,
"method": "tools/call",
"params": {
"name": "test_simple_text",
"arguments": {}
}
}'
```
## Conformance Testing
This server implements the MCP Server Conformance Requirements specified in `../../../SERVER_REQUIREMENTS.md`. All tools, resources, and prompts use standardized naming conventions for consistent testing across SDK implementations.
To run conformance tests against this server:
```bash
cd ../../../
npm run test:server -- --server-url http://localhost:3000/mcp --all
```
## Implementation Notes
- All tool, resource, and prompt names follow the standardized naming conventions (`test_*` for tools/prompts, `test://` for resources)
- Names are descriptive of the feature being tested (e.g., `test_image_content`, `test_tool_with_progress`)
- The server uses the TypeScript MCP SDK (`@modelcontextprotocol/sdk`) high-level API
- Uses `registerTool()`, `registerResource()`, and `registerPrompt()` methods
- Transport is Streamable HTTP (Express) for web-based testing compatibility
- Promise rejections from notifications are caught and handled gracefully
## For SDK Implementers
If you're implementing MCP in another language/SDK:
1. **Read the Requirements**: See `../../../SERVER_REQUIREMENTS.md` for complete specifications
2. **Use This as Reference**: This TypeScript implementation demonstrates all required features
3. **Follow Naming Conventions**: Use exact tool/resource/prompt names specified in requirements
4. **Implement Automatic Behaviors**: Dynamic registration after 2s, resource updates every 3s
5. **Handle Notifications Carefully**: Catch/ignore errors when no client is connected
**Goal**: All SDK example servers provide the same interface, enabling a single test suite to verify conformance across all implementations.

View file

@ -0,0 +1,748 @@
#!/usr/bin/env node
/**
* MCP Everything Server - Conformance Test Server
*
* Server implementing all MCP features for conformance testing based on Conformnace Server Specification.
* Should be using registerTool(), registerResource(), and registerPrompt().
* we use tool() instead of registerTool() as there is a bug with logging in registerTool().
*/
import { McpServer, ResourceTemplate } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { z } from 'zod';
import express from 'express';
import cors from 'cors';
import { randomUUID } from 'crypto';
// Server state
const resourceSubscriptions = new Set<string>();
const watchedResourceContent = 'Watched resource content';
// Session management
const transports: { [sessionId: string]: StreamableHTTPServerTransport } = {};
const servers: { [sessionId: string]: McpServer } = {};
// Sample base64 encoded 1x1 red PNG pixel for testing
const TEST_IMAGE_BASE64 = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8DwHwAFBQIAX8jx0gAAAABJRU5ErkJggg==';
// Sample base64 encoded minimal WAV file for testing
const TEST_AUDIO_BASE64 = 'UklGRiYAAABXQVZFZm10IBAAAAABAAEAQB8AAAB9AAACABAAZGF0YQIAAAA=';
// Function to create a new MCP server instance (one per session)
function createMcpServer() {
const mcpServer = new McpServer(
{
name: 'mcp-conformance-test-server',
version: '1.0.0'
},
{
capabilities: {
tools: {
listChanged: true
},
resources: {
subscribe: true,
listChanged: true
},
prompts: {
listChanged: true
},
logging: {},
completions: {}
}
}
);
// Helper to send log messages using the underlying server
function sendLog(
level: 'debug' | 'info' | 'notice' | 'warning' | 'error' | 'critical' | 'alert' | 'emergency',
message: string,
data?: any
) {
mcpServer.server
.notification({
method: 'notifications/message',
params: {
level,
logger: 'conformance-test-server',
data: data || message
}
})
.catch(() => {
// Ignore error if no client is connected
});
}
// ===== TOOLS =====
// Simple text tool
mcpServer.tool('test_simple_text', 'Tests simple text content response', {}, async () => {
return {
content: [{ type: 'text', text: 'This is a simple text response for testing.' }]
};
});
// Image content tool
mcpServer.registerTool(
'test_image_content',
{
description: 'Tests image content response'
},
async () => {
return {
content: [{ type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' }]
};
}
);
// Audio content tool
mcpServer.registerTool(
'test_audio_content',
{
description: 'Tests audio content response'
},
async () => {
return {
content: [{ type: 'audio', data: TEST_AUDIO_BASE64, mimeType: 'audio/wav' }]
};
}
);
// Embedded resource tool
mcpServer.registerTool(
'test_embedded_resource',
{
description: 'Tests embedded resource content response'
},
async () => {
return {
content: [
{
type: 'resource',
resource: {
uri: 'test://embedded-resource',
mimeType: 'text/plain',
text: 'This is an embedded resource content.'
}
}
]
};
}
);
// Multiple content types tool
mcpServer.registerTool(
'test_multiple_content_types',
{
description: 'Tests response with multiple content types (text, image, resource)'
},
async () => {
return {
content: [
{ type: 'text', text: 'Multiple content types test:' },
{ type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' },
{
type: 'resource',
resource: {
uri: 'test://mixed-content-resource',
mimeType: 'application/json',
text: JSON.stringify({ test: 'data', value: 123 })
}
}
]
};
}
);
// Tool with logging - registerTool with empty inputSchema to get (args, extra) signature
mcpServer.registerTool(
'test_tool_with_logging',
{
description: 'Tests tool that emits log messages during execution',
inputSchema: {} // Empty schema so callback gets (args, extra) instead of just (extra)
},
async (_args, { sendNotification }) => {
await sendNotification({
method: 'notifications/message',
params: {
level: 'info',
data: 'Tool execution started'
}
});
await new Promise(resolve => setTimeout(resolve, 50));
await sendNotification({
method: 'notifications/message',
params: {
level: 'info',
data: 'Tool processing data'
}
});
await new Promise(resolve => setTimeout(resolve, 50));
await sendNotification({
method: 'notifications/message',
params: {
level: 'info',
data: 'Tool execution completed'
}
});
return {
content: [{ type: 'text', text: 'Tool with logging executed successfully' }]
};
}
);
// Tool with progress - registerTool with empty inputSchema to get (args, extra) signature
mcpServer.registerTool(
'test_tool_with_progress',
{
description: 'Tests tool that reports progress notifications',
inputSchema: {} // Empty schema so callback gets (args, extra) instead of just (extra)
},
async (_args, { sendNotification, _meta }) => {
const progressToken = _meta?.progressToken ?? 0;
console.log('???? Progress token:', progressToken);
await sendNotification({
method: 'notifications/progress',
params: {
progressToken,
progress: 0,
total: 100,
message: `Completed step ${0} of ${100}`
}
});
await new Promise(resolve => setTimeout(resolve, 50));
await sendNotification({
method: 'notifications/progress',
params: {
progressToken,
progress: 50,
total: 100,
message: `Completed step ${50} of ${100}`
}
});
await new Promise(resolve => setTimeout(resolve, 50));
await sendNotification({
method: 'notifications/progress',
params: {
progressToken,
progress: 100,
total: 100,
message: `Completed step ${100} of ${100}`
}
});
return {
content: [{ type: 'text', text: String(progressToken) }]
};
}
);
// Error handling tool
mcpServer.registerTool(
'test_error_handling',
{
description: 'Tests error response handling'
},
async () => {
throw new Error('This tool intentionally returns an error for testing');
}
);
// Sampling tool - requests LLM completion from client
mcpServer.registerTool(
'test_sampling',
{
description: 'Tests server-initiated sampling (LLM completion request)',
inputSchema: {
prompt: z.string().describe('The prompt to send to the LLM')
}
},
async (args: { prompt: string }) => {
try {
// Request sampling from client
const result = await mcpServer.server.request(
{
method: 'sampling/createMessage',
params: {
messages: [
{
role: 'user',
content: {
type: 'text',
text: args.prompt
}
}
],
maxTokens: 100
}
},
z.object({ method: z.literal('sampling/createMessage') }).passthrough() as any
);
const samplingResult = result as any;
const modelResponse = samplingResult.content?.text || samplingResult.message?.content?.text || 'No response';
return {
content: [
{
type: 'text',
text: `LLM response: ${modelResponse}`
}
]
};
} catch (error: any) {
return {
content: [
{
type: 'text',
text: `Sampling not supported or error: ${error.message}`
}
]
};
}
}
);
// Elicitation tool - requests user input from client
mcpServer.registerTool(
'test_elicitation',
{
description: 'Tests server-initiated elicitation (user input request)',
inputSchema: {
message: z.string().describe('The message to show the user')
}
},
async (args: { message: string }) => {
try {
// Request user input from client
const result = await mcpServer.server.request(
{
method: 'elicitation/create',
params: {
message: args.message,
requestedSchema: {
type: 'object',
properties: {
response: {
type: 'string',
description: "User's response"
}
},
required: ['response']
}
}
},
z.object({ method: z.literal('elicitation/create') }).passthrough() as any
);
const elicitResult = result as any;
return {
content: [
{
type: 'text',
text: `User response: action=${elicitResult.action}, content=${JSON.stringify(elicitResult.content || {})}`
}
]
};
} catch (error: any) {
return {
content: [
{
type: 'text',
text: `Elicitation not supported or error: ${error.message}`
}
]
};
}
}
);
// Dynamic tool (registered later via timer)
// ===== RESOURCES =====
// Static text resource
mcpServer.registerResource(
'static-text',
'test://static-text',
{
title: 'Static Text Resource',
description: 'A static text resource for testing',
mimeType: 'text/plain'
},
async () => {
return {
contents: [
{
uri: 'test://static-text',
mimeType: 'text/plain',
text: 'This is the content of the static text resource.'
}
]
};
}
);
// Static binary resource
mcpServer.registerResource(
'static-binary',
'test://static-binary',
{
title: 'Static Binary Resource',
description: 'A static binary resource (image) for testing',
mimeType: 'image/png'
},
async () => {
return {
contents: [
{
uri: 'test://static-binary',
mimeType: 'image/png',
blob: TEST_IMAGE_BASE64
}
]
};
}
);
// Resource template
mcpServer.registerResource(
'template',
new ResourceTemplate('test://template/{id}/data', {
list: undefined
}),
{
title: 'Resource Template',
description: 'A resource template with parameter substitution',
mimeType: 'application/json'
},
async (uri, variables) => {
const id = variables.id;
return {
contents: [
{
uri: uri.toString(),
mimeType: 'application/json',
text: JSON.stringify({
id,
templateTest: true,
data: `Data for ID: ${id}`
})
}
]
};
}
);
// Watched resource
mcpServer.registerResource(
'watched-resource',
'test://watched-resource',
{
title: 'Watched Resource',
description: 'A resource that auto-updates every 3 seconds',
mimeType: 'text/plain'
},
async () => {
return {
contents: [
{
uri: 'test://watched-resource',
mimeType: 'text/plain',
text: watchedResourceContent
}
]
};
}
);
// Subscribe/Unsubscribe handlers
mcpServer.server.setRequestHandler(z.object({ method: z.literal('resources/subscribe') }).passthrough(), async (request: any) => {
const uri = request.params.uri;
resourceSubscriptions.add(uri);
sendLog('info', `Subscribed to resource: ${uri}`);
return {};
});
mcpServer.server.setRequestHandler(z.object({ method: z.literal('resources/unsubscribe') }).passthrough(), async (request: any) => {
const uri = request.params.uri;
resourceSubscriptions.delete(uri);
sendLog('info', `Unsubscribed from resource: ${uri}`);
return {};
});
// ===== PROMPTS =====
// Simple prompt
mcpServer.registerPrompt(
'test_simple_prompt',
{
title: 'Simple Test Prompt',
description: 'A simple prompt without arguments'
},
async () => {
return {
messages: [
{
role: 'user',
content: { type: 'text', text: 'This is a simple prompt for testing.' }
}
]
};
}
);
// Prompt with arguments
mcpServer.registerPrompt(
'test_prompt_with_arguments',
{
title: 'Prompt With Arguments',
description: 'A prompt with required arguments',
argsSchema: {
arg1: z.string().describe('First test argument'),
arg2: z.string().describe('Second test argument')
}
},
async args => {
const { arg1, arg2 } = args;
return {
messages: [
{
role: 'user',
content: {
type: 'text',
text: `Prompt with arguments: arg1='${arg1}', arg2='${arg2}'`
}
}
]
};
}
);
// Prompt with embedded resource
mcpServer.registerPrompt(
'test_prompt_with_embedded_resource',
{
title: 'Prompt With Embedded Resource',
description: 'A prompt that includes an embedded resource',
argsSchema: {
resourceUri: z.string().describe('URI of the resource to embed')
}
},
async args => {
const uri = args.resourceUri;
return {
messages: [
{
role: 'user',
content: {
type: 'resource',
resource: {
uri,
mimeType: 'text/plain',
text: 'Embedded resource content for testing.'
}
}
},
{
role: 'user',
content: { type: 'text', text: 'Please process the embedded resource above.' }
}
]
};
}
);
// Prompt with image
mcpServer.registerPrompt(
'test_prompt_with_image',
{
title: 'Prompt With Image',
description: 'A prompt that includes image content'
},
async () => {
return {
messages: [
{
role: 'user',
content: { type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' }
},
{
role: 'user',
content: { type: 'text', text: 'Please analyze the image above.' }
}
]
};
}
);
// ===== LOGGING =====
mcpServer.server.setRequestHandler(z.object({ method: z.literal('logging/setLevel') }).passthrough(), async (request: any) => {
const level = request.params.level;
sendLog('info', `Log level set to: ${level}`);
return {};
});
// ===== COMPLETION =====
mcpServer.server.setRequestHandler(z.object({ method: z.literal('completion/complete') }).passthrough(), async (_request: any) => {
// Basic completion support - returns empty array for conformance
// Real implementations would provide contextual suggestions
return {
completion: {
values: [],
total: 0,
hasMore: false
}
};
});
return mcpServer;
}
// Helper to check if request is an initialize request
function isInitializeRequest(body: any): boolean {
return body?.method === 'initialize';
}
// ===== EXPRESS APP =====
const app = express();
app.use(express.json());
// Configure CORS to expose Mcp-Session-Id header for browser-based clients
app.use(
cors({
origin: '*', // Allow all origins
exposedHeaders: ['Mcp-Session-Id'],
allowedHeaders: ['Content-Type', 'mcp-session-id', 'last-event-id']
})
);
// Handle POST requests - stateful mode
app.post('/mcp', async (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined;
try {
let transport: StreamableHTTPServerTransport;
if (sessionId && transports[sessionId]) {
// Reuse existing transport for established sessions
transport = transports[sessionId];
} else if (!sessionId && isInitializeRequest(req.body)) {
// Create new transport for initialization requests
const mcpServer = createMcpServer();
transport = new StreamableHTTPServerTransport({
sessionIdGenerator: () => randomUUID(),
onsessioninitialized: newSessionId => {
transports[newSessionId] = transport;
servers[newSessionId] = mcpServer;
console.log(`Session initialized with ID: ${newSessionId}`);
}
});
transport.onclose = () => {
const sid = transport.sessionId;
if (sid && transports[sid]) {
delete transports[sid];
if (servers[sid]) {
servers[sid].close();
delete servers[sid];
}
console.log(`Session ${sid} closed`);
}
};
await mcpServer.connect(transport);
await transport.handleRequest(req, res, req.body);
return;
} else {
res.status(400).json({
jsonrpc: '2.0',
error: {
code: -32000,
message: 'Invalid or missing session ID'
},
id: null
});
return;
}
await transport.handleRequest(req, res, req.body);
} catch (error) {
console.error('Error handling MCP request:', error);
if (!res.headersSent) {
res.status(500).json({
jsonrpc: '2.0',
error: {
code: -32603,
message: 'Internal server error'
},
id: null
});
}
}
});
// Handle GET requests - SSE streams for sessions
app.get('/mcp', async (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined;
if (!sessionId || !transports[sessionId]) {
res.status(400).send('Invalid or missing session ID');
return;
}
const lastEventId = req.headers['last-event-id'] as string | undefined;
if (lastEventId) {
console.log(`Client reconnecting with Last-Event-ID: ${lastEventId}`);
} else {
console.log(`Establishing SSE stream for session ${sessionId}`);
}
try {
const transport = transports[sessionId];
await transport.handleRequest(req, res);
} catch (error) {
console.error('Error handling SSE stream:', error);
if (!res.headersSent) {
res.status(500).send('Error establishing SSE stream');
}
}
});
// Handle DELETE requests - session termination
app.delete('/mcp', async (req, res) => {
const sessionId = req.headers['mcp-session-id'] as string | undefined;
if (!sessionId || !transports[sessionId]) {
res.status(400).send('Invalid or missing session ID');
return;
}
console.log(`Received session termination request for session ${sessionId}`);
try {
const transport = transports[sessionId];
await transport.handleRequest(req, res);
} catch (error) {
console.error('Error handling termination:', error);
if (!res.headersSent) {
res.status(500).send('Error processing session termination');
}
}
});
// Start server
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`MCP Conformance Test Server running on http://localhost:${PORT}`);
console.log(` - MCP endpoint: http://localhost:${PORT}/mcp`);
});

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,27 @@
{
"name": "mcp-conformance-test-server",
"version": "1.0.0",
"description": "Reference implementation of MCP server for conformance testing",
"type": "module",
"main": "everything-server.ts",
"scripts": {
"start": "tsx everything-server.ts",
"dev": "tsx watch everything-server.ts"
},
"keywords": [
"mcp",
"model-context-protocol",
"conformance",
"testing"
],
"dependencies": {
"@modelcontextprotocol/sdk": "^1.20.1",
"@types/cors": "^2.8.19",
"cors": "^2.8.5",
"express": "^5.1.0"
},
"devDependencies": {
"tsx": "^4.7.0",
"typescript": "^5.5.4"
}
}

View file

@ -0,0 +1,8 @@
{
"extends": "../../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "."
},
"include": ["*.ts"]
}

115
package-lock.json generated
View file

@ -16,6 +16,7 @@
"devDependencies": {
"@eslint/js": "^9.8.0",
"@jest/globals": "^29.7.0",
"@types/express": "^5.0.3",
"@types/jest": "^29.5.8",
"eslint": "^9.8.0",
"eslint-config-prettier": "^10.1.8",
@ -58,7 +59,6 @@
"integrity": "sha512-e7jT4DxYvIDLk1ZHmU/m/mB19rex9sv0c2ftBtjSBv+kVM/902eh0fINUzD7UwLLNR+jU585GxUJ8/EBfAM5fw==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/code-frame": "^7.27.1",
"@babel/generator": "^7.28.5",
@ -1787,6 +1787,27 @@
"@babel/types": "^7.28.2"
}
},
"node_modules/@types/body-parser": {
"version": "1.19.6",
"resolved": "https://registry.npmjs.org/@types/body-parser/-/body-parser-1.19.6.tgz",
"integrity": "sha512-HLFeCYgz89uk22N5Qg3dvGvsv46B8GLvKKo1zKG4NybA8U2DiEO3w9lqGg29t/tfLRJpJ6iQxnVw4OnB7MoM9g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/connect": "*",
"@types/node": "*"
}
},
"node_modules/@types/connect": {
"version": "3.4.38",
"resolved": "https://registry.npmjs.org/@types/connect/-/connect-3.4.38.tgz",
"integrity": "sha512-K6uROf1LD88uDQqJCktA4yzL1YYAK6NgfsI0v/mTgyPKWsX1CnJ0XPSDhViejru1GcRkLWb8RlzFYJRqGUbaug==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/estree": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz",
@ -1794,6 +1815,31 @@
"dev": true,
"license": "MIT"
},
"node_modules/@types/express": {
"version": "5.0.5",
"resolved": "https://registry.npmjs.org/@types/express/-/express-5.0.5.tgz",
"integrity": "sha512-LuIQOcb6UmnF7C1PCFmEU1u2hmiHL43fgFQX67sN3H4Z+0Yk0Neo++mFsBjhOAuLzvlQeqAAkeDOZrJs9rzumQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/body-parser": "*",
"@types/express-serve-static-core": "^5.0.0",
"@types/serve-static": "^1"
}
},
"node_modules/@types/express-serve-static-core": {
"version": "5.1.0",
"resolved": "https://registry.npmjs.org/@types/express-serve-static-core/-/express-serve-static-core-5.1.0.tgz",
"integrity": "sha512-jnHMsrd0Mwa9Cf4IdOzbz543y4XJepXrbia2T4b6+spXC2We3t1y6K44D3mR8XMFSXMCf3/l7rCgddfx7UNVBA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*",
"@types/qs": "*",
"@types/range-parser": "*",
"@types/send": "*"
}
},
"node_modules/@types/graceful-fs": {
"version": "4.1.9",
"resolved": "https://registry.npmjs.org/@types/graceful-fs/-/graceful-fs-4.1.9.tgz",
@ -1804,6 +1850,13 @@
"@types/node": "*"
}
},
"node_modules/@types/http-errors": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@types/http-errors/-/http-errors-2.0.5.tgz",
"integrity": "sha512-r8Tayk8HJnX0FztbZN7oVqGccWgw98T/0neJphO91KkmOzug1KkofZURD4UaD5uH8AqcFLfdPErnBod0u71/qg==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/istanbul-lib-coverage": {
"version": "2.0.6",
"resolved": "https://registry.npmjs.org/@types/istanbul-lib-coverage/-/istanbul-lib-coverage-2.0.6.tgz",
@ -1849,6 +1902,13 @@
"dev": true,
"license": "MIT"
},
"node_modules/@types/mime": {
"version": "1.3.5",
"resolved": "https://registry.npmjs.org/@types/mime/-/mime-1.3.5.tgz",
"integrity": "sha512-/pyBZWSLD2n0dcHE3hq8s8ZvcETHtEuF+3E7XVt0Ig2nvsVQXdghHVcEkIWjy9A0wKfTn97a/PSDYohKIlnP/w==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/node": {
"version": "24.10.0",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.10.0.tgz",
@ -1859,6 +1919,53 @@
"undici-types": "~7.16.0"
}
},
"node_modules/@types/qs": {
"version": "6.14.0",
"resolved": "https://registry.npmjs.org/@types/qs/-/qs-6.14.0.tgz",
"integrity": "sha512-eOunJqu0K1923aExK6y8p6fsihYEn/BYuQ4g0CxAAgFc4b/ZLN4CrsRZ55srTdqoiLzU2B2evC+apEIxprEzkQ==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/range-parser": {
"version": "1.2.7",
"resolved": "https://registry.npmjs.org/@types/range-parser/-/range-parser-1.2.7.tgz",
"integrity": "sha512-hKormJbkJqzQGhziax5PItDUTMAM9uE2XXQmM37dyd4hVM+5aVl7oVxMVUiVQn2oCQFN/LKCZdvSM0pFRqbSmQ==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/send": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/@types/send/-/send-1.2.1.tgz",
"integrity": "sha512-arsCikDvlU99zl1g69TcAB3mzZPpxgw0UQnaHeC1Nwb015xp8bknZv5rIfri9xTOcMuaVgvabfIRA7PSZVuZIQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/serve-static": {
"version": "1.15.10",
"resolved": "https://registry.npmjs.org/@types/serve-static/-/serve-static-1.15.10.tgz",
"integrity": "sha512-tRs1dB+g8Itk72rlSI2ZrW6vZg0YrLI81iQSTkMmOqnqCaNr/8Ek4VwWcN5vZgCYWbg/JJSGBlUaYGAOP73qBw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/http-errors": "*",
"@types/node": "*",
"@types/send": "<1"
}
},
"node_modules/@types/serve-static/node_modules/@types/send": {
"version": "0.17.6",
"resolved": "https://registry.npmjs.org/@types/send/-/send-0.17.6.tgz",
"integrity": "sha512-Uqt8rPBE8SY0RK8JB1EzVOIZ32uqy8HwdxCnoCOsYrvnswqmFZ/k+9Ikidlk/ImhsdvBsloHbAlewb2IEBV/Og==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/mime": "^1",
"@types/node": "*"
}
},
"node_modules/@types/stack-utils": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/@types/stack-utils/-/stack-utils-2.0.3.tgz",
@ -2412,9 +2519,9 @@
"license": "MIT"
},
"node_modules/baseline-browser-mapping": {
"version": "2.8.23",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.8.23.tgz",
"integrity": "sha512-616V5YX4bepJFzNyOfce5Fa8fDJMfoxzOIzDCZwaGL8MKVpFrXqfNUoIpRn9YMI5pXf/VKgzjB4htFMsFKKdiQ==",
"version": "2.8.24",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.8.24.tgz",
"integrity": "sha512-uUhTRDPXamakPyghwrUcjaGvvBqGrWvBHReoiULMIpOJVM9IYzQh83Xk2Onx5HlGI2o10NNCzcs9TG/S3TkwrQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {

View file

@ -4,6 +4,7 @@
"type": "module",
"scripts": {
"start": "tsx src/runner.ts",
"test:server": "tsx src/server-runner.ts",
"test": "jest",
"lint": "eslint src/ && prettier --check .",
"lint:fix": "eslint src/ --fix && prettier --write .",
@ -12,6 +13,7 @@
"devDependencies": {
"@eslint/js": "^9.8.0",
"@jest/globals": "^29.7.0",
"@types/express": "^5.0.3",
"@types/jest": "^29.5.8",
"eslint": "^9.8.0",
"eslint-config-prettier": "^10.1.8",

View file

@ -1,6 +1,4 @@
// Namespaced exports for client and server checks
// Namespaced exports for client checks
import * as client from './client.js';
import * as server from './server.js';
export const clientChecks = client;
export const serverChecks = server;

View file

@ -1,37 +0,0 @@
import { ConformanceCheck, CheckStatus } from '../types.js';
export function createServerInitializationCheck(initializeResponse: any, expectedSpecVersion: string = '2025-06-18'): ConformanceCheck {
const result = initializeResponse?.result;
const protocolVersion = result?.protocolVersion;
const serverInfo = result?.serverInfo;
const capabilities = result?.capabilities;
const errors: string[] = [];
if (!initializeResponse?.jsonrpc) errors.push('Missing jsonrpc field');
if (!initializeResponse?.id) errors.push('Missing id field');
if (!result) errors.push('Missing result field');
if (!protocolVersion) errors.push('Missing protocolVersion in result');
if (protocolVersion !== expectedSpecVersion)
errors.push(`Protocol version mismatch: expected ${expectedSpecVersion}, got ${protocolVersion}`);
if (!serverInfo) errors.push('Missing serverInfo in result');
if (!serverInfo?.name) errors.push('Missing server name in serverInfo');
if (!serverInfo?.version) errors.push('Missing server version in serverInfo');
if (capabilities === undefined) errors.push('Missing capabilities in result');
const status: CheckStatus = errors.length === 0 ? 'SUCCESS' : 'FAILURE';
return {
id: 'mcp-server-initialization',
name: 'MCPServerInitialization',
description: 'Validates that MCP server properly responds to initialize request',
status,
timestamp: new Date().toISOString(),
specReferences: [{ id: 'MCP-Lifecycle', url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle' }],
details: {
expectedSpecVersion,
response: initializeResponse
},
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
logs: errors.length > 0 ? errors : undefined
};
}

View file

@ -1,4 +1,3 @@
import { runConformanceTest } from './runner/index.js';
export { runConformanceTest };
export type { ScenarioConfig } from './runner/index.js';

View file

@ -2,7 +2,8 @@ import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprotocol/sdk/types.js';
import type { Scenario, ConformanceCheck } from '../../types.js';
import express from 'express';
// @ts-expect-error - express has incomplete type definitions
import express, { Request, Response, NextFunction } from 'express';
import { ScenarioUrls } from '../../types.js';
function createServer(checks: ConformanceCheck[]): express.Application {
@ -83,7 +84,7 @@ function createServer(checks: ConformanceCheck[]): express.Application {
const app = express();
app.use(express.json());
app.use((req, res, next) => {
app.use((req: Request, res: Response, next: NextFunction) => {
// Log incoming requests for debugging
// console.log(`Incoming request: ${req.method} ${req.url}`);
checks.push({
@ -105,13 +106,13 @@ function createServer(checks: ConformanceCheck[]): express.Application {
status: 'INFO',
timestamp: new Date().toISOString(),
details: {
// TODO: this isn't working
body: JSON.stringify(res.body)
// TODO: Response body not available in express middleware
statusCode: res.statusCode
}
});
});
app.post('/mcp', async (req, res) => {
app.post('/mcp', async (req: Request, res: Response) => {
const transport = new StreamableHTTPServerTransport({
sessionIdGenerator: undefined
});
@ -125,6 +126,7 @@ function createServer(checks: ConformanceCheck[]): express.Application {
export class ToolsCallScenario implements Scenario {
name = 'tools_call';
description = 'Tests calling tools with various parameter types';
private app: express.Application | null = null;
private httpServer: any = null;
private checks: ConformanceCheck[] = [];

View file

@ -1,14 +1,84 @@
import { Scenario, ClientScenario } from '../types.js';
import { InitializeScenario } from './client/initialize.js';
import { ToolsCallScenario } from './client/tools_call.js';
import { ServerInitializeClientScenario } from './server/server_initialize.js';
// Import all new server test scenarios
import { ServerInitializeScenario } from './server/lifecycle.js';
import { LoggingSetLevelScenario, CompletionCompleteScenario } from './server/utils.js';
import {
ToolsListScenario,
ToolsCallSimpleTextScenario,
ToolsCallImageScenario,
ToolsCallMultipleContentTypesScenario,
ToolsCallWithLoggingScenario,
ToolsCallErrorScenario,
ToolsCallWithProgressScenario,
ToolsCallSamplingScenario,
ToolsCallElicitationScenario,
ToolsCallAudioScenario,
ToolsCallEmbeddedResourceScenario
} from './server/tools.js';
import {
ResourcesListScenario,
ResourcesReadTextScenario,
ResourcesReadBinaryScenario,
ResourcesTemplateReadScenario,
ResourcesSubscribeScenario,
ResourcesUnsubscribeScenario
} from './server/resources.js';
import {
PromptsListScenario,
PromptsGetSimpleScenario,
PromptsGetWithArgsScenario,
PromptsGetEmbeddedResourceScenario,
PromptsGetWithImageScenario
} from './server/prompts.js';
export const scenarios = new Map<string, Scenario>([
['initialize', new InitializeScenario()],
['tools-call', new ToolsCallScenario()]
]);
export const clientScenarios = new Map<string, ClientScenario>([['initialize', new ServerInitializeClientScenario()]]);
export const clientScenarios = new Map<string, ClientScenario>([
// Lifecycle scenarios
['server-initialize', new ServerInitializeScenario()],
// Utilities scenarios
['logging-set-level', new LoggingSetLevelScenario()],
['completion-complete', new CompletionCompleteScenario()],
// Tools scenarios
['tools-list', new ToolsListScenario()],
['tools-call-simple-text', new ToolsCallSimpleTextScenario()],
['tools-call-image', new ToolsCallImageScenario()],
['tools-call-audio', new ToolsCallAudioScenario()],
['tools-call-embedded-resource', new ToolsCallEmbeddedResourceScenario()],
['tools-call-mixed-content', new ToolsCallMultipleContentTypesScenario()],
['tools-call-with-logging', new ToolsCallWithLoggingScenario()],
['tools-call-error', new ToolsCallErrorScenario()],
['tools-call-with-progress', new ToolsCallWithProgressScenario()],
['tools-call-sampling', new ToolsCallSamplingScenario()],
['tools-call-elicitation', new ToolsCallElicitationScenario()],
// Resources scenarios
['resources-list', new ResourcesListScenario()],
['resources-read-text', new ResourcesReadTextScenario()],
['resources-read-binary', new ResourcesReadBinaryScenario()],
['resources-templates-read', new ResourcesTemplateReadScenario()],
['resources-subscribe', new ResourcesSubscribeScenario()],
['resources-unsubscribe', new ResourcesUnsubscribeScenario()],
// Prompts scenarios
['prompts-list', new PromptsListScenario()],
['prompts-get-simple', new PromptsGetSimpleScenario()],
['prompts-get-with-args', new PromptsGetWithArgsScenario()],
['prompts-get-embedded-resource', new PromptsGetEmbeddedResourceScenario()],
['prompts-get-with-image', new PromptsGetWithImageScenario()]
]);
export function registerScenario(name: string, scenario: Scenario): void {
scenarios.set(name, scenario);

View file

@ -0,0 +1,83 @@
/**
* Helper utilities for creating MCP clients to test servers
*/
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
import { LoggingMessageNotificationSchema, ProgressNotificationSchema } from '@modelcontextprotocol/sdk/types.js';
export interface MCPClientConnection {
client: Client;
close: () => Promise<void>;
}
/**
* Create and connect an MCP client to a server
*/
export async function connectToServer(serverUrl: string): Promise<MCPClientConnection> {
const client = new Client(
{
name: 'conformance-test-client',
version: '1.0.0'
},
{
capabilities: {
// Client capabilities
sampling: {},
elicitation: {}
}
}
);
const transport = new StreamableHTTPClientTransport(new URL(serverUrl));
await client.connect(transport);
return {
client,
close: async () => {
await client.close();
}
};
}
/**
* Helper to collect notifications (logging and progress)
*/
export class NotificationCollector {
private loggingNotifications: any[] = [];
private progressNotifications: any[] = [];
constructor(client: Client) {
// Set up notification handler for logging messages
client.setNotificationHandler(LoggingMessageNotificationSchema, notification => {
this.loggingNotifications.push(notification);
});
// Set up notification handler for progress notifications
client.setNotificationHandler(ProgressNotificationSchema, notification => {
this.progressNotifications.push(notification);
});
}
/**
* Get all collected logging notifications
*/
getLoggingNotifications(): any[] {
return this.loggingNotifications;
}
/**
* Get all collected progress notifications
*/
getProgressNotifications(): any[] {
return this.progressNotifications;
}
/**
* Get all notifications (for backward compatibility)
*/
getNotifications(): any[] {
return this.loggingNotifications;
}
}

View file

@ -0,0 +1,58 @@
/**
* Lifecycle test scenarios for MCP servers
*/
import { ClientScenario, ConformanceCheck } from '../../types.js';
import { connectToServer } from './client-helper.js';
export class ServerInitializeScenario implements ClientScenario {
name = 'server-initialize';
description = 'Test basic server initialization handshake';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
// The connection process already does initialization
// Check that we have a connected client
checks.push({
id: 'server-initialize',
name: 'ServerInitialize',
description: 'Server responds to initialize request with valid structure',
status: 'SUCCESS',
timestamp: new Date().toISOString(),
specReferences: [
{
id: 'MCP-Initialize',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle#initialization'
}
],
details: {
serverUrl,
connected: true
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'server-initialize',
name: 'ServerInitialize',
description: 'Server responds to initialize request with valid structure',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed to initialize: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Initialize',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle#initialization'
}
]
});
}
return checks;
}
}

View file

@ -0,0 +1,350 @@
/**
* Prompts test scenarios for MCP servers
*/
import { ClientScenario, ConformanceCheck } from '../../types.js';
import { connectToServer } from './client-helper.js';
export class PromptsListScenario implements ClientScenario {
name = 'prompts-list';
description = 'Test listing available prompts';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.listPrompts();
// Validate response structure
const errors: string[] = [];
if (!result.prompts) {
errors.push('Missing prompts array');
} else {
if (!Array.isArray(result.prompts)) {
errors.push('prompts is not an array');
}
result.prompts.forEach((prompt, index) => {
if (!prompt.name) errors.push(`Prompt ${index}: missing name`);
if (!prompt.description) errors.push(`Prompt ${index}: missing description`);
});
}
checks.push({
id: 'prompts-list',
name: 'PromptsList',
description: 'Server lists available prompts with valid structure',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Prompts-List',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#listing-prompts'
}
],
details: {
promptCount: result.prompts?.length || 0,
prompts: result.prompts?.map(p => p.name)
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'prompts-list',
name: 'PromptsList',
description: 'Server lists available prompts with valid structure',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Prompts-List',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#listing-prompts'
}
]
});
}
return checks;
}
}
export class PromptsGetSimpleScenario implements ClientScenario {
name = 'prompts-get-simple';
description = 'Test getting a simple prompt without arguments';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.getPrompt({
name: 'test_simple_prompt'
});
// Validate response
const errors: string[] = [];
if (!result.messages) errors.push('Missing messages array');
if (!Array.isArray(result.messages)) errors.push('messages is not an array');
if (result.messages.length === 0) errors.push('messages array is empty');
result.messages.forEach((message: any, index: number) => {
if (!message.role) errors.push(`Message ${index}: missing role`);
if (!message.content) errors.push(`Message ${index}: missing content`);
});
checks.push({
id: 'prompts-get-simple',
name: 'PromptsGetSimple',
description: 'Get simple prompt successfully',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Prompts-Get',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#getting-prompts'
}
],
details: {
messageCount: result.messages?.length || 0
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'prompts-get-simple',
name: 'PromptsGetSimple',
description: 'Get simple prompt successfully',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Prompts-Get',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#getting-prompts'
}
]
});
}
return checks;
}
}
export class PromptsGetWithArgsScenario implements ClientScenario {
name = 'prompts-get-with-args';
description = 'Test parameterized prompt';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.getPrompt({
name: 'test_prompt_with_arguments',
arguments: {
arg1: 'testValue1',
arg2: 'testValue2'
}
});
// Validate response
const errors: string[] = [];
if (!result.messages) errors.push('Missing messages array');
if (result.messages.length === 0) errors.push('messages array is empty');
// Check if arguments were substituted
const messageText = JSON.stringify(result.messages);
if (!messageText.includes('testValue1')) {
errors.push('arg1 not substituted in prompt');
}
if (!messageText.includes('testValue2')) {
errors.push('arg2 not substituted in prompt');
}
checks.push({
id: 'prompts-get-with-args',
name: 'PromptsGetWithArgs',
description: 'Get parameterized prompt with argument substitution',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Prompts-Get',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#getting-prompts'
}
],
details: {
messageCount: result.messages?.length || 0,
messages: result.messages
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'prompts-get-with-args',
name: 'PromptsGetWithArgs',
description: 'Get parameterized prompt with argument substitution',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Prompts-Get',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#getting-prompts'
}
]
});
}
return checks;
}
}
export class PromptsGetEmbeddedResourceScenario implements ClientScenario {
name = 'prompts-get-embedded-resource';
description = 'Test prompt with embedded resource content';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.getPrompt({
name: 'test_prompt_with_embedded_resource',
arguments: {
resourceUri: 'test://example-resource'
}
});
// Validate response
const errors: string[] = [];
if (!result.messages) errors.push('Missing messages array');
// Look for resource content
const hasResourceContent = result.messages.some(
(msg: any) => msg.content?.type === 'resource' || msg.content?.resource !== undefined
);
if (!hasResourceContent) {
errors.push('No embedded resource found in prompt');
}
checks.push({
id: 'prompts-get-embedded-resource',
name: 'PromptsGetEmbeddedResource',
description: 'Get prompt with embedded resource',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Prompts-Embedded-Resources',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#embedded-resources'
}
],
details: {
messageCount: result.messages?.length || 0,
messages: result.messages
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'prompts-get-embedded-resource',
name: 'PromptsGetEmbeddedResource',
description: 'Get prompt with embedded resource',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Prompts-Embedded-Resources',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#embedded-resources'
}
]
});
}
return checks;
}
}
export class PromptsGetWithImageScenario implements ClientScenario {
name = 'prompts-get-with-image';
description = 'Test prompt with image content';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.getPrompt({
name: 'test_prompt_with_image'
});
// Validate response
const errors: string[] = [];
if (!result.messages) errors.push('Missing messages array');
// Look for image content
const hasImageContent = result.messages.some(
(msg: any) => msg.content?.type === 'image' && msg.content?.data && msg.content?.mimeType
);
if (!hasImageContent) {
errors.push('No image content found in prompt');
}
checks.push({
id: 'prompts-get-with-image',
name: 'PromptsGetWithImage',
description: 'Get prompt with image content',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Prompts-Image',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#image-content'
}
],
details: {
messageCount: result.messages?.length || 0
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'prompts-get-with-image',
name: 'PromptsGetWithImage',
description: 'Get prompt with image content',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Prompts-Image',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#image-content'
}
]
});
}
return checks;
}
}

View file

@ -0,0 +1,389 @@
/**
* Resources test scenarios for MCP servers
*/
import { ClientScenario, ConformanceCheck } from '../../types.js';
import { connectToServer } from './client-helper.js';
export class ResourcesListScenario implements ClientScenario {
name = 'resources-list';
description = 'Test listing available resources';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.listResources();
// Validate response structure
const errors: string[] = [];
if (!result.resources) {
errors.push('Missing resources array');
} else {
if (!Array.isArray(result.resources)) {
errors.push('resources is not an array');
}
result.resources.forEach((resource, index) => {
if (!resource.uri) errors.push(`Resource ${index}: missing uri`);
if (!resource.name) errors.push(`Resource ${index}: missing name`);
});
}
checks.push({
id: 'resources-list',
name: 'ResourcesList',
description: 'Server lists available resources with valid structure',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Resources-List',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#listing-resources'
}
],
details: {
resourceCount: result.resources?.length || 0,
resources: result.resources?.map(r => r.uri)
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'resources-list',
name: 'ResourcesList',
description: 'Server lists available resources with valid structure',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Resources-List',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#listing-resources'
}
]
});
}
return checks;
}
}
export class ResourcesReadTextScenario implements ClientScenario {
name = 'resources-read-text';
description = 'Test reading text resource';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.readResource({
uri: 'test://static-text'
});
// Validate response
const errors: string[] = [];
if (!result.contents) errors.push('Missing contents array');
if (!Array.isArray(result.contents)) errors.push('contents is not an array');
if (result.contents.length === 0) errors.push('contents array is empty');
const content = result.contents[0];
if (content) {
if (!content.uri) errors.push('Content missing uri');
if (!content.mimeType) errors.push('Content missing mimeType');
if (!content.text) errors.push('Content missing text field');
}
checks.push({
id: 'resources-read-text',
name: 'ResourcesReadText',
description: 'Read text resource successfully',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Resources-Read',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#reading-resources'
}
],
details: {
uri: content?.uri,
mimeType: content?.mimeType,
hasText: !!content?.text
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'resources-read-text',
name: 'ResourcesReadText',
description: 'Read text resource successfully',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Resources-Read',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#reading-resources'
}
]
});
}
return checks;
}
}
export class ResourcesReadBinaryScenario implements ClientScenario {
name = 'resources-read-binary';
description = 'Test reading binary resource';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.readResource({
uri: 'test://static-binary'
});
// Validate response
const errors: string[] = [];
if (!result.contents) errors.push('Missing contents array');
if (result.contents.length === 0) errors.push('contents array is empty');
const content = result.contents[0];
if (content) {
if (!content.uri) errors.push('Content missing uri');
if (!content.mimeType) errors.push('Content missing mimeType');
if (!content.blob) errors.push('Content missing blob field');
}
checks.push({
id: 'resources-read-binary',
name: 'ResourcesReadBinary',
description: 'Read binary resource successfully',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Resources-Read',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#reading-resources'
}
],
details: {
uri: content?.uri,
mimeType: content?.mimeType,
hasBlob: !!content?.blob
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'resources-read-binary',
name: 'ResourcesReadBinary',
description: 'Read binary resource successfully',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Resources-Read',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#reading-resources'
}
]
});
}
return checks;
}
}
export class ResourcesTemplateReadScenario implements ClientScenario {
name = 'resources-templates-read';
description = 'Test reading resource from template';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.readResource({
uri: 'test://template/123/data'
});
// Validate response
const errors: string[] = [];
if (!result.contents) errors.push('Missing contents array');
if (result.contents.length === 0) errors.push('contents array is empty');
const content = result.contents[0];
if (content) {
if (!content.uri) errors.push('Content missing uri');
if (!content.text && !content.blob) errors.push('Content missing text or blob');
// Check if parameter was substituted
const text = content.text || (content.blob ? '[binary]' : '');
if (typeof text === 'string' && !text.includes('123')) {
errors.push('Parameter substitution not reflected in content');
}
}
checks.push({
id: 'resources-templates-read',
name: 'ResourcesTemplateRead',
description: 'Read resource from template with parameter substitution',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Resources-Templates',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-templates'
}
],
details: {
uri: content?.uri,
content: content?.text || content?.blob
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'resources-templates-read',
name: 'ResourcesTemplateRead',
description: 'Read resource from template with parameter substitution',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Resources-Templates',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-templates'
}
]
});
}
return checks;
}
}
export class ResourcesSubscribeScenario implements ClientScenario {
name = 'resources-subscribe';
description = 'Test subscribing to resource updates';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
await connection.client.subscribeResource({
uri: 'test://watched-resource'
});
checks.push({
id: 'resources-subscribe',
name: 'ResourcesSubscribe',
description: 'Subscribe to resource successfully',
status: 'SUCCESS',
timestamp: new Date().toISOString(),
specReferences: [
{
id: 'MCP-Resources-Subscribe',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-subscriptions'
}
]
});
await connection.close();
} catch (error) {
checks.push({
id: 'resources-subscribe',
name: 'ResourcesSubscribe',
description: 'Subscribe to resource successfully',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Resources-Subscribe',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-subscriptions'
}
]
});
}
return checks;
}
}
export class ResourcesUnsubscribeScenario implements ClientScenario {
name = 'resources-unsubscribe';
description = 'Test unsubscribing from resource';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
// First subscribe
await connection.client.subscribeResource({
uri: 'test://watched-resource'
});
// Then unsubscribe
await connection.client.unsubscribeResource({
uri: 'test://watched-resource'
});
checks.push({
id: 'resources-unsubscribe',
name: 'ResourcesUnsubscribe',
description: 'Unsubscribe from resource successfully',
status: 'SUCCESS',
timestamp: new Date().toISOString(),
specReferences: [
{
id: 'MCP-Resources-Subscribe',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-subscriptions'
}
]
});
await connection.close();
} catch (error) {
checks.push({
id: 'resources-unsubscribe',
name: 'ResourcesUnsubscribe',
description: 'Unsubscribe from resource successfully',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Resources-Subscribe',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-subscriptions'
}
]
});
}
return checks;
}
}

View file

@ -0,0 +1,807 @@
/**
* Tools test scenarios for MCP servers
*/
import { ClientScenario, ConformanceCheck } from '../../types.js';
import { connectToServer, NotificationCollector } from './client-helper.js';
import { CallToolResultSchema, CreateMessageRequestSchema, ElicitRequestSchema, Progress } from '@modelcontextprotocol/sdk/types.js';
export class ToolsListScenario implements ClientScenario {
name = 'tools-list';
description = 'Test listing available tools';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.listTools();
// Validate response structure
const errors: string[] = [];
if (!result.tools) {
errors.push('Missing tools array');
} else {
if (!Array.isArray(result.tools)) {
errors.push('tools is not an array');
}
result.tools.forEach((tool, index) => {
if (!tool.name) errors.push(`Tool ${index}: missing name`);
if (!tool.description) errors.push(`Tool ${index}: missing description`);
if (!tool.inputSchema) errors.push(`Tool ${index}: missing inputSchema`);
});
}
checks.push({
id: 'tools-list',
name: 'ToolsList',
description: 'Server lists available tools with valid structure',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Tools-List',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#listing-tools'
}
],
details: {
toolCount: result.tools?.length || 0,
tools: result.tools?.map(t => t.name)
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-list',
name: 'ToolsList',
description: 'Server lists available tools with valid structure',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Tools-List',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#listing-tools'
}
]
});
}
return checks;
}
}
export class ToolsCallSimpleTextScenario implements ClientScenario {
name = 'tools-call-simple-text';
description = 'Test calling a tool that returns simple text';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.callTool({
name: 'test_simple_text',
arguments: {}
});
// Validate response
const errors: string[] = [];
const content = (result as any).content;
if (!content) errors.push('Missing content array');
if (!Array.isArray(content)) errors.push('content is not an array');
if (content && content.length === 0) errors.push('content array is empty');
const textContent = content && content.find((c: any) => c.type === 'text');
if (!textContent) errors.push('No text content found');
if (textContent && !textContent.text) errors.push('Text content missing text field');
checks.push({
id: 'tools-call-simple-text',
name: 'ToolsCallSimpleText',
description: 'Tool returns simple text content',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
],
details: {
result
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-simple-text',
name: 'ToolsCallSimpleText',
description: 'Tool returns simple text content',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
]
});
}
return checks;
}
}
export class ToolsCallImageScenario implements ClientScenario {
name = 'tools-call-image';
description = 'Test calling a tool that returns image content';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.callTool({
name: 'test_image_content',
arguments: {}
});
// Validate response
const errors: string[] = [];
const content = (result as any).content;
if (!content) errors.push('Missing content array');
const imageContent = content && content.find((c: any) => c.type === 'image');
if (!imageContent) errors.push('No image content found');
if (imageContent && !imageContent.data) errors.push('Image content missing data field');
if (imageContent && !imageContent.mimeType) errors.push('Image content missing mimeType');
checks.push({
id: 'tools-call-image',
name: 'ToolsCallImage',
description: 'Tool returns image content',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
],
details: {
mimeType: imageContent?.mimeType,
hasData: !!imageContent?.data
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-image',
name: 'ToolsCallImage',
description: 'Tool returns image content',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
]
});
}
return checks;
}
}
export class ToolsCallMultipleContentTypesScenario implements ClientScenario {
name = 'tools-call-mixed-content';
description = 'Test tool returning multiple content types';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.callTool({
name: 'test_multiple_content_types',
arguments: {}
});
// Validate response
const errors: string[] = [];
const content = (result as any).content;
if (!content) errors.push('Missing content array');
if (content && content.length < 2) errors.push('Expected multiple content items');
const hasText = content && content.some((c: any) => c.type === 'text');
const hasImage = content && content.some((c: any) => c.type === 'image');
const hasResource = content && content.some((c: any) => c.type === 'resource');
if (!hasText) errors.push('Missing text content');
if (!hasImage) errors.push('Missing image content');
if (!hasResource) errors.push('Missing resource content');
checks.push({
id: 'tools-call-mixed-content',
name: 'ToolsCallMixedContent',
description: 'Tool returns multiple content types',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
],
details: {
contentCount: content ? content.length : 0,
contentTypes: content ? content.map((c: any) => c.type) : []
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-mixed-content',
name: 'ToolsCallMixedContent',
description: 'Tool returns multiple content types',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
]
});
}
return checks;
}
}
export class ToolsCallWithLoggingScenario implements ClientScenario {
name = 'tools-call-with-logging';
description = 'Test tool that sends log messages during execution';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const notifications = new NotificationCollector(connection.client);
await connection.client.callTool({
name: 'test_tool_with_logging',
arguments: {}
});
// Wait a bit for notifications to arrive
await new Promise(resolve => setTimeout(resolve, 200));
// Check for log notifications from the NotificationCollector
const logNotifications = notifications.getNotifications();
const errors: string[] = [];
if (logNotifications.length === 0) {
errors.push('No log notifications received');
} else if (logNotifications.length < 3) {
errors.push(`Expected at least 3 log messages, got ${logNotifications.length}`);
}
checks.push({
id: 'tools-call-with-logging',
name: 'ToolsCallWithLogging',
description: 'Tool sends log messages during execution',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Logging',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/logging'
}
],
details: {
logCount: logNotifications.length,
logs: logNotifications.map((n: any) => n.params)
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-with-logging',
name: 'ToolsCallWithLogging',
description: 'Tool sends log messages during execution',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Logging',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/logging'
}
]
});
}
return checks;
}
}
export class ToolsCallErrorScenario implements ClientScenario {
name = 'tools-call-error';
description = 'Test tool error reporting';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result: any = await connection.client.callTool({
name: 'test_error_handling',
arguments: {}
});
// Check if result has isError flag
const hasIsError = result.isError === true;
const hasContent = result.content && result.content.length > 0;
const hasErrorMessage = hasContent && result.content[0].text;
checks.push({
id: 'tools-call-error',
name: 'ToolsCallError',
description: 'Tool returns error correctly',
status: hasIsError && hasErrorMessage ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: !hasIsError
? 'Tool did not return isError: true'
: !hasErrorMessage
? 'Error result missing error message'
: undefined,
specReferences: [
{
id: 'MCP-Error-Handling',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle'
}
],
details: {
result
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-error',
name: 'ToolsCallError',
description: 'Tool returns error correctly',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Error-Handling',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle'
}
]
});
}
return checks;
}
}
export class ToolsCallWithProgressScenario implements ClientScenario {
name = 'tools-call-with-progress';
description = 'Test tool that reports progress notifications';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const progressUpdates: Array<Progress> = [];
// TODO: investigate why await connection.client.callTool didn't work for progress.
const result = await connection.client.request(
{
method: 'tools/call',
params: {
name: 'test_tool_with_progress',
arguments: {},
_meta: {
progressToken: 'progress-test-1'
}
}
},
CallToolResultSchema,
{
onprogress: progress => {
progressUpdates.push(progress);
}
}
);
const errors: string[] = [];
if (progressUpdates.length === 0) {
errors.push('No progress notifications received');
} else if (progressUpdates.length < 3) {
errors.push(`Expected at least 3 progress notifications, got ${progressUpdates.length}`);
}
if (progressUpdates.length >= 3) {
const progress0 = progressUpdates[0].progress;
const progress1 = progressUpdates[1].progress;
const progress2 = progressUpdates[2].progress;
if (!(progress0 <= progress1 && progress1 <= progress2)) {
errors.push('Progress values should be increasing');
}
}
checks.push({
id: 'tools-call-with-progress',
name: 'ToolsCallWithProgress',
description: 'Tool reports progress notifications',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Progress',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/progress'
}
],
details: {
progressCount: progressUpdates.length,
progressNotifications: progressUpdates.map((n: Progress) => n),
result
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-with-progress',
name: 'ToolsCallWithProgress',
description: 'Tool reports progress notifications',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Progress',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/progress'
}
]
});
}
return checks;
}
}
export class ToolsCallSamplingScenario implements ClientScenario {
name = 'tools-call-sampling';
description = 'Test tool that requests LLM sampling from client';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
let samplingRequested = false;
connection.client.setRequestHandler(CreateMessageRequestSchema, async _request => {
samplingRequested = true;
return {
role: 'assistant',
content: {
type: 'text',
text: 'This is a test response from the client'
},
model: 'test-model',
stopReason: 'endTurn'
};
});
const result = await connection.client.callTool({
name: 'test_sampling',
arguments: {
prompt: 'Test prompt for sampling'
}
});
const errors: string[] = [];
if (!samplingRequested) {
errors.push('Server did not request sampling from client');
}
const content = (result as any).content;
if (!content || content.length === 0) {
errors.push('Tool did not return content');
}
checks.push({
id: 'tools-call-sampling',
name: 'ToolsCallSampling',
description: 'Tool requests LLM sampling from client',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Sampling',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/sampling'
}
],
details: {
samplingRequested,
result
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-sampling',
name: 'ToolsCallSampling',
description: 'Tool requests LLM sampling from client',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Sampling',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/sampling'
}
]
});
}
return checks;
}
}
export class ToolsCallElicitationScenario implements ClientScenario {
name = 'tools-call-elicitation';
description = 'Test tool that requests user input (elicitation) from client';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
let elicitationRequested = false;
connection.client.setRequestHandler(ElicitRequestSchema, async _request => {
elicitationRequested = true;
return {
action: 'accept',
content: {
username: 'testuser',
email: 'test@example.com'
}
};
});
const result = await connection.client.callTool({
name: 'test_elicitation',
arguments: {
message: 'Please provide your information'
}
});
const errors: string[] = [];
if (!elicitationRequested) {
errors.push('Server did not request elicitation from client');
}
const content = (result as any).content;
if (!content || content.length === 0) {
errors.push('Tool did not return content');
}
checks.push({
id: 'tools-call-elicitation',
name: 'ToolsCallElicitation',
description: 'Tool requests user input from client',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Elicitation',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/elicitation'
}
],
details: {
elicitationRequested,
result
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-elicitation',
name: 'ToolsCallElicitation',
description: 'Tool requests user input from client',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Elicitation',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/elicitation'
}
]
});
}
return checks;
}
}
export class ToolsCallAudioScenario implements ClientScenario {
name = 'tools-call-audio';
description = 'Test calling a tool that returns audio content';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.callTool({
name: 'test_audio_content',
arguments: {}
});
// Validate response
const errors: string[] = [];
const content = (result as any).content;
if (!content) errors.push('Missing content array');
if (!Array.isArray(content)) errors.push('content is not an array');
if (content && content.length === 0) errors.push('content array is empty');
const audioContent = content && content.find((c: any) => c.type === 'audio');
if (!audioContent) errors.push('No audio content found');
if (audioContent && !audioContent.data) errors.push('Audio content missing data field');
if (audioContent && !audioContent.mimeType) errors.push('Audio content missing mimeType field');
if (audioContent && audioContent.mimeType !== 'audio/wav')
errors.push(`Expected mimeType 'audio/wav', got '${audioContent.mimeType}'`);
checks.push({
id: 'tools-call-audio',
name: 'ToolsCallAudio',
description: 'Tool returns audio content',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
],
details: {
hasAudioContent: !!audioContent,
audioDataLength: audioContent?.data?.length || 0
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-audio',
name: 'ToolsCallAudio',
description: 'Tool returns audio content',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
]
});
}
return checks;
}
}
export class ToolsCallEmbeddedResourceScenario implements ClientScenario {
name = 'tools-call-embedded-resource';
description = 'Test calling a tool that returns embedded resource content';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
const result = await connection.client.callTool({
name: 'test_embedded_resource',
arguments: {}
});
// Validate response
const errors: string[] = [];
const content = (result as any).content;
if (!content) errors.push('Missing content array');
if (!Array.isArray(content)) errors.push('content is not an array');
if (content && content.length === 0) errors.push('content array is empty');
const resourceContent = content && content.find((c: any) => c.type === 'resource');
if (!resourceContent) errors.push('No resource content found');
if (resourceContent && !resourceContent.resource) errors.push('Resource content missing resource field');
if (resourceContent?.resource) {
if (!resourceContent.resource.uri) errors.push('Resource missing uri field');
if (!resourceContent.resource.mimeType) errors.push('Resource missing mimeType field');
if (!resourceContent.resource.text && !resourceContent.resource.blob) {
errors.push('Resource missing both text and blob fields');
}
}
checks.push({
id: 'tools-call-embedded-resource',
name: 'ToolsCallEmbeddedResource',
description: 'Tool returns embedded resource content',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
],
details: {
hasResourceContent: !!resourceContent,
resourceUri: resourceContent?.resource?.uri
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'tools-call-embedded-resource',
name: 'ToolsCallEmbeddedResource',
description: 'Tool returns embedded resource content',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Tools-Call',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
}
]
});
}
return checks;
}
}

View file

@ -0,0 +1,136 @@
/**
* Utilities test scenarios for MCP servers
*/
import { ClientScenario, ConformanceCheck } from '../../types.js';
import { connectToServer } from './client-helper.js';
export class LoggingSetLevelScenario implements ClientScenario {
name = 'logging-set-level';
description = 'Test setting logging level';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
// Send logging/setLevel request
const result = await connection.client.setLoggingLevel('info');
// Validate response (should return empty object {})
const errors: string[] = [];
if (result && Object.keys(result).length > 0) {
errors.push('Expected empty object {} response');
}
checks.push({
id: 'logging-set-level',
name: 'LoggingSetLevel',
description: 'Server accepts logging level setting',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Logging',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/logging'
}
],
details: {
result
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'logging-set-level',
name: 'LoggingSetLevel',
description: 'Server accepts logging level setting',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Logging',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/logging'
}
]
});
}
return checks;
}
}
export class CompletionCompleteScenario implements ClientScenario {
name = 'completion-complete';
description = 'Test completion endpoint';
async run(serverUrl: string): Promise<ConformanceCheck[]> {
const checks: ConformanceCheck[] = [];
try {
const connection = await connectToServer(serverUrl);
// Send completion/complete request
const result = await connection.client.complete({
ref: {
type: 'ref/prompt',
name: 'test_prompt_with_arguments'
},
argument: {
name: 'arg1',
value: 'test'
}
});
// Validate response structure
const errors: string[] = [];
if (!result.completion) {
errors.push('Missing completion field');
} else {
if (!result.completion.values) errors.push('Missing values array in completion');
if (!Array.isArray(result.completion.values)) errors.push('completion.values is not an array');
}
checks.push({
id: 'completion-complete',
name: 'CompletionComplete',
description: 'Server responds to completion requests',
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
specReferences: [
{
id: 'MCP-Completion',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/completion'
}
],
details: {
result
}
});
await connection.close();
} catch (error) {
checks.push({
id: 'completion-complete',
name: 'CompletionComplete',
description: 'Server responds to completion requests',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
specReferences: [
{
id: 'MCP-Completion',
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/completion'
}
]
});
}
return checks;
}
}

Some files were not shown because too many files have changed in this diff Show more