[compliance tests] initial server scenarios (#4)
* spec for servers * initial version of everything server and tests - tets are very simple * some clean up * lint etc. * fix package lock agian --------- Co-authored-by: Paul Carleton <paulc@anthropic.com>
This commit is contained in:
parent
44a2732b53
commit
a692dd7bba
21 changed files with 5678 additions and 78 deletions
|
|
@ -6,11 +6,19 @@ A framework for testing MCP (Model Context Protocol) client implementations agai
|
|||
|
||||
## Quick Start
|
||||
|
||||
### Testing Clients
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run start -- --command "tsx examples/clients/typescript/test1.ts" --scenario initialize
|
||||
```
|
||||
|
||||
### Testing Servers
|
||||
|
||||
```bash
|
||||
npm run test:server -- --server-url http://localhost:3000/mcp --all
|
||||
```
|
||||
|
||||
## Overview
|
||||
|
||||
The conformance test framework validates MCP client implementations by:
|
||||
|
|
|
|||
933
SERVER_REQUIREMENTS.md
Normal file
933
SERVER_REQUIREMENTS.md
Normal file
|
|
@ -0,0 +1,933 @@
|
|||
# MCP Server Conformance Requirements
|
||||
|
||||
This document specifies the requirements for building an MCP "Everything Server" for conformance testing. SDK maintainers should implement a server meeting these requirements to enable automated conformance testing.
|
||||
|
||||
## Purpose
|
||||
|
||||
The Everything Server is a reference implementation that:
|
||||
|
||||
- Demonstrates all MCP server features in a single, testable server
|
||||
- Uses standardized naming conventions for tools, resources, and prompts
|
||||
- Enables automated conformance testing across different SDK implementations
|
||||
- Serves as a working example for SDK users
|
||||
|
||||
## Protocol Version
|
||||
|
||||
<!-- TODO: need to change to the new version before the official release, probaby worth havign separate tests for new and old version? -->
|
||||
|
||||
**Target MCP Specification**: `2025-06-18`
|
||||
|
||||
## Transport
|
||||
|
||||
**Required Transport**: Streamable HTTP (for initial conformance testing)
|
||||
|
||||
## Specification Compliance
|
||||
|
||||
This document specifies requirements based on the MCP specification. All features and behaviors described are mandated by the MCP specification itself and enable automated conformance testing.
|
||||
|
||||
---
|
||||
|
||||
## Server Information
|
||||
|
||||
### Server Identity
|
||||
|
||||
Your server MUST provide:
|
||||
|
||||
```typescript
|
||||
{
|
||||
name: "mcp-conformance-test-server", // Or your SDK-specific name
|
||||
version: "1.0.0"
|
||||
}
|
||||
```
|
||||
|
||||
### Capabilities Declaration
|
||||
|
||||
Your server MUST declare these capabilities during initialization:
|
||||
|
||||
```typescript
|
||||
{
|
||||
capabilities: {
|
||||
tools: {
|
||||
listChanged: true
|
||||
},
|
||||
resources: {
|
||||
subscribe: true,
|
||||
listChanged: true
|
||||
},
|
||||
prompts: {
|
||||
listChanged: true
|
||||
},
|
||||
logging: {},
|
||||
completions: {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Note**: All capabilities listed are required for conformance testing. The MCP specification also supports optional `experimental` capabilities for non-standard features, but these are not required for conformance.
|
||||
|
||||
---
|
||||
|
||||
## 1. Lifecycle Requirements
|
||||
|
||||
### 1.1. Initialize Handshake
|
||||
|
||||
**Endpoint**: `initialize`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept `initialize` request with client info and capabilities
|
||||
- Return server info, protocol version, and capabilities
|
||||
- Protocol version MUST be `"2025-06-18"`
|
||||
|
||||
**Example Response**:
|
||||
|
||||
```json
|
||||
{
|
||||
"protocolVersion": "2025-06-18",
|
||||
"serverInfo": {
|
||||
"name": "mcp-conformance-test-server",
|
||||
"version": "1.0.0"
|
||||
},
|
||||
"capabilities": {
|
||||
/* as above */
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 1.2. Initialized Notification
|
||||
|
||||
**Notification**: `initialized`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept `initialized` notification from client after handshake
|
||||
- No response required (it's a notification)
|
||||
- Server should be ready for requests after receiving this
|
||||
|
||||
---
|
||||
|
||||
## 2. Tools Requirements
|
||||
|
||||
### 2.1. List Tools
|
||||
|
||||
**Endpoint**: `tools/list`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Return array of all available tools
|
||||
- Each tool MUST have:
|
||||
- `name` (string)
|
||||
- `description` (string)
|
||||
- `inputSchema` (valid JSON Schema object)
|
||||
|
||||
### 2.2. Call Tool
|
||||
|
||||
**Endpoint**: `tools/call`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept tool name and arguments
|
||||
- Execute tool and return result
|
||||
- Result MUST have `content` array
|
||||
- Support `_meta.progressToken` for progress reporting (if provided)
|
||||
|
||||
### 2.3. Required Tools
|
||||
|
||||
Implement these tools with exact names:
|
||||
|
||||
#### `test_simple_text`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Returns**: Text content
|
||||
|
||||
```typescript
|
||||
{
|
||||
content: [{ type: 'text', text: 'This is a simple text response for testing.' }];
|
||||
}
|
||||
```
|
||||
|
||||
#### `test_image_content`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Returns**: Image content with base64 data
|
||||
|
||||
```typescript
|
||||
{
|
||||
content: [
|
||||
{
|
||||
type: 'image',
|
||||
data: '<base64-encoded-png>',
|
||||
mimeType: 'image/png'
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation Note**: Use a minimal test image (e.g., 1x1 red pixel PNG)
|
||||
|
||||
#### `test_audio_content`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Returns**: Audio content with base64 data
|
||||
|
||||
```typescript
|
||||
{
|
||||
content: [
|
||||
{
|
||||
type: 'audio',
|
||||
data: '<base64-encoded-wav>',
|
||||
mimeType: 'audio/wav'
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation Note**: Use a minimal test audio file
|
||||
|
||||
#### `test_embedded_resource`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Returns**: Embedded resource content
|
||||
|
||||
```typescript
|
||||
{
|
||||
content: [
|
||||
{
|
||||
type: 'resource',
|
||||
resource: {
|
||||
uri: 'test://embedded-resource',
|
||||
mimeType: 'text/plain',
|
||||
text: 'This is an embedded resource content.'
|
||||
}
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
#### `test_multiple_content_types`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Returns**: Multiple content items (text + image + resource)
|
||||
|
||||
```typescript
|
||||
{
|
||||
content: [
|
||||
{ type: 'text', text: 'Multiple content types test:' },
|
||||
{ type: 'image', data: '<base64>', mimeType: 'image/png' },
|
||||
{
|
||||
type: 'resource',
|
||||
resource: {
|
||||
uri: 'test://mixed-content-resource',
|
||||
mimeType: 'application/json',
|
||||
text: '{"test":"data","value":123}'
|
||||
}
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
#### `test_tool_with_logging`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Behavior**: During execution, send 3 log notifications at info level:
|
||||
|
||||
1. "Tool execution started"
|
||||
2. "Tool processing data" (after ~50ms delay)
|
||||
3. "Tool execution completed" (after another ~50ms delay)
|
||||
|
||||
**Returns**: Text content confirming execution
|
||||
|
||||
**Implementation Note**: The delays are important to test that clients can receive multiple log notifications during tool execution
|
||||
|
||||
#### `test_tool_with_progress`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Behavior**: If `_meta.progressToken` is provided in request:
|
||||
|
||||
- Send progress notification: `0/100`
|
||||
- Wait ~50ms
|
||||
- Send progress notification: `50/100`
|
||||
- Wait ~50ms
|
||||
- Send progress notification: `100/100`
|
||||
|
||||
If no progress token provided, just execute with delays.
|
||||
|
||||
**Returns**: Text content confirming execution
|
||||
|
||||
**Progress Notification Format**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
method: "notifications/progress",
|
||||
params: {
|
||||
progressToken: <from request._meta.progressToken>,
|
||||
progress: 50,
|
||||
total: 100
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `test_error_handling`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Behavior**: Always throw an error
|
||||
|
||||
**Returns**: JSON-RPC error
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"code": -32000,
|
||||
"message": "This tool intentionally returns an error for testing"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `test_sampling`
|
||||
|
||||
**Arguments**:
|
||||
|
||||
- `prompt` (string, required) - The prompt to send to the LLM
|
||||
|
||||
**Behavior**: Request LLM sampling from the client using `sampling/createMessage`
|
||||
|
||||
**Sampling Request**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
"method": "sampling/createMessage",
|
||||
"params": {
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": { "type": "text", "text": "<prompt from arguments>" }
|
||||
}
|
||||
],
|
||||
"maxTokens": 100
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Returns**: Text content with the LLM's response
|
||||
|
||||
```typescript
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "LLM response: <response from sampling>"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation Note**: If the client doesn't support sampling (no `sampling` capability), return an error.
|
||||
|
||||
#### `test_elicitation`
|
||||
|
||||
**Arguments**:
|
||||
|
||||
- `message` (string, required) - The message to show the user
|
||||
|
||||
**Behavior**: Request user input from the client using `elicitation/create`
|
||||
|
||||
**Elicitation Request**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
"method": "elicitation/create",
|
||||
"params": {
|
||||
"message": "<message from arguments>",
|
||||
"requestedSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"response": {
|
||||
"type": "string",
|
||||
"description": "User's response"
|
||||
}
|
||||
},
|
||||
"required": ["response"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Returns**: Text content with the user's response
|
||||
|
||||
```typescript
|
||||
{
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "User response: <action: accept/decline/cancel, content: {...}>"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation Note**: If the client doesn't support elicitation (no `elicitation` capability), return an error.
|
||||
|
||||
---
|
||||
|
||||
## 3. Resources Requirements
|
||||
|
||||
### 3.1. List Resources
|
||||
|
||||
**Endpoint**: `resources/list`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Return array of all available **direct resources** (not templates)
|
||||
- Support optional cursor-based pagination (see Section 6.1)
|
||||
- Each resource MUST have:
|
||||
- `uri` (string)
|
||||
- `name` (string)
|
||||
- `description` (string)
|
||||
- `mimeType` (string, optional)
|
||||
|
||||
**Note**: Resource templates are listed via separate `resources/templates/list` endpoint (see 3.1a)
|
||||
|
||||
### 3.1a. List Resource Templates
|
||||
|
||||
**Endpoint**: `resources/templates/list`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Return array of all available **resource templates**
|
||||
- Support optional cursor-based pagination (see Section 6.1)
|
||||
- Each template MUST have:
|
||||
- `uriTemplate` (string) - RFC 6570 URI template
|
||||
- `name` (string)
|
||||
- `description` (string)
|
||||
- `mimeType` (string, optional)
|
||||
|
||||
### 3.2. Read Resource
|
||||
|
||||
**Endpoint**: `resources/read`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept resource URI
|
||||
- Return resource contents
|
||||
- Response MUST have `contents` array
|
||||
- Each content item MUST have `uri`, `mimeType`, and either `text` or `blob`
|
||||
|
||||
### 3.3. Required Resources
|
||||
|
||||
Implement these resources with exact URIs:
|
||||
|
||||
#### `test://static-text`
|
||||
|
||||
**Type**: Static text resource
|
||||
|
||||
**Metadata** (for `resources/list`):
|
||||
|
||||
```typescript
|
||||
{
|
||||
uri: "test://static-text",
|
||||
name: "Static Text Resource",
|
||||
description: "A static text resource for testing",
|
||||
mimeType: "text/plain"
|
||||
}
|
||||
```
|
||||
|
||||
**Content** (for `resources/read`):
|
||||
|
||||
```typescript
|
||||
{
|
||||
contents: [
|
||||
{
|
||||
uri: 'test://static-text',
|
||||
mimeType: 'text/plain',
|
||||
text: 'This is the content of the static text resource.'
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
#### `test://static-binary`
|
||||
|
||||
**Type**: Static binary resource (image)
|
||||
|
||||
**Metadata**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
uri: "test://static-binary",
|
||||
name: "Static Binary Resource",
|
||||
description: "A static binary resource (image) for testing",
|
||||
mimeType: "image/png"
|
||||
}
|
||||
```
|
||||
|
||||
**Content**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
contents: [
|
||||
{
|
||||
uri: 'test://static-binary',
|
||||
mimeType: 'image/png',
|
||||
blob: '<base64-encoded-png>'
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
#### `test://template/{id}/data`
|
||||
|
||||
**Type**: Resource template with parameter
|
||||
|
||||
**Metadata** (for `resources/list`):
|
||||
|
||||
```typescript
|
||||
{
|
||||
uriTemplate: "test://template/{id}/data",
|
||||
name: "Resource Template",
|
||||
description: "A resource template with parameter substitution",
|
||||
mimeType: "application/json"
|
||||
}
|
||||
```
|
||||
|
||||
**Behavior**: When client requests `test://template/123/data`, substitute `{id}` with `123`
|
||||
|
||||
**Content** (for `resources/read` with `uri: "test://template/123/data"`):
|
||||
|
||||
```typescript
|
||||
{
|
||||
contents: [
|
||||
{
|
||||
uri: 'test://template/123/data',
|
||||
mimeType: 'application/json',
|
||||
text: '{"id":"123","templateTest":true,"data":"Data for ID: 123"}'
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation Note**: Use RFC 6570 URI template syntax
|
||||
|
||||
#### `test://watched-resource`
|
||||
|
||||
**Type**: Subscribable resource
|
||||
|
||||
**Metadata**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
uri: "test://watched-resource",
|
||||
name: "Watched Resource",
|
||||
description: "A resource that can be subscribed to",
|
||||
mimeType: "text/plain"
|
||||
}
|
||||
```
|
||||
|
||||
**Content**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
contents: [
|
||||
{
|
||||
uri: 'test://watched-resource',
|
||||
mimeType: 'text/plain',
|
||||
text: 'Watched resource content'
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
### 3.4. Resource Subscription
|
||||
|
||||
**Endpoint**: `resources/subscribe`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept subscription request with URI
|
||||
- Track subscribed URIs
|
||||
- Send `notifications/resources/updated` when subscribed resources change
|
||||
- Return empty object `{}`
|
||||
|
||||
**Example Request**:
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "resources/subscribe",
|
||||
"params": {
|
||||
"uri": "test://watched-resource"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Endpoint**: `resources/unsubscribe`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept unsubscribe request with URI
|
||||
- Remove URI from subscriptions
|
||||
- Stop sending update notifications for that URI
|
||||
- Return empty object `{}`
|
||||
|
||||
---
|
||||
|
||||
## 4. Prompts Requirements
|
||||
|
||||
### 4.1. List Prompts
|
||||
|
||||
**Endpoint**: `prompts/list`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Return array of all available prompts
|
||||
- Each prompt MUST have:
|
||||
- `name` (string)
|
||||
- `description` (string)
|
||||
- `arguments` (array, optional) - list of required arguments
|
||||
|
||||
### 4.2. Get Prompt
|
||||
|
||||
**Endpoint**: `prompts/get`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept prompt name and arguments
|
||||
- Return prompt messages
|
||||
- Response MUST have `messages` array
|
||||
- Each message MUST have `role` and `content`
|
||||
|
||||
### 4.3. Required Prompts
|
||||
|
||||
Implement these prompts with exact names:
|
||||
|
||||
#### `test_simple_prompt`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Returns**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'text',
|
||||
text: 'This is a simple prompt for testing.'
|
||||
}
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
#### `test_prompt_with_arguments`
|
||||
|
||||
**Arguments**:
|
||||
|
||||
- `arg1` (string, required) - First test argument
|
||||
- `arg2` (string, required) - Second test argument
|
||||
|
||||
**Metadata** (for `prompts/list`):
|
||||
|
||||
```typescript
|
||||
{
|
||||
name: "test_prompt_with_arguments",
|
||||
description: "A prompt with required arguments",
|
||||
arguments: [
|
||||
{ name: "arg1", description: "First test argument", required: true },
|
||||
{ name: "arg2", description: "Second test argument", required: true }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Returns** (with args `{arg1: "hello", arg2: "world"}`):
|
||||
|
||||
```typescript
|
||||
{
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'text',
|
||||
text: "Prompt with arguments: arg1='hello', arg2='world'"
|
||||
}
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
#### `test_prompt_with_embedded_resource`
|
||||
|
||||
**Arguments**:
|
||||
|
||||
- `resourceUri` (string, required) - URI of the resource to embed
|
||||
|
||||
**Returns**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'resource',
|
||||
resource: {
|
||||
uri: '<resourceUri from arguments>',
|
||||
mimeType: 'text/plain',
|
||||
text: 'Embedded resource content for testing.'
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'text',
|
||||
text: 'Please process the embedded resource above.'
|
||||
}
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
#### `test_prompt_with_image`
|
||||
|
||||
**Arguments**: None
|
||||
|
||||
**Returns**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'image',
|
||||
data: '<base64-encoded-png>',
|
||||
mimeType: 'image/png'
|
||||
}
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'text',
|
||||
text: 'Please analyze the image above.'
|
||||
}
|
||||
}
|
||||
];
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Logging Requirements
|
||||
|
||||
### 5.1. Set Log Level
|
||||
|
||||
**Endpoint**: `logging/setLevel`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept log level setting
|
||||
- Filter subsequent log notifications based on level
|
||||
- Return empty object `{}`
|
||||
|
||||
**Log Levels** (in order of severity):
|
||||
|
||||
- `debug`
|
||||
- `info`
|
||||
- `notice`
|
||||
- `warning`
|
||||
- `error`
|
||||
- `critical`
|
||||
- `alert`
|
||||
- `emergency`
|
||||
|
||||
### 5.2. Log Notifications
|
||||
|
||||
**Notification**: `notifications/message`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Send log notifications during operations
|
||||
- Each log MUST have:
|
||||
- `level` (string) - one of the log levels above
|
||||
- `data` (any) - log message or structured data
|
||||
- `logger` (string, optional) - logger name
|
||||
|
||||
**Example**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
method: "notifications/message",
|
||||
params: {
|
||||
level: "info",
|
||||
logger: "conformance-test-server",
|
||||
data: "Tool execution started"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation Note**: When no client is connected, log notifications may fail. Handle this gracefully by catching promise rejections:
|
||||
|
||||
```typescript
|
||||
mcpServer.server
|
||||
.notification({
|
||||
method: 'notifications/message',
|
||||
params: { level, logger, data }
|
||||
})
|
||||
.catch(() => {
|
||||
// Ignore error if no client is connected
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Pagination (Utility Feature)
|
||||
|
||||
### 6.1. Cursor-Based Pagination
|
||||
|
||||
**Purpose**: Allow servers to return large result sets in manageable chunks.
|
||||
|
||||
**Applies To**:
|
||||
|
||||
- `tools/list`
|
||||
- `resources/list`
|
||||
- `resources/templates/list`
|
||||
- `prompts/list`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
**Request Format**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
"method": "tools/list", // or resources/list, prompts/list, etc.
|
||||
"params": {
|
||||
"cursor": "optional-opaque-token" // Optional: for subsequent pages
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response Format**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
"tools": [ /* array of tools */ ], // or resources, prompts, etc.
|
||||
"nextCursor": "optional-opaque-token" // Optional: indicates more results available
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation Requirements**:
|
||||
|
||||
- If `cursor` parameter is provided, return results starting after that position
|
||||
- If more results are available, include `nextCursor` in response
|
||||
- Cursor tokens MUST be opaque strings (format is server-defined)
|
||||
- Page size is determined by server (clients MUST NOT assume fixed page size)
|
||||
- For conformance testing, pagination is optional (all items can be returned in single response)
|
||||
|
||||
**Example**:
|
||||
|
||||
```typescript
|
||||
// First request (no cursor)
|
||||
Request: { "method": "tools/list" }
|
||||
Response: {
|
||||
"tools": [/* first 10 tools */],
|
||||
"nextCursor": "page2token"
|
||||
}
|
||||
|
||||
// Second request (with cursor)
|
||||
Request: { "method": "tools/list", "params": { "cursor": "page2token" } }
|
||||
Response: {
|
||||
"tools": [/* next 10 tools */],
|
||||
"nextCursor": "page3token"
|
||||
}
|
||||
|
||||
// Last page (no nextCursor)
|
||||
Request: { "method": "tools/list", "params": { "cursor": "page3token" } }
|
||||
Response: {
|
||||
"tools": [/* remaining tools */]
|
||||
// No nextCursor = end of results
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Completion Requirements
|
||||
|
||||
### 6.1. Complete Request
|
||||
|
||||
**Endpoint**: `completion/complete`
|
||||
|
||||
**Requirements**:
|
||||
|
||||
- Accept completion requests for prompt or resource template arguments
|
||||
- Provide contextual suggestions based on partial input
|
||||
- Return array of completion values ranked by relevance
|
||||
|
||||
**Request Format**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
"method": "completion/complete",
|
||||
"params": {
|
||||
"ref": {
|
||||
"type": "ref/prompt", // or "ref/resource"
|
||||
"name": "test_prompt_with_arguments" // or uri for resources
|
||||
},
|
||||
"argument": {
|
||||
"name": "arg1",
|
||||
"value": "par" // partial value to complete
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response Format**:
|
||||
|
||||
```typescript
|
||||
{
|
||||
"completion": {
|
||||
"values": ["paris", "park", "party"], // Up to 100 items
|
||||
"total": 150, // Optional: total available matches
|
||||
"hasMore": true // Optional: indicates more results exist
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation Note**: For conformance testing, completion support can be minimal or return empty arrays. The capability just needs to be declared and the endpoint must respond correctly.
|
||||
|
||||
---
|
||||
|
||||
## 8. Testing Your Server
|
||||
|
||||
### 8.1. Starting Your Server
|
||||
|
||||
```bash
|
||||
# Example for TypeScript
|
||||
cd examples/servers/typescript
|
||||
npm install
|
||||
npm start
|
||||
```
|
||||
|
||||
Server should output:
|
||||
|
||||
```
|
||||
MCP Conformance Test Server running on http://localhost:3000
|
||||
- MCP endpoint: http://localhost:3000/mcp
|
||||
```
|
||||
|
||||
### 8.2. Running Conformance Tests
|
||||
|
||||
```bash
|
||||
# Single test
|
||||
npm run test:server -- --server-url http://localhost:3000/mcp --scenario server-initialize
|
||||
|
||||
# All tests
|
||||
npm run test:server -- --server-url http://localhost:3000/mcp --all
|
||||
```
|
||||
173
examples/servers/typescript/README.md
Normal file
173
examples/servers/typescript/README.md
Normal file
|
|
@ -0,0 +1,173 @@
|
|||
# MCP Conformance Test Server
|
||||
|
||||
A reference implementation of an MCP server that implements all features required for conformance testing.
|
||||
|
||||
## Features
|
||||
|
||||
This server implements:
|
||||
|
||||
### Tools
|
||||
|
||||
- `test_simple_text` - Returns simple text content
|
||||
- `test_image_content` - Returns image content (base64 PNG)
|
||||
- `test_audio_content` - Returns audio content (base64 WAV)
|
||||
- `test_embedded_resource` - Returns embedded resource
|
||||
- `test_multiple_content_types` - Returns mixed content types
|
||||
- `test_tool_with_logging` - Emits log messages during execution
|
||||
- `test_tool_with_progress` - Reports progress notifications
|
||||
- `test_error_handling` - Returns error response
|
||||
- `test_sampling` - Requests LLM completion from client
|
||||
- `test_elicitation` - Requests user input from client
|
||||
- `test_dynamic_tool` - Dynamically added/removed tool
|
||||
|
||||
### Resources
|
||||
|
||||
- `test://static-text` - Static text resource
|
||||
- `test://static-binary` - Static binary resource (image)
|
||||
- `test://template/{id}/data` - Resource template with parameter
|
||||
- `test://watched-resource` - Subscribable resource with updates
|
||||
|
||||
### Prompts
|
||||
|
||||
- `test_simple_prompt` - Simple prompt without arguments
|
||||
- `test_prompt_with_arguments(arg1, arg2)` - Parameterized prompt
|
||||
- `test_prompt_with_embedded_resource(resourceUri)` - Prompt with embedded resource
|
||||
- `test_prompt_with_image` - Prompt with image content
|
||||
|
||||
### Other Capabilities
|
||||
|
||||
- Logging at all levels (debug, info, notice, warning, error, critical, alert, emergency)
|
||||
- Completion support for prompt and resource arguments
|
||||
- List changed notifications for tools, resources, and prompts
|
||||
- Resource subscription and update notifications
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
|
||||
## Running the Server
|
||||
|
||||
```bash
|
||||
npm start
|
||||
```
|
||||
|
||||
The server will start on `http://localhost:3000` (or the port specified in `PORT` environment variable).
|
||||
|
||||
## Endpoints
|
||||
|
||||
### MCP Endpoint
|
||||
|
||||
- `POST /mcp` - Main MCP protocol endpoint
|
||||
|
||||
### Health Check
|
||||
|
||||
- `GET /health` - Server health check
|
||||
|
||||
## Automatic Behaviors
|
||||
|
||||
The server automatically demonstrates dynamic capabilities:
|
||||
|
||||
- **Dynamic Tool** - `test_dynamic_tool` is automatically added 2 seconds after server starts
|
||||
- **Dynamic Resource** - `test://dynamic-resource` is automatically added 2 seconds after server starts
|
||||
- **Dynamic Prompt** - `test_dynamic_prompt` is automatically added 2 seconds after server starts
|
||||
- **Resource Updates** - `test://watched-resource` automatically updates every 3 seconds with new content
|
||||
|
||||
These behaviors allow testing of MCP notifications without requiring manual triggers.
|
||||
|
||||
## Example Usage
|
||||
|
||||
### Starting the Server
|
||||
|
||||
```bash
|
||||
npm start
|
||||
```
|
||||
|
||||
### Testing with MCP Inspector
|
||||
|
||||
```bash
|
||||
npx @modelcontextprotocol/inspector http://localhost:3000/mcp
|
||||
```
|
||||
|
||||
### Testing with curl
|
||||
|
||||
#### Initialize
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/mcp \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"jsonrpc": "2.0",
|
||||
"id": 1,
|
||||
"method": "initialize",
|
||||
"params": {
|
||||
"protocolVersion": "2025-06-18",
|
||||
"capabilities": {},
|
||||
"clientInfo": {
|
||||
"name": "test-client",
|
||||
"version": "1.0.0"
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
#### List Tools
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/mcp \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"jsonrpc": "2.0",
|
||||
"id": 2,
|
||||
"method": "tools/list"
|
||||
}'
|
||||
```
|
||||
|
||||
#### Call Tool
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/mcp \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"jsonrpc": "2.0",
|
||||
"id": 3,
|
||||
"method": "tools/call",
|
||||
"params": {
|
||||
"name": "test_simple_text",
|
||||
"arguments": {}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
## Conformance Testing
|
||||
|
||||
This server implements the MCP Server Conformance Requirements specified in `../../../SERVER_REQUIREMENTS.md`. All tools, resources, and prompts use standardized naming conventions for consistent testing across SDK implementations.
|
||||
|
||||
To run conformance tests against this server:
|
||||
|
||||
```bash
|
||||
cd ../../../
|
||||
npm run test:server -- --server-url http://localhost:3000/mcp --all
|
||||
```
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- All tool, resource, and prompt names follow the standardized naming conventions (`test_*` for tools/prompts, `test://` for resources)
|
||||
- Names are descriptive of the feature being tested (e.g., `test_image_content`, `test_tool_with_progress`)
|
||||
- The server uses the TypeScript MCP SDK (`@modelcontextprotocol/sdk`) high-level API
|
||||
- Uses `registerTool()`, `registerResource()`, and `registerPrompt()` methods
|
||||
- Transport is Streamable HTTP (Express) for web-based testing compatibility
|
||||
- Promise rejections from notifications are caught and handled gracefully
|
||||
|
||||
## For SDK Implementers
|
||||
|
||||
If you're implementing MCP in another language/SDK:
|
||||
|
||||
1. **Read the Requirements**: See `../../../SERVER_REQUIREMENTS.md` for complete specifications
|
||||
2. **Use This as Reference**: This TypeScript implementation demonstrates all required features
|
||||
3. **Follow Naming Conventions**: Use exact tool/resource/prompt names specified in requirements
|
||||
4. **Implement Automatic Behaviors**: Dynamic registration after 2s, resource updates every 3s
|
||||
5. **Handle Notifications Carefully**: Catch/ignore errors when no client is connected
|
||||
|
||||
**Goal**: All SDK example servers provide the same interface, enabling a single test suite to verify conformance across all implementations.
|
||||
748
examples/servers/typescript/everything-server.ts
Normal file
748
examples/servers/typescript/everything-server.ts
Normal file
|
|
@ -0,0 +1,748 @@
|
|||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* MCP Everything Server - Conformance Test Server
|
||||
*
|
||||
* Server implementing all MCP features for conformance testing based on Conformnace Server Specification.
|
||||
* Should be using registerTool(), registerResource(), and registerPrompt().
|
||||
* we use tool() instead of registerTool() as there is a bug with logging in registerTool().
|
||||
*/
|
||||
|
||||
import { McpServer, ResourceTemplate } from '@modelcontextprotocol/sdk/server/mcp.js';
|
||||
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
|
||||
import { z } from 'zod';
|
||||
import express from 'express';
|
||||
import cors from 'cors';
|
||||
import { randomUUID } from 'crypto';
|
||||
|
||||
// Server state
|
||||
const resourceSubscriptions = new Set<string>();
|
||||
const watchedResourceContent = 'Watched resource content';
|
||||
|
||||
// Session management
|
||||
const transports: { [sessionId: string]: StreamableHTTPServerTransport } = {};
|
||||
const servers: { [sessionId: string]: McpServer } = {};
|
||||
|
||||
// Sample base64 encoded 1x1 red PNG pixel for testing
|
||||
const TEST_IMAGE_BASE64 = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8DwHwAFBQIAX8jx0gAAAABJRU5ErkJggg==';
|
||||
|
||||
// Sample base64 encoded minimal WAV file for testing
|
||||
const TEST_AUDIO_BASE64 = 'UklGRiYAAABXQVZFZm10IBAAAAABAAEAQB8AAAB9AAACABAAZGF0YQIAAAA=';
|
||||
|
||||
// Function to create a new MCP server instance (one per session)
|
||||
function createMcpServer() {
|
||||
const mcpServer = new McpServer(
|
||||
{
|
||||
name: 'mcp-conformance-test-server',
|
||||
version: '1.0.0'
|
||||
},
|
||||
{
|
||||
capabilities: {
|
||||
tools: {
|
||||
listChanged: true
|
||||
},
|
||||
resources: {
|
||||
subscribe: true,
|
||||
listChanged: true
|
||||
},
|
||||
prompts: {
|
||||
listChanged: true
|
||||
},
|
||||
logging: {},
|
||||
completions: {}
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Helper to send log messages using the underlying server
|
||||
function sendLog(
|
||||
level: 'debug' | 'info' | 'notice' | 'warning' | 'error' | 'critical' | 'alert' | 'emergency',
|
||||
message: string,
|
||||
data?: any
|
||||
) {
|
||||
mcpServer.server
|
||||
.notification({
|
||||
method: 'notifications/message',
|
||||
params: {
|
||||
level,
|
||||
logger: 'conformance-test-server',
|
||||
data: data || message
|
||||
}
|
||||
})
|
||||
.catch(() => {
|
||||
// Ignore error if no client is connected
|
||||
});
|
||||
}
|
||||
|
||||
// ===== TOOLS =====
|
||||
|
||||
// Simple text tool
|
||||
mcpServer.tool('test_simple_text', 'Tests simple text content response', {}, async () => {
|
||||
return {
|
||||
content: [{ type: 'text', text: 'This is a simple text response for testing.' }]
|
||||
};
|
||||
});
|
||||
|
||||
// Image content tool
|
||||
mcpServer.registerTool(
|
||||
'test_image_content',
|
||||
{
|
||||
description: 'Tests image content response'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
content: [{ type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' }]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Audio content tool
|
||||
mcpServer.registerTool(
|
||||
'test_audio_content',
|
||||
{
|
||||
description: 'Tests audio content response'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
content: [{ type: 'audio', data: TEST_AUDIO_BASE64, mimeType: 'audio/wav' }]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Embedded resource tool
|
||||
mcpServer.registerTool(
|
||||
'test_embedded_resource',
|
||||
{
|
||||
description: 'Tests embedded resource content response'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'resource',
|
||||
resource: {
|
||||
uri: 'test://embedded-resource',
|
||||
mimeType: 'text/plain',
|
||||
text: 'This is an embedded resource content.'
|
||||
}
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Multiple content types tool
|
||||
mcpServer.registerTool(
|
||||
'test_multiple_content_types',
|
||||
{
|
||||
description: 'Tests response with multiple content types (text, image, resource)'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
content: [
|
||||
{ type: 'text', text: 'Multiple content types test:' },
|
||||
{ type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' },
|
||||
{
|
||||
type: 'resource',
|
||||
resource: {
|
||||
uri: 'test://mixed-content-resource',
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify({ test: 'data', value: 123 })
|
||||
}
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Tool with logging - registerTool with empty inputSchema to get (args, extra) signature
|
||||
mcpServer.registerTool(
|
||||
'test_tool_with_logging',
|
||||
{
|
||||
description: 'Tests tool that emits log messages during execution',
|
||||
inputSchema: {} // Empty schema so callback gets (args, extra) instead of just (extra)
|
||||
},
|
||||
async (_args, { sendNotification }) => {
|
||||
await sendNotification({
|
||||
method: 'notifications/message',
|
||||
params: {
|
||||
level: 'info',
|
||||
data: 'Tool execution started'
|
||||
}
|
||||
});
|
||||
await new Promise(resolve => setTimeout(resolve, 50));
|
||||
|
||||
await sendNotification({
|
||||
method: 'notifications/message',
|
||||
params: {
|
||||
level: 'info',
|
||||
data: 'Tool processing data'
|
||||
}
|
||||
});
|
||||
await new Promise(resolve => setTimeout(resolve, 50));
|
||||
|
||||
await sendNotification({
|
||||
method: 'notifications/message',
|
||||
params: {
|
||||
level: 'info',
|
||||
data: 'Tool execution completed'
|
||||
}
|
||||
});
|
||||
return {
|
||||
content: [{ type: 'text', text: 'Tool with logging executed successfully' }]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Tool with progress - registerTool with empty inputSchema to get (args, extra) signature
|
||||
mcpServer.registerTool(
|
||||
'test_tool_with_progress',
|
||||
{
|
||||
description: 'Tests tool that reports progress notifications',
|
||||
inputSchema: {} // Empty schema so callback gets (args, extra) instead of just (extra)
|
||||
},
|
||||
async (_args, { sendNotification, _meta }) => {
|
||||
const progressToken = _meta?.progressToken ?? 0;
|
||||
console.log('???? Progress token:', progressToken);
|
||||
await sendNotification({
|
||||
method: 'notifications/progress',
|
||||
params: {
|
||||
progressToken,
|
||||
progress: 0,
|
||||
total: 100,
|
||||
message: `Completed step ${0} of ${100}`
|
||||
}
|
||||
});
|
||||
await new Promise(resolve => setTimeout(resolve, 50));
|
||||
|
||||
await sendNotification({
|
||||
method: 'notifications/progress',
|
||||
params: {
|
||||
progressToken,
|
||||
progress: 50,
|
||||
total: 100,
|
||||
message: `Completed step ${50} of ${100}`
|
||||
}
|
||||
});
|
||||
await new Promise(resolve => setTimeout(resolve, 50));
|
||||
|
||||
await sendNotification({
|
||||
method: 'notifications/progress',
|
||||
params: {
|
||||
progressToken,
|
||||
progress: 100,
|
||||
total: 100,
|
||||
message: `Completed step ${100} of ${100}`
|
||||
}
|
||||
});
|
||||
|
||||
return {
|
||||
content: [{ type: 'text', text: String(progressToken) }]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Error handling tool
|
||||
mcpServer.registerTool(
|
||||
'test_error_handling',
|
||||
{
|
||||
description: 'Tests error response handling'
|
||||
},
|
||||
async () => {
|
||||
throw new Error('This tool intentionally returns an error for testing');
|
||||
}
|
||||
);
|
||||
|
||||
// Sampling tool - requests LLM completion from client
|
||||
mcpServer.registerTool(
|
||||
'test_sampling',
|
||||
{
|
||||
description: 'Tests server-initiated sampling (LLM completion request)',
|
||||
inputSchema: {
|
||||
prompt: z.string().describe('The prompt to send to the LLM')
|
||||
}
|
||||
},
|
||||
async (args: { prompt: string }) => {
|
||||
try {
|
||||
// Request sampling from client
|
||||
const result = await mcpServer.server.request(
|
||||
{
|
||||
method: 'sampling/createMessage',
|
||||
params: {
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'text',
|
||||
text: args.prompt
|
||||
}
|
||||
}
|
||||
],
|
||||
maxTokens: 100
|
||||
}
|
||||
},
|
||||
z.object({ method: z.literal('sampling/createMessage') }).passthrough() as any
|
||||
);
|
||||
|
||||
const samplingResult = result as any;
|
||||
const modelResponse = samplingResult.content?.text || samplingResult.message?.content?.text || 'No response';
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `LLM response: ${modelResponse}`
|
||||
}
|
||||
]
|
||||
};
|
||||
} catch (error: any) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `Sampling not supported or error: ${error.message}`
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Elicitation tool - requests user input from client
|
||||
mcpServer.registerTool(
|
||||
'test_elicitation',
|
||||
{
|
||||
description: 'Tests server-initiated elicitation (user input request)',
|
||||
inputSchema: {
|
||||
message: z.string().describe('The message to show the user')
|
||||
}
|
||||
},
|
||||
async (args: { message: string }) => {
|
||||
try {
|
||||
// Request user input from client
|
||||
const result = await mcpServer.server.request(
|
||||
{
|
||||
method: 'elicitation/create',
|
||||
params: {
|
||||
message: args.message,
|
||||
requestedSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
response: {
|
||||
type: 'string',
|
||||
description: "User's response"
|
||||
}
|
||||
},
|
||||
required: ['response']
|
||||
}
|
||||
}
|
||||
},
|
||||
z.object({ method: z.literal('elicitation/create') }).passthrough() as any
|
||||
);
|
||||
|
||||
const elicitResult = result as any;
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `User response: action=${elicitResult.action}, content=${JSON.stringify(elicitResult.content || {})}`
|
||||
}
|
||||
]
|
||||
};
|
||||
} catch (error: any) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `Elicitation not supported or error: ${error.message}`
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Dynamic tool (registered later via timer)
|
||||
|
||||
// ===== RESOURCES =====
|
||||
|
||||
// Static text resource
|
||||
mcpServer.registerResource(
|
||||
'static-text',
|
||||
'test://static-text',
|
||||
{
|
||||
title: 'Static Text Resource',
|
||||
description: 'A static text resource for testing',
|
||||
mimeType: 'text/plain'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri: 'test://static-text',
|
||||
mimeType: 'text/plain',
|
||||
text: 'This is the content of the static text resource.'
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Static binary resource
|
||||
mcpServer.registerResource(
|
||||
'static-binary',
|
||||
'test://static-binary',
|
||||
{
|
||||
title: 'Static Binary Resource',
|
||||
description: 'A static binary resource (image) for testing',
|
||||
mimeType: 'image/png'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri: 'test://static-binary',
|
||||
mimeType: 'image/png',
|
||||
blob: TEST_IMAGE_BASE64
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Resource template
|
||||
mcpServer.registerResource(
|
||||
'template',
|
||||
new ResourceTemplate('test://template/{id}/data', {
|
||||
list: undefined
|
||||
}),
|
||||
{
|
||||
title: 'Resource Template',
|
||||
description: 'A resource template with parameter substitution',
|
||||
mimeType: 'application/json'
|
||||
},
|
||||
async (uri, variables) => {
|
||||
const id = variables.id;
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri: uri.toString(),
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify({
|
||||
id,
|
||||
templateTest: true,
|
||||
data: `Data for ID: ${id}`
|
||||
})
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Watched resource
|
||||
mcpServer.registerResource(
|
||||
'watched-resource',
|
||||
'test://watched-resource',
|
||||
{
|
||||
title: 'Watched Resource',
|
||||
description: 'A resource that auto-updates every 3 seconds',
|
||||
mimeType: 'text/plain'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri: 'test://watched-resource',
|
||||
mimeType: 'text/plain',
|
||||
text: watchedResourceContent
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Subscribe/Unsubscribe handlers
|
||||
mcpServer.server.setRequestHandler(z.object({ method: z.literal('resources/subscribe') }).passthrough(), async (request: any) => {
|
||||
const uri = request.params.uri;
|
||||
resourceSubscriptions.add(uri);
|
||||
sendLog('info', `Subscribed to resource: ${uri}`);
|
||||
return {};
|
||||
});
|
||||
|
||||
mcpServer.server.setRequestHandler(z.object({ method: z.literal('resources/unsubscribe') }).passthrough(), async (request: any) => {
|
||||
const uri = request.params.uri;
|
||||
resourceSubscriptions.delete(uri);
|
||||
sendLog('info', `Unsubscribed from resource: ${uri}`);
|
||||
return {};
|
||||
});
|
||||
|
||||
// ===== PROMPTS =====
|
||||
|
||||
// Simple prompt
|
||||
mcpServer.registerPrompt(
|
||||
'test_simple_prompt',
|
||||
{
|
||||
title: 'Simple Test Prompt',
|
||||
description: 'A simple prompt without arguments'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: { type: 'text', text: 'This is a simple prompt for testing.' }
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Prompt with arguments
|
||||
mcpServer.registerPrompt(
|
||||
'test_prompt_with_arguments',
|
||||
{
|
||||
title: 'Prompt With Arguments',
|
||||
description: 'A prompt with required arguments',
|
||||
argsSchema: {
|
||||
arg1: z.string().describe('First test argument'),
|
||||
arg2: z.string().describe('Second test argument')
|
||||
}
|
||||
},
|
||||
async args => {
|
||||
const { arg1, arg2 } = args;
|
||||
return {
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'text',
|
||||
text: `Prompt with arguments: arg1='${arg1}', arg2='${arg2}'`
|
||||
}
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Prompt with embedded resource
|
||||
mcpServer.registerPrompt(
|
||||
'test_prompt_with_embedded_resource',
|
||||
{
|
||||
title: 'Prompt With Embedded Resource',
|
||||
description: 'A prompt that includes an embedded resource',
|
||||
argsSchema: {
|
||||
resourceUri: z.string().describe('URI of the resource to embed')
|
||||
}
|
||||
},
|
||||
async args => {
|
||||
const uri = args.resourceUri;
|
||||
return {
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: {
|
||||
type: 'resource',
|
||||
resource: {
|
||||
uri,
|
||||
mimeType: 'text/plain',
|
||||
text: 'Embedded resource content for testing.'
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: { type: 'text', text: 'Please process the embedded resource above.' }
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// Prompt with image
|
||||
mcpServer.registerPrompt(
|
||||
'test_prompt_with_image',
|
||||
{
|
||||
title: 'Prompt With Image',
|
||||
description: 'A prompt that includes image content'
|
||||
},
|
||||
async () => {
|
||||
return {
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: { type: 'image', data: TEST_IMAGE_BASE64, mimeType: 'image/png' }
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: { type: 'text', text: 'Please analyze the image above.' }
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
);
|
||||
|
||||
// ===== LOGGING =====
|
||||
|
||||
mcpServer.server.setRequestHandler(z.object({ method: z.literal('logging/setLevel') }).passthrough(), async (request: any) => {
|
||||
const level = request.params.level;
|
||||
sendLog('info', `Log level set to: ${level}`);
|
||||
return {};
|
||||
});
|
||||
|
||||
// ===== COMPLETION =====
|
||||
|
||||
mcpServer.server.setRequestHandler(z.object({ method: z.literal('completion/complete') }).passthrough(), async (_request: any) => {
|
||||
// Basic completion support - returns empty array for conformance
|
||||
// Real implementations would provide contextual suggestions
|
||||
return {
|
||||
completion: {
|
||||
values: [],
|
||||
total: 0,
|
||||
hasMore: false
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
return mcpServer;
|
||||
}
|
||||
|
||||
// Helper to check if request is an initialize request
|
||||
function isInitializeRequest(body: any): boolean {
|
||||
return body?.method === 'initialize';
|
||||
}
|
||||
|
||||
// ===== EXPRESS APP =====
|
||||
|
||||
const app = express();
|
||||
app.use(express.json());
|
||||
|
||||
// Configure CORS to expose Mcp-Session-Id header for browser-based clients
|
||||
app.use(
|
||||
cors({
|
||||
origin: '*', // Allow all origins
|
||||
exposedHeaders: ['Mcp-Session-Id'],
|
||||
allowedHeaders: ['Content-Type', 'mcp-session-id', 'last-event-id']
|
||||
})
|
||||
);
|
||||
|
||||
// Handle POST requests - stateful mode
|
||||
app.post('/mcp', async (req, res) => {
|
||||
const sessionId = req.headers['mcp-session-id'] as string | undefined;
|
||||
|
||||
try {
|
||||
let transport: StreamableHTTPServerTransport;
|
||||
|
||||
if (sessionId && transports[sessionId]) {
|
||||
// Reuse existing transport for established sessions
|
||||
transport = transports[sessionId];
|
||||
} else if (!sessionId && isInitializeRequest(req.body)) {
|
||||
// Create new transport for initialization requests
|
||||
const mcpServer = createMcpServer();
|
||||
|
||||
transport = new StreamableHTTPServerTransport({
|
||||
sessionIdGenerator: () => randomUUID(),
|
||||
onsessioninitialized: newSessionId => {
|
||||
transports[newSessionId] = transport;
|
||||
servers[newSessionId] = mcpServer;
|
||||
console.log(`Session initialized with ID: ${newSessionId}`);
|
||||
}
|
||||
});
|
||||
|
||||
transport.onclose = () => {
|
||||
const sid = transport.sessionId;
|
||||
if (sid && transports[sid]) {
|
||||
delete transports[sid];
|
||||
if (servers[sid]) {
|
||||
servers[sid].close();
|
||||
delete servers[sid];
|
||||
}
|
||||
console.log(`Session ${sid} closed`);
|
||||
}
|
||||
};
|
||||
|
||||
await mcpServer.connect(transport);
|
||||
await transport.handleRequest(req, res, req.body);
|
||||
return;
|
||||
} else {
|
||||
res.status(400).json({
|
||||
jsonrpc: '2.0',
|
||||
error: {
|
||||
code: -32000,
|
||||
message: 'Invalid or missing session ID'
|
||||
},
|
||||
id: null
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
await transport.handleRequest(req, res, req.body);
|
||||
} catch (error) {
|
||||
console.error('Error handling MCP request:', error);
|
||||
if (!res.headersSent) {
|
||||
res.status(500).json({
|
||||
jsonrpc: '2.0',
|
||||
error: {
|
||||
code: -32603,
|
||||
message: 'Internal server error'
|
||||
},
|
||||
id: null
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Handle GET requests - SSE streams for sessions
|
||||
app.get('/mcp', async (req, res) => {
|
||||
const sessionId = req.headers['mcp-session-id'] as string | undefined;
|
||||
|
||||
if (!sessionId || !transports[sessionId]) {
|
||||
res.status(400).send('Invalid or missing session ID');
|
||||
return;
|
||||
}
|
||||
|
||||
const lastEventId = req.headers['last-event-id'] as string | undefined;
|
||||
if (lastEventId) {
|
||||
console.log(`Client reconnecting with Last-Event-ID: ${lastEventId}`);
|
||||
} else {
|
||||
console.log(`Establishing SSE stream for session ${sessionId}`);
|
||||
}
|
||||
|
||||
try {
|
||||
const transport = transports[sessionId];
|
||||
await transport.handleRequest(req, res);
|
||||
} catch (error) {
|
||||
console.error('Error handling SSE stream:', error);
|
||||
if (!res.headersSent) {
|
||||
res.status(500).send('Error establishing SSE stream');
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Handle DELETE requests - session termination
|
||||
app.delete('/mcp', async (req, res) => {
|
||||
const sessionId = req.headers['mcp-session-id'] as string | undefined;
|
||||
|
||||
if (!sessionId || !transports[sessionId]) {
|
||||
res.status(400).send('Invalid or missing session ID');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`Received session termination request for session ${sessionId}`);
|
||||
|
||||
try {
|
||||
const transport = transports[sessionId];
|
||||
await transport.handleRequest(req, res);
|
||||
} catch (error) {
|
||||
console.error('Error handling termination:', error);
|
||||
if (!res.headersSent) {
|
||||
res.status(500).send('Error processing session termination');
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Start server
|
||||
const PORT = process.env.PORT || 3000;
|
||||
app.listen(PORT, () => {
|
||||
console.log(`MCP Conformance Test Server running on http://localhost:${PORT}`);
|
||||
console.log(` - MCP endpoint: http://localhost:${PORT}/mcp`);
|
||||
});
|
||||
1679
examples/servers/typescript/package-lock.json
generated
Normal file
1679
examples/servers/typescript/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load diff
27
examples/servers/typescript/package.json
Normal file
27
examples/servers/typescript/package.json
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
{
|
||||
"name": "mcp-conformance-test-server",
|
||||
"version": "1.0.0",
|
||||
"description": "Reference implementation of MCP server for conformance testing",
|
||||
"type": "module",
|
||||
"main": "everything-server.ts",
|
||||
"scripts": {
|
||||
"start": "tsx everything-server.ts",
|
||||
"dev": "tsx watch everything-server.ts"
|
||||
},
|
||||
"keywords": [
|
||||
"mcp",
|
||||
"model-context-protocol",
|
||||
"conformance",
|
||||
"testing"
|
||||
],
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "^1.20.1",
|
||||
"@types/cors": "^2.8.19",
|
||||
"cors": "^2.8.5",
|
||||
"express": "^5.1.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"tsx": "^4.7.0",
|
||||
"typescript": "^5.5.4"
|
||||
}
|
||||
}
|
||||
8
examples/servers/typescript/tsconfig.json
Normal file
8
examples/servers/typescript/tsconfig.json
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
{
|
||||
"extends": "../../../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"outDir": "./dist",
|
||||
"rootDir": "."
|
||||
},
|
||||
"include": ["*.ts"]
|
||||
}
|
||||
115
package-lock.json
generated
115
package-lock.json
generated
|
|
@ -16,6 +16,7 @@
|
|||
"devDependencies": {
|
||||
"@eslint/js": "^9.8.0",
|
||||
"@jest/globals": "^29.7.0",
|
||||
"@types/express": "^5.0.3",
|
||||
"@types/jest": "^29.5.8",
|
||||
"eslint": "^9.8.0",
|
||||
"eslint-config-prettier": "^10.1.8",
|
||||
|
|
@ -58,7 +59,6 @@
|
|||
"integrity": "sha512-e7jT4DxYvIDLk1ZHmU/m/mB19rex9sv0c2ftBtjSBv+kVM/902eh0fINUzD7UwLLNR+jU585GxUJ8/EBfAM5fw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/code-frame": "^7.27.1",
|
||||
"@babel/generator": "^7.28.5",
|
||||
|
|
@ -1787,6 +1787,27 @@
|
|||
"@babel/types": "^7.28.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/body-parser": {
|
||||
"version": "1.19.6",
|
||||
"resolved": "https://registry.npmjs.org/@types/body-parser/-/body-parser-1.19.6.tgz",
|
||||
"integrity": "sha512-HLFeCYgz89uk22N5Qg3dvGvsv46B8GLvKKo1zKG4NybA8U2DiEO3w9lqGg29t/tfLRJpJ6iQxnVw4OnB7MoM9g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/connect": "*",
|
||||
"@types/node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/connect": {
|
||||
"version": "3.4.38",
|
||||
"resolved": "https://registry.npmjs.org/@types/connect/-/connect-3.4.38.tgz",
|
||||
"integrity": "sha512-K6uROf1LD88uDQqJCktA4yzL1YYAK6NgfsI0v/mTgyPKWsX1CnJ0XPSDhViejru1GcRkLWb8RlzFYJRqGUbaug==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/estree": {
|
||||
"version": "1.0.8",
|
||||
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz",
|
||||
|
|
@ -1794,6 +1815,31 @@
|
|||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/express": {
|
||||
"version": "5.0.5",
|
||||
"resolved": "https://registry.npmjs.org/@types/express/-/express-5.0.5.tgz",
|
||||
"integrity": "sha512-LuIQOcb6UmnF7C1PCFmEU1u2hmiHL43fgFQX67sN3H4Z+0Yk0Neo++mFsBjhOAuLzvlQeqAAkeDOZrJs9rzumQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/body-parser": "*",
|
||||
"@types/express-serve-static-core": "^5.0.0",
|
||||
"@types/serve-static": "^1"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/express-serve-static-core": {
|
||||
"version": "5.1.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/express-serve-static-core/-/express-serve-static-core-5.1.0.tgz",
|
||||
"integrity": "sha512-jnHMsrd0Mwa9Cf4IdOzbz543y4XJepXrbia2T4b6+spXC2We3t1y6K44D3mR8XMFSXMCf3/l7rCgddfx7UNVBA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/node": "*",
|
||||
"@types/qs": "*",
|
||||
"@types/range-parser": "*",
|
||||
"@types/send": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/graceful-fs": {
|
||||
"version": "4.1.9",
|
||||
"resolved": "https://registry.npmjs.org/@types/graceful-fs/-/graceful-fs-4.1.9.tgz",
|
||||
|
|
@ -1804,6 +1850,13 @@
|
|||
"@types/node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/http-errors": {
|
||||
"version": "2.0.5",
|
||||
"resolved": "https://registry.npmjs.org/@types/http-errors/-/http-errors-2.0.5.tgz",
|
||||
"integrity": "sha512-r8Tayk8HJnX0FztbZN7oVqGccWgw98T/0neJphO91KkmOzug1KkofZURD4UaD5uH8AqcFLfdPErnBod0u71/qg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/istanbul-lib-coverage": {
|
||||
"version": "2.0.6",
|
||||
"resolved": "https://registry.npmjs.org/@types/istanbul-lib-coverage/-/istanbul-lib-coverage-2.0.6.tgz",
|
||||
|
|
@ -1849,6 +1902,13 @@
|
|||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/mime": {
|
||||
"version": "1.3.5",
|
||||
"resolved": "https://registry.npmjs.org/@types/mime/-/mime-1.3.5.tgz",
|
||||
"integrity": "sha512-/pyBZWSLD2n0dcHE3hq8s8ZvcETHtEuF+3E7XVt0Ig2nvsVQXdghHVcEkIWjy9A0wKfTn97a/PSDYohKIlnP/w==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/node": {
|
||||
"version": "24.10.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.10.0.tgz",
|
||||
|
|
@ -1859,6 +1919,53 @@
|
|||
"undici-types": "~7.16.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/qs": {
|
||||
"version": "6.14.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/qs/-/qs-6.14.0.tgz",
|
||||
"integrity": "sha512-eOunJqu0K1923aExK6y8p6fsihYEn/BYuQ4g0CxAAgFc4b/ZLN4CrsRZ55srTdqoiLzU2B2evC+apEIxprEzkQ==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/range-parser": {
|
||||
"version": "1.2.7",
|
||||
"resolved": "https://registry.npmjs.org/@types/range-parser/-/range-parser-1.2.7.tgz",
|
||||
"integrity": "sha512-hKormJbkJqzQGhziax5PItDUTMAM9uE2XXQmM37dyd4hVM+5aVl7oVxMVUiVQn2oCQFN/LKCZdvSM0pFRqbSmQ==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/send": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/@types/send/-/send-1.2.1.tgz",
|
||||
"integrity": "sha512-arsCikDvlU99zl1g69TcAB3mzZPpxgw0UQnaHeC1Nwb015xp8bknZv5rIfri9xTOcMuaVgvabfIRA7PSZVuZIQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/serve-static": {
|
||||
"version": "1.15.10",
|
||||
"resolved": "https://registry.npmjs.org/@types/serve-static/-/serve-static-1.15.10.tgz",
|
||||
"integrity": "sha512-tRs1dB+g8Itk72rlSI2ZrW6vZg0YrLI81iQSTkMmOqnqCaNr/8Ek4VwWcN5vZgCYWbg/JJSGBlUaYGAOP73qBw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/http-errors": "*",
|
||||
"@types/node": "*",
|
||||
"@types/send": "<1"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/serve-static/node_modules/@types/send": {
|
||||
"version": "0.17.6",
|
||||
"resolved": "https://registry.npmjs.org/@types/send/-/send-0.17.6.tgz",
|
||||
"integrity": "sha512-Uqt8rPBE8SY0RK8JB1EzVOIZ32uqy8HwdxCnoCOsYrvnswqmFZ/k+9Ikidlk/ImhsdvBsloHbAlewb2IEBV/Og==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/mime": "^1",
|
||||
"@types/node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/stack-utils": {
|
||||
"version": "2.0.3",
|
||||
"resolved": "https://registry.npmjs.org/@types/stack-utils/-/stack-utils-2.0.3.tgz",
|
||||
|
|
@ -2412,9 +2519,9 @@
|
|||
"license": "MIT"
|
||||
},
|
||||
"node_modules/baseline-browser-mapping": {
|
||||
"version": "2.8.23",
|
||||
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.8.23.tgz",
|
||||
"integrity": "sha512-616V5YX4bepJFzNyOfce5Fa8fDJMfoxzOIzDCZwaGL8MKVpFrXqfNUoIpRn9YMI5pXf/VKgzjB4htFMsFKKdiQ==",
|
||||
"version": "2.8.24",
|
||||
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.8.24.tgz",
|
||||
"integrity": "sha512-uUhTRDPXamakPyghwrUcjaGvvBqGrWvBHReoiULMIpOJVM9IYzQh83Xk2Onx5HlGI2o10NNCzcs9TG/S3TkwrQ==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@
|
|||
"type": "module",
|
||||
"scripts": {
|
||||
"start": "tsx src/runner.ts",
|
||||
"test:server": "tsx src/server-runner.ts",
|
||||
"test": "jest",
|
||||
"lint": "eslint src/ && prettier --check .",
|
||||
"lint:fix": "eslint src/ --fix && prettier --write .",
|
||||
|
|
@ -12,6 +13,7 @@
|
|||
"devDependencies": {
|
||||
"@eslint/js": "^9.8.0",
|
||||
"@jest/globals": "^29.7.0",
|
||||
"@types/express": "^5.0.3",
|
||||
"@types/jest": "^29.5.8",
|
||||
"eslint": "^9.8.0",
|
||||
"eslint-config-prettier": "^10.1.8",
|
||||
|
|
|
|||
|
|
@ -1,6 +1,4 @@
|
|||
// Namespaced exports for client and server checks
|
||||
// Namespaced exports for client checks
|
||||
import * as client from './client.js';
|
||||
import * as server from './server.js';
|
||||
|
||||
export const clientChecks = client;
|
||||
export const serverChecks = server;
|
||||
|
|
|
|||
|
|
@ -1,37 +0,0 @@
|
|||
import { ConformanceCheck, CheckStatus } from '../types.js';
|
||||
|
||||
export function createServerInitializationCheck(initializeResponse: any, expectedSpecVersion: string = '2025-06-18'): ConformanceCheck {
|
||||
const result = initializeResponse?.result;
|
||||
const protocolVersion = result?.protocolVersion;
|
||||
const serverInfo = result?.serverInfo;
|
||||
const capabilities = result?.capabilities;
|
||||
|
||||
const errors: string[] = [];
|
||||
if (!initializeResponse?.jsonrpc) errors.push('Missing jsonrpc field');
|
||||
if (!initializeResponse?.id) errors.push('Missing id field');
|
||||
if (!result) errors.push('Missing result field');
|
||||
if (!protocolVersion) errors.push('Missing protocolVersion in result');
|
||||
if (protocolVersion !== expectedSpecVersion)
|
||||
errors.push(`Protocol version mismatch: expected ${expectedSpecVersion}, got ${protocolVersion}`);
|
||||
if (!serverInfo) errors.push('Missing serverInfo in result');
|
||||
if (!serverInfo?.name) errors.push('Missing server name in serverInfo');
|
||||
if (!serverInfo?.version) errors.push('Missing server version in serverInfo');
|
||||
if (capabilities === undefined) errors.push('Missing capabilities in result');
|
||||
|
||||
const status: CheckStatus = errors.length === 0 ? 'SUCCESS' : 'FAILURE';
|
||||
|
||||
return {
|
||||
id: 'mcp-server-initialization',
|
||||
name: 'MCPServerInitialization',
|
||||
description: 'Validates that MCP server properly responds to initialize request',
|
||||
status,
|
||||
timestamp: new Date().toISOString(),
|
||||
specReferences: [{ id: 'MCP-Lifecycle', url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle' }],
|
||||
details: {
|
||||
expectedSpecVersion,
|
||||
response: initializeResponse
|
||||
},
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
logs: errors.length > 0 ? errors : undefined
|
||||
};
|
||||
}
|
||||
|
|
@ -1,4 +1,3 @@
|
|||
import { runConformanceTest } from './runner/index.js';
|
||||
|
||||
export { runConformanceTest };
|
||||
export type { ScenarioConfig } from './runner/index.js';
|
||||
|
|
|
|||
|
|
@ -2,7 +2,8 @@ import { Server } from '@modelcontextprotocol/sdk/server/index.js';
|
|||
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
|
||||
import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprotocol/sdk/types.js';
|
||||
import type { Scenario, ConformanceCheck } from '../../types.js';
|
||||
import express from 'express';
|
||||
// @ts-expect-error - express has incomplete type definitions
|
||||
import express, { Request, Response, NextFunction } from 'express';
|
||||
import { ScenarioUrls } from '../../types.js';
|
||||
|
||||
function createServer(checks: ConformanceCheck[]): express.Application {
|
||||
|
|
@ -83,7 +84,7 @@ function createServer(checks: ConformanceCheck[]): express.Application {
|
|||
const app = express();
|
||||
app.use(express.json());
|
||||
|
||||
app.use((req, res, next) => {
|
||||
app.use((req: Request, res: Response, next: NextFunction) => {
|
||||
// Log incoming requests for debugging
|
||||
// console.log(`Incoming request: ${req.method} ${req.url}`);
|
||||
checks.push({
|
||||
|
|
@ -105,13 +106,13 @@ function createServer(checks: ConformanceCheck[]): express.Application {
|
|||
status: 'INFO',
|
||||
timestamp: new Date().toISOString(),
|
||||
details: {
|
||||
// TODO: this isn't working
|
||||
body: JSON.stringify(res.body)
|
||||
// TODO: Response body not available in express middleware
|
||||
statusCode: res.statusCode
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
app.post('/mcp', async (req, res) => {
|
||||
app.post('/mcp', async (req: Request, res: Response) => {
|
||||
const transport = new StreamableHTTPServerTransport({
|
||||
sessionIdGenerator: undefined
|
||||
});
|
||||
|
|
@ -125,6 +126,7 @@ function createServer(checks: ConformanceCheck[]): express.Application {
|
|||
|
||||
export class ToolsCallScenario implements Scenario {
|
||||
name = 'tools_call';
|
||||
description = 'Tests calling tools with various parameter types';
|
||||
private app: express.Application | null = null;
|
||||
private httpServer: any = null;
|
||||
private checks: ConformanceCheck[] = [];
|
||||
|
|
|
|||
|
|
@ -1,14 +1,84 @@
|
|||
import { Scenario, ClientScenario } from '../types.js';
|
||||
import { InitializeScenario } from './client/initialize.js';
|
||||
import { ToolsCallScenario } from './client/tools_call.js';
|
||||
import { ServerInitializeClientScenario } from './server/server_initialize.js';
|
||||
|
||||
// Import all new server test scenarios
|
||||
import { ServerInitializeScenario } from './server/lifecycle.js';
|
||||
|
||||
import { LoggingSetLevelScenario, CompletionCompleteScenario } from './server/utils.js';
|
||||
|
||||
import {
|
||||
ToolsListScenario,
|
||||
ToolsCallSimpleTextScenario,
|
||||
ToolsCallImageScenario,
|
||||
ToolsCallMultipleContentTypesScenario,
|
||||
ToolsCallWithLoggingScenario,
|
||||
ToolsCallErrorScenario,
|
||||
ToolsCallWithProgressScenario,
|
||||
ToolsCallSamplingScenario,
|
||||
ToolsCallElicitationScenario,
|
||||
ToolsCallAudioScenario,
|
||||
ToolsCallEmbeddedResourceScenario
|
||||
} from './server/tools.js';
|
||||
|
||||
import {
|
||||
ResourcesListScenario,
|
||||
ResourcesReadTextScenario,
|
||||
ResourcesReadBinaryScenario,
|
||||
ResourcesTemplateReadScenario,
|
||||
ResourcesSubscribeScenario,
|
||||
ResourcesUnsubscribeScenario
|
||||
} from './server/resources.js';
|
||||
|
||||
import {
|
||||
PromptsListScenario,
|
||||
PromptsGetSimpleScenario,
|
||||
PromptsGetWithArgsScenario,
|
||||
PromptsGetEmbeddedResourceScenario,
|
||||
PromptsGetWithImageScenario
|
||||
} from './server/prompts.js';
|
||||
|
||||
export const scenarios = new Map<string, Scenario>([
|
||||
['initialize', new InitializeScenario()],
|
||||
['tools-call', new ToolsCallScenario()]
|
||||
]);
|
||||
|
||||
export const clientScenarios = new Map<string, ClientScenario>([['initialize', new ServerInitializeClientScenario()]]);
|
||||
export const clientScenarios = new Map<string, ClientScenario>([
|
||||
// Lifecycle scenarios
|
||||
['server-initialize', new ServerInitializeScenario()],
|
||||
|
||||
// Utilities scenarios
|
||||
['logging-set-level', new LoggingSetLevelScenario()],
|
||||
['completion-complete', new CompletionCompleteScenario()],
|
||||
|
||||
// Tools scenarios
|
||||
['tools-list', new ToolsListScenario()],
|
||||
['tools-call-simple-text', new ToolsCallSimpleTextScenario()],
|
||||
['tools-call-image', new ToolsCallImageScenario()],
|
||||
['tools-call-audio', new ToolsCallAudioScenario()],
|
||||
['tools-call-embedded-resource', new ToolsCallEmbeddedResourceScenario()],
|
||||
['tools-call-mixed-content', new ToolsCallMultipleContentTypesScenario()],
|
||||
['tools-call-with-logging', new ToolsCallWithLoggingScenario()],
|
||||
['tools-call-error', new ToolsCallErrorScenario()],
|
||||
['tools-call-with-progress', new ToolsCallWithProgressScenario()],
|
||||
['tools-call-sampling', new ToolsCallSamplingScenario()],
|
||||
['tools-call-elicitation', new ToolsCallElicitationScenario()],
|
||||
|
||||
// Resources scenarios
|
||||
['resources-list', new ResourcesListScenario()],
|
||||
['resources-read-text', new ResourcesReadTextScenario()],
|
||||
['resources-read-binary', new ResourcesReadBinaryScenario()],
|
||||
['resources-templates-read', new ResourcesTemplateReadScenario()],
|
||||
['resources-subscribe', new ResourcesSubscribeScenario()],
|
||||
['resources-unsubscribe', new ResourcesUnsubscribeScenario()],
|
||||
|
||||
// Prompts scenarios
|
||||
['prompts-list', new PromptsListScenario()],
|
||||
['prompts-get-simple', new PromptsGetSimpleScenario()],
|
||||
['prompts-get-with-args', new PromptsGetWithArgsScenario()],
|
||||
['prompts-get-embedded-resource', new PromptsGetEmbeddedResourceScenario()],
|
||||
['prompts-get-with-image', new PromptsGetWithImageScenario()]
|
||||
]);
|
||||
|
||||
export function registerScenario(name: string, scenario: Scenario): void {
|
||||
scenarios.set(name, scenario);
|
||||
|
|
|
|||
83
src/scenarios/server/client-helper.ts
Normal file
83
src/scenarios/server/client-helper.ts
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
/**
|
||||
* Helper utilities for creating MCP clients to test servers
|
||||
*/
|
||||
|
||||
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
|
||||
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
|
||||
import { LoggingMessageNotificationSchema, ProgressNotificationSchema } from '@modelcontextprotocol/sdk/types.js';
|
||||
|
||||
export interface MCPClientConnection {
|
||||
client: Client;
|
||||
close: () => Promise<void>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create and connect an MCP client to a server
|
||||
*/
|
||||
export async function connectToServer(serverUrl: string): Promise<MCPClientConnection> {
|
||||
const client = new Client(
|
||||
{
|
||||
name: 'conformance-test-client',
|
||||
version: '1.0.0'
|
||||
},
|
||||
{
|
||||
capabilities: {
|
||||
// Client capabilities
|
||||
sampling: {},
|
||||
elicitation: {}
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
const transport = new StreamableHTTPClientTransport(new URL(serverUrl));
|
||||
|
||||
await client.connect(transport);
|
||||
|
||||
return {
|
||||
client,
|
||||
close: async () => {
|
||||
await client.close();
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Helper to collect notifications (logging and progress)
|
||||
*/
|
||||
export class NotificationCollector {
|
||||
private loggingNotifications: any[] = [];
|
||||
private progressNotifications: any[] = [];
|
||||
|
||||
constructor(client: Client) {
|
||||
// Set up notification handler for logging messages
|
||||
client.setNotificationHandler(LoggingMessageNotificationSchema, notification => {
|
||||
this.loggingNotifications.push(notification);
|
||||
});
|
||||
|
||||
// Set up notification handler for progress notifications
|
||||
client.setNotificationHandler(ProgressNotificationSchema, notification => {
|
||||
this.progressNotifications.push(notification);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all collected logging notifications
|
||||
*/
|
||||
getLoggingNotifications(): any[] {
|
||||
return this.loggingNotifications;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all collected progress notifications
|
||||
*/
|
||||
getProgressNotifications(): any[] {
|
||||
return this.progressNotifications;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all notifications (for backward compatibility)
|
||||
*/
|
||||
getNotifications(): any[] {
|
||||
return this.loggingNotifications;
|
||||
}
|
||||
}
|
||||
58
src/scenarios/server/lifecycle.ts
Normal file
58
src/scenarios/server/lifecycle.ts
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
/**
|
||||
* Lifecycle test scenarios for MCP servers
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../types.js';
|
||||
import { connectToServer } from './client-helper.js';
|
||||
|
||||
export class ServerInitializeScenario implements ClientScenario {
|
||||
name = 'server-initialize';
|
||||
description = 'Test basic server initialization handshake';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
// The connection process already does initialization
|
||||
// Check that we have a connected client
|
||||
checks.push({
|
||||
id: 'server-initialize',
|
||||
name: 'ServerInitialize',
|
||||
description: 'Server responds to initialize request with valid structure',
|
||||
status: 'SUCCESS',
|
||||
timestamp: new Date().toISOString(),
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Initialize',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle#initialization'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
serverUrl,
|
||||
connected: true
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'server-initialize',
|
||||
name: 'ServerInitialize',
|
||||
description: 'Server responds to initialize request with valid structure',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed to initialize: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Initialize',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle#initialization'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
350
src/scenarios/server/prompts.ts
Normal file
350
src/scenarios/server/prompts.ts
Normal file
|
|
@ -0,0 +1,350 @@
|
|||
/**
|
||||
* Prompts test scenarios for MCP servers
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../types.js';
|
||||
import { connectToServer } from './client-helper.js';
|
||||
|
||||
export class PromptsListScenario implements ClientScenario {
|
||||
name = 'prompts-list';
|
||||
description = 'Test listing available prompts';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.listPrompts();
|
||||
|
||||
// Validate response structure
|
||||
const errors: string[] = [];
|
||||
if (!result.prompts) {
|
||||
errors.push('Missing prompts array');
|
||||
} else {
|
||||
if (!Array.isArray(result.prompts)) {
|
||||
errors.push('prompts is not an array');
|
||||
}
|
||||
|
||||
result.prompts.forEach((prompt, index) => {
|
||||
if (!prompt.name) errors.push(`Prompt ${index}: missing name`);
|
||||
if (!prompt.description) errors.push(`Prompt ${index}: missing description`);
|
||||
});
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'prompts-list',
|
||||
name: 'PromptsList',
|
||||
description: 'Server lists available prompts with valid structure',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-List',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#listing-prompts'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
promptCount: result.prompts?.length || 0,
|
||||
prompts: result.prompts?.map(p => p.name)
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'prompts-list',
|
||||
name: 'PromptsList',
|
||||
description: 'Server lists available prompts with valid structure',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-List',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#listing-prompts'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class PromptsGetSimpleScenario implements ClientScenario {
|
||||
name = 'prompts-get-simple';
|
||||
description = 'Test getting a simple prompt without arguments';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.getPrompt({
|
||||
name: 'test_simple_prompt'
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
if (!result.messages) errors.push('Missing messages array');
|
||||
if (!Array.isArray(result.messages)) errors.push('messages is not an array');
|
||||
if (result.messages.length === 0) errors.push('messages array is empty');
|
||||
|
||||
result.messages.forEach((message: any, index: number) => {
|
||||
if (!message.role) errors.push(`Message ${index}: missing role`);
|
||||
if (!message.content) errors.push(`Message ${index}: missing content`);
|
||||
});
|
||||
|
||||
checks.push({
|
||||
id: 'prompts-get-simple',
|
||||
name: 'PromptsGetSimple',
|
||||
description: 'Get simple prompt successfully',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-Get',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#getting-prompts'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
messageCount: result.messages?.length || 0
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'prompts-get-simple',
|
||||
name: 'PromptsGetSimple',
|
||||
description: 'Get simple prompt successfully',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-Get',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#getting-prompts'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class PromptsGetWithArgsScenario implements ClientScenario {
|
||||
name = 'prompts-get-with-args';
|
||||
description = 'Test parameterized prompt';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.getPrompt({
|
||||
name: 'test_prompt_with_arguments',
|
||||
arguments: {
|
||||
arg1: 'testValue1',
|
||||
arg2: 'testValue2'
|
||||
}
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
if (!result.messages) errors.push('Missing messages array');
|
||||
if (result.messages.length === 0) errors.push('messages array is empty');
|
||||
|
||||
// Check if arguments were substituted
|
||||
const messageText = JSON.stringify(result.messages);
|
||||
if (!messageText.includes('testValue1')) {
|
||||
errors.push('arg1 not substituted in prompt');
|
||||
}
|
||||
if (!messageText.includes('testValue2')) {
|
||||
errors.push('arg2 not substituted in prompt');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'prompts-get-with-args',
|
||||
name: 'PromptsGetWithArgs',
|
||||
description: 'Get parameterized prompt with argument substitution',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-Get',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#getting-prompts'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
messageCount: result.messages?.length || 0,
|
||||
messages: result.messages
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'prompts-get-with-args',
|
||||
name: 'PromptsGetWithArgs',
|
||||
description: 'Get parameterized prompt with argument substitution',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-Get',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#getting-prompts'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class PromptsGetEmbeddedResourceScenario implements ClientScenario {
|
||||
name = 'prompts-get-embedded-resource';
|
||||
description = 'Test prompt with embedded resource content';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.getPrompt({
|
||||
name: 'test_prompt_with_embedded_resource',
|
||||
arguments: {
|
||||
resourceUri: 'test://example-resource'
|
||||
}
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
if (!result.messages) errors.push('Missing messages array');
|
||||
|
||||
// Look for resource content
|
||||
const hasResourceContent = result.messages.some(
|
||||
(msg: any) => msg.content?.type === 'resource' || msg.content?.resource !== undefined
|
||||
);
|
||||
|
||||
if (!hasResourceContent) {
|
||||
errors.push('No embedded resource found in prompt');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'prompts-get-embedded-resource',
|
||||
name: 'PromptsGetEmbeddedResource',
|
||||
description: 'Get prompt with embedded resource',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-Embedded-Resources',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#embedded-resources'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
messageCount: result.messages?.length || 0,
|
||||
messages: result.messages
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'prompts-get-embedded-resource',
|
||||
name: 'PromptsGetEmbeddedResource',
|
||||
description: 'Get prompt with embedded resource',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-Embedded-Resources',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#embedded-resources'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class PromptsGetWithImageScenario implements ClientScenario {
|
||||
name = 'prompts-get-with-image';
|
||||
description = 'Test prompt with image content';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.getPrompt({
|
||||
name: 'test_prompt_with_image'
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
if (!result.messages) errors.push('Missing messages array');
|
||||
|
||||
// Look for image content
|
||||
const hasImageContent = result.messages.some(
|
||||
(msg: any) => msg.content?.type === 'image' && msg.content?.data && msg.content?.mimeType
|
||||
);
|
||||
|
||||
if (!hasImageContent) {
|
||||
errors.push('No image content found in prompt');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'prompts-get-with-image',
|
||||
name: 'PromptsGetWithImage',
|
||||
description: 'Get prompt with image content',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-Image',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#image-content'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
messageCount: result.messages?.length || 0
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'prompts-get-with-image',
|
||||
name: 'PromptsGetWithImage',
|
||||
description: 'Get prompt with image content',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Prompts-Image',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/prompts#image-content'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
389
src/scenarios/server/resources.ts
Normal file
389
src/scenarios/server/resources.ts
Normal file
|
|
@ -0,0 +1,389 @@
|
|||
/**
|
||||
* Resources test scenarios for MCP servers
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../types.js';
|
||||
import { connectToServer } from './client-helper.js';
|
||||
|
||||
export class ResourcesListScenario implements ClientScenario {
|
||||
name = 'resources-list';
|
||||
description = 'Test listing available resources';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.listResources();
|
||||
|
||||
// Validate response structure
|
||||
const errors: string[] = [];
|
||||
if (!result.resources) {
|
||||
errors.push('Missing resources array');
|
||||
} else {
|
||||
if (!Array.isArray(result.resources)) {
|
||||
errors.push('resources is not an array');
|
||||
}
|
||||
|
||||
result.resources.forEach((resource, index) => {
|
||||
if (!resource.uri) errors.push(`Resource ${index}: missing uri`);
|
||||
if (!resource.name) errors.push(`Resource ${index}: missing name`);
|
||||
});
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'resources-list',
|
||||
name: 'ResourcesList',
|
||||
description: 'Server lists available resources with valid structure',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-List',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#listing-resources'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
resourceCount: result.resources?.length || 0,
|
||||
resources: result.resources?.map(r => r.uri)
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'resources-list',
|
||||
name: 'ResourcesList',
|
||||
description: 'Server lists available resources with valid structure',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-List',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#listing-resources'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ResourcesReadTextScenario implements ClientScenario {
|
||||
name = 'resources-read-text';
|
||||
description = 'Test reading text resource';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.readResource({
|
||||
uri: 'test://static-text'
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
if (!result.contents) errors.push('Missing contents array');
|
||||
if (!Array.isArray(result.contents)) errors.push('contents is not an array');
|
||||
if (result.contents.length === 0) errors.push('contents array is empty');
|
||||
|
||||
const content = result.contents[0];
|
||||
if (content) {
|
||||
if (!content.uri) errors.push('Content missing uri');
|
||||
if (!content.mimeType) errors.push('Content missing mimeType');
|
||||
if (!content.text) errors.push('Content missing text field');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'resources-read-text',
|
||||
name: 'ResourcesReadText',
|
||||
description: 'Read text resource successfully',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Read',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#reading-resources'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
uri: content?.uri,
|
||||
mimeType: content?.mimeType,
|
||||
hasText: !!content?.text
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'resources-read-text',
|
||||
name: 'ResourcesReadText',
|
||||
description: 'Read text resource successfully',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Read',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#reading-resources'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ResourcesReadBinaryScenario implements ClientScenario {
|
||||
name = 'resources-read-binary';
|
||||
description = 'Test reading binary resource';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.readResource({
|
||||
uri: 'test://static-binary'
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
if (!result.contents) errors.push('Missing contents array');
|
||||
if (result.contents.length === 0) errors.push('contents array is empty');
|
||||
|
||||
const content = result.contents[0];
|
||||
if (content) {
|
||||
if (!content.uri) errors.push('Content missing uri');
|
||||
if (!content.mimeType) errors.push('Content missing mimeType');
|
||||
if (!content.blob) errors.push('Content missing blob field');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'resources-read-binary',
|
||||
name: 'ResourcesReadBinary',
|
||||
description: 'Read binary resource successfully',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Read',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#reading-resources'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
uri: content?.uri,
|
||||
mimeType: content?.mimeType,
|
||||
hasBlob: !!content?.blob
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'resources-read-binary',
|
||||
name: 'ResourcesReadBinary',
|
||||
description: 'Read binary resource successfully',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Read',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#reading-resources'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ResourcesTemplateReadScenario implements ClientScenario {
|
||||
name = 'resources-templates-read';
|
||||
description = 'Test reading resource from template';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.readResource({
|
||||
uri: 'test://template/123/data'
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
if (!result.contents) errors.push('Missing contents array');
|
||||
if (result.contents.length === 0) errors.push('contents array is empty');
|
||||
|
||||
const content = result.contents[0];
|
||||
if (content) {
|
||||
if (!content.uri) errors.push('Content missing uri');
|
||||
if (!content.text && !content.blob) errors.push('Content missing text or blob');
|
||||
|
||||
// Check if parameter was substituted
|
||||
const text = content.text || (content.blob ? '[binary]' : '');
|
||||
if (typeof text === 'string' && !text.includes('123')) {
|
||||
errors.push('Parameter substitution not reflected in content');
|
||||
}
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'resources-templates-read',
|
||||
name: 'ResourcesTemplateRead',
|
||||
description: 'Read resource from template with parameter substitution',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Templates',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-templates'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
uri: content?.uri,
|
||||
content: content?.text || content?.blob
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'resources-templates-read',
|
||||
name: 'ResourcesTemplateRead',
|
||||
description: 'Read resource from template with parameter substitution',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Templates',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-templates'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ResourcesSubscribeScenario implements ClientScenario {
|
||||
name = 'resources-subscribe';
|
||||
description = 'Test subscribing to resource updates';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
await connection.client.subscribeResource({
|
||||
uri: 'test://watched-resource'
|
||||
});
|
||||
|
||||
checks.push({
|
||||
id: 'resources-subscribe',
|
||||
name: 'ResourcesSubscribe',
|
||||
description: 'Subscribe to resource successfully',
|
||||
status: 'SUCCESS',
|
||||
timestamp: new Date().toISOString(),
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Subscribe',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-subscriptions'
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'resources-subscribe',
|
||||
name: 'ResourcesSubscribe',
|
||||
description: 'Subscribe to resource successfully',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Subscribe',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-subscriptions'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ResourcesUnsubscribeScenario implements ClientScenario {
|
||||
name = 'resources-unsubscribe';
|
||||
description = 'Test unsubscribing from resource';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
// First subscribe
|
||||
await connection.client.subscribeResource({
|
||||
uri: 'test://watched-resource'
|
||||
});
|
||||
|
||||
// Then unsubscribe
|
||||
await connection.client.unsubscribeResource({
|
||||
uri: 'test://watched-resource'
|
||||
});
|
||||
|
||||
checks.push({
|
||||
id: 'resources-unsubscribe',
|
||||
name: 'ResourcesUnsubscribe',
|
||||
description: 'Unsubscribe from resource successfully',
|
||||
status: 'SUCCESS',
|
||||
timestamp: new Date().toISOString(),
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Subscribe',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-subscriptions'
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'resources-unsubscribe',
|
||||
name: 'ResourcesUnsubscribe',
|
||||
description: 'Unsubscribe from resource successfully',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Resources-Subscribe',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/resources#resource-subscriptions'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
807
src/scenarios/server/tools.ts
Normal file
807
src/scenarios/server/tools.ts
Normal file
|
|
@ -0,0 +1,807 @@
|
|||
/**
|
||||
* Tools test scenarios for MCP servers
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../types.js';
|
||||
import { connectToServer, NotificationCollector } from './client-helper.js';
|
||||
import { CallToolResultSchema, CreateMessageRequestSchema, ElicitRequestSchema, Progress } from '@modelcontextprotocol/sdk/types.js';
|
||||
|
||||
export class ToolsListScenario implements ClientScenario {
|
||||
name = 'tools-list';
|
||||
description = 'Test listing available tools';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.listTools();
|
||||
|
||||
// Validate response structure
|
||||
const errors: string[] = [];
|
||||
if (!result.tools) {
|
||||
errors.push('Missing tools array');
|
||||
} else {
|
||||
if (!Array.isArray(result.tools)) {
|
||||
errors.push('tools is not an array');
|
||||
}
|
||||
|
||||
result.tools.forEach((tool, index) => {
|
||||
if (!tool.name) errors.push(`Tool ${index}: missing name`);
|
||||
if (!tool.description) errors.push(`Tool ${index}: missing description`);
|
||||
if (!tool.inputSchema) errors.push(`Tool ${index}: missing inputSchema`);
|
||||
});
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'tools-list',
|
||||
name: 'ToolsList',
|
||||
description: 'Server lists available tools with valid structure',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-List',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#listing-tools'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
toolCount: result.tools?.length || 0,
|
||||
tools: result.tools?.map(t => t.name)
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-list',
|
||||
name: 'ToolsList',
|
||||
description: 'Server lists available tools with valid structure',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-List',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#listing-tools'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallSimpleTextScenario implements ClientScenario {
|
||||
name = 'tools-call-simple-text';
|
||||
description = 'Test calling a tool that returns simple text';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.callTool({
|
||||
name: 'test_simple_text',
|
||||
arguments: {}
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
const content = (result as any).content;
|
||||
if (!content) errors.push('Missing content array');
|
||||
if (!Array.isArray(content)) errors.push('content is not an array');
|
||||
if (content && content.length === 0) errors.push('content array is empty');
|
||||
|
||||
const textContent = content && content.find((c: any) => c.type === 'text');
|
||||
if (!textContent) errors.push('No text content found');
|
||||
if (textContent && !textContent.text) errors.push('Text content missing text field');
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-simple-text',
|
||||
name: 'ToolsCallSimpleText',
|
||||
description: 'Tool returns simple text content',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
result
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-simple-text',
|
||||
name: 'ToolsCallSimpleText',
|
||||
description: 'Tool returns simple text content',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallImageScenario implements ClientScenario {
|
||||
name = 'tools-call-image';
|
||||
description = 'Test calling a tool that returns image content';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.callTool({
|
||||
name: 'test_image_content',
|
||||
arguments: {}
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
const content = (result as any).content;
|
||||
if (!content) errors.push('Missing content array');
|
||||
|
||||
const imageContent = content && content.find((c: any) => c.type === 'image');
|
||||
if (!imageContent) errors.push('No image content found');
|
||||
if (imageContent && !imageContent.data) errors.push('Image content missing data field');
|
||||
if (imageContent && !imageContent.mimeType) errors.push('Image content missing mimeType');
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-image',
|
||||
name: 'ToolsCallImage',
|
||||
description: 'Tool returns image content',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
mimeType: imageContent?.mimeType,
|
||||
hasData: !!imageContent?.data
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-image',
|
||||
name: 'ToolsCallImage',
|
||||
description: 'Tool returns image content',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallMultipleContentTypesScenario implements ClientScenario {
|
||||
name = 'tools-call-mixed-content';
|
||||
description = 'Test tool returning multiple content types';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.callTool({
|
||||
name: 'test_multiple_content_types',
|
||||
arguments: {}
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
const content = (result as any).content;
|
||||
if (!content) errors.push('Missing content array');
|
||||
if (content && content.length < 2) errors.push('Expected multiple content items');
|
||||
|
||||
const hasText = content && content.some((c: any) => c.type === 'text');
|
||||
const hasImage = content && content.some((c: any) => c.type === 'image');
|
||||
const hasResource = content && content.some((c: any) => c.type === 'resource');
|
||||
|
||||
if (!hasText) errors.push('Missing text content');
|
||||
if (!hasImage) errors.push('Missing image content');
|
||||
if (!hasResource) errors.push('Missing resource content');
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-mixed-content',
|
||||
name: 'ToolsCallMixedContent',
|
||||
description: 'Tool returns multiple content types',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
contentCount: content ? content.length : 0,
|
||||
contentTypes: content ? content.map((c: any) => c.type) : []
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-mixed-content',
|
||||
name: 'ToolsCallMixedContent',
|
||||
description: 'Tool returns multiple content types',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallWithLoggingScenario implements ClientScenario {
|
||||
name = 'tools-call-with-logging';
|
||||
description = 'Test tool that sends log messages during execution';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
const notifications = new NotificationCollector(connection.client);
|
||||
|
||||
await connection.client.callTool({
|
||||
name: 'test_tool_with_logging',
|
||||
arguments: {}
|
||||
});
|
||||
|
||||
// Wait a bit for notifications to arrive
|
||||
await new Promise(resolve => setTimeout(resolve, 200));
|
||||
|
||||
// Check for log notifications from the NotificationCollector
|
||||
const logNotifications = notifications.getNotifications();
|
||||
|
||||
const errors: string[] = [];
|
||||
if (logNotifications.length === 0) {
|
||||
errors.push('No log notifications received');
|
||||
} else if (logNotifications.length < 3) {
|
||||
errors.push(`Expected at least 3 log messages, got ${logNotifications.length}`);
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-with-logging',
|
||||
name: 'ToolsCallWithLogging',
|
||||
description: 'Tool sends log messages during execution',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Logging',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/logging'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
logCount: logNotifications.length,
|
||||
logs: logNotifications.map((n: any) => n.params)
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-with-logging',
|
||||
name: 'ToolsCallWithLogging',
|
||||
description: 'Tool sends log messages during execution',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Logging',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/logging'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallErrorScenario implements ClientScenario {
|
||||
name = 'tools-call-error';
|
||||
description = 'Test tool error reporting';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result: any = await connection.client.callTool({
|
||||
name: 'test_error_handling',
|
||||
arguments: {}
|
||||
});
|
||||
|
||||
// Check if result has isError flag
|
||||
const hasIsError = result.isError === true;
|
||||
const hasContent = result.content && result.content.length > 0;
|
||||
const hasErrorMessage = hasContent && result.content[0].text;
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-error',
|
||||
name: 'ToolsCallError',
|
||||
description: 'Tool returns error correctly',
|
||||
status: hasIsError && hasErrorMessage ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: !hasIsError
|
||||
? 'Tool did not return isError: true'
|
||||
: !hasErrorMessage
|
||||
? 'Error result missing error message'
|
||||
: undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Error-Handling',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
result
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-error',
|
||||
name: 'ToolsCallError',
|
||||
description: 'Tool returns error correctly',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Error-Handling',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallWithProgressScenario implements ClientScenario {
|
||||
name = 'tools-call-with-progress';
|
||||
description = 'Test tool that reports progress notifications';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
const progressUpdates: Array<Progress> = [];
|
||||
// TODO: investigate why await connection.client.callTool didn't work for progress.
|
||||
const result = await connection.client.request(
|
||||
{
|
||||
method: 'tools/call',
|
||||
params: {
|
||||
name: 'test_tool_with_progress',
|
||||
arguments: {},
|
||||
_meta: {
|
||||
progressToken: 'progress-test-1'
|
||||
}
|
||||
}
|
||||
},
|
||||
CallToolResultSchema,
|
||||
{
|
||||
onprogress: progress => {
|
||||
progressUpdates.push(progress);
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
const errors: string[] = [];
|
||||
if (progressUpdates.length === 0) {
|
||||
errors.push('No progress notifications received');
|
||||
} else if (progressUpdates.length < 3) {
|
||||
errors.push(`Expected at least 3 progress notifications, got ${progressUpdates.length}`);
|
||||
}
|
||||
|
||||
if (progressUpdates.length >= 3) {
|
||||
const progress0 = progressUpdates[0].progress;
|
||||
const progress1 = progressUpdates[1].progress;
|
||||
const progress2 = progressUpdates[2].progress;
|
||||
|
||||
if (!(progress0 <= progress1 && progress1 <= progress2)) {
|
||||
errors.push('Progress values should be increasing');
|
||||
}
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-with-progress',
|
||||
name: 'ToolsCallWithProgress',
|
||||
description: 'Tool reports progress notifications',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Progress',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/progress'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
progressCount: progressUpdates.length,
|
||||
progressNotifications: progressUpdates.map((n: Progress) => n),
|
||||
result
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-with-progress',
|
||||
name: 'ToolsCallWithProgress',
|
||||
description: 'Tool reports progress notifications',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Progress',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/progress'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallSamplingScenario implements ClientScenario {
|
||||
name = 'tools-call-sampling';
|
||||
description = 'Test tool that requests LLM sampling from client';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
let samplingRequested = false;
|
||||
connection.client.setRequestHandler(CreateMessageRequestSchema, async _request => {
|
||||
samplingRequested = true;
|
||||
return {
|
||||
role: 'assistant',
|
||||
content: {
|
||||
type: 'text',
|
||||
text: 'This is a test response from the client'
|
||||
},
|
||||
model: 'test-model',
|
||||
stopReason: 'endTurn'
|
||||
};
|
||||
});
|
||||
|
||||
const result = await connection.client.callTool({
|
||||
name: 'test_sampling',
|
||||
arguments: {
|
||||
prompt: 'Test prompt for sampling'
|
||||
}
|
||||
});
|
||||
|
||||
const errors: string[] = [];
|
||||
if (!samplingRequested) {
|
||||
errors.push('Server did not request sampling from client');
|
||||
}
|
||||
|
||||
const content = (result as any).content;
|
||||
if (!content || content.length === 0) {
|
||||
errors.push('Tool did not return content');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-sampling',
|
||||
name: 'ToolsCallSampling',
|
||||
description: 'Tool requests LLM sampling from client',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Sampling',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/sampling'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
samplingRequested,
|
||||
result
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-sampling',
|
||||
name: 'ToolsCallSampling',
|
||||
description: 'Tool requests LLM sampling from client',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Sampling',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/sampling'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallElicitationScenario implements ClientScenario {
|
||||
name = 'tools-call-elicitation';
|
||||
description = 'Test tool that requests user input (elicitation) from client';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
let elicitationRequested = false;
|
||||
connection.client.setRequestHandler(ElicitRequestSchema, async _request => {
|
||||
elicitationRequested = true;
|
||||
return {
|
||||
action: 'accept',
|
||||
content: {
|
||||
username: 'testuser',
|
||||
email: 'test@example.com'
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
const result = await connection.client.callTool({
|
||||
name: 'test_elicitation',
|
||||
arguments: {
|
||||
message: 'Please provide your information'
|
||||
}
|
||||
});
|
||||
|
||||
const errors: string[] = [];
|
||||
if (!elicitationRequested) {
|
||||
errors.push('Server did not request elicitation from client');
|
||||
}
|
||||
|
||||
const content = (result as any).content;
|
||||
if (!content || content.length === 0) {
|
||||
errors.push('Tool did not return content');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-elicitation',
|
||||
name: 'ToolsCallElicitation',
|
||||
description: 'Tool requests user input from client',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Elicitation',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/elicitation'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
elicitationRequested,
|
||||
result
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-elicitation',
|
||||
name: 'ToolsCallElicitation',
|
||||
description: 'Tool requests user input from client',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Elicitation',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/elicitation'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallAudioScenario implements ClientScenario {
|
||||
name = 'tools-call-audio';
|
||||
description = 'Test calling a tool that returns audio content';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.callTool({
|
||||
name: 'test_audio_content',
|
||||
arguments: {}
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
const content = (result as any).content;
|
||||
if (!content) errors.push('Missing content array');
|
||||
if (!Array.isArray(content)) errors.push('content is not an array');
|
||||
if (content && content.length === 0) errors.push('content array is empty');
|
||||
|
||||
const audioContent = content && content.find((c: any) => c.type === 'audio');
|
||||
if (!audioContent) errors.push('No audio content found');
|
||||
if (audioContent && !audioContent.data) errors.push('Audio content missing data field');
|
||||
if (audioContent && !audioContent.mimeType) errors.push('Audio content missing mimeType field');
|
||||
if (audioContent && audioContent.mimeType !== 'audio/wav')
|
||||
errors.push(`Expected mimeType 'audio/wav', got '${audioContent.mimeType}'`);
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-audio',
|
||||
name: 'ToolsCallAudio',
|
||||
description: 'Tool returns audio content',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
hasAudioContent: !!audioContent,
|
||||
audioDataLength: audioContent?.data?.length || 0
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-audio',
|
||||
name: 'ToolsCallAudio',
|
||||
description: 'Tool returns audio content',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class ToolsCallEmbeddedResourceScenario implements ClientScenario {
|
||||
name = 'tools-call-embedded-resource';
|
||||
description = 'Test calling a tool that returns embedded resource content';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
const result = await connection.client.callTool({
|
||||
name: 'test_embedded_resource',
|
||||
arguments: {}
|
||||
});
|
||||
|
||||
// Validate response
|
||||
const errors: string[] = [];
|
||||
const content = (result as any).content;
|
||||
if (!content) errors.push('Missing content array');
|
||||
if (!Array.isArray(content)) errors.push('content is not an array');
|
||||
if (content && content.length === 0) errors.push('content array is empty');
|
||||
|
||||
const resourceContent = content && content.find((c: any) => c.type === 'resource');
|
||||
if (!resourceContent) errors.push('No resource content found');
|
||||
if (resourceContent && !resourceContent.resource) errors.push('Resource content missing resource field');
|
||||
if (resourceContent?.resource) {
|
||||
if (!resourceContent.resource.uri) errors.push('Resource missing uri field');
|
||||
if (!resourceContent.resource.mimeType) errors.push('Resource missing mimeType field');
|
||||
if (!resourceContent.resource.text && !resourceContent.resource.blob) {
|
||||
errors.push('Resource missing both text and blob fields');
|
||||
}
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'tools-call-embedded-resource',
|
||||
name: 'ToolsCallEmbeddedResource',
|
||||
description: 'Tool returns embedded resource content',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
hasResourceContent: !!resourceContent,
|
||||
resourceUri: resourceContent?.resource?.uri
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'tools-call-embedded-resource',
|
||||
name: 'ToolsCallEmbeddedResource',
|
||||
description: 'Tool returns embedded resource content',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Tools-Call',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/tools#calling-tools'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
136
src/scenarios/server/utils.ts
Normal file
136
src/scenarios/server/utils.ts
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
/**
|
||||
* Utilities test scenarios for MCP servers
|
||||
*/
|
||||
|
||||
import { ClientScenario, ConformanceCheck } from '../../types.js';
|
||||
import { connectToServer } from './client-helper.js';
|
||||
|
||||
export class LoggingSetLevelScenario implements ClientScenario {
|
||||
name = 'logging-set-level';
|
||||
description = 'Test setting logging level';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
// Send logging/setLevel request
|
||||
const result = await connection.client.setLoggingLevel('info');
|
||||
|
||||
// Validate response (should return empty object {})
|
||||
const errors: string[] = [];
|
||||
if (result && Object.keys(result).length > 0) {
|
||||
errors.push('Expected empty object {} response');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'logging-set-level',
|
||||
name: 'LoggingSetLevel',
|
||||
description: 'Server accepts logging level setting',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Logging',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/logging'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
result
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'logging-set-level',
|
||||
name: 'LoggingSetLevel',
|
||||
description: 'Server accepts logging level setting',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Logging',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/logging'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
|
||||
export class CompletionCompleteScenario implements ClientScenario {
|
||||
name = 'completion-complete';
|
||||
description = 'Test completion endpoint';
|
||||
|
||||
async run(serverUrl: string): Promise<ConformanceCheck[]> {
|
||||
const checks: ConformanceCheck[] = [];
|
||||
|
||||
try {
|
||||
const connection = await connectToServer(serverUrl);
|
||||
|
||||
// Send completion/complete request
|
||||
const result = await connection.client.complete({
|
||||
ref: {
|
||||
type: 'ref/prompt',
|
||||
name: 'test_prompt_with_arguments'
|
||||
},
|
||||
argument: {
|
||||
name: 'arg1',
|
||||
value: 'test'
|
||||
}
|
||||
});
|
||||
|
||||
// Validate response structure
|
||||
const errors: string[] = [];
|
||||
if (!result.completion) {
|
||||
errors.push('Missing completion field');
|
||||
} else {
|
||||
if (!result.completion.values) errors.push('Missing values array in completion');
|
||||
if (!Array.isArray(result.completion.values)) errors.push('completion.values is not an array');
|
||||
}
|
||||
|
||||
checks.push({
|
||||
id: 'completion-complete',
|
||||
name: 'CompletionComplete',
|
||||
description: 'Server responds to completion requests',
|
||||
status: errors.length === 0 ? 'SUCCESS' : 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: errors.length > 0 ? errors.join('; ') : undefined,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Completion',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/completion'
|
||||
}
|
||||
],
|
||||
details: {
|
||||
result
|
||||
}
|
||||
});
|
||||
|
||||
await connection.close();
|
||||
} catch (error) {
|
||||
checks.push({
|
||||
id: 'completion-complete',
|
||||
name: 'CompletionComplete',
|
||||
description: 'Server responds to completion requests',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
specReferences: [
|
||||
{
|
||||
id: 'MCP-Completion',
|
||||
url: 'https://modelcontextprotocol.io/specification/2025-06-18/server/utilities/completion'
|
||||
}
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
return checks;
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue