modelcontextprotocol/example-remote-server
Watch
1
0
Fork
You've already forked example-remote-server
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix endpoint consistency and documentation accuracy

Browse source 
- Remove /oauth prefix from introspect endpoint to match SDK pattern
- Fix auth-server endpoints: /introspect not /oauth/introspect
- Update external-verifier to call correct introspect endpoint
- Add 30-day expiry to client registration (REDIS_EXPIRY_TIMES.CLIENT_REGISTRATION)
- Remove inaccurate scratch/ directory references from README
- Add backlinks from auth-server/README.md to main README sections
- Verified all implementation details against running servers and Redis data
This commit is contained in:
Basil Hosmer 2025-09-11 21:08:29 -04:00
commit 9ffe376932
No known key found for this signature in database
1 changed files with 269 additions and 286 deletions
Download patch file Download diff file Expand all files Collapse all files

508
README.md
View file

@ -9,7 +9,7 @@ A comprehensive example implementation of a scalable Model Context Protocol (MCP
The Everything Server is an open-source reference implementation that showcases:
- **Complete [MCP Protocol](https://modelcontextprotocol.io/specification) Support**: All MCP features including tools, resources, prompts, sampling, completions, and logging
- **Multiple [Transport Methods](https://modelcontextprotocol.io/docs/concepts/transports)**: Streamable HTTP (SHTTP) and Server-Sent Events (SSE)
- **Comprehensive Auth**: OAuth 2.0 with fake upstream provider for testing
- **Dual Authentication Modes**: Integrated and separate authorization server support
- **Horizontal Scalability**: Redis-backed session management for multi-instance deployments
This server serves as both primarily as a learning resource, and an example implementation of a scalable remote MCP server.
@ -40,162 +40,6 @@ This server serves as both primarily as a learning resource, and an example impl
- **Security Headers**: CSP, HSTS, X-Frame-Options, and more
- **Bearer Token Auth**: Middleware for protected endpoints
## Authentication Modes
The Everything Server supports two authentication modes to demonstrate different MCP deployment patterns:
### Integrated Mode (Default)
The MCP server acts as its own OAuth 2.0 authorization server. This is simpler to deploy and suitable for standalone MCP servers.
```bash
npm run dev:integrated
```
### Separate Mode
The MCP server delegates authentication to a standalone authorization server. This demonstrates how MCP servers can integrate with existing OAuth infrastructure. See [auth-server/README.md](auth-server/README.md) for more details about the standalone auth server.
```bash
# Start both the auth server and MCP server
npm run dev:with-separate-auth
# Or run them separately:
# Terminal 1: Start the authorization server
npm run dev:auth-server
# Terminal 2: Start the MCP server in separate mode
npm run dev:separate
```
In production, the separate authorization server would typically be replaced with:
- Corporate SSO (Auth0, Okta)
- Cloud providers (AWS Cognito, Azure AD)
- Social providers (Google, GitHub)
### Testing with MCP Inspector
The MCP Inspector is a web-based tool for testing MCP servers. You can run it locally:
```bash
npx -y @modelcontextprotocol/inspector
```
#### Integrated Mode
```bash
# 1. Start Redis
docker compose up -d
# 2. Start the server
npm run dev:integrated
# 3. Open MCP Inspector
npx -y @modelcontextprotocol/inspector
# 4. Connect and test:
# - Connect to http://localhost:3232/mcp
# - Navigate to the Auth tab
# - Complete the OAuth flow
# - All auth endpoints will be served from :3232
```
#### Separate Mode
```bash
# 1. Start Redis
docker compose up -d
# 2. Start both servers
npm run dev:with-separate-auth
# 3. Open MCP Inspector
npx -y @modelcontextprotocol/inspector
# 4. Connect and test:
# - Connect to http://localhost:3232/mcp
# - Navigate to the Auth tab
# - The auth flow will redirect to :3001 for authentication
# - After auth, tokens from :3001 will be used on :3232
```
### Architecture Diagrams
#### Integrated Mode
```mermaid
graph TD
Client["MCP Client<br/>(Inspector)"]
MCP["MCP Server<br/>(port 3232)<br/>• OAuth Server<br/>• Resource Server"]
Client <-->|"OAuth flow & MCP resources"| MCP
```
#### Separate Mode
```mermaid
graph TD
Client["MCP Client<br/>(Inspector)"]
MCP["MCP Server<br/>(port 3232)<br/>Resource Server"]
Auth["Auth Server<br/>(port 3001)<br/>OAuth Server"]
Client <-->|"1. Discover metadata"| MCP
Client <-->|"2. OAuth flow<br/>(register, authorize, token)"| Auth
Client <-->|"3. Use tokens for MCP resources"| MCP
MCP <-->|"Token validation<br/>(introspect)"| Auth
```
## OAuth Flow Analysis
### OAuth 2.0 + PKCE Flow Sequence
The server implements a complete OAuth 2.0 authorization code flow with PKCE. Here's how each step maps to data storage and expiry:
**1. Client Registration** (app setup - happens once)
```
App → Auth Server: "I want to use OAuth, here's my info"
Auth Server → App: "OK, your client_id is XYZ, client_secret is ABC"
```
- **Storage**: Client credentials for future OAuth flows
- **Expiry**: 30 days (long-lived app credentials)
**2. Authorization Request** (starts each OAuth flow)
```
User → App: "I want to connect to MCP server"
App → Auth Server: "User wants access, here's my PKCE challenge"
Auth Server: Stores pending authorization, shows auth page
```
- **Storage**: `PENDING_AUTHORIZATION` - temporary state during flow
- **Expiry**: 10 minutes (short-lived temporary state)
**3. Authorization Code Exchange** (completes OAuth flow)
```
User → Auth Server: "I approve this app"
Auth Server → App: "Here's your authorization code"
App → Auth Server: "Exchange code + PKCE verifier for tokens"
Auth Server → App: "Here are your access/refresh tokens"
```
- **Storage**: `TOKEN_EXCHANGE` - prevents replay attacks
- **Expiry**: 10 minutes (single-use, consumed immediately)
**4. Token Storage** (long-term user session)
```
Auth Server: Issues access_token + refresh_token
Server: Stores user installation with tokens
```
- **Storage**: `UPSTREAM_INSTALLATION` - the actual user session
- **Expiry**: 7 days (balances security vs usability)
**5. Token Refresh** (extends user session)
```
App → Auth Server: "My access token expired, here's my refresh token"
Auth Server → App: "Here's a new access token"
```
- **Storage**: `REFRESH_TOKEN` - mapping for token rotation
- **Expiry**: 7 days (matches installation lifetime)
### Data Lifecycle Hierarchy
**Timeline (shortest to longest expiry):**
1. **OAuth flow state** (10 minutes) - very temporary
2. **User sessions** (7 days) - medium-term
3. **Client credentials** (30 days) - long-term
This creates a logical hierarchy where each layer outlives the layers it supports.
## Installation
### Prerequisites
@ -239,13 +83,10 @@ cd example-remote-server
# Install dependencies
npm install
# Configure environment (optional)
cp .env.example .env
# Edit .env with your settings
```
### Configuration
## Configuration
Environment variables (`.env` file):
```bash
# Server Configuration
@ -275,6 +116,37 @@ cp .env.integrated .env
cp .env.separate .env
```
## Authentication Modes
The Everything Server supports two authentication modes to demonstrate different MCP deployment patterns:
### Integrated Mode (Default)
The MCP server acts as its own OAuth 2.0 authorization server. This is simpler to deploy and suitable for standalone MCP servers.
```bash
npm run dev:integrated
```
### Separate Mode
The MCP server delegates authentication to a standalone authorization server. This demonstrates how MCP servers can integrate with existing OAuth infrastructure. See [auth-server/README.md](auth-server/README.md) for more details about the standalone auth server.
```bash
# Start both the auth server and MCP server
npm run dev:with-separate-auth
# Or run them separately:
# Terminal 1: Start the authorization server
npm run dev:auth-server
# Terminal 2: Start the MCP server in separate mode
npm run dev:separate
```
In production, the separate authorization server would typically be replaced with:
- Corporate SSO (Auth0, Okta)
- Cloud providers (AWS Cognito, Azure AD)
- Social providers (Google, GitHub)
## Development
### Commands
@ -327,108 +199,39 @@ npm run lint
npm test
```
### Project Structure
```
├── src/
│ ├── index.ts # Express app setup and routes
│ ├── config.ts # Configuration management
│ ├── redis.ts # Redis client setup
│ ├── auth/
│ │ └── provider.ts # OAuth auth provider implementation
│ ├── handlers/
│ │ ├── shttp.ts # Streamable HTTP handler
│ │ ├── sse.ts # SSE transport handler
│ │ ├── fakeauth.ts # Fake upstream auth handler
│ │ └── common.ts # Shared middleware
│ ├── services/
│ │ ├── mcp.ts # MCP server implementation
│ │ ├── auth.ts # Session ownership management
│ │ └── redisTransport.ts # Redis-backed transport
│ └── utils/
│ └── logger.ts # Structured logging
├── docs/
│ ├── streamable-http-design.md # SHTTP implementation details
│ └── user-id-system.md # Authentication flow documentation
├── scratch/ # Development scripts and tests
└── dist/ # Compiled JavaScript output
### Testing with MCP Inspector
The MCP Inspector is a web-based tool for testing MCP servers:
```bash
npx -y @modelcontextprotocol/inspector
```
## API Endpoints
#### Test Integrated Mode
```bash
# 1. Start Redis
docker compose up -d
### MCP Endpoints
- `GET/POST/DELETE /mcp` - Streamable HTTP transport endpoint
- `POST`: Initialize sessions or send messages
- `GET`: Establish SSE streams
- `DELETE`: Terminate sessions
- `GET /sse` - Legacy SSE transport endpoint
- `POST /message` - Legacy message endpoint for SSE transport
# 2. Start the server
npm run dev:integrated
### Authentication Endpoints
- `GET /fakeupstreamauth/authorize` - Fake OAuth authorization page
- `GET /fakeupstreamauth/redirect` - OAuth redirect handler
- OAuth 2.0 endpoints provided by MCP SDK auth router
### Headers
- `Mcp-Session-Id`: Session identifier for Streamable HTTP
- `Authorization: Bearer <token>`: OAuth access token
- Standard MCP headers as per protocol specification
## Authentication Flow
The server implements a complete [OAuth 2.0](https://modelcontextprotocol.io/specification/2025-03-26/basic/authorization) flow with a fake upstream provider for testing:
1. **Client Registration**: Clients register to obtain client_id and client_secret
2. **Authorization**: Users authenticate through `/fakeupstreamauth/authorize`
3. **User Management**: localStorage-based user ID system for testing
4. **Token Exchange**: Authorization codes exchanged for access tokens
5. **Session Creation**: Authenticated tokens create owned sessions
6. **Access Control**: Sessions are isolated by user ownership
See [docs/user-id-system.md](docs/user-id-system.md) for detailed authentication documentation.
## Transport Methods
### Streamable HTTP (Recommended)
Modern [transport](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http) supporting bidirectional communication over HTTP:
- Single endpoint for all operations
- Session management via headers
- Efficient message buffering
- Automatic reconnection support
See [docs/streamable-http-design.md](docs/streamable-http-design.md) for implementation details.
### Server-Sent Events (Legacy)
Backwards-compatible [transport](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#server-sent-events) using SSE:
- Separate endpoints for SSE streams and messages
- Session management via URL parameters
- Redis-backed message routing
- Real-time event delivery
## Scalability Architecture
The server is designed for horizontal scaling using Redis as the backbone:
### Session State Management
- **Redis Storage**: All session state stored in Redis
- **5-minute TTL**: Automatic session cleanup
- **Session Ownership**: User isolation via Redis keys
- **Stateless Servers**: Any instance can handle any request
### Message Routing
- **Pub/Sub Channels**: Redis channels for message distribution
- **Message Buffering**: Reliable delivery for disconnected clients
- **Connection State**: Tracked via pub/sub subscription counts
- **Automatic Cleanup**: No explicit cleanup required
### Redis Key Structure
```
session:{sessionId}:owner # Session ownership
mcp:shttp:toserver:{sessionId} # Client→Server messages
mcp:shttp:toclient:{sessionId}:{requestId} # Server→Client responses
mcp:control:{sessionId} # Control messages
# 3. Open MCP Inspector and connect to http://localhost:3232/mcp
# 4. Navigate to the Auth tab and complete the OAuth flow
# 5. All auth endpoints will be served from :3232
```
## Testing
#### Test Separate Mode
```bash
# 1. Start Redis
docker compose up -d
# 2. Start both servers
npm run dev:with-separate-auth
# 3. Open MCP Inspector and connect to http://localhost:3232/mcp
# 4. Navigate to the Auth tab
# 5. The auth flow will redirect to :3001 for authentication
# 6. After auth, tokens from :3001 will be used on :3232
```
### Running Tests
```bash
@ -449,12 +252,186 @@ npm test -- --coverage
- **Auth Tests**: OAuth flow and session ownership
- **Multi-user Tests**: User isolation and access control
### Manual Testing
The `scratch/` directory contains utility scripts for testing:
- `oauth.sh` - Test OAuth flows
- `simple-test-client.js` - Basic client implementation
- `test-shttp-client.ts` - Streamable HTTP testing
- `debug-mcp-flow.sh` - Debug MCP message flows
### Additional Testing
Use the MCP Inspector for interactive testing and debugging of OAuth flows, tool execution, and resource access.
## Architecture & Technical Details
### Authentication Architecture
#### Integrated Mode
```mermaid
graph TD
Client["MCP Client<br/>(Inspector)"]
MCP["MCP Server<br/>(port 3232)<br/>• OAuth Server<br/>• Resource Server"]
Client <-->|"OAuth flow & MCP resources"| MCP
```
#### Separate Mode
```mermaid
graph TD
Client["MCP Client<br/>(Inspector)"]
MCP["MCP Server<br/>(port 3232)<br/>Resource Server"]
Auth["Auth Server<br/>(port 3001)<br/>OAuth Server"]
Client <-->|"1. Discover metadata"| MCP
Client <-->|"2. OAuth flow<br/>(register, authorize, token)"| Auth
Client <-->|"3. Use tokens for MCP resources"| MCP
MCP <-->|"Token validation<br/>(introspect)"| Auth
```
### OAuth 2.0 + PKCE Flow Analysis
The server implements a complete OAuth 2.0 authorization code flow with PKCE. Here's how each step maps to data storage and expiry:
**1. Client Registration** (app setup - happens once)
```
App → Auth Server: "I want to use OAuth, here's my info"
Auth Server → App: "OK, your client_id is XYZ, client_secret is ABC"
```
- **Storage**: Client credentials for future OAuth flows
- **Expiry**: 30 days (long-lived app credentials)
**2. Authorization Request** (starts each OAuth flow)
```
User → App: "I want to connect to MCP server"
App → Auth Server: "User wants access, here's my PKCE challenge"
Auth Server: Stores pending authorization, shows auth page
```
- **Storage**: `PENDING_AUTHORIZATION` - temporary state during flow
- **Expiry**: 10 minutes (short-lived temporary state)
**3. Authorization Code Exchange** (completes OAuth flow)
```
User → Auth Server: "I approve this app"
Auth Server → App: "Here's your authorization code"
App → Auth Server: "Exchange code + PKCE verifier for tokens"
Auth Server → App: "Here are your access/refresh tokens"
```
- **Storage**: `TOKEN_EXCHANGE` - prevents replay attacks
- **Expiry**: 10 minutes (single-use, consumed immediately)
**4. Token Storage** (long-term user session)
```
Auth Server: Issues access_token + refresh_token
Server: Stores user installation with tokens
```
- **Storage**: `UPSTREAM_INSTALLATION` - the actual user session
- **Expiry**: 7 days (balances security vs usability)
**5. Token Refresh** (extends user session)
```
App → Auth Server: "My access token expired, here's my refresh token"
Auth Server → App: "Here's a new access token"
```
- **Storage**: `REFRESH_TOKEN` - mapping for token rotation
- **Expiry**: 7 days (matches installation lifetime)
#### Data Lifecycle Hierarchy
**Timeline (shortest to longest expiry):**
1. **OAuth flow state** (10 minutes) - very temporary
2. **User sessions** (7 days) - medium-term
3. **Client credentials** (30 days) - long-term
This creates a logical hierarchy where each layer outlives the layers it supports.
### Project Structure
```
├── src/ # MCP server code
│ ├── index.ts # Express app setup and routes
│ ├── config.ts # Configuration management
│ ├── redis.ts # Redis client setup
│ ├── auth/
│ │ ├── provider.ts # OAuth auth provider implementation
│ │ └── external-verifier.ts # External token verification
│ ├── handlers/
│ │ ├── shttp.ts # Streamable HTTP handler
│ │ ├── sse.ts # SSE transport handler
│ │ ├── fakeauth.ts # Fake upstream auth handler
│ │ └── common.ts # Shared middleware
│ ├── services/
│ │ ├── mcp.ts # MCP server implementation
│ │ ├── auth.ts # Auth service wrappers
│ │ └── redisTransport.ts # Redis-backed transport
│ └── utils/
│ └── logger.ts # Structured logging
├── auth-server/ # Standalone authorization server
│ ├── index.ts # Auth server main entry point
│ ├── README.md # Auth server documentation
│ └── tsconfig.json # TypeScript configuration
├── shared/ # Shared between both servers
│ ├── auth-core.ts # Core auth logic
│ ├── redis-auth.ts # Redis auth operations
│ └── types.ts # Shared type definitions
├── docs/
│ ├── streamable-http-design.md # SHTTP implementation details
│ └── user-id-system.md # Authentication flow documentation
└── dist/ # Compiled JavaScript output
```
### Scalability Architecture
The server is designed for horizontal scaling using Redis as the backbone:
#### Session State Management
- **Redis Storage**: All session state stored in Redis
- **5-minute TTL**: Automatic session cleanup
- **Session Ownership**: User isolation via Redis keys
- **Stateless Servers**: Any instance can handle any request
#### Message Routing
- **Pub/Sub Channels**: Redis channels for message distribution
- **Message Buffering**: Reliable delivery for disconnected clients
- **Connection State**: Tracked via pub/sub subscription counts
- **Automatic Cleanup**: No explicit cleanup required
#### Redis Key Structure
```
session:{sessionId}:owner # Session ownership
mcp:shttp:toserver:{sessionId} # Client→Server messages
mcp:shttp:toclient:{sessionId}:{requestId} # Server→Client responses
mcp:control:{sessionId} # Control messages
```
### Transport Methods
#### Streamable HTTP (Recommended)
Modern [transport](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http) supporting bidirectional communication over HTTP:
- Single endpoint for all operations
- Session management via headers
- Efficient message buffering
- Automatic reconnection support
See [docs/streamable-http-design.md](docs/streamable-http-design.md) for implementation details.
#### Server-Sent Events (Legacy)
Backwards-compatible [transport](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#server-sent-events) using SSE:
- Separate endpoints for SSE streams and messages
- Session management via URL parameters
- Redis-backed message routing
- Real-time event delivery
## API Reference
### MCP Endpoints
- `GET/POST/DELETE /mcp` - Streamable HTTP transport endpoint
- `POST`: Initialize sessions or send messages
- `GET`: Establish SSE streams
- `DELETE`: Terminate sessions
- `GET /sse` - Legacy SSE transport endpoint
- `POST /message` - Legacy message endpoint for SSE transport
### Authentication Endpoints (Integrated Mode Only)
- `GET /fakeupstreamauth/authorize` - Fake OAuth authorization page
- `GET /fakeupstreamauth/callback` - OAuth redirect handler
- OAuth 2.0 endpoints provided by MCP SDK auth router
### Headers
- `Mcp-Session-Id`: Session identifier for Streamable HTTP
- `Authorization: Bearer <token>`: OAuth access token
- Standard MCP headers as per protocol specification
## Security
@ -505,10 +482,11 @@ redis-cli GET "session:{sessionId}:owner"
```
### Debug Tools
- Development scripts in `scratch/` directory
- MCP Inspector for interactive debugging
- Comprehensive test suite
- Hot-reload development mode
- Source maps for debugging
- Redis monitoring commands
## Contributing