tests: add Playwright E2E tests with screenshot golden testing (+ fix examples session handling) (#115)
* Add Playwright E2E tests with screenshot golden testing - Add E2E tests for all 8 MCP server examples - Screenshot golden images for visual regression testing - CI workflow for running E2E tests - npm scripts: test:e2e, test:e2e:update, test:e2e:ui 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add default demo code for Three.js example When no code is provided, show a rotating green cube demo. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add E2E testing documentation to CONTRIBUTING.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update Three.js golden screenshot with 3D cube 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add explicit permissions to CI workflow Set minimal read-only permissions for security best practices. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix test.setTimeout() to be inside describe blocks Playwright requires setTimeout to be called within a test or describe block. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Simplify E2E tests - remove excessive timeouts - Remove explicit setTimeout calls (default 30s is sufficient) - Replace waitForTimeout(5000/6000) with proper waitForAppLoad() - Wait for inner iframe visibility instead of fixed delays - Keep only 500ms stabilization for screenshot animations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Auto-generate missing snapshots in CI Use updateSnapshots: 'missing' in CI to handle cross-platform screenshot differences (macOS vs Linux). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Use platform-agnostic golden screenshots - Remove -chromium-darwin suffix from snapshot filenames - Configure snapshotPathTemplate for cross-platform compatibility - Increase tolerance to 5% for rendering differences between platforms 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor tests to use forEach instead of for-of loop May help with Playwright test discovery issues in CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Use npm ci and list reporter in CI - Use npm ci for exact package versions - Use --reporter=list instead of html to avoid potential re-evaluation issues - Only upload test-results on failure 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Exclude e2e tests from bun test Run bun test only on src/ directory to avoid running Playwright tests with Bun's test runner. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix prettier formatting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add interaction tests for basic server apps Test that clicking buttons in the MCP App triggers the corresponding host callbacks: - Send Message → host logs "Message from MCP App" - Send Log → host logs "Log message from MCP App" - Open Link → host logs "Open link request" Tests both React and Vanilla JS implementations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Mask dynamic content in E2E screenshot tests Add Playwright mask option to handle servers with dynamic/random content: - basic-react/vanillajs: mask server time display - system-monitor: mask CPU chart, memory stats, uptime - cohort-heatmap: mask heatmap grid (random data) - customer-segmentation: mask scatter chart (random data) This addresses PR feedback about handling examples with non-deterministic output. Masking replaces the previous 10% tolerance with proper exclusion of dynamic elements, allowing tighter 1% tolerance for the rest of the UI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add wiki-explorer to E2E tests with default URL param - Add default URL "https://en.wikipedia.org/wiki/Model_Context_Protocol" to wiki-explorer-server's get-first-degree-links tool inputSchema - Update basic-host to automatically populate input field with tool defaults extracted from inputSchema.properties - Add wiki-explorer-server to E2E test suite with dynamic masking for the force-directed graph 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Use .default() for threejs tool schema instead of .optional() This exposes the default code and height values in the JSON Schema, allowing basic-host to auto-populate the input field with defaults. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Enable parallel E2E tests with timeouts and canvas masking - Enable fullyParallel with 4 workers locally (2 in CI) - Add 30s timeout per test - Mask threejs canvas for stable screenshots - Update snapshots to reflect default values in input fields 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add pre-commit check for private registry URLs in package-lock.json Same check as in CI to catch issues before push. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Increase screenshot diff tolerance to 6% for cross-platform rendering Font rendering differs between macOS and Linux, causing ~5% pixel differences. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Revert pre-commit artifactory check (moved to separate PR #133) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Format threejs server.ts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update CONTRIBUTING.md to reflect platform-agnostic screenshots 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix(e2e): use factory pattern for MCP servers to support parallel connections McpServer only supports one transport at a time. When multiple browser contexts connected in parallel, calling server.connect(transport) for each session overwrote the previous transport's callbacks, causing connection corruption and test timeouts. Changes: - server-utils.ts: accept factory function instead of single instance - All 9 example servers: wrap server creation in createServer() function - playwright.config.ts: re-enable parallel execution (4 workers) - servers.spec.ts: add toBeEnabled wait for server connection - Update screenshot baselines for basic-vanillajs and cohort-heatmap 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Olivier Chafik
GitHub
parent
cfd1d1dc43
commit
c9e37204de
1 changed files with 3045 additions and 1457 deletions
|
|
@ -24,21 +24,31 @@ export interface ServerOptions {
|
|||
|
||||
type Transport = StreamableHTTPServerTransport | SSEServerTransport;
|
||||
|
||||
/** Session state: transport + its dedicated server instance */
|
||||
interface Session {
|
||||
transport: Transport;
|
||||
server: McpServer;
|
||||
}
|
||||
|
||||
/**
|
||||
* Starts an MCP server with HTTP transports.
|
||||
*
|
||||
* Provides:
|
||||
* - /mcp (GET/POST/DELETE): Streamable HTTP with stateful sessions
|
||||
* - /sse (GET) + /messages (POST): Legacy SSE for older clients
|
||||
*
|
||||
* @param createServer - Factory function that creates a new McpServer instance per session.
|
||||
* Each session needs its own server because McpServer only supports
|
||||
* one transport at a time.
|
||||
*/
|
||||
export async function startServer(
|
||||
server: McpServer,
|
||||
createServer: () => McpServer,
|
||||
options: ServerOptions,
|
||||
): Promise<void> {
|
||||
const { port, name = "MCP Server" } = options;
|
||||
|
||||
// Unified session store for both transport types
|
||||
const sessions = new Map<string, Transport>();
|
||||
// Session store: each session has its own transport AND server instance
|
||||
const sessions = new Map<string, Session>();
|
||||
|
||||
// Express app - bind to all interfaces for development/testing
|
||||
const app = createMcpExpressApp({ host: "0.0.0.0" });
|
||||
|
|
@ -52,12 +62,13 @@ export async function startServer(
|
|||
app.all("/mcp", async (req: Request, res: Response) => {
|
||||
try {
|
||||
const sessionId = req.headers["mcp-session-id"] as string | undefined;
|
||||
let transport = sessionId
|
||||
? (sessions.get(sessionId) as StreamableHTTPServerTransport | undefined)
|
||||
: undefined;
|
||||
let session = sessionId ? sessions.get(sessionId) : undefined;
|
||||
|
||||
// Session exists but wrong transport type
|
||||
if (sessionId && sessions.has(sessionId) && !transport) {
|
||||
if (
|
||||
session &&
|
||||
!(session.transport instanceof StreamableHTTPServerTransport)
|
||||
) {
|
||||
return res.status(400).json({
|
||||
jsonrpc: "2.0",
|
||||
error: { code: -32000, message: "Session uses different transport" },
|
||||
|
|
@ -66,7 +77,7 @@ export async function startServer(
|
|||
}
|
||||
|
||||
// New session requires initialize request
|
||||
if (!transport) {
|
||||
if (!session) {
|
||||
if (req.method !== "POST" || !isInitializeRequest(req.body)) {
|
||||
return res.status(400).json({
|
||||
jsonrpc: "2.0",
|
||||
|
|
@ -75,20 +86,27 @@ export async function startServer(
|
|||
});
|
||||
}
|
||||
|
||||
transport = new StreamableHTTPServerTransport({
|
||||
// Create new server instance for this session (McpServer supports only one transport)
|
||||
const serverInstance = createServer();
|
||||
|
||||
const transport = new StreamableHTTPServerTransport({
|
||||
sessionIdGenerator: () => randomUUID(),
|
||||
onsessioninitialized: (id) => {
|
||||
sessions.set(id, transport!);
|
||||
sessions.set(id, { transport, server: serverInstance });
|
||||
},
|
||||
});
|
||||
const t = transport;
|
||||
t.onclose = () => {
|
||||
if (t.sessionId) sessions.delete(t.sessionId);
|
||||
transport.onclose = () => {
|
||||
if (transport.sessionId) sessions.delete(transport.sessionId);
|
||||
};
|
||||
await server.connect(transport);
|
||||
await serverInstance.connect(transport);
|
||||
session = { transport, server: serverInstance };
|
||||
}
|
||||
|
||||
await transport.handleRequest(req, res, req.body);
|
||||
await (session.transport as StreamableHTTPServerTransport).handleRequest(
|
||||
req,
|
||||
res,
|
||||
req.body,
|
||||
);
|
||||
} catch (error) {
|
||||
console.error("MCP error:", error);
|
||||
if (!res.headersSent) {
|
||||
|
|
@ -104,10 +122,12 @@ export async function startServer(
|
|||
// Legacy SSE
|
||||
app.get("/sse", async (_req: Request, res: Response) => {
|
||||
try {
|
||||
// Create new server instance for this session (McpServer supports only one transport)
|
||||
const serverInstance = createServer();
|
||||
const transport = new SSEServerTransport("/messages", res);
|
||||
sessions.set(transport.sessionId, transport);
|
||||
sessions.set(transport.sessionId, { transport, server: serverInstance });
|
||||
res.on("close", () => sessions.delete(transport.sessionId));
|
||||
await server.connect(transport);
|
||||
await serverInstance.connect(transport);
|
||||
} catch (error) {
|
||||
console.error("SSE error:", error);
|
||||
if (!res.headersSent) res.status(500).end();
|
||||
|
|
@ -116,15 +136,15 @@ export async function startServer(
|
|||
|
||||
app.post("/messages", async (req: Request, res: Response) => {
|
||||
try {
|
||||
const transport = sessions.get(req.query.sessionId as string);
|
||||
if (!(transport instanceof SSEServerTransport)) {
|
||||
const session = sessions.get(req.query.sessionId as string);
|
||||
if (!session || !(session.transport instanceof SSEServerTransport)) {
|
||||
return res.status(404).json({
|
||||
jsonrpc: "2.0",
|
||||
error: { code: -32001, message: "Session not found" },
|
||||
id: null,
|
||||
});
|
||||
}
|
||||
await transport.handlePostMessage(req, res, req.body);
|
||||
await session.transport.handlePostMessage(req, res, req.body);
|
||||
} catch (error) {
|
||||
console.error("Message error:", error);
|
||||
if (!res.headersSent) {
|
||||
|
|
@ -151,7 +171,7 @@ export async function startServer(
|
|||
|
||||
const shutdown = () => {
|
||||
console.log("\nShutting down...");
|
||||
sessions.forEach((t) => t.close().catch(() => {}));
|
||||
sessions.forEach((session) => session.transport.close().catch(() => {}));
|
||||
httpServer.close(() => process.exit(0));
|
||||
};
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue