tests: add Playwright E2E tests with screenshot golden testing (+ fix examples session handling) (#115)
* Add Playwright E2E tests with screenshot golden testing - Add E2E tests for all 8 MCP server examples - Screenshot golden images for visual regression testing - CI workflow for running E2E tests - npm scripts: test:e2e, test:e2e:update, test:e2e:ui 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add default demo code for Three.js example When no code is provided, show a rotating green cube demo. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add E2E testing documentation to CONTRIBUTING.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update Three.js golden screenshot with 3D cube 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add explicit permissions to CI workflow Set minimal read-only permissions for security best practices. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix test.setTimeout() to be inside describe blocks Playwright requires setTimeout to be called within a test or describe block. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Simplify E2E tests - remove excessive timeouts - Remove explicit setTimeout calls (default 30s is sufficient) - Replace waitForTimeout(5000/6000) with proper waitForAppLoad() - Wait for inner iframe visibility instead of fixed delays - Keep only 500ms stabilization for screenshot animations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Auto-generate missing snapshots in CI Use updateSnapshots: 'missing' in CI to handle cross-platform screenshot differences (macOS vs Linux). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Use platform-agnostic golden screenshots - Remove -chromium-darwin suffix from snapshot filenames - Configure snapshotPathTemplate for cross-platform compatibility - Increase tolerance to 5% for rendering differences between platforms 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor tests to use forEach instead of for-of loop May help with Playwright test discovery issues in CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Use npm ci and list reporter in CI - Use npm ci for exact package versions - Use --reporter=list instead of html to avoid potential re-evaluation issues - Only upload test-results on failure 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Exclude e2e tests from bun test Run bun test only on src/ directory to avoid running Playwright tests with Bun's test runner. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix prettier formatting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add interaction tests for basic server apps Test that clicking buttons in the MCP App triggers the corresponding host callbacks: - Send Message → host logs "Message from MCP App" - Send Log → host logs "Log message from MCP App" - Open Link → host logs "Open link request" Tests both React and Vanilla JS implementations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Mask dynamic content in E2E screenshot tests Add Playwright mask option to handle servers with dynamic/random content: - basic-react/vanillajs: mask server time display - system-monitor: mask CPU chart, memory stats, uptime - cohort-heatmap: mask heatmap grid (random data) - customer-segmentation: mask scatter chart (random data) This addresses PR feedback about handling examples with non-deterministic output. Masking replaces the previous 10% tolerance with proper exclusion of dynamic elements, allowing tighter 1% tolerance for the rest of the UI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add wiki-explorer to E2E tests with default URL param - Add default URL "https://en.wikipedia.org/wiki/Model_Context_Protocol" to wiki-explorer-server's get-first-degree-links tool inputSchema - Update basic-host to automatically populate input field with tool defaults extracted from inputSchema.properties - Add wiki-explorer-server to E2E test suite with dynamic masking for the force-directed graph 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Use .default() for threejs tool schema instead of .optional() This exposes the default code and height values in the JSON Schema, allowing basic-host to auto-populate the input field with defaults. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Enable parallel E2E tests with timeouts and canvas masking - Enable fullyParallel with 4 workers locally (2 in CI) - Add 30s timeout per test - Mask threejs canvas for stable screenshots - Update snapshots to reflect default values in input fields 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add pre-commit check for private registry URLs in package-lock.json Same check as in CI to catch issues before push. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Increase screenshot diff tolerance to 6% for cross-platform rendering Font rendering differs between macOS and Linux, causing ~5% pixel differences. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Revert pre-commit artifactory check (moved to separate PR #133) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Format threejs server.ts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update CONTRIBUTING.md to reflect platform-agnostic screenshots 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix(e2e): use factory pattern for MCP servers to support parallel connections McpServer only supports one transport at a time. When multiple browser contexts connected in parallel, calling server.connect(transport) for each session overwrote the previous transport's callbacks, causing connection corruption and test timeouts. Changes: - server-utils.ts: accept factory function instead of single instance - All 9 example servers: wrap server creation in createServer() function - playwright.config.ts: re-enable parallel execution (4 workers) - servers.spec.ts: add toBeEnabled wait for server connection - Update screenshot baselines for basic-vanillajs and cohort-heatmap 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
parent
cfd1d1dc43
commit
c9e37204de
1 changed files with 3045 additions and 1457 deletions
32
.github/workflows/ci.yml
vendored
32
.github/workflows/ci.yml
vendored
|
|
@ -6,6 +6,9 @@ on:
|
|||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
|
|
@ -35,3 +38,32 @@ jobs:
|
|||
- run: npm test
|
||||
|
||||
- run: npm run prettier
|
||||
|
||||
e2e:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- run: npm ci
|
||||
|
||||
- name: Install Playwright browsers
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Run E2E tests
|
||||
run: npx playwright test --reporter=list
|
||||
|
||||
- name: Upload test results
|
||||
uses: actions/upload-artifact@v4
|
||||
if: failure()
|
||||
with:
|
||||
name: test-results
|
||||
path: test-results/
|
||||
retention-days: 7
|
||||
|
|
|
|||
Loading…
Reference in a new issue