Watch

Olivier Chafik 6bb63c3bae

examples: add transcript-server example with live speech transcription (#240 )

* feat: add transcript-server example with live speech transcription

- New example: transcript-server with Web Speech API transcription
  - Start/stop recording with timer display
  - Live transcript with interim results
  - Send button to send transcript as ui/message to host
  - Copy button to copy full transcript
  - Sent messages marked with divider, greyed out
  - Experimental ui/update-model-context for live context updates
  - Transparent background, respects host theme

- Enable microphone and clipboard in basic-host sandbox iframes
  - Add allow="microphone; clipboard-write" to both outer and inner iframes
  - Required for Web Speech API, audio capture, and clipboard access

- Add alert popup in basic-host for ui/message from apps

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: format transcript-server files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add e2e test and golden for transcript-server

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: use @types/dom-speech-recognition instead of hand-rolled types

Removes ~50 lines of manually defined Web Speech API types in favor of
the community-maintained type package.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add newlines between transcript chunks when copying to clipboard

* refactor: unify transcript formatting with timestamps

- Add formatEntry() helper that includes [timestamp] prefix
- Add formatEntries() helper that joins entries with newlines
- Use same format for clipboard copy, send message, and model context
- DRY up duplicate code in send button handler

* fix: log model context update failures instead of silently swallowing

* fix: log model context updates for debugging

* refactor: pass permissions through loadSandboxProxy like csp

- Add permissions parameter to loadSandboxProxy in implementation.ts
- Build iframe allow attribute from permissions instead of hardcoding
- Update sandbox.ts to not hardcode allow attribute (set via notification)
- Add microphone + clipboardWrite permissions to transcript-server resource
- Update README to reflect permission configuration via resource _meta.ui

* fix(transcript-server): auto-expand instead of scroll

- Remove min-height: 100vh and overflow-y: auto
- Let content grow naturally with autoResize (default)
- Remove unnecessary scrollTop calls

* test: remove transcript timer mask (no longer dynamic)

* refactor: pass permissions through loadSandboxProxy like csp

* docs: document updateModelContext for transitional speech

* test: update e2e screenshots

* refactor: use app.updateModelContext() instead of raw request

* package-lock.json

* src/generated/schema.json

* test: update e2e screenshots from CI

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-12 21:27:31 +00:00

2.4 KiB

Raw Permalink Blame History

Transcript Server

An MCP App Server for live speech transcription using the Web Speech API.

Features

Live Transcription: Real-time speech-to-text using browser's Web Speech API
Transitional Model Context: Streams interim transcriptions to the model via ui/update-model-context, allowing the model to see what the user is saying as they speak
Audio Level Indicator: Visual feedback showing microphone input levels
Send to Host: Button to send completed transcriptions as a ui/message to the MCP host
Start/Stop Control: Toggle listening on and off
Clear Transcript: Reset the transcript area

Setup

Prerequisites

Node.js 18+
Chrome, Edge, or Safari (Web Speech API support)

Installation

npm install

Running

# Development mode (with hot reload)
npm run dev

# Production build and serve
npm run start

Usage

The server exposes a single tool:

`transcribe`

Opens a live speech transcription interface.

Parameters: None

Example:

{
  "name": "transcribe",
  "arguments": {}
}

How It Works

Click Start to begin listening
Speak into your microphone
Watch your speech appear as text in real-time (interim text is streamed to model context via ui/update-model-context)
Click Send to send the transcript as a ui/message to the host (clears the model context)
Click Clear to reset the transcript

Architecture

transcript-server/
├── server.ts          # MCP server with transcribe tool
├── server-utils.ts    # HTTP transport utilities
├── mcp-app.html       # Transcript UI entry point
├── src/
│   ├── mcp-app.ts     # App logic, Web Speech API integration
│   ├── mcp-app.css    # Transcript UI styles
│   └── global.css     # Base styles
└── dist/              # Built output (single HTML file)

Notes

Microphone Permission: Requires allow="microphone" on the sandbox iframe (configured via permissions: { microphone: {} } in the resource _meta.ui)
Browser Support: Web Speech API is well-supported in Chrome/Edge, with Safari support. Firefox has limited support.
Continuous Mode: Recognition automatically restarts when it ends, for seamless transcription

Future Enhancements

Language selection dropdown
Whisper-based offline transcription (see TRANSCRIPTION.md)
Export transcript to file
Timestamps toggle

2.4 KiB Raw Permalink Blame History