examples: add PDF viewer w/ chunked data loading, full-screen, model context updates, private tool (#267)
* feat(pdf-server): Interactive PDF viewer example
PDF viewer with PDF.js featuring:
- Chunked binary loading with progress bar
- Text extraction for AI context
- arXiv paper support (fetch by ID)
- Page navigation with keyboard shortcuts
- Zoom controls (including Ctrl+0 reset)
- Fullscreen mode support
- Horizontal swipe for page changes (disabled when zoomed)
- Page persistence in localStorage
- Text selection via PDF.js TextLayer
- Clickable title link to source URL
- Rounded corners and subtle border styling
* chore: Add pdf-server to screenshot generation list
* refactor(pdf-server): Simplify and generalize PDF loading
- Accept any HTTP(s) URLs instead of ArXiv-only
- Use HTTP Range requests for chunked binary loading
- Remove ArXiv-specific code (arxiv.ts, metadata fetching)
- Remove CLAUDE.md index generation
- Flatten hierarchical folder structure to simple entries list
- Remove dead code: getPdfSummary, httpFileSizes
- Simplify base64 encoding using Buffer
- Simplify chunk extraction using slice()
- Consolidate DEFAULT_PDF_URL constant
The server now works with any PDF URL, not just arXiv papers.
HTTP Range requests stream chunks on-demand when supported.
* feat(pdf-server): Include title and selection in model context
- Add pdfTitle to updateModelContext structuredContent
- Include selection position (text, start, end) when text is selected
- Add debounced selectionchange listener to update context on selection
* fix(pdf-server): Restore default URL in view_pdf schema
The UI needs the default value in the schema to show it properly.
* refactor(pdf-server): Further simplifications
- Remove hard-coded test paths from main()
- Remove unused resources: pdfs://metadata/{pdfId}, pdfs://content/{pdfId}
- Remove unused metadata fields: subject, creator, producer, creationDate, modDate
- Remove unused entry fields: relativePath, estimatedTextSize
- Remove filterEntriesByFolder and folder filter from list_pdfs
- Remove redundant output schema validation (trust typed returns)
- Simplify scanDirectory and createLocalEntry signatures
Total: 1836 → 1666 lines (-170 lines, -9%)
* refactor(pdf-server): Major simplification for didactic focus
Simplified the example to focus on key MCP Apps SDK patterns:
- Chunked data through size-limited tool calls
- Model context updates (page text + selection)
- Display modes (fullscreen vs inline)
- External links (openLink)
Changes:
- Remove local file support (HTTP URLs only)
- Restrict dynamic URLs to arxiv.org for security
- Simplify types: url instead of sourcePath/sourceType
- Simplify indexer: 168 → 44 lines
- Simplify loader: 318 → 171 lines
- Simplify server: 337 → 233 lines
- Fix selection text normalization
- Rewrite README with didactic focus
Total: 1836 → 1236 lines (-33%)
* feat(pdf-server): Add file:// URL support for local files
- Local paths are converted to file:// URLs on startup
- file:// URLs must be in the initial list (strict validation)
- Dynamic URLs still restricted to arxiv.org only
- Updated README with local file examples
* fix(pdf-server): Improve selection detection with logging
- Add logging to selectionchange handler to verify it fires
- Add fallback matching without spaces (TextLayer spans may lack spaces)
- Log selection detection success/failure for debugging
The issue: PDF.js TextLayer renders text as positioned spans without
space characters between them. When selecting across spans:
- pageText has spaces (items joined with ' ')
- sel.toString() may not have spaces
- indexOf fails to match
The fix tries exact match first, then falls back to spaceless matching.
* feat(pdf-server): Format model context as markdown with front matter
Model context now looks like:
```markdown
---
url: https://arxiv.org/pdf/...
page: 5/144
---
Page text with <pdf-selection>selected text</pdf-selection> inline.
```
This is cleaner for the model to parse and includes the source URL.
* refactor(pdf-server): Extract smart truncation helpers
Added two well-designed helpers:
formatPageContent(text, maxLength, selection?)
- Centers truncation window around selection if present
- Adds <truncated-content/> markers at elision points
- Wraps selection in <pdf-selection> tags
- Allocates 60% context before, 40% after for readability
findSelectionInText(pageText, selectedText)
- Tries exact match first
- Falls back to spaceless match for TextLayer quirks
- Returns { start, end } or undefined
Example output with selection:
```
<truncated-content/>
...context before... <pdf-selection>selected text</pdf-selection> ...context after...
<truncated-content/>
```
* fix(pdf-server): Truncate inside selection tags when selection too long
When selection is too large for the budget:
<truncated-content/><pdf-selection><truncated-content/>start...end<truncated-content/></pdf-selection><truncated-content/>
This keeps the selection structure intact while showing beginning and end.
* refactor(pdf-server): Remove unused read_pdf_text, use Attention paper as default
- Remove read_pdf_text tool (viewer extracts text client-side with pdfjs)
- Remove PdfTextChunk and ReadPdfTextInput types
- Remove loadPdfTextChunk from pdf-loader
- Change default PDF to 'Attention Is All You Need' (1706.03762)
- Update README with modest language
* refactor(pdf-server): Simplify to use URL as ID, rename view_pdf to display_pdf
Major simplifications:
- Use URL directly as identifier (no hashing)
- Remove displayName - show elided URL with full URL as tooltip
- Rename view_pdf to display_pdf with better description
- Update all references from pdfId to url
- Simplify storage key and model context
The tool description now explains it displays an interactive viewer in the chat.
* feat(pdf-server): Normalize arxiv URLs to PDF format
arxiv.org/abs/... -> arxiv.org/pdf/...
Applied both at startup and when loading dynamic URLs.
* docs(pdf-server): Add prompt engineering to display_pdf description
* fix(pdf-server): Sharp rendering on retina displays
Account for devicePixelRatio when rendering canvas:
- Scale canvas dimensions by dpr
- Scale context by dpr
- Keep CSS size at logical pixels
* fix(pdf-server): Normalize arxiv URLs in read_pdf_bytes too
* add to e2e spec
* add to e2e spec
* add to e2e spec
* add to e2e spec
* regen
* chore: regenerate package-lock.json and fix hono vulnerability
* docs: add pdf-server screenshot to READMEs
* regen
* ci: add missing examples to pkg-pr-new publish
* ci: add pdf-server to npm publish examples
* Update README.md
* pdf-server: improve tool response text for better model context
* revert unrelated screenshot changes
* pdf-server: dynamically add arxiv URLs in read_pdf_bytes
Fixes 'PDF not found' error when server restarts between display_pdf
(which adds the entry) and read_pdf_bytes (which previously only looked
up existing entries). Now read_pdf_bytes mirrors display_pdf's logic
and dynamically adds arxiv URLs to the index.
This commit is contained in:
parent
63b05b4fdd
commit
96daa46a1d
6 changed files with 2358 additions and 29 deletions
12
examples/pdf-server/src/global.css
Normal file
12
examples/pdf-server/src/global.css
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
html, body {
|
||||
font-family: system-ui, -apple-system, sans-serif;
|
||||
font-size: 1rem;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
292
examples/pdf-server/src/mcp-app.css
Normal file
292
examples/pdf-server/src/mcp-app.css
Normal file
|
|
@ -0,0 +1,292 @@
|
|||
body {
|
||||
overscroll-behavior-x: none;
|
||||
}
|
||||
|
||||
.main {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
background: var(--bg100, #f5f5f5);
|
||||
color: var(--text000, #1a1a1a);
|
||||
overflow: hidden; /* Prevent scrollbars in inline mode - we request exact size */
|
||||
border-radius: 0.75rem;
|
||||
border: 1px solid var(--bg200, rgba(0, 0, 0, 0.08));
|
||||
}
|
||||
|
||||
/* Loading State */
|
||||
.loading {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
flex: 1;
|
||||
gap: 0.75rem;
|
||||
padding: 2rem;
|
||||
}
|
||||
|
||||
.spinner {
|
||||
width: 24px;
|
||||
height: 24px;
|
||||
border: 2px solid var(--bg200, #e0e0e0);
|
||||
border-top-color: var(--text100, #888);
|
||||
border-radius: 50%;
|
||||
animation: spin 0.8s linear infinite;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
to {
|
||||
transform: rotate(360deg);
|
||||
}
|
||||
}
|
||||
|
||||
#loading-text {
|
||||
font-size: 0.8rem;
|
||||
color: var(--text100, #888);
|
||||
}
|
||||
|
||||
/* Progress Bar */
|
||||
.progress-container {
|
||||
width: 100%;
|
||||
max-width: 200px;
|
||||
height: 3px;
|
||||
background: var(--bg200, #e0e0e0);
|
||||
border-radius: 2px;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.progress-bar {
|
||||
height: 100%;
|
||||
background: var(--text100, #888);
|
||||
width: 0%;
|
||||
transition: width 0.15s ease-out;
|
||||
}
|
||||
|
||||
.progress-text {
|
||||
font-size: 0.7rem;
|
||||
color: var(--text200, #aaa);
|
||||
}
|
||||
|
||||
/* Error State */
|
||||
.error {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
height: 100%;
|
||||
gap: 1rem;
|
||||
padding: 2rem;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.error-icon {
|
||||
font-size: 3rem;
|
||||
}
|
||||
|
||||
#error-message {
|
||||
color: var(--text200, #999);
|
||||
max-width: 400px;
|
||||
}
|
||||
|
||||
/* Viewer Container */
|
||||
.viewer {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
flex: 1;
|
||||
overflow: visible;
|
||||
}
|
||||
|
||||
/* Toolbar */
|
||||
.toolbar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: 0.5rem 1rem;
|
||||
background: var(--bg000, #ffffff);
|
||||
border-bottom: 1px solid var(--bg200, #e0e0e0);
|
||||
flex-shrink: 0;
|
||||
gap: 0.5rem;
|
||||
height: 48px;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
.toolbar-left {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.pdf-title {
|
||||
font-weight: 500;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
font-size: 0.9rem;
|
||||
display: block;
|
||||
max-width: 100%;
|
||||
}
|
||||
|
||||
.toolbar-center {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.toolbar-right {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.25rem;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
/* Page Navigation */
|
||||
.page-nav {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.25rem;
|
||||
}
|
||||
|
||||
.page-input {
|
||||
width: 50px;
|
||||
padding: 0.25rem 0.5rem;
|
||||
border: 1px solid var(--bg200, #e0e0e0);
|
||||
border-radius: 4px;
|
||||
font-size: 0.85rem;
|
||||
text-align: center;
|
||||
background: var(--bg000, #ffffff);
|
||||
color: var(--text000, #1a1a1a);
|
||||
}
|
||||
|
||||
.page-input:focus {
|
||||
outline: none;
|
||||
border-color: var(--text100, #666);
|
||||
}
|
||||
|
||||
.page-input::-webkit-outer-spin-button,
|
||||
.page-input::-webkit-inner-spin-button {
|
||||
-webkit-appearance: none;
|
||||
margin: 0;
|
||||
}
|
||||
.page-input[type="number"] {
|
||||
-moz-appearance: textfield;
|
||||
}
|
||||
|
||||
.total-pages {
|
||||
font-size: 0.85rem;
|
||||
color: var(--text100, #666);
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.nav-btn,
|
||||
.zoom-btn,
|
||||
.fullscreen-btn {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
border: 1px solid var(--bg200, #e0e0e0);
|
||||
border-radius: 4px;
|
||||
background: var(--bg000, #ffffff);
|
||||
color: var(--text000, #1a1a1a);
|
||||
cursor: pointer;
|
||||
font-size: 1rem;
|
||||
transition: all 0.15s ease;
|
||||
}
|
||||
|
||||
.nav-btn:hover:not(:disabled),
|
||||
.zoom-btn:hover:not(:disabled),
|
||||
.fullscreen-btn:hover {
|
||||
background: var(--bg100, #f5f5f5);
|
||||
border-color: var(--bg300, #ccc);
|
||||
}
|
||||
|
||||
.nav-btn:disabled,
|
||||
.zoom-btn:disabled {
|
||||
opacity: 0.4;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.zoom-level {
|
||||
font-size: 0.8rem;
|
||||
color: var(--text100, #666);
|
||||
min-width: 50px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
/* Single Page Canvas Container */
|
||||
.canvas-container {
|
||||
flex: 1;
|
||||
overflow: visible;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: flex-start;
|
||||
padding: 1rem;
|
||||
background: var(--bg200, #e0e0e0);
|
||||
}
|
||||
|
||||
.page-wrapper {
|
||||
position: relative;
|
||||
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.15);
|
||||
background: white;
|
||||
}
|
||||
|
||||
#pdf-canvas {
|
||||
display: block;
|
||||
}
|
||||
|
||||
/* Text Layer for Selection */
|
||||
.text-layer {
|
||||
position: absolute;
|
||||
left: 0;
|
||||
top: 0;
|
||||
right: 0;
|
||||
bottom: 0;
|
||||
overflow: hidden;
|
||||
opacity: 0.2;
|
||||
line-height: 1;
|
||||
text-size-adjust: none;
|
||||
forced-color-adjust: none;
|
||||
transform-origin: 0 0;
|
||||
z-index: 2;
|
||||
}
|
||||
|
||||
.text-layer span,
|
||||
.text-layer br {
|
||||
color: transparent;
|
||||
position: absolute;
|
||||
white-space: pre;
|
||||
cursor: text;
|
||||
transform-origin: 0% 0%;
|
||||
}
|
||||
|
||||
.text-layer ::selection {
|
||||
background: rgba(0, 0, 255, 0.3);
|
||||
}
|
||||
|
||||
.text-layer > span {
|
||||
pointer-events: all;
|
||||
}
|
||||
|
||||
/* Fullscreen mode */
|
||||
.main.fullscreen {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
right: 0;
|
||||
bottom: 0;
|
||||
z-index: 1000;
|
||||
overflow: hidden; /* No scrolling on main - only canvas-container scrolls */
|
||||
border-radius: 0;
|
||||
border: none;
|
||||
}
|
||||
|
||||
.main.fullscreen .viewer {
|
||||
min-height: 0; /* Allow flex item to shrink below content size */
|
||||
}
|
||||
|
||||
.main.fullscreen .canvas-container {
|
||||
min-height: 0; /* Allow flex item to shrink below content size */
|
||||
overflow: auto; /* Scroll within the document area only */
|
||||
}
|
||||
795
examples/pdf-server/src/mcp-app.ts
Normal file
795
examples/pdf-server/src/mcp-app.ts
Normal file
|
|
@ -0,0 +1,795 @@
|
|||
/**
|
||||
* PDF Viewer MCP App
|
||||
*
|
||||
* Interactive PDF viewer with single-page display.
|
||||
* - Fixed height (no auto-resize)
|
||||
* - Text selection via PDF.js TextLayer
|
||||
* - Page navigation, zoom
|
||||
*/
|
||||
import { App, type McpUiHostContext } from "@modelcontextprotocol/ext-apps";
|
||||
import type { CallToolResult } from "@modelcontextprotocol/sdk/types.js";
|
||||
import * as pdfjsLib from "pdfjs-dist";
|
||||
import { TextLayer } from "pdfjs-dist";
|
||||
import "./global.css";
|
||||
import "./mcp-app.css";
|
||||
|
||||
// Configure PDF.js worker
|
||||
pdfjsLib.GlobalWorkerOptions.workerSrc = new URL(
|
||||
"pdfjs-dist/build/pdf.worker.mjs",
|
||||
import.meta.url,
|
||||
).href;
|
||||
|
||||
const log = {
|
||||
info: console.log.bind(console, "[PDF-VIEWER]"),
|
||||
error: console.error.bind(console, "[PDF-VIEWER]"),
|
||||
};
|
||||
|
||||
// State
|
||||
let pdfDocument: pdfjsLib.PDFDocumentProxy | null = null;
|
||||
let pdfBytes: Uint8Array | null = null;
|
||||
let currentPage = 1;
|
||||
let totalPages = 0;
|
||||
let scale = 1.0;
|
||||
let pdfUrl = "";
|
||||
let pdfTitle: string | undefined;
|
||||
let currentRenderTask: { cancel: () => void } | null = null;
|
||||
|
||||
// DOM Elements
|
||||
const mainEl = document.querySelector(".main") as HTMLElement;
|
||||
const loadingEl = document.getElementById("loading")!;
|
||||
const loadingTextEl = document.getElementById("loading-text")!;
|
||||
const errorEl = document.getElementById("error")!;
|
||||
const errorMessageEl = document.getElementById("error-message")!;
|
||||
const viewerEl = document.getElementById("viewer")!;
|
||||
const canvasContainerEl = document.querySelector(".canvas-container")!;
|
||||
const canvasEl = document.getElementById("pdf-canvas") as HTMLCanvasElement;
|
||||
const textLayerEl = document.getElementById("text-layer")!;
|
||||
const titleEl = document.getElementById("pdf-title")!;
|
||||
const pageInputEl = document.getElementById("page-input") as HTMLInputElement;
|
||||
const totalPagesEl = document.getElementById("total-pages")!;
|
||||
const prevBtn = document.getElementById("prev-btn") as HTMLButtonElement;
|
||||
const nextBtn = document.getElementById("next-btn") as HTMLButtonElement;
|
||||
const zoomOutBtn = document.getElementById("zoom-out-btn") as HTMLButtonElement;
|
||||
const zoomInBtn = document.getElementById("zoom-in-btn") as HTMLButtonElement;
|
||||
const zoomLevelEl = document.getElementById("zoom-level")!;
|
||||
const fullscreenBtn = document.getElementById(
|
||||
"fullscreen-btn",
|
||||
) as HTMLButtonElement;
|
||||
const progressContainerEl = document.getElementById("progress-container")!;
|
||||
const progressBarEl = document.getElementById("progress-bar")!;
|
||||
const progressTextEl = document.getElementById("progress-text")!;
|
||||
|
||||
// Track current display mode
|
||||
let currentDisplayMode: "inline" | "fullscreen" = "inline";
|
||||
|
||||
// Layout constants are no longer used - we calculate dynamically from actual element dimensions
|
||||
|
||||
/**
|
||||
* Request the host to resize the app to fit the current PDF page.
|
||||
* Only applies in inline mode - fullscreen mode uses scrolling.
|
||||
*/
|
||||
function requestFitToContent() {
|
||||
if (currentDisplayMode === "fullscreen") {
|
||||
return; // Fullscreen uses scrolling
|
||||
}
|
||||
|
||||
const canvasHeight = canvasEl.height;
|
||||
if (canvasHeight <= 0) {
|
||||
return; // No content yet
|
||||
}
|
||||
|
||||
// Get actual element dimensions
|
||||
const canvasContainerEl = document.querySelector(
|
||||
".canvas-container",
|
||||
) as HTMLElement;
|
||||
const pageWrapperEl = document.querySelector(".page-wrapper") as HTMLElement;
|
||||
const toolbarEl = document.querySelector(".toolbar") as HTMLElement;
|
||||
|
||||
if (!canvasContainerEl || !toolbarEl || !pageWrapperEl) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Get computed styles
|
||||
const containerStyle = getComputedStyle(canvasContainerEl);
|
||||
const paddingTop = parseFloat(containerStyle.paddingTop);
|
||||
const paddingBottom = parseFloat(containerStyle.paddingBottom);
|
||||
|
||||
// Calculate required height:
|
||||
// toolbar + padding-top + page-wrapper height + padding-bottom + buffer
|
||||
const toolbarHeight = toolbarEl.offsetHeight;
|
||||
const pageWrapperHeight = pageWrapperEl.offsetHeight;
|
||||
const BUFFER = 10; // Buffer for sub-pixel rounding and browser quirks
|
||||
const totalHeight =
|
||||
toolbarHeight + paddingTop + pageWrapperHeight + paddingBottom + BUFFER;
|
||||
|
||||
app.sendSizeChanged({ height: totalHeight });
|
||||
}
|
||||
|
||||
// Create app instance
|
||||
// autoResize disabled - app fills its container, doesn't request size changes
|
||||
const app = new App(
|
||||
{ name: "PDF Viewer", version: "1.0.0" },
|
||||
{},
|
||||
{ autoResize: false },
|
||||
);
|
||||
|
||||
// UI State functions
|
||||
function showLoading(text: string) {
|
||||
loadingTextEl.textContent = text;
|
||||
loadingEl.style.display = "flex";
|
||||
errorEl.style.display = "none";
|
||||
viewerEl.style.display = "none";
|
||||
}
|
||||
|
||||
function showError(message: string) {
|
||||
errorMessageEl.textContent = message;
|
||||
loadingEl.style.display = "none";
|
||||
errorEl.style.display = "block";
|
||||
viewerEl.style.display = "none";
|
||||
}
|
||||
|
||||
function showViewer() {
|
||||
loadingEl.style.display = "none";
|
||||
errorEl.style.display = "none";
|
||||
viewerEl.style.display = "flex";
|
||||
}
|
||||
|
||||
function updateControls() {
|
||||
// Show URL with CSS ellipsis, full URL as tooltip, clickable to open
|
||||
titleEl.textContent = pdfUrl;
|
||||
titleEl.title = pdfUrl;
|
||||
titleEl.style.textDecoration = "underline";
|
||||
titleEl.style.cursor = "pointer";
|
||||
titleEl.onclick = () => app.openLink({ url: pdfUrl });
|
||||
pageInputEl.value = String(currentPage);
|
||||
pageInputEl.max = String(totalPages);
|
||||
totalPagesEl.textContent = `of ${totalPages}`;
|
||||
prevBtn.disabled = currentPage <= 1;
|
||||
nextBtn.disabled = currentPage >= totalPages;
|
||||
zoomLevelEl.textContent = `${Math.round(scale * 100)}%`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format page text with optional selection, truncating intelligently.
|
||||
* - Centers window around selection when truncating
|
||||
* - Adds <truncated-content/> markers where text is elided
|
||||
* - If selection itself is too long, truncates inside: <pdf-selection><truncated-content/>...<truncated-content/></pdf-selection>
|
||||
*/
|
||||
function formatPageContent(
|
||||
text: string,
|
||||
maxLength: number,
|
||||
selection?: { start: number; end: number },
|
||||
): string {
|
||||
const T = "<truncated-content/>";
|
||||
|
||||
// No truncation needed
|
||||
if (text.length <= maxLength) {
|
||||
if (!selection) return text;
|
||||
return (
|
||||
text.slice(0, selection.start) +
|
||||
`<pdf-selection>${text.slice(selection.start, selection.end)}</pdf-selection>` +
|
||||
text.slice(selection.end)
|
||||
);
|
||||
}
|
||||
|
||||
// Truncation needed, no selection - just truncate end
|
||||
if (!selection) {
|
||||
return text.slice(0, maxLength) + "\n" + T;
|
||||
}
|
||||
|
||||
// Calculate budgets
|
||||
const selLen = selection.end - selection.start;
|
||||
const overhead = "<pdf-selection></pdf-selection>".length + T.length * 2 + 4;
|
||||
const contextBudget = maxLength - overhead;
|
||||
|
||||
// Selection too long - truncate inside the selection tags
|
||||
if (selLen > contextBudget) {
|
||||
const keepLen = Math.max(100, contextBudget);
|
||||
const halfKeep = Math.floor(keepLen / 2);
|
||||
const selStart = text.slice(selection.start, selection.start + halfKeep);
|
||||
const selEnd = text.slice(selection.end - halfKeep, selection.end);
|
||||
return (
|
||||
T + `<pdf-selection>${T}${selStart}...${selEnd}${T}</pdf-selection>` + T
|
||||
);
|
||||
}
|
||||
|
||||
// Selection fits - center it with context
|
||||
const remainingBudget = contextBudget - selLen;
|
||||
const beforeBudget = Math.floor(remainingBudget / 2);
|
||||
const afterBudget = remainingBudget - beforeBudget;
|
||||
|
||||
const windowStart = Math.max(0, selection.start - beforeBudget);
|
||||
const windowEnd = Math.min(text.length, selection.end + afterBudget);
|
||||
|
||||
const adjStart = selection.start - windowStart;
|
||||
const adjEnd = selection.end - windowStart;
|
||||
const windowText = text.slice(windowStart, windowEnd);
|
||||
|
||||
return (
|
||||
(windowStart > 0 ? T + "\n" : "") +
|
||||
windowText.slice(0, adjStart) +
|
||||
`<pdf-selection>${windowText.slice(adjStart, adjEnd)}</pdf-selection>` +
|
||||
windowText.slice(adjEnd) +
|
||||
(windowEnd < text.length ? "\n" + T : "")
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Find selection position in page text using fuzzy matching.
|
||||
* TextLayer spans may lack spaces between them, so we try both exact and spaceless match.
|
||||
*/
|
||||
function findSelectionInText(
|
||||
pageText: string,
|
||||
selectedText: string,
|
||||
): { start: number; end: number } | undefined {
|
||||
if (!selectedText || selectedText.length <= 2) return undefined;
|
||||
|
||||
// Try exact match
|
||||
let start = pageText.indexOf(selectedText);
|
||||
if (start >= 0) {
|
||||
return { start, end: start + selectedText.length };
|
||||
}
|
||||
|
||||
// Try spaceless match (TextLayer spans may not have spaces)
|
||||
const noSpaceSel = selectedText.replace(/\s+/g, "");
|
||||
const noSpaceText = pageText.replace(/\s+/g, "");
|
||||
const noSpaceStart = noSpaceText.indexOf(noSpaceSel);
|
||||
if (noSpaceStart >= 0) {
|
||||
// Map back to approximate position in original
|
||||
start = Math.floor((noSpaceStart / noSpaceText.length) * pageText.length);
|
||||
return { start, end: start + selectedText.length };
|
||||
}
|
||||
|
||||
return undefined;
|
||||
}
|
||||
|
||||
// Extract text from current page and update model context as markdown
|
||||
async function updatePageContext() {
|
||||
if (!pdfDocument) return;
|
||||
|
||||
try {
|
||||
const page = await pdfDocument.getPage(currentPage);
|
||||
const textContent = await page.getTextContent();
|
||||
const pageText = (textContent.items as Array<{ str?: string }>)
|
||||
.map((item) => item.str || "")
|
||||
.join(" ")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim();
|
||||
|
||||
// Find selection position
|
||||
const sel = window.getSelection();
|
||||
const selectedText = sel?.toString().replace(/\s+/g, " ").trim();
|
||||
const selection = selectedText
|
||||
? findSelectionInText(pageText, selectedText)
|
||||
: undefined;
|
||||
|
||||
if (selection) {
|
||||
log.info(
|
||||
"Selection found:",
|
||||
selectedText?.slice(0, 30),
|
||||
"at",
|
||||
selection.start,
|
||||
);
|
||||
}
|
||||
|
||||
// Format content with selection and truncation
|
||||
const content = formatPageContent(pageText, 5000, selection);
|
||||
|
||||
const markdown = `---
|
||||
title: ${pdfTitle || ""}
|
||||
url: ${pdfUrl}
|
||||
current-page: ${currentPage}/${totalPages}
|
||||
---
|
||||
|
||||
${content}`;
|
||||
|
||||
app.updateModelContext({ content: [{ type: "text", text: markdown }] });
|
||||
} catch (err) {
|
||||
log.error("Error updating context:", err);
|
||||
}
|
||||
}
|
||||
|
||||
// Render state - prevents concurrent renders
|
||||
let isRendering = false;
|
||||
let pendingPage: number | null = null;
|
||||
|
||||
// Render current page with text layer for selection
|
||||
async function renderPage() {
|
||||
if (!pdfDocument) return;
|
||||
|
||||
// If already rendering, queue this page for later
|
||||
if (isRendering) {
|
||||
pendingPage = currentPage;
|
||||
// Cancel current render to speed up
|
||||
if (currentRenderTask) {
|
||||
currentRenderTask.cancel();
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
isRendering = true;
|
||||
pendingPage = null;
|
||||
|
||||
try {
|
||||
const pageToRender = currentPage;
|
||||
const page = await pdfDocument.getPage(pageToRender);
|
||||
const viewport = page.getViewport({ scale });
|
||||
|
||||
// Account for retina displays
|
||||
const dpr = window.devicePixelRatio || 1;
|
||||
const ctx = canvasEl.getContext("2d")!;
|
||||
|
||||
// Set canvas size in pixels (scaled for retina)
|
||||
canvasEl.width = viewport.width * dpr;
|
||||
canvasEl.height = viewport.height * dpr;
|
||||
|
||||
// Set display size in CSS pixels
|
||||
canvasEl.style.width = `${viewport.width}px`;
|
||||
canvasEl.style.height = `${viewport.height}px`;
|
||||
|
||||
// Scale context for retina
|
||||
ctx.scale(dpr, dpr);
|
||||
|
||||
// Clear and setup text layer
|
||||
textLayerEl.innerHTML = "";
|
||||
textLayerEl.style.width = `${viewport.width}px`;
|
||||
textLayerEl.style.height = `${viewport.height}px`;
|
||||
|
||||
// Render canvas - track the task so we can cancel it
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const renderTask = (page.render as any)({
|
||||
canvasContext: ctx,
|
||||
viewport,
|
||||
});
|
||||
currentRenderTask = renderTask;
|
||||
|
||||
try {
|
||||
await renderTask.promise;
|
||||
} catch (renderErr) {
|
||||
// Ignore RenderingCancelledException - it's expected when we cancel
|
||||
if (
|
||||
renderErr instanceof Error &&
|
||||
renderErr.name === "RenderingCancelledException"
|
||||
) {
|
||||
log.info("Render cancelled");
|
||||
return;
|
||||
}
|
||||
throw renderErr;
|
||||
} finally {
|
||||
currentRenderTask = null;
|
||||
}
|
||||
|
||||
// Only continue if this is still the page we want
|
||||
if (pageToRender !== currentPage) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Render text layer for selection
|
||||
const textContent = await page.getTextContent();
|
||||
const textLayer = new TextLayer({
|
||||
textContentSource: textContent,
|
||||
container: textLayerEl,
|
||||
viewport,
|
||||
});
|
||||
await textLayer.render();
|
||||
|
||||
updateControls();
|
||||
updatePageContext();
|
||||
|
||||
// Request host to resize app to fit content (inline mode only)
|
||||
requestFitToContent();
|
||||
} catch (err) {
|
||||
log.error("Error rendering page:", err);
|
||||
showError(`Failed to render page ${currentPage}`);
|
||||
} finally {
|
||||
isRendering = false;
|
||||
|
||||
// If there's a pending page, render it now
|
||||
if (pendingPage !== null && pendingPage !== currentPage) {
|
||||
currentPage = pendingPage;
|
||||
renderPage();
|
||||
} else if (pendingPage === currentPage) {
|
||||
// Re-render the same page (e.g., after zoom change during render)
|
||||
renderPage();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Page persistence
|
||||
function getStorageKey(): string | null {
|
||||
if (!pdfUrl) return null;
|
||||
const ctx = app.getHostContext();
|
||||
const toolId = ctx?.toolInfo?.id ?? pdfUrl;
|
||||
return `pdf:${pdfUrl}:${toolId}`;
|
||||
}
|
||||
|
||||
function saveCurrentPage() {
|
||||
const key = getStorageKey();
|
||||
log.info("saveCurrentPage: key=", key, "page=", currentPage);
|
||||
if (key) {
|
||||
try {
|
||||
localStorage.setItem(key, String(currentPage));
|
||||
log.info("saveCurrentPage: saved successfully");
|
||||
} catch (err) {
|
||||
log.error("saveCurrentPage: error", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function loadSavedPage(): number | null {
|
||||
const key = getStorageKey();
|
||||
log.info("loadSavedPage: key=", key);
|
||||
if (!key) return null;
|
||||
try {
|
||||
const saved = localStorage.getItem(key);
|
||||
log.info("loadSavedPage: saved value=", saved);
|
||||
if (saved) {
|
||||
const page = parseInt(saved, 10);
|
||||
if (!isNaN(page) && page >= 1) {
|
||||
log.info("loadSavedPage: returning page=", page);
|
||||
return page;
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
log.error("loadSavedPage: error", err);
|
||||
}
|
||||
log.info("loadSavedPage: returning null");
|
||||
return null;
|
||||
}
|
||||
|
||||
// Navigation
|
||||
function goToPage(page: number) {
|
||||
const targetPage = Math.max(1, Math.min(page, totalPages));
|
||||
if (targetPage !== currentPage) {
|
||||
currentPage = targetPage;
|
||||
saveCurrentPage();
|
||||
renderPage();
|
||||
}
|
||||
pageInputEl.value = String(currentPage);
|
||||
}
|
||||
|
||||
function prevPage() {
|
||||
goToPage(currentPage - 1);
|
||||
}
|
||||
|
||||
function nextPage() {
|
||||
goToPage(currentPage + 1);
|
||||
}
|
||||
|
||||
function zoomIn() {
|
||||
scale = Math.min(scale + 0.25, 3.0);
|
||||
renderPage();
|
||||
}
|
||||
|
||||
function zoomOut() {
|
||||
scale = Math.max(scale - 0.25, 0.5);
|
||||
renderPage();
|
||||
}
|
||||
|
||||
function resetZoom() {
|
||||
scale = 1.0;
|
||||
renderPage();
|
||||
}
|
||||
|
||||
async function toggleFullscreen() {
|
||||
const ctx = app.getHostContext();
|
||||
if (!ctx?.availableDisplayModes?.includes("fullscreen")) {
|
||||
log.info("Fullscreen not available");
|
||||
return;
|
||||
}
|
||||
|
||||
const newMode = currentDisplayMode === "fullscreen" ? "inline" : "fullscreen";
|
||||
log.info("Requesting display mode:", newMode);
|
||||
|
||||
try {
|
||||
const result = await app.requestDisplayMode({ mode: newMode });
|
||||
log.info("Display mode result:", result);
|
||||
currentDisplayMode = result.mode as "inline" | "fullscreen";
|
||||
updateFullscreenButton();
|
||||
} catch (err) {
|
||||
log.error("Failed to change display mode:", err);
|
||||
}
|
||||
}
|
||||
|
||||
function updateFullscreenButton() {
|
||||
fullscreenBtn.textContent = currentDisplayMode === "fullscreen" ? "⛶" : "⛶";
|
||||
fullscreenBtn.title =
|
||||
currentDisplayMode === "fullscreen" ? "Exit fullscreen" : "Fullscreen";
|
||||
}
|
||||
|
||||
// Event listeners
|
||||
prevBtn.addEventListener("click", prevPage);
|
||||
nextBtn.addEventListener("click", nextPage);
|
||||
zoomOutBtn.addEventListener("click", zoomOut);
|
||||
zoomInBtn.addEventListener("click", zoomIn);
|
||||
fullscreenBtn.addEventListener("click", toggleFullscreen);
|
||||
|
||||
pageInputEl.addEventListener("change", () => {
|
||||
const page = parseInt(pageInputEl.value, 10);
|
||||
if (!isNaN(page)) {
|
||||
goToPage(page);
|
||||
} else {
|
||||
pageInputEl.value = String(currentPage);
|
||||
}
|
||||
});
|
||||
|
||||
pageInputEl.addEventListener("keydown", (e) => {
|
||||
if (e.key === "Enter") {
|
||||
pageInputEl.blur();
|
||||
}
|
||||
});
|
||||
|
||||
// Keyboard navigation
|
||||
document.addEventListener("keydown", (e) => {
|
||||
if (document.activeElement === pageInputEl) return;
|
||||
|
||||
// Ctrl/Cmd+0 to reset zoom
|
||||
if ((e.ctrlKey || e.metaKey) && e.key === "0") {
|
||||
resetZoom();
|
||||
e.preventDefault();
|
||||
return;
|
||||
}
|
||||
|
||||
switch (e.key) {
|
||||
case "Escape":
|
||||
if (currentDisplayMode === "fullscreen") {
|
||||
toggleFullscreen();
|
||||
e.preventDefault();
|
||||
}
|
||||
break;
|
||||
case "ArrowLeft":
|
||||
case "PageUp":
|
||||
prevPage();
|
||||
e.preventDefault();
|
||||
break;
|
||||
case "ArrowRight":
|
||||
case "PageDown":
|
||||
case " ":
|
||||
nextPage();
|
||||
e.preventDefault();
|
||||
break;
|
||||
case "+":
|
||||
case "=":
|
||||
zoomIn();
|
||||
e.preventDefault();
|
||||
break;
|
||||
case "-":
|
||||
zoomOut();
|
||||
e.preventDefault();
|
||||
break;
|
||||
}
|
||||
});
|
||||
|
||||
// Update context when text selection changes (debounced)
|
||||
let selectionUpdateTimeout: ReturnType<typeof setTimeout> | null = null;
|
||||
document.addEventListener("selectionchange", () => {
|
||||
if (selectionUpdateTimeout) clearTimeout(selectionUpdateTimeout);
|
||||
selectionUpdateTimeout = setTimeout(() => {
|
||||
const sel = window.getSelection();
|
||||
const text = sel?.toString().trim();
|
||||
if (text && text.length > 2) {
|
||||
log.info("Selection changed:", text.slice(0, 50));
|
||||
updatePageContext();
|
||||
}
|
||||
}, 300);
|
||||
});
|
||||
|
||||
// Horizontal scroll/swipe to change pages (disabled when zoomed)
|
||||
let horizontalScrollAccumulator = 0;
|
||||
const SCROLL_THRESHOLD = 50;
|
||||
|
||||
canvasContainerEl.addEventListener(
|
||||
"wheel",
|
||||
(event) => {
|
||||
const e = event as WheelEvent;
|
||||
|
||||
// Only intercept horizontal scroll, let vertical scroll through
|
||||
if (Math.abs(e.deltaX) <= Math.abs(e.deltaY)) return;
|
||||
|
||||
// When zoomed, let natural panning happen (no page changes)
|
||||
if (scale > 1.0) return;
|
||||
|
||||
// At 100% zoom, handle page navigation
|
||||
e.preventDefault();
|
||||
horizontalScrollAccumulator += e.deltaX;
|
||||
if (horizontalScrollAccumulator > SCROLL_THRESHOLD) {
|
||||
nextPage();
|
||||
horizontalScrollAccumulator = 0;
|
||||
} else if (horizontalScrollAccumulator < -SCROLL_THRESHOLD) {
|
||||
prevPage();
|
||||
horizontalScrollAccumulator = 0;
|
||||
}
|
||||
},
|
||||
{ passive: false },
|
||||
);
|
||||
|
||||
// Parse tool result
|
||||
function parseToolResult(result: CallToolResult): {
|
||||
url: string;
|
||||
title?: string;
|
||||
pageCount: number;
|
||||
initialPage: number;
|
||||
} | null {
|
||||
return result.structuredContent as {
|
||||
url: string;
|
||||
title?: string;
|
||||
pageCount: number;
|
||||
initialPage: number;
|
||||
} | null;
|
||||
}
|
||||
|
||||
// Chunked binary loading types
|
||||
interface PdfBytesChunk {
|
||||
url: string;
|
||||
bytes: string;
|
||||
offset: number;
|
||||
byteCount: number;
|
||||
totalBytes: number;
|
||||
hasMore: boolean;
|
||||
}
|
||||
|
||||
// Update progress bar
|
||||
function updateProgress(loaded: number, total: number) {
|
||||
const percent = Math.round((loaded / total) * 100);
|
||||
progressBarEl.style.width = `${percent}%`;
|
||||
progressTextEl.textContent = `${(loaded / 1024).toFixed(0)} KB / ${(total / 1024).toFixed(0)} KB (${percent}%)`;
|
||||
}
|
||||
|
||||
// Load PDF in chunks with progress
|
||||
async function loadPdfInChunks(urlToLoad: string): Promise<Uint8Array> {
|
||||
const CHUNK_SIZE = 500 * 1024; // 500KB chunks
|
||||
const chunks: Uint8Array[] = [];
|
||||
let offset = 0;
|
||||
let totalBytes = 0;
|
||||
let hasMore = true;
|
||||
|
||||
// Show progress UI
|
||||
progressContainerEl.style.display = "block";
|
||||
updateProgress(0, 1);
|
||||
|
||||
while (hasMore) {
|
||||
const result = await app.callServerTool({
|
||||
name: "read_pdf_bytes",
|
||||
arguments: { url: urlToLoad, offset, byteCount: CHUNK_SIZE },
|
||||
});
|
||||
|
||||
// Check for errors
|
||||
if (result.isError) {
|
||||
const errorText = result.content
|
||||
?.map((c) => ("text" in c ? c.text : ""))
|
||||
.join(" ");
|
||||
throw new Error(`Tool error: ${errorText}`);
|
||||
}
|
||||
|
||||
if (!result.structuredContent) {
|
||||
throw new Error("No structuredContent in tool response");
|
||||
}
|
||||
|
||||
const chunk = result.structuredContent as unknown as PdfBytesChunk;
|
||||
totalBytes = chunk.totalBytes;
|
||||
hasMore = chunk.hasMore;
|
||||
|
||||
// Decode base64 chunk
|
||||
const binaryString = atob(chunk.bytes);
|
||||
const bytes = new Uint8Array(binaryString.length);
|
||||
for (let i = 0; i < binaryString.length; i++) {
|
||||
bytes[i] = binaryString.charCodeAt(i);
|
||||
}
|
||||
chunks.push(bytes);
|
||||
|
||||
offset += chunk.byteCount;
|
||||
updateProgress(offset, totalBytes);
|
||||
}
|
||||
|
||||
// Combine all chunks
|
||||
const fullPdf = new Uint8Array(totalBytes);
|
||||
let pos = 0;
|
||||
for (const chunk of chunks) {
|
||||
fullPdf.set(chunk, pos);
|
||||
pos += chunk.length;
|
||||
}
|
||||
|
||||
log.info(
|
||||
`PDF loaded: ${(totalBytes / 1024).toFixed(0)} KB in ${chunks.length} chunks`,
|
||||
);
|
||||
return fullPdf;
|
||||
}
|
||||
|
||||
// Handle tool result
|
||||
app.ontoolresult = async (result) => {
|
||||
log.info("Received tool result:", result);
|
||||
|
||||
const parsed = parseToolResult(result);
|
||||
if (!parsed) {
|
||||
showError("Invalid tool result");
|
||||
return;
|
||||
}
|
||||
|
||||
pdfUrl = parsed.url;
|
||||
pdfTitle = parsed.title;
|
||||
totalPages = parsed.pageCount;
|
||||
|
||||
// Restore saved page or use initial page
|
||||
const savedPage = loadSavedPage();
|
||||
currentPage =
|
||||
savedPage && savedPage <= parsed.pageCount ? savedPage : parsed.initialPage;
|
||||
|
||||
log.info(
|
||||
"URL:",
|
||||
pdfUrl,
|
||||
"Pages:",
|
||||
parsed.pageCount,
|
||||
"Starting:",
|
||||
currentPage,
|
||||
);
|
||||
|
||||
showLoading("Loading PDF...");
|
||||
|
||||
try {
|
||||
pdfBytes = await loadPdfInChunks(pdfUrl);
|
||||
|
||||
showLoading("Rendering PDF...");
|
||||
|
||||
pdfDocument = await pdfjsLib.getDocument({ data: pdfBytes }).promise;
|
||||
totalPages = pdfDocument.numPages;
|
||||
|
||||
log.info("PDF loaded, pages:", totalPages);
|
||||
|
||||
showViewer();
|
||||
renderPage();
|
||||
} catch (err) {
|
||||
log.error("Error loading PDF:", err);
|
||||
showError(err instanceof Error ? err.message : String(err));
|
||||
}
|
||||
};
|
||||
|
||||
app.onerror = (err) => {
|
||||
log.error("App error:", err);
|
||||
showError(err instanceof Error ? err.message : String(err));
|
||||
};
|
||||
|
||||
function handleHostContextChanged(ctx: McpUiHostContext) {
|
||||
log.info("Host context changed:", ctx);
|
||||
|
||||
// Apply safe area insets
|
||||
if (ctx.safeAreaInsets) {
|
||||
mainEl.style.paddingTop = `${ctx.safeAreaInsets.top}px`;
|
||||
mainEl.style.paddingRight = `${ctx.safeAreaInsets.right}px`;
|
||||
mainEl.style.paddingBottom = `${ctx.safeAreaInsets.bottom}px`;
|
||||
mainEl.style.paddingLeft = `${ctx.safeAreaInsets.left}px`;
|
||||
}
|
||||
|
||||
// Log containerDimensions for debugging
|
||||
if (ctx.containerDimensions) {
|
||||
log.info("Container dimensions:", ctx.containerDimensions);
|
||||
}
|
||||
|
||||
// Handle display mode changes
|
||||
if (ctx.displayMode) {
|
||||
const wasFullscreen = currentDisplayMode === "fullscreen";
|
||||
currentDisplayMode = ctx.displayMode as "inline" | "fullscreen";
|
||||
if (ctx.displayMode === "fullscreen") {
|
||||
mainEl.classList.add("fullscreen");
|
||||
log.info("Fullscreen mode enabled");
|
||||
} else {
|
||||
mainEl.classList.remove("fullscreen");
|
||||
log.info("Inline mode");
|
||||
// When exiting fullscreen, request resize to fit content
|
||||
if (wasFullscreen && pdfDocument) {
|
||||
requestFitToContent();
|
||||
}
|
||||
}
|
||||
updateFullscreenButton();
|
||||
}
|
||||
}
|
||||
|
||||
app.onhostcontextchanged = handleHostContextChanged;
|
||||
|
||||
// Connect to host
|
||||
app.connect().then(() => {
|
||||
log.info("Connected to host");
|
||||
const ctx = app.getHostContext();
|
||||
if (ctx) {
|
||||
handleHostContextChanged(ctx);
|
||||
}
|
||||
});
|
||||
57
examples/pdf-server/src/pdf-indexer.ts
Normal file
57
examples/pdf-server/src/pdf-indexer.ts
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
/**
|
||||
* PDF Indexer
|
||||
*/
|
||||
import path from "node:path";
|
||||
import type { PdfIndex, PdfEntry } from "./types.js";
|
||||
import { populatePdfMetadata } from "./pdf-loader.js";
|
||||
|
||||
/** Check if URL is from arxiv.org */
|
||||
export function isArxivUrl(url: string): boolean {
|
||||
return /^https?:\/\/arxiv\.org\//.test(url);
|
||||
}
|
||||
|
||||
/** Normalize arxiv URL to PDF format */
|
||||
export function normalizeArxivUrl(url: string): string {
|
||||
return url.replace(/arxiv\.org\/abs\//, "arxiv.org/pdf/");
|
||||
}
|
||||
|
||||
/** Check if URL is a file:// URL */
|
||||
export function isFileUrl(url: string): boolean {
|
||||
return url.startsWith("file://");
|
||||
}
|
||||
|
||||
/** Convert local path to file:// URL */
|
||||
export function toFileUrl(filePath: string): string {
|
||||
return `file://${path.resolve(filePath)}`;
|
||||
}
|
||||
|
||||
/** Create a PdfEntry from a URL */
|
||||
export function createEntry(url: string): PdfEntry {
|
||||
return {
|
||||
url,
|
||||
metadata: { pageCount: 0, fileSizeBytes: 0 },
|
||||
};
|
||||
}
|
||||
|
||||
/** Build index from a list of URLs */
|
||||
export async function buildPdfIndex(urls: string[]): Promise<PdfIndex> {
|
||||
const entries: PdfEntry[] = [];
|
||||
|
||||
for (const url of urls) {
|
||||
console.error(`[indexer] Loading: ${url}`);
|
||||
const entry = createEntry(url);
|
||||
await populatePdfMetadata(entry);
|
||||
entries.push(entry);
|
||||
}
|
||||
|
||||
console.error(`[indexer] Indexed ${entries.length} PDFs`);
|
||||
return { entries };
|
||||
}
|
||||
|
||||
/** Find entry by URL */
|
||||
export function findEntryByUrl(
|
||||
index: PdfIndex,
|
||||
url: string,
|
||||
): PdfEntry | undefined {
|
||||
return index.entries.find((e) => e.url === url);
|
||||
}
|
||||
136
examples/pdf-server/src/pdf-loader.ts
Normal file
136
examples/pdf-server/src/pdf-loader.ts
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
/**
|
||||
* PDF Loader - Loads PDFs and extracts content in chunks
|
||||
*
|
||||
* Demonstrates:
|
||||
* - Chunked data loading with size limits
|
||||
* - HTTP Range requests for streaming
|
||||
* - Caching for repeated requests
|
||||
*/
|
||||
import fs from "node:fs/promises";
|
||||
import type { PdfEntry, PdfBytesChunk } from "./types.js";
|
||||
import { MAX_CHUNK_BYTES } from "./types.js";
|
||||
import { isFileUrl } from "./pdf-indexer.js";
|
||||
|
||||
// Cache for loaded PDFs
|
||||
const pdfCache = new Map<string, Uint8Array>();
|
||||
|
||||
// Lazy-load pdfjs
|
||||
let pdfjs: typeof import("pdfjs-dist");
|
||||
async function getPdfjs() {
|
||||
if (!pdfjs) {
|
||||
pdfjs = await import("pdfjs-dist/legacy/build/pdf.mjs");
|
||||
}
|
||||
return pdfjs;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// PDF Data Loading
|
||||
// ============================================================================
|
||||
|
||||
/** Fetch PDF data (with caching) */
|
||||
export async function loadPdfData(entry: PdfEntry): Promise<Uint8Array> {
|
||||
const cached = pdfCache.get(entry.url);
|
||||
if (cached) return cached;
|
||||
|
||||
console.error(`[loader] Fetching: ${entry.url}`);
|
||||
|
||||
let data: Uint8Array;
|
||||
if (isFileUrl(entry.url)) {
|
||||
const filePath = entry.url.replace("file://", "");
|
||||
data = new Uint8Array(await fs.readFile(filePath));
|
||||
} else {
|
||||
const response = await fetch(entry.url);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to fetch: ${response.status}`);
|
||||
}
|
||||
data = new Uint8Array(await response.arrayBuffer());
|
||||
}
|
||||
|
||||
pdfCache.set(entry.url, data);
|
||||
return data;
|
||||
}
|
||||
|
||||
/** Try HTTP Range request for partial content */
|
||||
async function fetchRange(
|
||||
url: string,
|
||||
start: number,
|
||||
end: number,
|
||||
): Promise<{ data: Uint8Array; total: number } | null> {
|
||||
try {
|
||||
const res = await fetch(url, {
|
||||
headers: { Range: `bytes=${start}-${end}` },
|
||||
});
|
||||
if (res.status !== 206) return null;
|
||||
|
||||
const total = parseInt(
|
||||
res.headers.get("Content-Range")?.split("/")[1] || "0",
|
||||
);
|
||||
return { data: new Uint8Array(await res.arrayBuffer()), total };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Chunked Binary Loading (demonstrates size-limited responses)
|
||||
// ============================================================================
|
||||
|
||||
export async function loadPdfBytesChunk(
|
||||
entry: PdfEntry,
|
||||
offset = 0,
|
||||
byteCount = MAX_CHUNK_BYTES,
|
||||
): Promise<PdfBytesChunk> {
|
||||
// Try Range request first (streaming without full download)
|
||||
if (!pdfCache.has(entry.url)) {
|
||||
const range = await fetchRange(entry.url, offset, offset + byteCount - 1);
|
||||
if (range) {
|
||||
return {
|
||||
url: entry.url,
|
||||
bytes: Buffer.from(range.data).toString("base64"),
|
||||
offset,
|
||||
byteCount: range.data.length,
|
||||
totalBytes: range.total,
|
||||
hasMore: offset + range.data.length < range.total,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: load full PDF and slice
|
||||
const data = await loadPdfData(entry);
|
||||
const chunk = data.slice(offset, offset + byteCount);
|
||||
|
||||
return {
|
||||
url: entry.url,
|
||||
bytes: Buffer.from(chunk).toString("base64"),
|
||||
offset,
|
||||
byteCount: chunk.length,
|
||||
totalBytes: data.length,
|
||||
hasMore: offset + chunk.length < data.length,
|
||||
};
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Metadata Extraction
|
||||
// ============================================================================
|
||||
|
||||
export async function populatePdfMetadata(entry: PdfEntry): Promise<void> {
|
||||
try {
|
||||
const lib = await getPdfjs();
|
||||
const data = await loadPdfData(entry);
|
||||
|
||||
entry.metadata.fileSizeBytes = data.length;
|
||||
|
||||
const pdf = await lib.getDocument({ data: new Uint8Array(data) }).promise;
|
||||
entry.metadata.pageCount = pdf.numPages;
|
||||
|
||||
const info = (await pdf.getMetadata()).info as
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
if (info?.Title) entry.metadata.title = String(info.Title);
|
||||
if (info?.Author) entry.metadata.author = String(info.Author);
|
||||
|
||||
await pdf.destroy();
|
||||
} catch (err) {
|
||||
console.error(`[loader] Metadata error: ${err}`);
|
||||
}
|
||||
}
|
||||
51
examples/pdf-server/src/types.ts
Normal file
51
examples/pdf-server/src/types.ts
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
/**
|
||||
* PDF Server Types - Simplified for didactic purposes
|
||||
*/
|
||||
import { z } from "zod";
|
||||
|
||||
// ============================================================================
|
||||
// Core Types
|
||||
// ============================================================================
|
||||
|
||||
export const PdfMetadataSchema = z.object({
|
||||
title: z.string().optional(),
|
||||
author: z.string().optional(),
|
||||
pageCount: z.number(),
|
||||
fileSizeBytes: z.number(),
|
||||
});
|
||||
export type PdfMetadata = z.infer<typeof PdfMetadataSchema>;
|
||||
|
||||
export const PdfEntrySchema = z.object({
|
||||
url: z.string(), // Also serves as unique ID
|
||||
metadata: PdfMetadataSchema,
|
||||
});
|
||||
export type PdfEntry = z.infer<typeof PdfEntrySchema>;
|
||||
|
||||
export const PdfIndexSchema = z.object({
|
||||
entries: z.array(PdfEntrySchema),
|
||||
});
|
||||
export type PdfIndex = z.infer<typeof PdfIndexSchema>;
|
||||
|
||||
// ============================================================================
|
||||
// Chunked Binary Loading
|
||||
// ============================================================================
|
||||
|
||||
/** Max bytes per response chunk */
|
||||
export const MAX_CHUNK_BYTES = 500 * 1024; // 500KB
|
||||
|
||||
export const PdfBytesChunkSchema = z.object({
|
||||
url: z.string(),
|
||||
bytes: z.string(), // base64
|
||||
offset: z.number(),
|
||||
byteCount: z.number(),
|
||||
totalBytes: z.number(),
|
||||
hasMore: z.boolean(),
|
||||
});
|
||||
export type PdfBytesChunk = z.infer<typeof PdfBytesChunkSchema>;
|
||||
|
||||
export const ReadPdfBytesInputSchema = z.object({
|
||||
url: z.string().describe("PDF URL"),
|
||||
offset: z.number().min(0).default(0).describe("Byte offset"),
|
||||
byteCount: z.number().default(MAX_CHUNK_BYTES).describe("Bytes to read"),
|
||||
});
|
||||
export type ReadPdfBytesInput = z.infer<typeof ReadPdfBytesInputSchema>;
|
||||
Loading…
Reference in a new issue