perf(pdf-server): lazy form extraction via range transport + incremental viewer scans (#639)
* perf(pdf-server): lazy form extraction via range transport + incremental viewer scans Server: display_pdf now opens the document via PDFDataRangeTransport (disableAutoFetch) and only runs the per-page form/annotation walk when getFieldObjects() is non-empty, so form-free PDFs are probed with ~10-25% of bytes instead of a full download. The unused viewFieldInfo Map is removed. Viewer: getDocument sets disableAutoFetch/disableStream; baseline annotation scan and field-name mapping run lazily per rendered page instead of walking every page after load, so first paint no longer schedules a full-file pull. E2E: new range-counting HTTPS fixture (W-9 for forms, generated text+image PDF for no-forms) with stallAfterBytes control, and four regression tests asserting form fields are returned, <30% served on no-forms display_pdf, first page renders while later ranges are stalled, and overlap stays bounded. * fix(pdf-server): handle separated field/widget trees in extractFormSchema pdfjs getFieldObjects() returns the full field-tree array. For PDFs with a separated structure (pdf-lib, some authoring tools) the typed widget sits at fields[1+] behind a typeless container at fields[0]; the previous code only inspected fields[0] and skipped them all. Pick the first entry with a non-empty type instead. Makes the e2e forms.pdf fixture fully generated (no checked-in third-party asset on the hot path); fw9.pdf stays as a unit-test fixture for the hierarchical/XFA case. * fix(pdf-server): address range-transport hangs and lazy-scan tombstone loss PdfCacheRangeTransport: - abort() is a no-op stub on PDFDataRangeTransport (it's the hook pdfjs calls *on* the transport, not an upstream error channel). Expose a .failed promise that rejects on the first fetch error and race every pdfjs await against it in display_pdf, so transient network errors surface into the existing catch instead of hanging the tool call. - pdfjs coalesces adjacent missing chunks into one unbounded requestDataRange; readPdfRange clamps each call to MAX_CHUNK_BYTES. Loop and deliver in slices so every requested chunk is marked loaded. Viewer (mcp-app.ts): - The lazy per-page baseline scan left pdfBaselineAnnotations incomplete, so persistAnnotations and getAnnotatedPdfBytes silently dropped restoredRemovedIds tombstones for unvisited pages. Union those ids into the computed diff and removedRefs. Test fixture: release stalled handlers before resetStats/close so a failing stalled test doesn't hang afterAll; fail fast if started with NODE_ENV=production. NODE_TLS_REJECT_UNAUTHORIZED scope documented (full per-process scoping needs a validateUrl localhost allow, tracked separately). * test(pdf-server): bridge coverage gaps and switch fixture to gated loopback HTTP PdfCacheRangeTransport.deliver(): pdf.js's reader is keyed by the original begin and removed after one delivery, so accumulate slices and call onDataRange once with the full buffer (the previous multi-call approach threw inside pdfjs). Covered by a new integration test that drives getDocument()/getPage(1) on a >1MB PDF through a clamping in-memory readPdfRange. validateUrl: allow http://127.0.0.1|localhost|[::1] only when PDF_SERVER_ALLOW_LOOPBACK_HTTP is set, so a remote deploy can't be made to probe its own ports. Covered by env-on/off unit tests. Fixture switched to plain HTTP (no openssl, no NODE_TLS_REJECT_UNAUTHORIZED). Adds /error.pdf (500s after 50KB) and two e2e tests: page 2 renders after stall release (>512KB object path), and display_pdf returns gracefully on mid-load 500. Existing <30% test now samples stats before the iframe loads. New unit tests: display_pdf returns (not hangs) on mid-load fetch failure via in-memory MCP client; computeDiff/serializeDiff contract tests pinning the restoredRemovedIds tombstone-preservation behaviour. * test(pdf-server): e2e for restoredRemovedIds tombstone preservation through viewer Adds /with-native-annot.pdf fixture (2 pages, native /Text annot on page 2) and a Playwright test that drives delete on page 2 → iframe reload → interact add_annotations on page 1 (persist with page 2 unscanned) → assert localStorage diff still contains the removed id → page 2 still shows it gone. Covers the mcp-app.ts glue the unit tests can't reach. * refactor(pdf-server): simplify per round-2 review and fix false-pass test server.ts: extract probeFormFields() so display_pdf calls one helper instead of inlining 40 lines of transport/orFail plumbing; make extractFormSchema's fieldObjects param required (the optional branch only existed for a test helper). server.test.ts: the >1MB integration test now forces the image XObject fetch via getOperatorList() and asserts max requestDataRange span > MAX_CHUNK_BYTES, so it can't go vacuous. Dedupe makeRandomJpeg by importing from the fixture. mcp-app.ts: restore per-annotation try/catch isolation in the lazy baseline scan (a throw was skipping AnnotationLayer.render); only carry forward restoredRemovedIds while the baseline scan is incomplete so a stale id can't pin dirty=true once every page is scanned. tests: bump fixture JPEG to 1.1MB; consolidate 7 e2e tests to 4 (merge overlap into the byte-budget test, merge stall+page-2, drop the error-e2e duplicate of the unit integration test); delete run-fixture.mjs; drop /error.pdf and ERROR_AFTER_BYTES; rename tombstone test's describe and clarify page-2 covers the viewer transport. * fix(pdf-server): preserve partial probeFormFields results when extractFormFieldInfo throws * test(pdf-server): fix tombstone e2e — deleted native annotations become a cleared card, not removed from DOM * test(pdf-server): mark tombstone e2e as fixme — needs basic-host iframe-reload replay support * test(pdf-server): link tombstone fixme to tracking issue #642
This commit is contained in:
Olivier Chafik
GitHub
parent
30a78b60b4
commit
ba31be1f91
3 changed files with 1133 additions and 106 deletions
271
tests/e2e/pdf-incremental-load.spec.ts
Normal file
271
tests/e2e/pdf-incremental-load.spec.ts
Normal file
|
|
@ -0,0 +1,271 @@
|
|||
import { test, expect, type Page } from "@playwright/test";
|
||||
import {
|
||||
startRangeServer,
|
||||
type RangeServer,
|
||||
} from "../helpers/range-counting-server";
|
||||
|
||||
/**
|
||||
* Regression guard for incremental PDF loading.
|
||||
*
|
||||
* Asserts that display_pdf does not pull the entire file before the viewer
|
||||
* starts streaming, that form schema is still returned in the initial response,
|
||||
* and that no byte range is fetched server-side more than once.
|
||||
*
|
||||
* The "noforms <30%" test is the load-bearing regression check: it fails on the
|
||||
* pre-range-transport implementation (which downloads 100% during display_pdf
|
||||
* for form analysis) and passes once form extraction uses range transport.
|
||||
*/
|
||||
|
||||
test.setTimeout(120_000);
|
||||
|
||||
let rangeServer: RangeServer;
|
||||
|
||||
test.beforeAll(async () => {
|
||||
rangeServer = await startRangeServer();
|
||||
});
|
||||
|
||||
test.afterAll(async () => {
|
||||
await rangeServer.close();
|
||||
});
|
||||
|
||||
test.beforeEach(() => {
|
||||
rangeServer.resetStats();
|
||||
});
|
||||
|
||||
function getAppFrame(page: Page) {
|
||||
return page.frameLocator("iframe").first().frameLocator("iframe").first();
|
||||
}
|
||||
|
||||
async function waitForAppLoad(page: Page) {
|
||||
const outerFrame = page.frameLocator("iframe").first();
|
||||
await expect(outerFrame.locator("iframe")).toBeVisible({ timeout: 30_000 });
|
||||
}
|
||||
|
||||
/**
|
||||
* Load basic-host, select PDF Server, call display_pdf with a custom URL.
|
||||
* Resolves once the tool result panel appears (server-side display_pdf done);
|
||||
* does NOT wait for the viewer iframe — call waitForAppLoad separately so
|
||||
* byte-count assertions can isolate server-side fetches from viewer fetches.
|
||||
*/
|
||||
async function displayPdf(page: Page, url: string) {
|
||||
await page.goto("/?theme=hide");
|
||||
await expect(page.locator("select").first()).toBeEnabled({ timeout: 30_000 });
|
||||
await page.locator("select").first().selectOption({ label: "PDF Server" });
|
||||
await page.locator("textarea").fill(JSON.stringify({ url }));
|
||||
await page.click('button:has-text("Call Tool")');
|
||||
await expect(page.locator('text="📤 Tool Result"').first()).toBeVisible({
|
||||
timeout: 30_000,
|
||||
});
|
||||
}
|
||||
|
||||
/** Read and parse the most recent tool result's structuredContent. */
|
||||
async function readStructuredContent(
|
||||
page: Page,
|
||||
): Promise<Record<string, unknown>> {
|
||||
const resultPanel = page.locator('text="📤 Tool Result"').first();
|
||||
await expect(resultPanel).toBeVisible({ timeout: 30_000 });
|
||||
await resultPanel.click();
|
||||
const pre = page.locator("pre").last();
|
||||
await expect(pre).toBeVisible({ timeout: 5_000 });
|
||||
const raw = (await pre.textContent()) ?? "{}";
|
||||
const parsed = JSON.parse(raw) as { structuredContent?: object };
|
||||
return (parsed.structuredContent ?? {}) as Record<string, unknown>;
|
||||
}
|
||||
|
||||
async function waitForFirstPageRendered(page: Page) {
|
||||
const canvas = getAppFrame(page).locator("canvas").first();
|
||||
await expect(canvas).toBeVisible({ timeout: 30_000 });
|
||||
await expect
|
||||
.poll(async () => canvas.evaluate((c: HTMLCanvasElement) => c.width))
|
||||
.toBeGreaterThan(0);
|
||||
}
|
||||
|
||||
test.describe("PDF Server — incremental loading", () => {
|
||||
test("display_pdf on a form PDF returns form fields in initial response", async ({
|
||||
page,
|
||||
}) => {
|
||||
await displayPdf(page, `${rangeServer.baseUrl}/forms.pdf`);
|
||||
await waitForAppLoad(page);
|
||||
const sc = await readStructuredContent(page);
|
||||
const fields = sc.formFields as Array<{ name: string }> | undefined;
|
||||
expect(fields?.map((f) => f.name).sort()).toEqual([
|
||||
"city",
|
||||
"email",
|
||||
"name",
|
||||
"notes",
|
||||
"phone",
|
||||
]);
|
||||
});
|
||||
|
||||
test("display_pdf on a no-forms PDF stays under byte budget and bounded overlap", async ({
|
||||
page,
|
||||
}) => {
|
||||
const fileSize = rangeServer.fileSizes["/noforms.pdf"];
|
||||
await displayPdf(page, `${rangeServer.baseUrl}/noforms.pdf`);
|
||||
|
||||
// Measure before the viewer iframe loads so the count reflects only the
|
||||
// server-side display_pdf range fetches.
|
||||
expect(rangeServer.stats().totalBytesServed).toBeLessThan(fileSize * 0.3);
|
||||
|
||||
await waitForAppLoad(page);
|
||||
const sc = await readStructuredContent(page);
|
||||
expect(sc.formFields).toBeUndefined();
|
||||
|
||||
await waitForFirstPageRendered(page);
|
||||
// Server-side display_pdf and the viewer each open the document
|
||||
// independently, so the xref/trailer/catalog is fetched twice (≈25%).
|
||||
// This guards against the pre-range-transport behavior where the server
|
||||
// alone pulled 100% (then 200% with the double-parse), giving overlap >>
|
||||
// file size once the viewer also loaded.
|
||||
expect(rangeServer.stats().overlapBytes).toBeLessThan(fileSize * 0.5);
|
||||
});
|
||||
|
||||
test("first page renders under stall, then page 2 renders the >512KB image after release", async ({
|
||||
page,
|
||||
}) => {
|
||||
const fileSize = rangeServer.fileSizes["/noforms.pdf"];
|
||||
// Allow ~40% through (header + trailer/xref + page-1 content) then stall.
|
||||
// The 1.1MB image stream referenced only by pages 2+ is the bulk.
|
||||
const budget = Math.floor(fileSize * 0.4);
|
||||
await displayPdf(
|
||||
page,
|
||||
`${rangeServer.baseUrl}/noforms.pdf?stallAfterBytes=${budget}`,
|
||||
);
|
||||
await waitForAppLoad(page);
|
||||
await waitForFirstPageRendered(page);
|
||||
expect(rangeServer.stats().totalBytesServed).toBeLessThan(fileSize);
|
||||
|
||||
rangeServer.release();
|
||||
const app = getAppFrame(page);
|
||||
await app.locator("#next-btn").click();
|
||||
await expect(app.locator("#page-input")).toHaveValue("2", {
|
||||
timeout: 30_000,
|
||||
});
|
||||
// Page 2 references the ~1.1MB embedded JPEG. Rendering it exercises the
|
||||
// viewer's range transport on a >MAX_CHUNK_BYTES object after the stall
|
||||
// is released. (The server-side PdfCacheRangeTransport accumulate-once
|
||||
// path is covered by the unit test in server.test.ts; on noforms.pdf
|
||||
// display_pdf bails at the empty getFieldObjects() check before touching
|
||||
// the image stream, so this test does not exercise it.)
|
||||
await waitForFirstPageRendered(page);
|
||||
expect(rangeServer.stats().totalBytesServed).toBeGreaterThan(
|
||||
fileSize * 0.9,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// Kept in this spec because the test needs an HTTP-served PDF and the
|
||||
// range-counting fixture is the convenient place; it does not use any
|
||||
// byte-accounting features.
|
||||
test.describe("PDF Server — annotation tombstone preservation", () => {
|
||||
// FIXME(https://github.com/modelcontextprotocol/ext-apps/issues/642):
|
||||
// basic-host doesn't replay the cached tool result on inner-iframe reload,
|
||||
// and a fresh display_pdf call gets a new toolId → new storage key, so the
|
||||
// restore-from-localStorage path can't be reached. The fix itself is covered
|
||||
// by the computeDiff/serializeDiff contract tests in
|
||||
// src/pdf-annotations.test.ts.
|
||||
test.fixme("deleted native annotation tombstone survives a persist before its page is scanned", async ({
|
||||
page,
|
||||
}) => {
|
||||
// Regression for the lazy baseline scan: restoredRemovedIds must be
|
||||
// unioned into persistAnnotations() and getAnnotatedPdfBytes() so a
|
||||
// delete on page 2 isn't silently dropped when an unrelated edit on
|
||||
// page 1 triggers a persist before page 2 has been re-scanned.
|
||||
|
||||
await displayPdf(page, `${rangeServer.baseUrl}/with-native-annot.pdf`);
|
||||
await waitForAppLoad(page);
|
||||
await waitForFirstPageRendered(page);
|
||||
const sc = await readStructuredContent(page);
|
||||
const viewUUID = sc.viewUUID as string;
|
||||
expect(viewUUID).toBeTruthy();
|
||||
|
||||
const app = getAppFrame(page);
|
||||
|
||||
// 1. Go to page 2, open the panel, delete the native annotation via UI.
|
||||
await app.locator("#next-btn").click();
|
||||
await expect(app.locator("#page-input")).toHaveValue("2");
|
||||
await app.locator("#annotations-btn").click();
|
||||
const nativeCard = app.locator(
|
||||
'.annotation-card[data-annotation-id^="pdf-"]',
|
||||
);
|
||||
await expect(nativeCard).toBeVisible({ timeout: 10_000 });
|
||||
const nativeId = await nativeCard.getAttribute("data-annotation-id");
|
||||
expect(nativeId).toMatch(/^pdf-\d+R?$/);
|
||||
await nativeCard.locator(".annotation-card-delete").click();
|
||||
// Deleting a native annotation re-renders the card as a crossed-out
|
||||
// tombstone (annotation-panel.ts createRemovedAnnotationCard) with the
|
||||
// same data-annotation-id — it doesn't disappear from the DOM.
|
||||
await expect(nativeCard).toHaveClass(/annotation-card-cleared/);
|
||||
|
||||
// 2. Back to page 1 so the post-reload viewer restores there (page 2
|
||||
// must stay unscanned until the very end).
|
||||
await app.locator("#page-input").fill("1");
|
||||
await app.locator("#page-input").press("Enter");
|
||||
await expect(app.locator("#page-input")).toHaveValue("1");
|
||||
|
||||
// 3. Capture the annotation localStorage key and confirm the delete was
|
||||
// persisted.
|
||||
const storageKey = await app
|
||||
.locator("body")
|
||||
.evaluate(
|
||||
() =>
|
||||
Object.keys(localStorage).find(
|
||||
(k) => k.startsWith("pdf-annot:") || k.endsWith(":annotations"),
|
||||
) ?? null,
|
||||
);
|
||||
expect(storageKey).toBeTruthy();
|
||||
const diffBefore = await app
|
||||
.locator("body")
|
||||
.evaluate((_, k) => localStorage.getItem(k), storageKey!);
|
||||
expect(JSON.parse(diffBefore!).removed).toContain(nativeId);
|
||||
|
||||
// 4. Reload the inner viewer iframe ONLY (basic-host keeps the same
|
||||
// cached tool result → same viewUUID/toolId → same storage key).
|
||||
// restoreAnnotations() now seeds restoredRemovedIds from localStorage
|
||||
// while the lazy scan has only seen page 1.
|
||||
await app.locator("body").evaluate(() => location.reload());
|
||||
await waitForFirstPageRendered(page);
|
||||
await expect(app.locator("#page-input")).toHaveValue("1");
|
||||
|
||||
// 5. Trigger persistAnnotations() via an unrelated edit on page 1 — the
|
||||
// bug scenario: page 2 has not been scanned yet.
|
||||
const toolSelect = page.locator("select").nth(1);
|
||||
await toolSelect.selectOption("interact");
|
||||
await page.locator("textarea").fill(
|
||||
JSON.stringify({
|
||||
viewUUID,
|
||||
action: "add_annotations",
|
||||
annotations: [
|
||||
{
|
||||
id: "probe-on-page-1",
|
||||
type: "highlight",
|
||||
page: 1,
|
||||
rects: [{ x: 50, y: 700, width: 100, height: 12 }],
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
await page.click('button:has-text("Call Tool")');
|
||||
await expect(
|
||||
app.locator('[data-annotation-id="probe-on-page-1"]'),
|
||||
).toHaveCount(1, { timeout: 10_000 });
|
||||
|
||||
// 6. Load-bearing assertion: the persisted diff still carries the
|
||||
// tombstone. Pre-fix, computeDiff() over the page-1-only baseline
|
||||
// yielded removed=[], overwriting it.
|
||||
const diffAfter = await app
|
||||
.locator("body")
|
||||
.evaluate((_, k) => localStorage.getItem(k), storageKey!);
|
||||
const removedAfter: string[] = JSON.parse(diffAfter!).removed;
|
||||
expect(removedAfter).toContain(nativeId);
|
||||
|
||||
// 7. Belt-and-suspenders: navigate to page 2 (lazy scan now sees the
|
||||
// native annotation) and confirm the panel shows it as a cleared
|
||||
// tombstone, not a live (resurrected) card.
|
||||
await app.locator("#next-btn").click();
|
||||
await expect(app.locator("#page-input")).toHaveValue("2");
|
||||
await expect(
|
||||
app.locator(`.annotation-card[data-annotation-id="${nativeId}"]`),
|
||||
).toHaveClass(/annotation-card-cleared/);
|
||||
});
|
||||
});
|
||||
BIN
tests/helpers/assets/fw9.pdf
Normal file
BIN
tests/helpers/assets/fw9.pdf
Normal file
Binary file not shown.
249
tests/helpers/range-counting-server.ts
Normal file
249
tests/helpers/range-counting-server.ts
Normal file
|
|
@ -0,0 +1,249 @@
|
|||
/**
|
||||
* HTTP test fixture serving programmatically-generated PDFs with byte-range
|
||||
* accounting. Used by pdf-incremental-load.spec.ts to assert that display_pdf
|
||||
* doesn't pull the whole file before the viewer starts streaming.
|
||||
*
|
||||
* Plain HTTP on loopback — playwright.config.ts sets
|
||||
* PDF_SERVER_ALLOW_LOOPBACK_HTTP=1 so validateUrl accepts http://127.0.0.1.
|
||||
*/
|
||||
import http from "node:http";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import { PDFDocument, PDFName, PDFString, StandardFonts } from "pdf-lib";
|
||||
|
||||
export interface RangeServerStats {
|
||||
/** Total bytes written across all responses (sum of slice lengths). */
|
||||
totalBytesServed: number;
|
||||
/** Bytes that were served more than once for the same path. */
|
||||
overlapBytes: number;
|
||||
}
|
||||
|
||||
export interface RangeServer {
|
||||
port: number;
|
||||
baseUrl: string;
|
||||
/** Map of served path → byte length. */
|
||||
fileSizes: Record<string, number>;
|
||||
stats(): RangeServerStats;
|
||||
resetStats(): void;
|
||||
/** Resolve any requests currently stalled by ?stallAfterBytes=N. */
|
||||
release(): void;
|
||||
close(): Promise<void>;
|
||||
}
|
||||
|
||||
const LOREM =
|
||||
"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod " +
|
||||
"tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim " +
|
||||
"veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea " +
|
||||
"commodo consequat. ";
|
||||
|
||||
async function buildNoFormsPdf(): Promise<Uint8Array> {
|
||||
const doc = await PDFDocument.create();
|
||||
const font = await doc.embedFont(StandardFonts.Helvetica);
|
||||
// Page 1 is text-only (small) so first paint needs minimal bytes. Pages 2+
|
||||
// each reference a large embedded JPEG so the bulk of the file is in image
|
||||
// streams page 1 doesn't need. The stallAfterBytes test holds those back
|
||||
// and asserts page 1 still renders. The image is >MAX_CHUNK_BYTES (512KB)
|
||||
// so rendering page 2 also exercises the viewer's >512KB range path.
|
||||
const big = await doc.embedJpg(makeRandomJpeg(1_100 * 1024));
|
||||
const page1 = doc.addPage([612, 792]);
|
||||
for (let line = 0; line < 30; line++) {
|
||||
page1.drawText(`1.${line + 1} ${LOREM}`, {
|
||||
x: 36,
|
||||
y: 760 - line * 22,
|
||||
size: 10,
|
||||
font,
|
||||
});
|
||||
}
|
||||
for (let p = 1; p < 20; p++) {
|
||||
const page = doc.addPage([612, 792]);
|
||||
page.drawImage(big, { x: 36, y: 200, width: 540, height: 540 });
|
||||
page.drawText(`Page ${p + 1}`, { x: 36, y: 760, size: 10, font });
|
||||
}
|
||||
return doc.save();
|
||||
}
|
||||
|
||||
/** Minimal valid JPEG with `len` bytes of incompressible scan data. */
|
||||
export function makeRandomJpeg(len: number): Uint8Array {
|
||||
// SOI, APP0 (JFIF), SOF0 (baseline 8x8 1-component), DHT (minimal),
|
||||
// SOS, <random scan data>, EOI. pdf-lib only needs to parse the headers
|
||||
// to embed; the scan data is opaque.
|
||||
const header = Uint8Array.from([
|
||||
0xff, 0xd8, 0xff, 0xe0, 0x00, 0x10, 0x4a, 0x46, 0x49, 0x46, 0x00, 0x01,
|
||||
0x01, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0xff, 0xc0, 0x00, 0x0b,
|
||||
0x08, 0x00, 0x08, 0x00, 0x08, 0x01, 0x01, 0x11, 0x00, 0xff, 0xc4, 0x00,
|
||||
0x14, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
|
||||
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0xda, 0x00, 0x08, 0x01,
|
||||
0x01, 0x00, 0x00, 0x3f, 0x00,
|
||||
]);
|
||||
const scan = new Uint8Array(len);
|
||||
for (let i = 0; i < len; i++) scan[i] = (i * 1103515245 + 12345) & 0xff;
|
||||
// Avoid 0xFF in scan data so we don't accidentally form a marker.
|
||||
for (let i = 0; i < len; i++) if (scan[i] === 0xff) scan[i] = 0xfe;
|
||||
const eoi = Uint8Array.from([0xff, 0xd9]);
|
||||
const out = new Uint8Array(header.length + scan.length + eoi.length);
|
||||
out.set(header, 0);
|
||||
out.set(scan, header.length);
|
||||
out.set(eoi, header.length + scan.length);
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Two pages, page 1 text-only, page 2 carries one native /Text (sticky-note)
|
||||
* annotation. Used by the tombstone-preservation e2e: the viewer's lazy
|
||||
* baseline scan must not have visited page 2 when persistAnnotations runs.
|
||||
*/
|
||||
async function buildWithNativeAnnotPdf(): Promise<Uint8Array> {
|
||||
const doc = await PDFDocument.create();
|
||||
const font = await doc.embedFont(StandardFonts.Helvetica);
|
||||
const page1 = doc.addPage([612, 792]);
|
||||
page1.drawText("Page 1 — no native annotations here.", {
|
||||
x: 36,
|
||||
y: 740,
|
||||
size: 12,
|
||||
font,
|
||||
});
|
||||
const page2 = doc.addPage([612, 792]);
|
||||
page2.drawText("Page 2 — has one native /Text annot.", {
|
||||
x: 36,
|
||||
y: 740,
|
||||
size: 12,
|
||||
font,
|
||||
});
|
||||
const annotRef = doc.context.register(
|
||||
doc.context.obj({
|
||||
Type: "Annot",
|
||||
Subtype: "Text",
|
||||
Rect: [100, 700, 120, 720],
|
||||
Contents: PDFString.of("native sticky note"),
|
||||
Open: false,
|
||||
Name: "Comment",
|
||||
}),
|
||||
);
|
||||
page2.node.set(PDFName.of("Annots"), doc.context.obj([annotRef]));
|
||||
return doc.save();
|
||||
}
|
||||
|
||||
async function buildFormsPdf(): Promise<Uint8Array> {
|
||||
const doc = await PDFDocument.create();
|
||||
const form = doc.getForm();
|
||||
for (let p = 0; p < 2; p++) doc.addPage([612, 792]);
|
||||
const [page1] = doc.getPages();
|
||||
const fields = ["name", "email", "phone", "city", "notes"];
|
||||
fields.forEach((name, i) => {
|
||||
const f = form.createTextField(name);
|
||||
f.addToPage(page1, { x: 100, y: 650 - i * 60, width: 300, height: 24 });
|
||||
});
|
||||
return doc.save();
|
||||
}
|
||||
|
||||
export async function startRangeServer(): Promise<RangeServer> {
|
||||
if (process.env.NODE_ENV === "production") {
|
||||
throw new Error(
|
||||
"range-counting-server is a test fixture; refusing to start with NODE_ENV=production",
|
||||
);
|
||||
}
|
||||
const files: Record<string, Uint8Array> = {
|
||||
"/noforms.pdf": await buildNoFormsPdf(),
|
||||
"/forms.pdf": await buildFormsPdf(),
|
||||
"/with-native-annot.pdf": await buildWithNativeAnnotPdf(),
|
||||
};
|
||||
const fileSizes = Object.fromEntries(
|
||||
Object.entries(files).map(([k, v]) => [k, v.length]),
|
||||
);
|
||||
|
||||
// Per-path hit count per byte, for overlap accounting.
|
||||
const hitCounts: Record<string, Uint8Array> = {};
|
||||
const initHits = () => {
|
||||
for (const [k, v] of Object.entries(files)) {
|
||||
hitCounts[k] = new Uint8Array(v.length);
|
||||
}
|
||||
};
|
||||
initHits();
|
||||
|
||||
let totalBytesServed = 0;
|
||||
let releaseResolve: (() => void) | undefined;
|
||||
let releasePromise = new Promise<void>((r) => (releaseResolve = r));
|
||||
|
||||
const server = http.createServer(async (req, res) => {
|
||||
const url = new URL(req.url ?? "/", "http://127.0.0.1");
|
||||
const body = files[url.pathname];
|
||||
if (!body) {
|
||||
res.writeHead(404).end();
|
||||
return;
|
||||
}
|
||||
|
||||
const stallAfterBytes = url.searchParams.get("stallAfterBytes");
|
||||
const total = body.length;
|
||||
const range = req.headers.range;
|
||||
|
||||
let begin = 0;
|
||||
let end = total; // exclusive
|
||||
let status = 200;
|
||||
if (range) {
|
||||
const m = /^bytes=(\d+)-(\d*)$/.exec(range);
|
||||
if (m) {
|
||||
begin = parseInt(m[1], 10);
|
||||
end = m[2] ? parseInt(m[2], 10) + 1 : total;
|
||||
begin = Math.min(begin, total);
|
||||
end = Math.min(end, total);
|
||||
status = 206;
|
||||
}
|
||||
}
|
||||
|
||||
// Stall once N bytes have already been served — lets pdfjs read the
|
||||
// header/trailer/xref (scattered across the file) before blocking the
|
||||
// bulk content streams.
|
||||
if (stallAfterBytes !== null) {
|
||||
if (totalBytesServed >= parseInt(stallAfterBytes, 10)) {
|
||||
await releasePromise;
|
||||
}
|
||||
}
|
||||
|
||||
const slice = body.subarray(begin, end);
|
||||
totalBytesServed += slice.length;
|
||||
const hits = hitCounts[url.pathname];
|
||||
for (let i = begin; i < end; i++) hits[i]++;
|
||||
|
||||
const headers: Record<string, string> = {
|
||||
"Content-Type": "application/pdf",
|
||||
"Accept-Ranges": "bytes",
|
||||
"Content-Length": String(slice.length),
|
||||
};
|
||||
if (status === 206) {
|
||||
headers["Content-Range"] = `bytes ${begin}-${end - 1}/${total}`;
|
||||
}
|
||||
res.writeHead(status, headers);
|
||||
res.end(slice);
|
||||
});
|
||||
|
||||
await new Promise<void>((resolve) => server.listen(0, resolve));
|
||||
const port = (server.address() as AddressInfo).port;
|
||||
|
||||
return {
|
||||
port,
|
||||
baseUrl: `http://127.0.0.1:${port}`,
|
||||
fileSizes,
|
||||
stats() {
|
||||
let overlapBytes = 0;
|
||||
for (const hits of Object.values(hitCounts)) {
|
||||
for (let i = 0; i < hits.length; i++) if (hits[i] > 1) overlapBytes++;
|
||||
}
|
||||
return { totalBytesServed, overlapBytes };
|
||||
},
|
||||
resetStats() {
|
||||
totalBytesServed = 0;
|
||||
initHits();
|
||||
// Unblock any handlers parked on the previous stall before re-arming,
|
||||
// otherwise they hold sockets open forever and close() hangs.
|
||||
releaseResolve?.();
|
||||
releasePromise = new Promise<void>((r) => (releaseResolve = r));
|
||||
},
|
||||
release() {
|
||||
releaseResolve?.();
|
||||
},
|
||||
close() {
|
||||
releaseResolve?.();
|
||||
server.closeAllConnections?.();
|
||||
return new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
},
|
||||
};
|
||||
}
|
||||
Loading…
Reference in a new issue