feat(pdf-server): rasterize imported annotations + form/save consistency follow-ups (#593)

* feat(pdf-server): rasterize imported annotations via annotationCanvasMap

Adds an 'imported' annotation type for anything in a loaded PDF that we
either don't model (Ink, Polygon, Caret, FileAttachment, ...) or can't
faithfully re-render (Stamp with an appearance stream, e.g. an image
signature). These now:
- Appear in the annotation panel as '<Subtype> (from PDF)'
- Render in our layer as a positioned div whose body is the per-
  annotation canvas pdf.js produced via page.render({annotationCanvasMap})
  - if pdf.js didn't divert it (no hasOwnCanvas), the box is transparent
    over the main-canvas pixel and just captures clicks
- Are selectable and draggable (resize/rotate disabled - bitmap would
  just stretch)
- Are skipped by addAnnotationDicts; getAnnotatedPdfBytes already
  filters baseline ids, so save leaves the original in the PDF.
  Move/delete are UI-only for now (documented).

Link (2) and Popup (16) are excluded - navigational/auxiliary, not
markup.

importPdfjsAnnotation tests added for unsupported-type and
appearance-stamp paths; computeDiff round-trip for 'imported'.

* fix(pdf-server): isEditing on page.render so stamps always divert to annotationCanvasMap

StampAnnotation.hasOwnCanvas defaults to noRotate; stamps without that
flag composited onto the main canvas, so deleting the 'imported' overlay
left an unclickable pixel. isEditing forces hasOwnCanvas=true for stamps
(via mustBeViewedWhenEditing) so the appearance lands in the per-id
canvas and our DOM element is the only render.

* fix(pdf-server): refresh annotation panel after save rebases baseline

setDirty(false) updated the title and save button but the panel kept
showing pending-change badges because it diffs against
pdfBaselineAnnotations/FormValues and was never re-rendered.

* fix(pdf-server): imported stamp canvases cropped/2x on retina

renderPage applied devicePixelRatio via ctx.scale(dpr,dpr) instead of
page.render's transform parameter. pdf.js sizes annotationCanvasMap
backing buffers as rectW * outputScaleX * viewport.scale, and
outputScaleX is read from transform[0] (defaults 1). So on retina the
per-annotation canvas got a 1x backing while its internal setTransform
(from the SVD of the already-dpr-scaled ctx) was 2x - the appearance
rendered at 2x into a half-sized buffer, showing only the top-left
quarter.

Pass dpr via transform: [dpr,0,0,dpr,0,0] so outputScaleX matches.
Also filter AnnotationLayer.render() to Widget annotations only so it
stops creating empty pointer-events:auto sections for stamps in
#form-layer that could steal clicks from our overlays.

* feat(pdf-server): strip deleted baseline annotations from /Annots on save

buildAnnotatedPdfBytes takes a removedRefs list and walks each page's
/Annots array (backwards) removing matching PDFRef entries.
getAnnotatedPdfBytes computes that list from baseline annotations no
longer in annotationMap, parsing the ref back from our id via
parseAnnotationRef (handles both pdf-<num>-<gen> and pdf-<num>R).

Panel: removed baseline annotations now stay listed as crossed-out cards
with a revert button (mirrors cleared form fields), so the user can see
and undo the pending deletion before save commits it.

* fix(pdf-server): re-render after seeding storage from AcroForm tree

buildFieldNameMap runs AFTER the first renderPage (perf: don't block the
canvas on an O(numPages) scan). When it detects a widget/field-tree
mismatch and pushes the field-tree value into annotationStorage, the
form layer has already rendered the stale widget value. Re-render once
when that happens so the input shows the AcroForm truth.

* fix(pdf-server): reload PDF after successful save instead of rebasing in place

Rebasing baselines while keeping the old pdfDocument drifts: subsequent
renders rasterize annotations that were just stripped from disk, and the
field/widget split pdf-lib's save can create isn't visible until reload.
Reload makes 'viewer == disk' an invariant. localStorage cleared first;
file_changed echo suppressed by lastSavedMtime as before.

examples/pdf-server/src/annotation-panel.ts

@@ -289,10 +289,23 @@ function panelFieldNames(): Set<string> {
   return new Set([...formFieldValues.keys(), ...pdfBaselineFormValues.keys()]);
 }
 
+/** Baseline annotations the user has deleted. Shown crossed-out in the panel
+ *  (mirroring cleared form fields) so they can be reverted, and so save knows
+ *  to strip their refs from /Annots. */
+function removedBaselineAnnotations(): PdfAnnotationDef[] {
+  return deps
+    .state()
+    .pdfBaselineAnnotations.filter((a) => !annotationMap.has(a.id));
+}
+
 /** Total count of annotations + form fields for the sidebar badge.
  *  Uses the union so cleared baseline items still contribute. */
 function sidebarItemCount(): number {
-  return annotationMap.size + panelFieldNames().size;
+  return (
+    annotationMap.size +
+    removedBaselineAnnotations().length +
+    panelFieldNames().size
+  );
 }
 
 export function updateAnnotationsBadge(): void {
@@ -338,6 +351,8 @@ export function getAnnotationLabel(def: PdfAnnotationDef): string {
       return "Line";
     case "image":
       return "Image";
+    case "imported":
+      return `${def.subtype} (from PDF)`;
   }
 }
 
@@ -381,6 +396,8 @@ export function getAnnotationColor(def: PdfAnnotationDef): string {
       return "#333";
     case "image":
       return "#999";
+    case "imported":
+      return "#666";
   }
 }
 
@@ -425,6 +442,14 @@ export function renderAnnotationPanel(): void {
     byPage.get(page)!.push(tracked);
   }
 
+  // Removed baseline annotations: still listed (crossed-out) so they can be
+  // reverted before save strips them from the file.
+  const removedByPage = new Map<number, PdfAnnotationDef[]>();
+  for (const def of removedBaselineAnnotations()) {
+    if (!removedByPage.has(def.page)) removedByPage.set(def.page, []);
+    removedByPage.get(def.page)!.push(def);
+  }
+
   // Group form fields by page — iterate the UNION so cleared baseline
   // fields remain visible (crossed out) with a per-item revert button.
   const fieldsByPage = new Map<number, string[]>();
@@ -441,7 +466,11 @@ export function renderAnnotationPanel(): void {
   }
 
   // Collect all pages that have annotations or form fields
-  const allPages = new Set([...byPage.keys(), ...fieldsByPage.keys()]);
+  const allPages = new Set([
+    ...byPage.keys(),
+    ...removedByPage.keys(),
+    ...fieldsByPage.keys(),
+  ]);
   const sortedPages = [...allPages].sort((a, b) => a - b);
 
   // Sort annotations within each page by Y position (descending = top-first in PDF coords)
@@ -466,8 +495,9 @@ export function renderAnnotationPanel(): void {
     const sectionKey = `page-${pageNum}`;
     const isOpen = panelState.openAccordionSection === sectionKey;
     const annotations = byPage.get(pageNum) ?? [];
+    const removed = removedByPage.get(pageNum) ?? [];
     const fields = fieldsByPage.get(pageNum) ?? [];
-    const itemCount = annotations.length + fields.length;
+    const itemCount = annotations.length + removed.length + fields.length;
 
     appendAccordionSection(
       `Page ${pageNum} (${itemCount})`,
@@ -483,6 +513,10 @@ export function renderAnnotationPanel(): void {
         for (const tracked of annotations) {
           body.appendChild(createAnnotationCard(tracked));
         }
+        // Then removed baseline annotations (crossed-out, revertable)
+        for (const def of removed) {
+          body.appendChild(createRemovedAnnotationCard(def));
+        }
       },
     );
   }
@@ -648,6 +682,55 @@ function createAnnotationCard(tracked: TrackedAnnotation): HTMLElement {
   return card;
 }
 
+/**
+ * Card for a baseline annotation the user deleted: crossed-out, no select/
+ * navigate (it has no DOM on the page anymore), revert button puts it back
+ * into `annotationMap` so it renders again and save leaves it in the file.
+ */
+function createRemovedAnnotationCard(def: PdfAnnotationDef): HTMLElement {
+  const card = document.createElement("div");
+  card.className = "annotation-card annotation-card-cleared";
+  card.dataset.annotationId = def.id;
+
+  const row = document.createElement("div");
+  row.className = "annotation-card-row";
+
+  const swatch = document.createElement("div");
+  swatch.className = "annotation-card-swatch annotation-card-swatch-cleared";
+  swatch.innerHTML = `<svg width="10" height="10" viewBox="0 0 10 10" stroke="${getAnnotationColor(def)}" stroke-width="1.5" stroke-linecap="round"><path d="M2 2l6 6M8 2L2 8"/></svg>`;
+  row.appendChild(swatch);
+
+  const typeLabel = document.createElement("span");
+  typeLabel.className = "annotation-card-type";
+  typeLabel.textContent = getAnnotationLabel(def);
+  row.appendChild(typeLabel);
+
+  const preview = getAnnotationPreview(def);
+  if (preview) {
+    const previewEl = document.createElement("span");
+    previewEl.className = "annotation-card-preview";
+    previewEl.textContent = preview;
+    row.appendChild(previewEl);
+  }
+
+  const revertBtn = document.createElement("button");
+  revertBtn.className = "annotation-card-delete";
+  revertBtn.title = "Restore annotation from file";
+  revertBtn.innerHTML = REVERT_SVG;
+  revertBtn.addEventListener("click", (e) => {
+    e.stopPropagation();
+    annotationMap.set(def.id, { def: { ...def }, elements: [] });
+    updateAnnotationsBadge();
+    renderAnnotationPanel();
+    deps.renderPage();
+    deps.persistAnnotations();
+  });
+  row.appendChild(revertBtn);
+
+  card.appendChild(row);
+  return card;
+}
+
 /** Revert one field to its PDF-stored baseline value. */
 function revertFieldToBaseline(name: string): void {
   const base = pdfBaselineFormValues.get(name);

examples/pdf-server/src/mcp-app.css

@@ -590,6 +590,21 @@ body {
   user-select: none;
 }
 
+/* Annotation imported verbatim from the PDF (Stamp/Ink/etc. via
+ * annotationCanvasMap). The body is the rasterized appearance canvas;
+ * if pdf.js didn't divert it (no hasOwnCanvas), the box stays transparent
+ * over the main-canvas pixel and just captures clicks. */
+.annotation-imported {
+  position: absolute;
+  pointer-events: auto;
+  cursor: grab;
+  user-select: none;
+}
+.annotation-imported:hover {
+  outline: 1px dashed var(--accent, #2563eb);
+  outline-offset: 1px;
+}
+
 /* Selection visuals */
 .annotation-selected {
   outline: 2px solid var(--accent, #2563eb);

examples/pdf-server/src/mcp-app.ts

@@ -25,6 +25,7 @@ import {
   type LineAnnotation,
   type StampAnnotation,
   type ImageAnnotation,
+  type ImportedAnnotation,
   type NoteAnnotation,
   type FreetextAnnotation,
   cssColorToRgb,
@@ -34,6 +35,7 @@ import {
   computeDiff,
   isDiffEmpty,
   buildAnnotatedPdfBytes,
+  parseAnnotationRef,
   importPdfjsAnnotation,
   uint8ArrayToBase64,
   convertFromModelCoords,
@@ -1383,6 +1385,10 @@ const DRAGGABLE_TYPES = new Set<string>([
   "stamp",
   "note",
   "image",
+  // "imported" is draggable in the UI but the move does NOT persist to the
+  // PDF on save (addAnnotationDicts skips it). Resize/rotate stay disabled
+  // — the appearance bitmap would just stretch.
+  "imported",
 ]);
 
 function setupAnnotationInteraction(
@@ -1890,6 +1896,25 @@ function paintAnnotationsOnCanvas(
         }
         break;
       }
+
+      case "imported": {
+        const s = pdfRectToScreen(
+          { x: def.x, y: def.y, width: def.width, height: def.height },
+          viewport,
+        );
+        const bmp = annotationCanvasMap.get(def.pdfjsId);
+        ctx.save();
+        if (bmp) {
+          ctx.drawImage(bmp, s.left, s.top, s.width, s.height);
+        } else {
+          ctx.strokeStyle = "#666";
+          ctx.lineWidth = 1;
+          ctx.setLineDash([3, 3]);
+          ctx.strokeRect(s.left, s.top, s.width, s.height);
+        }
+        ctx.restore();
+        break;
+      }
     }
   }
 }
@@ -1987,6 +2012,8 @@ function renderAnnotation(
       return [renderLineAnnotation(def, viewport)];
     case "image":
       return [renderImageAnnotation(def, viewport)];
+    case "imported":
+      return [renderImportedAnnotation(def, viewport)];
   }
 }
 
@@ -2175,6 +2202,45 @@ function renderImageAnnotation(
   return el;
 }
 
+/**
+ * Per-annotation appearance bitmaps from page.render(). Keyed by pdf.js
+ * annotation id (e.g. "118R"). Populated for the current page only —
+ * cleared at the start of each renderPage().
+ */
+const annotationCanvasMap = new Map<string, HTMLCanvasElement>();
+
+function renderImportedAnnotation(
+  def: ImportedAnnotation,
+  viewport: { width: number; height: number; scale: number },
+): HTMLElement {
+  const screen = pdfRectToScreen(
+    { x: def.x, y: def.y, width: def.width, height: def.height },
+    viewport,
+  );
+  const el = document.createElement("div");
+  el.className = "annotation-imported";
+  el.style.left = `${screen.left}px`;
+  el.style.top = `${screen.top}px`;
+  el.style.width = `${screen.width}px`;
+  el.style.height = `${screen.height}px`;
+  el.title = `${def.subtype} (from PDF)`;
+
+  // page.render() may or may not have produced a separate canvas for this
+  // annotation (hasOwnCanvas depends on the PDF's flags). When it did, use
+  // it as a pixel-faithful body; when it didn't, the appearance is on the
+  // main canvas already, so leave the box transparent — it still captures
+  // clicks for select/delete.
+  const canvas = annotationCanvasMap.get(def.pdfjsId);
+  if (canvas) {
+    canvas.style.width = "100%";
+    canvas.style.height = "100%";
+    canvas.style.display = "block";
+    canvas.style.pointerEvents = "none";
+    el.appendChild(canvas);
+  }
+  return el;
+}
+
 // =============================================================================
 // Annotation CRUD
 // =============================================================================
@@ -2761,7 +2827,8 @@ function normaliseFieldValue(
  */
 async function buildFieldNameMap(
   doc: pdfjsLib.PDFDocumentProxy,
-): Promise<void> {
+): Promise<boolean> {
+  let pushedToStorage = false;
   fieldNameToIds.clear();
   radioButtonValues.clear();
   fieldNameToPage.clear();
@@ -2857,12 +2924,16 @@ async function buildFieldNameMap(
       if (!widgetIds) continue; // no widget → not rendered anyway
 
       // Type comes from getFieldObjects (widget annot data doesn't have it).
-      // Value comes from the widget annotation (fall back to field-dict if
+      // Value: prefer the AcroForm field-tree value over the widget's
-      // the widget didn't expose one).
+      // fieldValue. pdf-lib's save() can leave a page widget pointing at a
+      // stale /V while the field tree has the new one (seen with comb text
+      // fields), and getAnnotations() reads the widget. If the two disagree
+      // we push the field-tree value into annotationStorage so the rendered
+      // input matches what's actually in /AcroForm.
       const type = fieldArr.find((f) => f.type)?.type;
-      const raw = widgetFieldValues.has(name)
+      const fieldTreeRaw = fieldArr.find((f) => f.value != null)?.value;
-        ? widgetFieldValues.get(name)
+      const widgetRaw = widgetFieldValues.get(name);
-        : fieldArr.find((f) => f.value != null)?.value;
+      const raw = fieldTreeRaw ?? widgetRaw;
       const v = normaliseFieldValue(type, raw);
       if (v !== null) {
         pdfBaselineFormValues.set(name, v);
@@ -2870,6 +2941,13 @@ async function buildFieldNameMap(
         // restored localStorage diff (applied in restoreAnnotations) will
         // overwrite specific fields the user changed.
         if (!formFieldValues.has(name)) formFieldValues.set(name, v);
+        // Widget out of sync with field tree → force storage so
+        // AnnotationLayer renders the field-tree value, not the stale
+        // widget. (syncFormValuesToStorage skips baseline==current.)
+        if (fieldTreeRaw != null && fieldTreeRaw !== widgetRaw) {
+          setFieldInStorage(name, v);
+          pushedToStorage = true;
+        }
       }
 
       // Skip parent entries with no concrete id (radio groups: the /T tree
@@ -2884,6 +2962,7 @@ async function buildFieldNameMap(
   }
 
   log.info(`Built field name map: ${fieldNameToIds.size} fields`);
+  return pushedToStorage;
 }
 
 /**
@@ -2971,6 +3050,14 @@ async function getAnnotatedPdfBytes(): Promise<Uint8Array> {
     }
   }
 
+  // Baseline annotations the user deleted: strip their refs from /Annots so
+  // they don't reappear on reload. Ids without a recoverable ref (page-index
+  // fallback) can't be removed by-ref and are skipped.
+  const removedRefs = pdfBaselineAnnotations
+    .filter((a) => !annotationMap.has(a.id))
+    .map((a) => parseAnnotationRef(a.id))
+    .filter((r): r is NonNullable<typeof r> => r !== null);
+
   // Only write fields that actually changed vs. what's already in the PDF.
   // Unchanged fields are no-ops at best, and at worst trip pdf-lib edge
   // cases (max-length text, missing /Yes appearance, …) on fields the user
@@ -2995,6 +3082,7 @@ async function getAnnotatedPdfBytes(): Promise<Uint8Array> {
     fullBytes as Uint8Array,
     annotations,
     formFieldsOut,
+    removedRefs,
   );
 }
 
@@ -3036,15 +3124,6 @@ async function savePdf(): Promise<void> {
       const sc = result.structuredContent as { mtimeMs?: number } | undefined;
       lastSavedMtime = sc?.mtimeMs ?? null;
 
-      // Rebase: the file on disk now contains our annotations + form values.
-      // Update the baseline so future diffs are relative to what was saved.
-      pdfBaselineAnnotations = [...annotationMap.values()].map((t) => ({
-        ...t.def,
-      }));
-      pdfBaselineFormValues.clear();
-      for (const [k, v] of formFieldValues) pdfBaselineFormValues.set(k, v);
-
-      setDirty(false); // → updateSaveBtn() disables button
       const key = annotationStorageKey();
       if (key) {
         try {
@@ -3053,6 +3132,13 @@ async function savePdf(): Promise<void> {
           /* ignore */
         }
       }
+      // Reload from the bytes we just wrote. The previous approach (rebase
+      // baselines but keep the old pdfDocument) drifts: subsequent renders
+      // still rasterize stripped annotations from the old bytes, and the
+      // field/widget split that pdf-lib's save can create isn't reflected
+      // until reload anyway. Reload makes "what you see = what's on disk"
+      // an invariant. (file_changed echo is suppressed by lastSavedMtime.)
+      await reloadPdf();
     }
   } catch (err) {
     log.error("Save failed:", err);
@@ -3156,8 +3242,15 @@ async function renderPage() {
     // resize so the size handoff is atomic.
     pageWrapperEl.style.transform = "";
 
-    // Scale context for retina
+    // Retina: pass dpr via page.render's `transform` (NOT ctx.scale).
-    ctx.scale(dpr, dpr);
+    // pdf.js sizes per-annotation canvases as
+    //   width = rectW * outputScaleX * viewport.scale
+    // and outputScaleX is read from transform[0] (defaults to 1). A bare
+    // ctx.scale(dpr,dpr) leaves outputScaleX at 1, so the
+    // annotationCanvasMap canvases get a half-sized backing store on
+    // retina while their internal setTransform IS dpr-aware → the
+    // appearance renders 2× too big into a 1× buffer → cropped/shifted.
+    const dprTransform = dpr !== 1 ? [dpr, 0, 0, dpr, 0, 0] : undefined;
 
     // Clear and setup text layer
     textLayerEl.innerHTML = "";
@@ -3166,11 +3259,29 @@ async function renderPage() {
     // Set --scale-factor so CSS font-size/transform rules work correctly.
     textLayerEl.style.setProperty("--scale-factor", `${scale}`);
 
-    // Render canvas - track the task so we can cancel it
+    // Render canvas - track the task so we can cancel it.
+    //
+    // annotationCanvasMap: pdf.js diverts annotations whose appearance needs
+    // its own bitmap (Stamp/Ink/FreeText/etc. with hasOwnCanvas) into
+    // per-id canvases instead of compositing onto the main canvas.
+    // renderImportedAnnotation() pulls from this map so those annotations
+    // become movable DOM elements with pixel-faithful visuals — instead of
+    // unselectable canvas pixels (the old "ghost annotation" problem) or
+    // our lossy text-label re-render.
+    annotationCanvasMap.clear();
     // eslint-disable-next-line @typescript-eslint/no-explicit-any
     const renderTask = (page.render as any)({
       canvasContext: ctx,
       viewport,
+      transform: dprTransform,
+      annotationCanvasMap,
+      // isEditing forces hasOwnCanvas=true for stamps regardless of /F
+      // NoRotate (StampAnnotation.mustBeViewedWhenEditing in pdf.worker).
+      // Without this, stamps without NoRotate composite onto the main canvas
+      // and deleting the "imported" overlay leaves an unclickable pixel
+      // behind. Other markup types still gate on noRotate; for those the
+      // overlay stays a transparent click-box (delete is UI-only until save).
+      isEditing: true,
     });
     currentRenderTask = renderTask;
 
@@ -3251,8 +3362,15 @@ async function renderPage() {
           commentManager: null,
         } as any);
         // eslint-disable-next-line @typescript-eslint/no-explicit-any
+        // Only feed Widgets (form fields) here. Markup annotations are
+        // owned by #annotation-layer; letting AnnotationLayer create
+        // <section> elements for them in #form-layer adds invisible
+        // pointer-events:auto boxes that steal clicks from our overlays.
+        const widgetAnns = annotations.filter(
+          (a: { subtype?: string }) => a.subtype === "Widget",
+        );
         await annotationLayer.render({
-          annotations,
+          annotations: widgetAnns,
           div: formLayerEl,
           page,
           viewport,
@@ -4313,8 +4431,9 @@ async function reloadPdf(): Promise<void> {
     await renderPage();
 
     await loadBaselineAnnotations(document);
-    await buildFieldNameMap(document);
+    const seeded = await buildFieldNameMap(document);
     syncFormValuesToStorage();
+    if (seeded) await renderPage();
     updateAnnotationsBadge();
     renderAnnotationPanel();
 
@@ -4539,9 +4658,16 @@ app.ontoolresult = async (result: CallToolResult) => {
     restoreAnnotations();
 
     // Build field name → annotation ID mapping for form filling
-    await buildFieldNameMap(document);
+    const seeded = await buildFieldNameMap(document);
     // Pre-populate annotationStorage from restored formFieldValues
     syncFormValuesToStorage();
+    // buildFieldNameMap may have pushed AcroForm-tree values into storage
+    // (when the page widget's /V is stale vs the field dict — pdf-lib's save
+    // can leave them split). The first renderPage above ran BEFORE that, so
+    // the form layer shows the stale widget value. Re-render so it picks up
+    // storage. Only when something was actually seeded — most PDFs don't hit
+    // this and the extra render would be pure waste.
+    if (seeded) await renderPage();
 
     updateAnnotationsBadge();
     // Save button visibility driven by setDirty()/updateSaveBtn();

examples/pdf-server/src/pdf-annotations.test.ts

@@ -10,6 +10,7 @@ import {
   defaultColor,
   importPdfjsAnnotation,
   buildAnnotatedPdfBytes,
+  parseAnnotationRef,
   base64ToUint8Array,
   uint8ArrayToBase64,
   convertFromModelCoords,
@@ -549,6 +550,76 @@ describe("importPdfjsAnnotation", () => {
     expect(result!.page).toBe(2);
   });
 
+  it("imports an unsupported subtype as 'imported' (placement only)", () => {
+    // annotationType 15 = Ink, not in PDFJS_TYPE_MAP. We keep it as a
+    // placement-only "imported" record so it's listed in the panel and
+    // rendered from annotationCanvasMap instead of being dropped.
+    const ann = {
+      annotationType: 15,
+      subtype: "Ink",
+      id: "200R",
+      rect: [100, 200, 180, 260],
+    };
+    const result = importPdfjsAnnotation(ann, 3, 0);
+    expect(result).not.toBeNull();
+    expect(result!.type).toBe("imported");
+    expect(result!.page).toBe(3);
+    expect((result as any).pdfjsId).toBe("200R");
+    expect((result as any).subtype).toBe("Ink");
+    expect((result as any).width).toBeCloseTo(80);
+    expect((result as any).height).toBeCloseTo(60);
+  });
+
+  it("imports an appearance-stream stamp as 'imported' (not text-label)", () => {
+    // A Stamp with hasAppearance carries a custom visual (e.g. an image
+    // signature) that our text-label StampAnnotation can't reproduce.
+    const ann = {
+      annotationType: 13,
+      subtype: "Stamp",
+      id: "118R",
+      rect: [420, 760, 514, 792],
+      hasAppearance: true,
+      contentsObj: { str: "DRAFT" },
+    };
+    const result = importPdfjsAnnotation(ann, 1, 0);
+    expect(result!.type).toBe("imported");
+    expect((result as any).pdfjsId).toBe("118R");
+    expect((result as any).subtype).toBe("Stamp");
+  });
+
+  it("computeDiff: 'imported' present in both baseline and current → no diff", () => {
+    const imp: PdfAnnotationDef = {
+      type: "imported",
+      id: "pdf-118R",
+      page: 1,
+      x: 420,
+      y: 760,
+      width: 94,
+      height: 32,
+      pdfjsId: "118R",
+      subtype: "Stamp",
+    };
+    const diff = computeDiff([imp], [imp], new Map());
+    expect(diff.added).toHaveLength(0);
+    expect(diff.removed).toHaveLength(0);
+  });
+
+  it("computeDiff: deleting an 'imported' annotation lists it in removed", () => {
+    const imp: PdfAnnotationDef = {
+      type: "imported",
+      id: "pdf-118R",
+      page: 1,
+      x: 420,
+      y: 760,
+      width: 94,
+      height: 32,
+      pdfjsId: "118R",
+      subtype: "Stamp",
+    };
+    const diff = computeDiff([imp], [], new Map());
+    expect(diff.removed).toEqual(["pdf-118R"]);
+  });
+
   it("imports a note (Text) annotation", () => {
     const ann = {
       annotationType: 1,
@@ -718,6 +789,29 @@ describe("base64 helpers", () => {
 // PDF Annotation Dict Creation (integration test with pdf-lib)
 // =============================================================================
 
+describe("parseAnnotationRef", () => {
+  it("parses pdf-<num>-<gen> ids", () => {
+    expect(parseAnnotationRef("pdf-118-0")).toEqual({
+      objectNumber: 118,
+      generationNumber: 0,
+    });
+    expect(parseAnnotationRef("pdf-5-2")).toEqual({
+      objectNumber: 5,
+      generationNumber: 2,
+    });
+  });
+  it("parses pdf-<num>R ids (pdf.js string id, gen=0)", () => {
+    expect(parseAnnotationRef("pdf-118R")).toEqual({
+      objectNumber: 118,
+      generationNumber: 0,
+    });
+  });
+  it("returns null for page-index fallback ids", () => {
+    expect(parseAnnotationRef("pdf-1-idx-3")).toBeNull();
+    expect(parseAnnotationRef("user-abc")).toBeNull();
+  });
+});
+
 describe("buildAnnotatedPdfBytes", () => {
   let blankPdfBytes: Uint8Array;
 
@@ -738,6 +832,53 @@ describe("buildAnnotatedPdfBytes", () => {
     expect(header).toBe("%PDF-");
   });
 
+  it("strips removedRefs entries from each page's /Annots array", async () => {
+    // Seed: add two highlights, save, capture their object refs.
+    const seeded = await buildAnnotatedPdfBytes(
+      blankPdfBytes,
+      [
+        {
+          type: "highlight",
+          id: "h1",
+          page: 1,
+          rects: [{ x: 72, y: 700, width: 100, height: 12 }],
+          color: "#ffff00",
+        },
+        {
+          type: "highlight",
+          id: "h2",
+          page: 1,
+          rects: [{ x: 72, y: 680, width: 100, height: 12 }],
+          color: "#ffff00",
+        },
+      ],
+      new Map(),
+    );
+    const seededDoc = await PDFDocument.load(seeded);
+    const annots = seededDoc.getPage(0).node.Annots()!;
+    expect(annots.size()).toBe(2);
+    const ref0 = annots.get(0) as unknown as {
+      objectNumber: number;
+      generationNumber: number;
+    };
+
+    // Now remove the first one by ref.
+    const stripped = await buildAnnotatedPdfBytes(seeded, [], new Map(), [
+      { objectNumber: ref0.objectNumber, generationNumber: ref0.generationNumber },
+    ]);
+    const strippedDoc = await PDFDocument.load(stripped);
+    const remaining = strippedDoc.getPage(0).node.Annots();
+    expect(remaining?.size() ?? 0).toBe(1);
+  });
+
+  it("removedRefs ignores refs not present in /Annots", async () => {
+    const out = await buildAnnotatedPdfBytes(blankPdfBytes, [], new Map(), [
+      { objectNumber: 9999, generationNumber: 0 },
+    ]);
+    const doc = await PDFDocument.load(out);
+    expect(doc.getPage(0).node.Annots()?.size() ?? 0).toBe(0);
+  });
+
   it("adds highlight annotation to PDF", async () => {
     const annotations: PdfAnnotationDef[] = [
       {

examples/pdf-server/src/pdf-annotations.ts

@@ -130,6 +130,32 @@ export interface ImageAnnotation extends AnnotationBase {
   aspect?: "preserve" | "ignore";
 }
 
+/**
+ * An annotation that already exists in the loaded PDF and that we render
+ * verbatim from its appearance stream (via pdf.js's annotationCanvasMap)
+ * rather than re-modeling it with one of our shape types.
+ *
+ * Covers two cases:
+ *  - subtypes we don't model (Ink, Polygon, Caret, FileAttachment, …)
+ *  - subtypes we *could* model but whose appearance carries information
+ *    our model would drop (e.g. Stamp with an image signature)
+ *
+ * The rasterized canvas is supplied at render time by mcp-app.ts; this
+ * struct only carries placement and identity. Coords are PDF user-space
+ * (origin bottom-left), matching the other rect-shaped types.
+ */
+export interface ImportedAnnotation extends AnnotationBase {
+  type: "imported";
+  x: number;
+  y: number;
+  width: number;
+  height: number;
+  /** pdf.js getAnnotations() id (e.g. "118R") — key into annotationCanvasMap. */
+  pdfjsId: string;
+  /** Original PDF /Subtype (e.g. "Stamp", "Ink") for the panel label. */
+  subtype: string;
+}
+
 export type PdfAnnotationDef =
   | HighlightAnnotation
   | UnderlineAnnotation
@@ -140,7 +166,8 @@ export type PdfAnnotationDef =
   | LineAnnotation
   | FreetextAnnotation
   | StampAnnotation
-  | ImageAnnotation;
+  | ImageAnnotation
+  | ImportedAnnotation;
 
 // =============================================================================
 // Coordinate Conversion (model ↔ internal PDF coords)
@@ -174,6 +201,7 @@ export function convertFromModelCoords(
     case "rectangle":
     case "circle":
     case "image":
+    case "imported":
       return { ...def, y: pageHeight - def.y - def.height };
     case "line":
       return {
@@ -374,6 +402,7 @@ export function defaultColor(type: PdfAnnotationDef["type"]): string {
     case "stamp":
       return "#cc0000";
     case "image":
+    case "imported":
       return "#00000000";
   }
 }
@@ -495,6 +524,11 @@ export async function addAnnotationDicts(
   const pages = pdfDoc.getPages();
 
   for (const def of annotations) {
+    // "imported" annotations are already in the source PDF — never
+    // re-serialize them. Deletion is handled by buildAnnotatedPdfBytes'
+    // removedRefs strip; moving an imported annotation is still UI-only.
+    if (def.type === "imported") continue;
+
     const pageIdx = def.page - 1;
     if (pageIdx < 0 || pageIdx >= pages.length) continue;
     const page = pages[pageIdx];
@@ -836,17 +870,62 @@ function setButtonGroupValue(
   }
 }
 
+/**
+ * Recover the PDF object reference from an annotation id assigned by
+ * `makeAnnotationId`. Handles both `pdf-<num>-<gen>` (from `ann.ref`) and
+ * `pdf-<num>R` (from pdf.js's string `ann.id`, gen always 0). Returns null for
+ * page-index fallback ids — those have no stable ref to remove by.
+ */
+export function parseAnnotationRef(
+  id: string,
+): { objectNumber: number; generationNumber: number } | null {
+  let m = /^pdf-(\d+)-(\d+)$/.exec(id);
+  if (m) return { objectNumber: +m[1], generationNumber: +m[2] };
+  m = /^pdf-(\d+)R$/.exec(id);
+  if (m) return { objectNumber: +m[1], generationNumber: 0 };
+  return null;
+}
+
 /**
  * Build annotated PDF bytes from the original document.
- * Applies user annotations and form fills, returns Uint8Array of the new PDF.
+ * Applies user annotations and form fills, removes baseline annotations the
+ * user deleted, returns Uint8Array of the new PDF.
  */
 export async function buildAnnotatedPdfBytes(
   pdfBytes: Uint8Array,
   annotations: PdfAnnotationDef[],
   formFields: Map<string, string | boolean>,
+  removedRefs: { objectNumber: number; generationNumber: number }[] = [],
 ): Promise<Uint8Array> {
   const pdfDoc = await PDFDocument.load(pdfBytes, { ignoreEncryption: true });
 
+  // Strip baseline annotations the user deleted. We match on object number +
+  // generation against each page's /Annots array entries (PDFRef). We don't
+  // know which page they were on, so scan every page — /Annots arrays are
+  // small and this only runs at save time.
+  if (removedRefs.length > 0) {
+    const wanted = new Set(
+      removedRefs.map((r) => `${r.objectNumber} ${r.generationNumber}`),
+    );
+    for (const page of pdfDoc.getPages()) {
+      const annots = page.node.Annots();
+      if (!annots) continue;
+      // Walk backwards so .remove(idx) doesn't shift unprocessed entries.
+      for (let i = annots.size() - 1; i >= 0; i--) {
+        const ref = annots.get(i) as {
+          objectNumber?: number;
+          generationNumber?: number;
+        };
+        if (
+          ref?.objectNumber !== undefined &&
+          wanted.has(`${ref.objectNumber} ${ref.generationNumber ?? 0}`)
+        ) {
+          annots.remove(i);
+        }
+      }
+    }
+  }
+
   // Add proper PDF annotation objects
   await addAnnotationDicts(pdfDoc, annotations);
 
@@ -1023,14 +1102,44 @@ export function importPdfjsAnnotation(
   pageNum: number,
   index: number,
 ): PdfAnnotationDef | null {
-  const ourType = PDFJS_TYPE_MAP[ann.annotationType];
+  // Skip form widgets (they're handled separately by AnnotationLayer) and
-  if (!ourType) return null;
+  // auxiliary types that aren't user-visible markup:
-
+  //   2 = Link (navigational, AnnotationLayer handles the click target)
-  // Skip form widgets (they're handled separately by AnnotationLayer)
+  //  16 = Popup (the speech-bubble UI for a parent annotation, not content)
-  if (ann.annotationType === 20) return null;
+  if (
+    ann.annotationType === 20 ||
+    ann.annotationType === 2 ||
+    ann.annotationType === 16
+  ) {
+    return null;
+  }
 
   const id = makeAnnotationId(ann, pageNum, index);
   const color = pdfjsColorToHex(ann.color);
+  const ourType = PDFJS_TYPE_MAP[ann.annotationType];
+
+  // Anything we don't model — and stamps whose visual is an appearance
+  // stream we can't reproduce as a text label — are kept as "imported":
+  // a placement-only record that the renderer fills with the rasterized
+  // appearance from pdf.js's annotationCanvasMap. This keeps Ink, Polygon,
+  // image-signature stamps, etc. visible AND selectable in our layer.
+  const importAsBitmap =
+    !ourType || (ourType === "stamp" && ann.hasAppearance);
+  if (importAsBitmap) {
+    if (!ann.rect) return null;
+    const r = pdfjsRectToRect(ann.rect);
+    return {
+      type: "imported",
+      id,
+      page: pageNum,
+      x: r.x,
+      y: r.y,
+      width: r.width,
+      height: r.height,
+      pdfjsId: String(ann.id ?? ""),
+      subtype: String(ann.subtype ?? `type${ann.annotationType}`),
+    };
+  }
 
   switch (ourType) {
     case "highlight":