modelcontextprotocol/ext-apps
Watch
1
0
Fork
You've already forked ext-apps
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(pdf-server): radio + dropdown in fill_form/save (#577)

Browse source 
* fix(pdf-server): fail fast on never-polled viewer + surface batch errors

interact's get_screenshot/get_text waited the full 45s for a viewer that
never mounted (host went idle before iframe reached startPolling), then
timed out with a generic message. Now tracks viewsPolled and fails in ~8s
with a recovery hint. Batch errors were also ordered success-first —
hosts that flatten isError to content[0] showed "Queued: Filled N fields"
and dropped the actual failure entirely. Viewer side: poll even when PDF
load throws, and catch connect() rejection instead of silently blanking.

* fix(pdf-server): radio button + dropdown handling in fill_form and save

Three related bugs around non-text form fields:

Viewer fill_form (mcp-app.ts): writing a string to annotationStorage on
every radio widget trips pdf.js's inverted string coercion
(`value !== buttonValue` → true) — the first-rendered widget checks
itself and clears the rest, so whatever you ask for, you get the first
option. Fix: extract a setFieldInStorage() helper that writes
{value: true} only on the widget whose buttonValue matches, {value: false}
on siblings, matching pdf.js's own change handler. Also covers the
syncFormValuesToStorage() path (localStorage restore).

Save (pdf-annotations.ts): buildAnnotatedPdfBytes only knew about
getTextField/getCheckBox — dropdowns and radios threw in the try block
and fell through the catch, so they were silently dropped on save.
Fix: instanceof dispatch on getFieldMaybe(). For radios, accept either
the option label or a numeric buttonValue (indexed into getOptions()),
since the viewer stores the latter for PDFs with /Opt.

display_pdf output (server.ts): radio groups showed as N identical
`name [Btn]` lines with no way to tell them apart; dropdowns didn't
list their options. Fix: surface buttonValue as exportValue and the
options array from the annotation data already in hand.
This commit is contained in:
Olivier Chafik 2026-04-01 22:28:28 +01:00 • committed by GitHub
commit 38700569e7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 434 additions and 72 deletions
Download patch file Download diff file Expand all files Collapse all files

104
examples/pdf-server/server.ts
View file

@ -196,6 +196,14 @@ export type { PdfCommand };
// reject first and return a real error instead of the client cancelling us.
const GET_PAGES_TIMEOUT_MS = 45_000;
/**
* Grace period for the viewer's first poll. If interact() arrives before the
* iframe has ever polled, we wait this long for it to show up (iframe mount +
* PDF load + startPolling). If no poll comes, the viewer almost certainly
* never rendered — failing fast beats a silent 45s hang.
*/
const VIEWER_FIRST_POLL_GRACE_MS = 8_000;
interface PageDataEntry {
page: number;
text?: string;
@ -232,6 +240,33 @@ function waitForPageData(
});
}
/**
* Wait for the viewer's first poll_pdf_commands call.
*
* Called before waitForPageData() so a viewer that never mounted fails in ~8s
* with a specific message instead of a generic 45s "Timeout waiting for page
* data" that gives no hint why.
*
* Intentionally does NOT touch pollWaiters: piggybacking on that single-slot
* Map races with poll_pdf_commands' batch-wait branch (which never cancels the
* prior waiter) and with concurrent interact calls (which would overwrite each
* other). A plain check loop on viewsPolled is stateless — multiple callers
* can wait independently and all observe the same add() when it happens.
*/
async function ensureViewerIsPolling(uuid: string): Promise<void> {
const deadline = Date.now() + VIEWER_FIRST_POLL_GRACE_MS;
while (!viewsPolled.has(uuid)) {
if (Date.now() >= deadline) {
throw new Error(
`Viewer never connected for viewUUID ${uuid} (no poll within ${VIEWER_FIRST_POLL_GRACE_MS / 1000}s). ` +
`The iframe likely failed to mount — this happens when the conversation ` +
`goes idle before the viewer finishes loading. Call display_pdf again to get a fresh viewUUID.`,
);
}
await new Promise((r) => setTimeout(r, 100));
}
}
interface QueueEntry {
commands: PdfCommand[];
/** Timestamp of the most recent enqueue or dequeue */
@ -243,6 +278,15 @@ const commandQueues = new Map<string, QueueEntry>();
/** Waiters for long-poll: resolve callback wakes up a blocked poll_pdf_commands */
const pollWaiters = new Map<string, () => void>();
/**
* viewUUIDs that have been polled at least once. A view missing from this set
* means the iframe never reached startPolling() — usually because it wasn't
* mounted yet, or ontoolresult threw before the poll loop started. Used to
* fail fast in get_screenshot/get_text instead of waiting the full 45s for
* a viewer that was never there.
*/
const viewsPolled = new Set<string>();
/** Valid form field names per viewer UUID (populated during display_pdf) */
const viewFieldNames = new Map<string, Set<string>>();
@ -288,6 +332,7 @@ function pruneStaleQueues(): void {
commandQueues.delete(uuid);
viewFieldNames.delete(uuid);
viewFieldInfo.delete(uuid);
viewsPolled.delete(uuid);
stopFileWatch(uuid);
}
}
@ -812,6 +857,10 @@ interface FormFieldInfo {
y: number;
width: number;
height: number;
/** Radio button export value (buttonValue) — distinguishes widgets that share a field name. */
exportValue?: string;
/** Dropdown/listbox option values, as seen in the widget's `options` array. */
options?: string[];
}
/**
@ -864,6 +913,16 @@ async function extractFormFieldInfo(
// Convert to model coords (top-left origin): modelY = pageHeight - pdfY - height
const modelY = pageHeight - y2;
// Choice widgets (combo/listbox) carry `options` as
// [{exportValue, displayValue}]. Expose export values — that's
// what fill_form needs.
let options: string[] | undefined;
if (Array.isArray(ann.options) && ann.options.length > 0) {
options = ann.options
.map((o: { exportValue?: string }) => o?.exportValue)
.filter((v: unknown): v is string => typeof v === "string");
}
fields.push({
name: fieldName,
type: fieldType,
@ -873,6 +932,12 @@ async function extractFormFieldInfo(
width: Math.round(width),
height: Math.round(height),
...(ann.alternativeText ? { label: ann.alternativeText } : undefined),
// Radio: buttonValue is the per-widget export value — the only
// thing distinguishing three `size [Btn]` lines from each other.
...(ann.radioButton && ann.buttonValue != null
? { exportValue: String(ann.buttonValue) }
: undefined),
...(options?.length ? { options } : undefined),
});
}
}
@ -1282,6 +1347,14 @@ Set \`elicit_form_inputs\` to true to prompt the user to fill form fields before
y: z.number(),
width: z.number(),
height: z.number(),
exportValue: z
.string()
.optional()
.describe("Radio button value — pass this to fill_form"),
options: z
.array(z.string())
.optional()
.describe("Dropdown/listbox option values"),
}),
)
.optional()
@ -1453,8 +1526,14 @@ URL: ${normalized}`,
for (const f of fields) {
const label = f.label ? ` "${f.label}"` : "";
const nameStr = f.name || "(unnamed)";
// Radio: =<exportValue> tells the model what value to pass.
// Dropdown: options:[...] lists valid choices.
const exportSuffix = f.exportValue ? `=${f.exportValue}` : "";
const optsSuffix = f.options
? ` options:[${f.options.join(", ")}]`
: "";
lines.push(
` ${nameStr}${label} [${f.type}] at (${f.x},${f.y}) ${f.width}×${f.height}`,
` ${nameStr}${exportSuffix}${label} [${f.type}] at (${f.x},${f.y}) ${f.width}×${f.height}${optsSuffix}`,
);
}
}
@ -1925,6 +2004,7 @@ URL: ${normalized}`,
let pageData: PageDataEntry[];
try {
await ensureViewerIsPolling(uuid);
pageData = await waitForPageData(requestId, signal);
} catch (err) {
return {
@ -1973,6 +2053,7 @@ URL: ${normalized}`,
let pageData: PageDataEntry[];
try {
await ensureViewerIsPolling(uuid);
pageData = await waitForPageData(requestId, signal);
} catch (err) {
return {
@ -2202,7 +2283,7 @@ Example — add a signature image and a stamp, then screenshot to verify:
// Process commands sequentially, collecting all content parts
const allContent: ContentPart[] = [];
let hasError = false;
let failedAt = -1;
for (let i = 0; i < commandList.length; i++) {
const result = await processInteractCommand(
@ -2211,15 +2292,27 @@ Example — add a signature image and a stamp, then screenshot to verify:
extra.signal,
);
if (result.isError) {
hasError = true;
// Error content first. Some hosts flatten isError results to
// content[0].text — if we push the error after prior successes,
// the user sees "Queued: Filled 7 fields" with isError:true and
// the actual failure is silently dropped.
allContent.unshift(...result.content);
failedAt = i;
break;
}
allContent.push(...result.content);
if (hasError) break; // Stop on first error
}
if (failedAt >= 0 && commandList.length > 1) {
allContent.unshift({
type: "text",
text: `Batch failed at step ${failedAt + 1}/${commandList.length} (${commandList[failedAt].action}):`,
});
}
return {
content: allContent,
...(hasError ? { isError: true } : {}),
...(failedAt >= 0 ? { isError: true } : {}),
};
},
);
@ -2280,6 +2373,7 @@ Example — add a signature image and a stamp, then screenshot to verify:
_meta: { ui: { visibility: ["app"] } },
},
async ({ viewUUID: uuid }): Promise<CallToolResult> => {
viewsPolled.add(uuid);
// If commands are already queued, wait briefly to let more accumulate
if (commandQueues.has(uuid)) {
await new Promise((r) => setTimeout(r, POLL_BATCH_WAIT_MS));