modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

fix: state every test-gate timeout instead of inheriting a library default

Browse source 
Three of the six Vitest projects ran on Vitest's own testTimeout: 5000 and
five on its hookTimeout/teardownTimeout: 10000, and all 788 Testing Library
waits in the web unit project ran on asyncUtilTimeout: 1000. None of those
numbers was chosen here, and all of them were sized for an idle machine rather
than for three or four concurrent worktree sessions each running a full
local:gate on eight cores. #2292, #2278, #2250, #1942 and #1742 were each that
class, diagnosed one site at a time.

This is the complement of #1596, not a retreat from it: every fix that issue
produced stands, and no race is hidden here. What changes is that a correct,
deterministic test is no longer cut off by a budget nobody picked.

- vitest.shared.mts exports frozen TIMEOUTS / INTEGRATION_TIMEOUTS; all six
  projects spread one of them, clients/launcher importing the shared module for
  the first time to do so.
- Both web projects configure Testing Library's asyncUtilTimeout. The storybook
  one needs its own setup file, deliberately outside .storybook/ and free of
  setProjectAnnotations so the automatic preview-annotation provisioning
  (#1898) is untouched.
- 28 per-suite restatements of the integration project's own budget are gone,
  so raising the shared value now moves every site under it. The handful that
  genuinely differ — cross-process lock contention, a full OAuth round trip —
  keep their budget as a named constant that says why.
- AppRenderer's waitFor no longer asks for exactly its enclosing budget, which
  it could never win (#2292's shape); cli's e2e child timer drops below the
  project budget so the bound carrying the diagnostic is the one that fires.
- The transition auto-settle's slack term is named RAF_SLACK_MS and raised;
  the term derived from HEADER_ANIM_MS is untouched. The two answer to
  different things and must not be scaled as one number.
- The 19 Playwright budgets scattered across three web smokes become four
  named constants in scripts/lib/browser-timeouts.mjs.

scripts/verify-test-timeouts.mjs resolves each project through Vitest itself
and asserts the budgets, the asyncUtilTimeout sites, and that retry stays
unset. It was observed red against the pre-change config: unit/tui/launcher at
testTimeout 5000, five projects at hookTimeout 10000, both asyncUtilTimeout
sites unmet.

Closes #2323

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsmUAZ6aLUAFqenjJKSSRp
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-12 08:32:34 -04:00
commit 58e53d5a92
7 changed files with 1276 additions and 394 deletions
Download patch file Download diff file Expand all files Collapse all files

49
scripts/lib/browser-timeouts.mjs Normal file
View file

@ -0,0 +1,49 @@
/**
* Playwright wall-clock budgets for the web smokes (#2323).
*
* Before this, `smoke-web-tabs.mjs`, `smoke-web-elicitation.mjs` and
* `smoke-web-browser.mjs` carried 15 locator budgets between them across three
* independently-chosen scales, with no config anywhere to raise. They are the
* same three or four decisions repeated, so they are named once here and the
* scripts import them — the same shape `render-smoke.mjs`'s `DEFAULTS` already
* has, and for the same reason: a budget nobody can find is a budget nobody
* revisits.
*
* These are generous on purpose and cost a passing run nothing. Playwright
* budgets are ceilings on a poll, not sleeps: every one of them returns the
* instant its locator condition holds. What they have to absorb is a smoke
* running against a cold `dist/` build on a machine already carrying three
* other agent sessions' gates.
*
* ⚠️ Raising one of these does NOT make a smoke wait longer for a server that
* is never coming: the launcher-death race in `smoke-web-browser.mjs` and the
* `waitForStage` diagnostics in the other two are what report that, and they
* report it with a cause rather than a timeout.
*/
export const BROWSER_TIMEOUTS = Object.freeze({
/**
* Load the page, or act on a control the app is already showing — a click on
* a rendered tab or list row, a wait for the first meaningful frame. What it
* absorbs is the browser's own scheduling under load, not a server.
*/
ui: 30_000,
/**
* Wait for something that only appears after a round trip to an MCP server:
* an elicitation modal, a tool result panel. Longer than `ui` because a test
* server has to boot, negotiate and answer before the DOM can change.
*/
roundTrip: 45_000,
/**
* Wait on a sub-element of something already on screen — a row inside a
* populated panel, a section of an open accordion. The enclosing wait has
* already paid for the round trip, so this only covers the render.
*/
nested: 15_000,
/**
* A wait whose expiry is caught and ignored, so it must stay short: it is
* pure added latency on every passing run. `networkidle` is the only one —
* the Google Fonts request may never idle on a restricted network.
*/
bestEffort: 5_000,
});

27
scripts/lib/browser-timeouts.test.mjs Normal file
View file

@ -0,0 +1,27 @@
import test from "node:test";
import assert from "node:assert/strict";
import { BROWSER_TIMEOUTS } from "./browser-timeouts.mjs";
test("every budget is a positive finite number of milliseconds", () => {
const entries = Object.entries(BROWSER_TIMEOUTS);
assert.ok(entries.length > 0);
for (const [name, ms] of entries) {
assert.equal(typeof ms, "number", `${name} is not a number`);
assert.ok(Number.isFinite(ms) && ms > 0, `${name} is not a usable budget`);
}
});
test("the object is frozen, so a smoke cannot mutate a shared budget", () => {
assert.ok(Object.isFrozen(BROWSER_TIMEOUTS));
});
test("the budgets are ordered by what each one has to absorb", () => {
// Not decoration: the ordering IS the semantics documented on each key. A
// `nested` wait that outlived its enclosing `roundTrip` one, or a
// `bestEffort` wait long enough to be felt on every passing run, would mean
// the names had stopped describing the values.
assert.ok(BROWSER_TIMEOUTS.bestEffort < BROWSER_TIMEOUTS.nested);
assert.ok(BROWSER_TIMEOUTS.nested < BROWSER_TIMEOUTS.ui);
assert.ok(BROWSER_TIMEOUTS.ui < BROWSER_TIMEOUTS.roundTrip);
});

7
scripts/smoke-web-browser.mjs
View file

@ -64,6 +64,7 @@ import { setTimeout as delay } from "node:timers/promises";
import { fileURLToPath } from "node:url";
import { dirname, resolve } from "node:path";
import { startProdWebServer } from "./lib/prod-web-server.mjs";
import { BROWSER_TIMEOUTS } from "./lib/browser-timeouts.mjs";
import {
attachPageDiagnostics,
loadBrowser,
@ -135,7 +136,7 @@ try {
// authenticates without a query param.
const response = await page.goto(server.baseUrl, {
waitUntil: "domcontentloaded",
timeout: 30_000,
timeout: BROWSER_TIMEOUTS.ui,
});
if (!response || !response.ok()) {
throw new Error(
@ -145,12 +146,12 @@ try {
// First meaningful frame: the always-present "Add Servers" control.
await page
.getByRole("button", { name: /Add Servers/ })
.waitFor({ state: "visible", timeout: 30_000 });
.waitFor({ state: "visible", timeout: BROWSER_TIMEOUTS.ui });
// Settle window: let lazily-evaluated chunks that throw a tick after first
// paint surface before we assert a clean boot. networkidle is best-effort
// (the Google-Fonts request may never idle on a restricted network).
await page
.waitForLoadState("networkidle", { timeout: 5_000 })
.waitForLoadState("networkidle", { timeout: BROWSER_TIMEOUTS.bestEffort })
.catch(() => {});
await delay(500);
};

28
scripts/smoke-web-elicitation.mjs
View file

@ -57,6 +57,7 @@ import { setTimeout as delay } from "node:timers/promises";
import { join, resolve } from "node:path";
import { startProdWebServer } from "./lib/prod-web-server.mjs";
import { stopChild } from "./lib/child-cleanup.mjs";
import { BROWSER_TIMEOUTS } from "./lib/browser-timeouts.mjs";
import {
attachPageDiagnostics,
loadBrowser,
@ -203,13 +204,16 @@ async function runTool(page) {
// The main-view tabs are a Mantine SegmentedControl: a visually-hidden radio
// plus a sibling <label for>. The label is the clickable element — there is
// no role="tab" here, and the radio itself has no hit box.
await page.locator('label[for$="-Tools"]').first().click({ timeout: 30_000 });
await page
.locator('label[for$="-Tools"]')
.first()
.click({ timeout: BROWSER_TIMEOUTS.ui });
await page
.getByRole("button", { name: TOOL, exact: true })
.click({ timeout: 30_000 });
.click({ timeout: BROWSER_TIMEOUTS.ui });
await page
.getByRole("button", { name: /execute tool/i })
.click({ timeout: 30_000 });
.click({ timeout: BROWSER_TIMEOUTS.ui });
}
try {
@ -244,7 +248,10 @@ try {
'[data-testid="app-elicitation"][data-app-elicitation-status="ready"]',
);
try {
await modal.waitFor({ state: "attached", timeout: 45_000 });
await modal.waitFor({
state: "attached",
timeout: BROWSER_TIMEOUTS.roundTrip,
});
} catch {
const any = page.locator('[data-testid="app-elicitation"]');
const last = (await any.count())
@ -261,14 +268,16 @@ try {
const app = page
.frameLocator('[data-testid="app-elicitation"] iframe')
.frameLocator("iframe");
await app.locator('[data-testid="choose-a"]').click({ timeout: 30_000 });
await app
.locator('[data-testid="choose-a"]')
.click({ timeout: BROWSER_TIMEOUTS.ui });
// The tool echoes the ElicitResult it received, so the result pane proves
// the app's standard result reached the SERVER — not merely the host.
await page
.getByText(/"action":"accept".*"choice":"option-a"/)
.first()
.waitFor({ state: "attached", timeout: 45_000 });
.waitFor({ state: "attached", timeout: BROWSER_TIMEOUTS.roundTrip });
await shot(page, "app-elicitation-result");
// ── 2. Not negotiated: the same tool falls back to the native form ─────
@ -279,11 +288,14 @@ try {
const nativeDialog = page.getByRole("dialog", {
name: /elicitation request/i,
});
await nativeDialog.waitFor({ state: "visible", timeout: 45_000 });
await nativeDialog.waitFor({
state: "visible",
timeout: BROWSER_TIMEOUTS.roundTrip,
});
await nativeDialog
.getByText("Choose option A or B.")
.first()
.waitFor({ state: "visible", timeout: 15_000 });
.waitFor({ state: "visible", timeout: BROWSER_TIMEOUTS.nested });
// The modal fades in; a screenshot taken on the first visible frame catches
// a transparent overlay. Nothing is asserted on this delay.
await delay(600);

17
scripts/smoke-web-tabs.mjs
View file

@ -91,6 +91,7 @@ import {
connectViaDeepLink,
} from "./lib/deep-link-connect.mjs";
import { startMcpAppServer } from "./lib/mcp-app-flow.mjs";
import { BROWSER_TIMEOUTS } from "./lib/browser-timeouts.mjs";
const repoRoot = resolve(import.meta.dirname, "..");
@ -173,7 +174,7 @@ async function openTab(page, name) {
await page
.locator(`label[for$="-${name}"]`)
.first()
.click({ timeout: 30_000 });
.click({ timeout: BROWSER_TIMEOUTS.ui });
}
/**
@ -184,7 +185,7 @@ async function waitForStage(page, { selector, screen, attrs, what }) {
try {
await page
.locator(selector)
.waitFor({ state: "attached", timeout: 45_000 });
.waitFor({ state: "attached", timeout: BROWSER_TIMEOUTS.roundTrip });
} catch {
const root = page.locator(screen);
const present = (await root.count()) > 0;
@ -244,10 +245,10 @@ try {
});
await page
.getByRole("button", { name: TOOL, exact: true })
.click({ timeout: 30_000 });
.click({ timeout: BROWSER_TIMEOUTS.ui });
await page
.getByRole("button", { name: /execute tool/i })
.click({ timeout: 30_000 });
.click({ timeout: BROWSER_TIMEOUTS.ui });
await waitForStage(page, {
selector: '[data-testid="tools-screen"][data-call-status="ok"]',
screen: '[data-testid="tools-screen"]',
@ -259,7 +260,7 @@ try {
// the data.
await page
.locator('[data-testid="structured-output"]')
.waitFor({ state: "attached", timeout: 15_000 });
.waitFor({ state: "attached", timeout: BROWSER_TIMEOUTS.nested });
// ── Resources ──────────────────────────────────────────────────────────
await openTab(page, "Resources");
@ -280,12 +281,12 @@ try {
// is closed rather than clicking blind, which would toggle it shut.
const uris = page.getByRole("button", { name: /^URIs/ });
if ((await uris.getAttribute("aria-expanded")) !== "true") {
await uris.click({ timeout: 15_000 });
await uris.click({ timeout: BROWSER_TIMEOUTS.nested });
}
// Selecting a resource reads it — there is no separate Read button here.
await page
.getByRole("button", { name: RESOURCE, exact: true })
.click({ timeout: 30_000 });
.click({ timeout: BROWSER_TIMEOUTS.ui });
await waitForStage(page, {
// Both the RPC status AND the rendered preview. The status flips the
// moment `resources/read` resolves, so stopping there would stay green
@ -311,7 +312,7 @@ try {
// submit step follows.
await page
.getByRole("button", { name: new RegExp(`^${PROMPT}`) })
.click({ timeout: 30_000 });
.click({ timeout: BROWSER_TIMEOUTS.ui });
await waitForStage(page, {
// As with the resource preview: the messages panel must have rendered,
// not merely `prompts/get` returned (Copilot, #2148).

290
scripts/verify-test-timeouts.mjs Normal file
View file

@ -0,0 +1,290 @@
/**
* Guards the wall-clock budgets every Vitest project in this repo runs under
* (#2323), and the two decisions that came with them.
*
* The class it encodes against: a budget nobody chose. Three of the six
* projects ran on Vitest's own `testTimeout: 5000` and five on its
* `hookTimeout` / `teardownTimeout: 10000` — values sized for an idle machine,
* not for the one this team works on (three or four concurrent agent sessions
* in separate worktrees, each free to run the full `npm run local:gate`, on
* eight logical cores). A correct, deterministic test cut off mid-flight by
* such a budget fails a gate its diff did not break, which trains people to
* re-run rather than read — the same argument AGENTS.md's "Lint has no warning
* tier" makes from the other direction. #2292, #2278, #2250, #1942 and #1742
* were each that class, found and fixed one site at a time.
*
* Three properties, which is what makes this worth a guard rather than a
* comment:
*
* 1. **Resolved, not declared.** It asks Vitest to resolve each project and
* reads the number a test actually gets. Asserting that a key is absent from
* some config block would pass just as happily on a config that had stopped
* being loaded at all.
* 2. **Unknown projects fail loudly.** A seventh project added without a row
* here is exactly the drift this exists to prevent, so it is an error rather
* than something the table silently skips.
* 3. **`retry` must stay unset.** A retry converts a load-induced red into a
* silent green on the only pre-push gate this repo has, and would re-open
* #1596 by hiding a real race behind a second attempt. That was a deliberate
* "do not raise" decision, so it is enforced rather than remembered.
*
* It also asserts the Testing Library half, which no Vitest config can see:
* `asyncUtilTimeout` governs every `waitFor` / `findBy*` in the web projects,
* defaults to 1000ms, and becomes the *binding* constraint on an async
* assertion once the enclosing per-test budget rises.
*
* ⚠️ Observed to FAIL against the unfixed config before it was trusted: on
* `origin/v2/main` it reports `unit`, `tui` and `launcher` at
* `testTimeout: 5000`, five of the six projects at `hookTimeout: 10000`, and
* both `asyncUtilTimeout` assertions unmet.
*/
import { existsSync, readFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
/**
* The budgets, restated here rather than imported from `vitest.shared.mts`.
*
* That duplication is the point: importing the same object the configs spread
* in would make this guard agree with any value at all, including a default
* someone reinstated by deleting the spread. A guard has to state the
* expectation independently or it is only asserting that a file parses.
*/
export const EXPECTED_TIMEOUTS = Object.freeze({
testTimeout: 15_000,
hookTimeout: 30_000,
teardownTimeout: 30_000,
});
/** Web's `integration` project pays for real servers, sockets and OAuth flows. */
export const EXPECTED_INTEGRATION_TIMEOUTS = Object.freeze({
...EXPECTED_TIMEOUTS,
testTimeout: 30_000,
});
/**
* Every Vitest project in the repo, keyed by the resolved project name. Vitest
* appends the browser instance to a browser project's name, so `storybook`
* resolves as `storybook (chromium)`; match on the leading segment.
*/
export const EXPECTED_PROJECTS = Object.freeze({
unit: EXPECTED_TIMEOUTS,
integration: EXPECTED_INTEGRATION_TIMEOUTS,
storybook: EXPECTED_TIMEOUTS,
cli: EXPECTED_TIMEOUTS,
tui: EXPECTED_TIMEOUTS,
launcher: EXPECTED_TIMEOUTS,
});
/**
* The four Vitest configs to resolve, and the project names each must yield.
*
* The node clients name no project, so their single project resolves with an
* empty name — `projects` supplies the name this guard checks it under. Web's
* three name themselves, so its entry lists them and the resolver matches by
* name instead of by position.
*/
export const CONFIG_ROOTS = Object.freeze([
{ root: "clients/web", projects: ["unit", "integration", "storybook"] },
{ root: "clients/cli", projects: ["cli"] },
{ root: "clients/tui", projects: ["tui"] },
{ root: "clients/launcher", projects: ["launcher"] },
]);
/**
* Files that must configure Testing Library's `asyncUtilTimeout`, with the
* import each has to configure it through. Storybook instruments its own copy
* of Testing Library so its interactions panel can trace each step, so
* configuring `@testing-library/*` there would configure a copy no play
* function calls.
*/
export const ASYNC_UTIL_SITES = Object.freeze([
{
file: "clients/web/src/test/setup.ts",
from: "@testing-library/react",
project: "unit",
},
{
file: "clients/web/src/test/storybookSetup.ts",
from: "storybook/test",
project: "storybook",
},
]);
/**
* Compare one resolved project against its row.
*
* @param {string} name project name as this guard knows it
* @param {{testTimeout?: unknown, hookTimeout?: unknown, teardownTimeout?: unknown, retry?: unknown}} config
* @param {Record<string, Readonly<Record<string, number>>>} [expected]
* @returns {string[]} one message per violation; empty when the project is fine
*/
export function checkProject(name, config, expected = EXPECTED_PROJECTS) {
const row = expected[name];
if (!row) {
return [
`project "${name}" has no row in EXPECTED_PROJECTS — a project whose budgets ` +
`nobody stated is exactly the drift this guard exists to prevent. Add it.`,
];
}
const failures = [];
for (const [key, want] of Object.entries(row)) {
const got = config[key];
if (got !== want) {
failures.push(
`project "${name}" resolves ${key} to ${String(got)}, expected ${want}`,
);
}
}
// Vitest leaves `retry` undefined when nothing sets it; 0 is the same
// decision written out.
const retry = config.retry;
if (retry !== undefined && retry !== 0) {
failures.push(
`project "${name}" sets retry to ${String(retry)} — a retry turns a ` +
`load-induced red into a silent green on the only pre-push gate here (#1596)`,
);
}
return failures;
}
/**
* Does this source configure `asyncUtilTimeout`, through the right import?
*
* Deliberately a source check and not a runtime one: the value only exists
* inside a running test environment, and a `configure` call reached by no
* project would satisfy a runtime probe of this module just as well.
*
* @param {string} source
* @param {string} from module the `configure` must come from
* @returns {string[]}
*/
export function checkAsyncUtilSource(source, from) {
const failures = [];
if (!new RegExp(`from\\s+["']${from.replace("/", "\\/")}["']`).test(source)) {
failures.push(`does not import from "${from}"`);
}
if (!/\bconfigure\s*\(/.test(source)) {
failures.push("does not call configure()");
} else if (!/asyncUtilTimeout\s*:\s*\d+/.test(source)) {
failures.push("calls configure() without an asyncUtilTimeout");
}
return failures;
}
/**
* Resolve every project of one config, via Vitest's own resolver.
*
* @param {string} root absolute directory holding the `vitest.config.ts` or `vite.config.ts`
* @returns {Promise<{name: string, config: Record<string, unknown>}[]>}
*/
async function resolveProjects(root) {
const { createVitest } = await import("vitest/node");
const config = ["vitest.config.ts", "vite.config.ts"]
.map((f) => join(root, f))
.find((f) => existsSync(f));
if (!config) {
throw new Error(`no vitest/vite config under ${root}`);
}
const vitest = await createVitest("test", {
watch: false,
run: true,
root,
config,
});
try {
return vitest.projects.map((p) => ({
name: p.name,
config: /** @type {Record<string, unknown>} */ (p.config),
}));
} finally {
await vitest.close();
}
}
/**
* Match a resolved project to the name this guard knows it by.
*
* A node client's single project resolves nameless, so its config entry
* supplies the name. Web's three name themselves, but a browser project's name
* carries its instance (`storybook (chromium)`), so compare the leading
* segment rather than the whole string.
*
* @param {{name: string}} project
* @param {string[]} expectedNames
* @returns {string | undefined}
*/
export function identifyProject(project, expectedNames) {
if (expectedNames.length === 1 && !project.name) return expectedNames[0];
const base = project.name.replace(/\s*\(.*\)$/, "");
return expectedNames.find((n) => n === base);
}
async function main() {
const failures = [];
let checked = 0;
for (const { root, projects: expectedNames } of CONFIG_ROOTS) {
const resolved = await resolveProjects(resolve(repoRoot, root));
const seen = new Set();
for (const project of resolved) {
const name = identifyProject(project, expectedNames);
if (!name) {
failures.push(
`${root}: resolved an unexpected project "${project.name}" — add it to ` +
`CONFIG_ROOTS and EXPECTED_PROJECTS rather than leaving its budgets unstated`,
);
continue;
}
seen.add(name);
checked += 1;
failures.push(
...checkProject(name, project.config).map((f) => `${root}: ${f}`),
);
}
for (const name of expectedNames) {
if (!seen.has(name)) {
failures.push(`${root}: project "${name}" did not resolve at all`);
}
}
}
for (const { file, from, project } of ASYNC_UTIL_SITES) {
const abs = resolve(repoRoot, file);
if (!existsSync(abs)) {
failures.push(
`${file} is missing — the "${project}" project has no asyncUtilTimeout`,
);
continue;
}
failures.push(
...checkAsyncUtilSource(readFileSync(abs, "utf-8"), from).map(
(f) => `${file} (${project} project) ${f}`,
),
);
}
if (failures.length > 0) {
console.error("verify:test-timeouts FAILED\n");
for (const f of failures) console.error(` - ${f}`);
console.error(
"\nEvery test-gate budget must be a value someone chose, sized for a machine\n" +
"running three or four concurrent worktree gates (#2323). The shared values live\n" +
"in `vitest.shared.mts` (TIMEOUTS / INTEGRATION_TIMEOUTS) and every project\n" +
"spreads one of them; Testing Library's own asyncUtilTimeout is configured in\n" +
"each web project's setup file. Raising a budget is a decision to state there,\n" +
"not a per-suite argument to add — and `retry` stays unset.",
);
process.exit(1);
}
console.log(
`verify:test-timeouts OK — ${checked} Vitest projects on stated budgets, ` +
`no retry, ${ASYNC_UTIL_SITES.length} asyncUtilTimeout sites configured.`,
);
}
if (process.argv[1] === fileURLToPath(import.meta.url)) main();

150
scripts/verify-test-timeouts.test.mjs Normal file
View file

@ -0,0 +1,150 @@
/**
* Unit tests for `verify-test-timeouts.mjs`'s pure decision logic.
*
* The guard's own resolution half needs a real Vitest and four real configs, so
* it is exercised by running it (it is in `validate`). What is tested here is
* everything that decides PASS or FAIL once a project is resolved — including
* the two failure modes a green repo can never produce on its own: an unknown
* project, and a `retry` someone added.
*
* ⚠️ Keep the filename exactly `verify-test-timeouts.test.mjs`. `node --test`
* silently SKIPS a file its glob misses and still exits 0, so a typo here reads
* as a passing suite.
*/
import test from "node:test";
import assert from "node:assert/strict";
import {
ASYNC_UTIL_SITES,
CONFIG_ROOTS,
EXPECTED_INTEGRATION_TIMEOUTS,
EXPECTED_PROJECTS,
EXPECTED_TIMEOUTS,
checkAsyncUtilSource,
checkProject,
identifyProject,
} from "./verify-test-timeouts.mjs";
const ok = { ...EXPECTED_TIMEOUTS };
test("a project on the stated budgets passes", () => {
assert.deepEqual(checkProject("unit", ok), []);
assert.deepEqual(checkProject("cli", ok), []);
assert.deepEqual(
checkProject("integration", { ...EXPECTED_INTEGRATION_TIMEOUTS }),
[],
);
});
test("each budget is checked independently", () => {
for (const key of Object.keys(EXPECTED_TIMEOUTS)) {
const failures = checkProject("unit", { ...ok, [key]: 1234 });
assert.equal(failures.length, 1, `${key} was not checked`);
assert.match(failures[0], new RegExp(`${key} to 1234, expected`));
}
});
test("a budget left on a library default is a failure, not an omission", () => {
// The exact shape this guard exists for: Vitest's own 5000/10000, which is
// what five of the six projects resolved to before #2323.
const failures = checkProject("tui", {
testTimeout: 5000,
hookTimeout: 10000,
teardownTimeout: 10000,
});
assert.equal(failures.length, 3);
});
test("an absent budget fails rather than being treated as fine", () => {
// `undefined !== 15000`. Worth pinning: a config that stopped being loaded
// resolves every key to undefined, and that must not read as a pass.
assert.equal(checkProject("unit", {}).length, 3);
});
test("a project with no row fails loudly", () => {
const failures = checkProject("brand-new", ok);
assert.equal(failures.length, 1);
assert.match(failures[0], /no row in EXPECTED_PROJECTS/);
});
test("retry is allowed to be unset or 0, and nothing else", () => {
assert.deepEqual(checkProject("unit", { ...ok }), []);
assert.deepEqual(checkProject("unit", { ...ok, retry: 0 }), []);
for (const retry of [1, 2]) {
const failures = checkProject("unit", { ...ok, retry });
assert.equal(failures.length, 1);
assert.match(failures[0], /retry/);
}
});
test("every project in the table is one the configs are asked for", () => {
const declared = CONFIG_ROOTS.flatMap((c) => c.projects).sort();
assert.deepEqual(declared, Object.keys(EXPECTED_PROJECTS).sort());
});
test("a nameless single project takes its name from the config entry", () => {
// The node clients set no `name`, so their one project resolves as "".
assert.equal(identifyProject({ name: "" }, ["cli"]), "cli");
});
test("a browser project is matched past its instance suffix", () => {
assert.equal(
identifyProject({ name: "storybook (chromium)" }, [
"unit",
"integration",
"storybook",
]),
"storybook",
);
});
test("an unrecognized project name is not silently matched", () => {
assert.equal(
identifyProject({ name: "e2e" }, ["unit", "storybook"]),
undefined,
);
// Nameless only resolves when the entry expects exactly one project —
// otherwise the guard cannot tell which row it belongs to.
assert.equal(identifyProject({ name: "" }, ["unit", "storybook"]), undefined);
});
test("a setup file that configures asyncUtilTimeout passes", () => {
const source = `import { cleanup, configure } from "@testing-library/react";
configure({ asyncUtilTimeout: 5000 });`;
assert.deepEqual(checkAsyncUtilSource(source, "@testing-library/react"), []);
});
test("configure() without the timeout is caught", () => {
const source = `import { configure } from "storybook/test";
configure({ testIdAttribute: "data-testid" });`;
assert.deepEqual(checkAsyncUtilSource(source, "storybook/test"), [
"calls configure() without an asyncUtilTimeout",
]);
});
test("the import is checked, not just the call", () => {
// Storybook instruments its own Testing Library copy, so configuring
// `@testing-library/*` from a story setup configures a copy no play function
// ever calls — green by inspection, dead in practice.
const source = `import { configure } from "@testing-library/dom";
configure({ asyncUtilTimeout: 5000 });`;
assert.deepEqual(checkAsyncUtilSource(source, "storybook/test"), [
'does not import from "storybook/test"',
]);
});
test("a setup file with no configure() at all is caught", () => {
const failures = checkAsyncUtilSource(
"export const nothing = 1;\n",
"storybook/test",
);
assert.equal(failures.length, 2);
});
test("both web projects have an asyncUtilTimeout site", () => {
assert.deepEqual(ASYNC_UTIL_SITES.map((s) => s.project).sort(), [
"storybook",
"unit",
]);
});