fix: address Copilot review round 5

Both findings are the same shape as round 2's: a guard that documents an
invariant while checking one narrow expression of it.

`retry` was only rejected at the project level, so `it("x", { retry: 2 })` and a
`--retry` flag in an npm script both walked through a guard whose own header
said "no retry anywhere". Enforcement is extended rather than the claim
narrowed, because the decision is about the behavior, not about a config key —
all three places Vitest accepts one are now scanned, and the header names the
one that remains out of reach (a human typing --retry into their own shell),
rather than overclaiming.

The test-definition scan is deliberately positional: the retry must sit in an
options object following the test name, which is the only place Vitest reads it
from. A bare word match would fire on fixture data, on `client.retry`, on an
unrelated helper's options and on a mock — all of which exist here, and a guard
that cries wolf gets disabled. Both directions are tested. The first pattern
also missed `it.each([...])("name", { retry })` entirely, so every parameterized
suite was a blind spot; the table argument is now part of the pattern.

Config discovery named only the two `.ts` spellings this repo happens to use,
while Vitest loads twelve. A `clients/foo/vitest.config.mts` was therefore
invisible, and an invisible config yields no project to reject — the same hole
round 1 found in the hand-written CONFIG_ROOTS, one level down. Both discovery
and resolution now use the full set, ordered as Vitest orders it.

Both checks were observed firing before being trusted: a planted retry in a
launcher test and a planted vitest.config.mts each fail the guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsmUAZ6aLUAFqenjJKSSRp
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-12 14:41:29 -04:00
commit f4d5b6a292

View file

@ -27,8 +27,11 @@ import {
checkProject,
checkSetupFilesWiring,
discoverConfigRoots,
findScriptRetries,
findTestLevelRetries,
identifyProject,
stripComments,
VITEST_CONFIG_FILENAMES,
} from "./verify-test-timeouts.mjs";
const ok = { ...EXPECTED_TIMEOUTS };
@ -311,3 +314,75 @@ test("discovery finds exactly the configs this guard is set up to check", () =>
);
assert.deepEqual(checkConfigRootCoverage(discoverConfigRoots()), []);
});
test("a retry on an individual test or suite is caught", () => {
// The project-level check cannot see these, and the decision is about the
// behavior rather than about one config key (Copilot).
for (const src of [
`it("x", { retry: 2 }, async () => {});`,
`test("x", { retry: 1 }, () => {});`,
`describe("x", { retry: 3 }, () => {});`,
`it.each([1])("x %i", { retry: 2 }, () => {});`,
'describe.each([[1, 2]])("x", { retry: 1 }, () => {});',
`it('single quoted', { timeout: 100, retry: 2 }, () => {});`,
]) {
assert.equal(findTestLevelRetries(src).length, 1, src);
}
});
test("a retry that is not a test option is not a false positive", () => {
// All four shapes exist in this repo. A bare /\bretry\b/ scan would fail on
// every one of them, and a guard that cries wolf gets disabled.
for (const src of [
`const fixture = { retry: 2 };`,
`expect(client.retry).toBe(2);`,
`it("x", async () => { await withRetries({ retry: 2 }); });`,
`vi.mock("x", () => ({ retry: 2 }));`,
]) {
assert.deepEqual(findTestLevelRetries(src), [], src);
}
});
test("a retry hidden in a comment is not reported", () => {
assert.deepEqual(
findTestLevelRetries(`// it("x", { retry: 2 }, () => {});`),
[],
);
});
test("a --retry flag in an npm script is caught", () => {
assert.deepEqual(findScriptRetries({ test: "vitest run --retry=2" }), [
"test: vitest run --retry=2",
]);
assert.equal(findScriptRetries({ test: "vitest run --retry 2" }).length, 1);
});
test("a script that merely contains the word retry is not a false positive", () => {
assert.deepEqual(
findScriptRetries({ test: "vitest run retry-helper.ts" }),
[],
);
assert.deepEqual(findScriptRetries({ test: "vitest run --no-retry-x" }), []);
assert.deepEqual(findScriptRetries({}), []);
assert.deepEqual(findScriptRetries(undefined), []);
});
test("discovery knows every filename Vitest loads a config from", () => {
// Deny-by-default only holds if discovery sees every config Vitest would; a
// `vitest.config.mts` this list did not name would be invisible (Copilot).
for (const ext of ["ts", "mts", "cts", "js", "mjs", "cjs"]) {
assert.ok(
VITEST_CONFIG_FILENAMES.includes(`vitest.config.${ext}`),
`vitest.config.${ext}`,
);
assert.ok(
VITEST_CONFIG_FILENAMES.includes(`vite.config.${ext}`),
`vite.config.${ext}`,
);
}
// vitest.config.* is preferred over vite.config.*, as Vitest itself does.
assert.ok(
VITEST_CONFIG_FILENAMES.indexOf("vitest.config.cjs") <
VITEST_CONFIG_FILENAMES.indexOf("vite.config.ts"),
);
});