fix: address Copilot review round 4
The comment stripper had a real hole: the line-comment regex was anchored, so it
only removed `//` at the start of a line and a call disabled after code —
`const disabled = true; // configure({ asyncUtilTimeout: 1000 });` — survived it
and the guard passed with no effective configuration. Replaced with a scanner,
because a regex gets this wrong in both directions: it has to strip a comment
that follows code, and it must NOT treat the `//` inside a string literal as one,
which would silently truncate any line holding a URL. Quote tracking (with
escapes) is the only way to have both. Three regression tests.
Two documentation claims were also wrong. The guard header and quality-gate.md
cited #2278 and #2250 as instances of a budget nobody chose; they were not —
#2278 was a missing condition wait around a geometry read and #2250 a real race
in a test's own timing, both fixed by making the test wait for the right thing.
Presenting a race fix as evidence for a larger ceiling argues against #1596,
which this guard exists to uphold. Only the budget and poll-ceiling cases are
cited now, and both files say explicitly why those two are excluded.
And browser-timeouts.mjs overstated its own behavior twice: `bestEffort` can
spend its full budget on an otherwise passing smoke precisely because its expiry
is caught, and raising any budget does delay the report when the thing waited on
never arrives. The diagnostics improve the eventual message; they do not make
the wait shorter. Both now stated plainly, so the header is something to weigh a
change against rather than a reassurance.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PsmUAZ6aLUAFqenjJKSSRp
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
1cc1b48c30
commit
7b794de0b0
3 changed files with 104 additions and 19 deletions
|
|
@ -10,16 +10,23 @@
|
|||
* has, and for the same reason: a budget nobody can find is a budget nobody
|
||||
* revisits.
|
||||
*
|
||||
* These are generous on purpose and cost a passing run nothing. Playwright
|
||||
* budgets are ceilings on a poll, not sleeps: every one of them returns the
|
||||
* instant its locator condition holds. What they have to absorb is a smoke
|
||||
* running against a cold `dist/` build on a machine already carrying three
|
||||
* other agent sessions' gates.
|
||||
* Three of the four are ceilings on a poll rather than sleeps — `ui`,
|
||||
* `roundTrip` and `nested` each return the instant their locator condition
|
||||
* holds, so a passing smoke pays only the time the app actually took. What they
|
||||
* absorb is a smoke running against a cold `dist/` build on a machine already
|
||||
* carrying other agent sessions' gates.
|
||||
*
|
||||
* ⚠️ Raising one of these does NOT make a smoke wait longer for a server that
|
||||
* is never coming: the launcher-death race in `smoke-web-browser.mjs` and the
|
||||
* `waitForStage` diagnostics in the other two are what report that, and they
|
||||
* report it with a cause rather than a timeout.
|
||||
* ⚠️ `bestEffort` is the exception and is not free: its expiry is caught and
|
||||
* ignored, so a smoke that never reaches network idle spends the full budget on
|
||||
* an otherwise **passing** run. That is why it is the smallest of the four, and
|
||||
* why it should stay that way.
|
||||
*
|
||||
* ⚠️ Raising any of these does delay the report when the thing waited on never
|
||||
* arrives — a larger budget is a longer wait before the failure. What the
|
||||
* launcher-death race in `smoke-web-browser.mjs` and the `waitForStage`
|
||||
* diagnostics in the other two buy is a better *eventual* message, naming a
|
||||
* cause instead of a locator; they do not make the wait shorter. Weigh both
|
||||
* when changing a value here.
|
||||
*/
|
||||
|
||||
export const BROWSER_TIMEOUTS = Object.freeze({
|
||||
|
|
|
|||
|
|
@ -10,8 +10,13 @@
|
|||
* eight logical cores). A correct, deterministic test cut off mid-flight by
|
||||
* such a budget fails a gate its diff did not break, which trains people to
|
||||
* re-run rather than read — the same argument AGENTS.md's "Lint has no warning
|
||||
* tier" makes from the other direction. #2292, #2278, #2250, #1942 and #1742
|
||||
* were each that class, found and fixed one site at a time.
|
||||
* tier" makes from the other direction. #2292, #1942 and #1742 were each that
|
||||
* class — a budget or poll ceiling nobody chose — found and fixed one site at a
|
||||
* time. ⚠️ Not every timing failure in this repo's history belongs here: #2278
|
||||
* was a missing condition wait around a geometry read and #2250 a genuine race
|
||||
* in a test's own timing, both fixed by making the test wait for the right
|
||||
* thing. Citing those as evidence for a larger ceiling would argue against
|
||||
* #1596, which this guard is meant to uphold rather than erode (Copilot).
|
||||
*
|
||||
* Three properties, which is what makes this worth a guard rather than a
|
||||
* comment:
|
||||
|
|
@ -172,22 +177,67 @@ export function checkProject(name, config, expected = EXPECTED_PROJECTS) {
|
|||
}
|
||||
|
||||
/**
|
||||
* Strip `//` and block comments so a commented-out call cannot satisfy a check.
|
||||
* Strip comments so a disabled call cannot satisfy a check.
|
||||
*
|
||||
* Comment-aware rather than exact, deliberately: a naive scan of the raw source
|
||||
* would accept a `// configure({ asyncUtilTimeout: 1000 })` left behind by
|
||||
* someone disabling it, which is the most likely way this stops being
|
||||
* configured (Copilot). String literals are not parsed out — a `configure(` in
|
||||
* a string would still be accepted — but these two files are fifteen lines each
|
||||
* and the false-positive that matters is the commented one.
|
||||
* configured (Copilot).
|
||||
*
|
||||
* A scanner rather than a regex, for two reasons that a regex gets wrong in
|
||||
* opposite directions. It must strip a comment that **follows code** on a line
|
||||
* — `const disabled = true; // configure({ … })` — which an anchored
|
||||
* line-comment pattern misses entirely (Copilot); and it must NOT treat the
|
||||
* `//` inside a string literal as the start of one, which an unanchored pattern
|
||||
* would, silently truncating any line holding a URL. Tracking quotes is the
|
||||
* only way to have both.
|
||||
*
|
||||
* @param {string} source
|
||||
* @returns {string}
|
||||
*/
|
||||
export function stripComments(source) {
|
||||
return source
|
||||
.replace(/\/\*[\s\S]*?\*\//g, "")
|
||||
.replace(/(^|\n)\s*\/\/[^\n]*/g, "$1");
|
||||
let out = "";
|
||||
let quote = null;
|
||||
for (let i = 0; i < source.length; i += 1) {
|
||||
const c = source[i];
|
||||
const next = source[i + 1];
|
||||
if (quote) {
|
||||
// A backslash escapes the next character, so a `\"` cannot close a `"`.
|
||||
if (c === "\\") {
|
||||
out += c + (next ?? "");
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
if (c === quote) quote = null;
|
||||
out += c;
|
||||
continue;
|
||||
}
|
||||
if (c === '"' || c === "'" || c === "`") {
|
||||
quote = c;
|
||||
out += c;
|
||||
continue;
|
||||
}
|
||||
if (c === "/" && next === "/") {
|
||||
while (i < source.length && source[i] !== "\n") i += 1;
|
||||
// Keep the newline, so line structure (and any anchored match a caller
|
||||
// makes) survives the strip.
|
||||
out += "\n";
|
||||
continue;
|
||||
}
|
||||
if (c === "/" && next === "*") {
|
||||
i += 2;
|
||||
while (
|
||||
i < source.length &&
|
||||
!(source[i] === "*" && source[i + 1] === "/")
|
||||
) {
|
||||
i += 1;
|
||||
}
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
out += c;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -222,6 +222,34 @@ configure({ asyncUtilTimeout: 1_000 });`;
|
|||
assert.deepEqual(checkAsyncUtilSource(source, "storybook/test"), []);
|
||||
});
|
||||
|
||||
test("a call disabled in a TRAILING line comment does not satisfy the check", () => {
|
||||
// The anchored regex this replaced only stripped `//` at the start of a line,
|
||||
// so a call commented out after code survived and the guard passed with no
|
||||
// effective configuration (Copilot).
|
||||
const source = `import { configure } from "storybook/test";
|
||||
const disabled = true; // configure({ asyncUtilTimeout: 1000 });`;
|
||||
assert.deepEqual(checkAsyncUtilSource(source, "storybook/test"), [
|
||||
"does not call configure()",
|
||||
]);
|
||||
});
|
||||
|
||||
test("a // inside a string literal is not treated as a comment", () => {
|
||||
// The failure mode in the other direction: an unanchored strip would truncate
|
||||
// any line holding a URL, which could remove the real call.
|
||||
const source = `import { configure } from "storybook/test";
|
||||
const docs = "https://testing-library.com/docs";
|
||||
configure({ asyncUtilTimeout: 1000 });`;
|
||||
assert.deepEqual(checkAsyncUtilSource(source, "storybook/test"), []);
|
||||
assert.match(stripComments(source), /https:\/\/testing-library/);
|
||||
});
|
||||
|
||||
test("stripComments handles escapes and mid-line block comments", () => {
|
||||
assert.equal(stripComments("a /* x */ b"), "a b");
|
||||
// A backslash-escaped quote must not close the string early, or everything
|
||||
// after it would be scanned as code.
|
||||
assert.match(stripComments('const s = "a\\" // b";\nkeep'), /keep/);
|
||||
});
|
||||
|
||||
test("stripComments leaves executable code alone", () => {
|
||||
const source = `/** header */
|
||||
import { configure } from "storybook/test"; // trailing
|
||||
|
|
|
|||
Loading…
Reference in a new issue