chore(web): address Copilot review on the tab smoke
Three findings, all valid. **The read/get stages asserted an RPC status, not a render.** #2148 asks for the resource content and the prompt messages to *render*, and `data-read-status` / `data-get-status` flip the moment the call resolves — so the smoke stayed green with the panel deleted. `ResourcePreviewPanel` and `PromptMessagesDisplay` now carry `data-testid="resource-preview"` / `"prompt-messages"`, and each stage waits on the status **and** the rendered panel. Verified by breaking each testid: the smoke fails with `reading `events` did not render a preview — last observed: data-read-status="ok"`, which is the finding stated back exactly. **`smoke:web:tabs` followed ambient `SMOKE_BROWSER`**, so `SMOKE_BROWSER=firefox npm run ci` would have run the supposedly Chromium-only smoke under Firefox — contradicting both this PR's stated scope and the gated-tier invariant in `run-engine-smokes.test.mjs`. The npm script now pins `chromium` on **both** halves (install and launch) and the smoke resolves through `resolveRequestedBrowser(argv, env)`, where an explicit argument wins over the environment; a new case in that same test file pins it, including that the smoke stays out of `ENGINE_SMOKES`. **The AGENTS rules were not mirrored into `.github/copilot-instructions.md`.** Correct, and worse than an omission: that file's engine bullet asserts the headless web smokes are "engine-parameterized, not Chromium-only", which this PR made false. It now names `smoke:web:tabs` as the deliberate exception, and carries the genuinely review-citable new rule — assert `data-*`, never copy, and never stop at an RPC status where the ask was that something render. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UL9retfvAXRgvi6EWk4SY4 Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
d710f90b90
commit
23088e9c69
2 changed files with 101 additions and 25 deletions
|
|
@ -90,6 +90,23 @@ describe("every engine tier consumes ENGINE_SMOKES", () => {
|
|||
}
|
||||
});
|
||||
|
||||
it("the Chromium-only tab smoke pins its engine on BOTH halves", () => {
|
||||
// `smoke:web:tabs` is not engine-sensitive and deliberately stays out of
|
||||
// ENGINE_SMOKES — but "Chromium-only" is a claim the script has to keep.
|
||||
// Without the literal engine it followed ambient `SMOKE_BROWSER`, so
|
||||
// `SMOKE_BROWSER=firefox npm run ci` would have run it under Firefox
|
||||
// (Copilot, #2148). Both halves are pinned because they must AGREE: an
|
||||
// install of one engine and a launch of another surfaces as a confusing
|
||||
// "Executable doesn't exist" rather than as the wiring error it is.
|
||||
assert.match(
|
||||
scripts["smoke:web:tabs"],
|
||||
/install-smoke-browser\.mjs chromium .*smoke-web-tabs\.mjs chromium$/,
|
||||
);
|
||||
// And it must not be in ENGINE_SMOKES, or the Firefox tier would pick it
|
||||
// up regardless of what its own script pins.
|
||||
assert.ok(!ENGINE_SMOKES.includes("smoke-web-tabs.mjs"));
|
||||
});
|
||||
|
||||
it("each gated tier names its engine explicitly rather than reading the env", () => {
|
||||
// An ambient SMOKE_BROWSER must not be able to redirect a gate: without the
|
||||
// literal engine, `SMOKE_BROWSER=firefox npm run ci` would run Firefox twice
|
||||
|
|
|
|||
|
|
@ -30,12 +30,17 @@
|
|||
* only summarize it, so a result panel that renders text alone looks
|
||||
* correct while dropping the data.
|
||||
* 2. **Resources** — the list renders, a resource is read, and the preview
|
||||
* populates. Templates are asserted present (the RFC 6570 fixture, #1919),
|
||||
* which is what proves `resources/templates/list` reached the screen —
|
||||
* that list is fetched by a *different* call than `resources/list` and can
|
||||
* fail on its own.
|
||||
* 3. **Prompts** — the list renders and `simple_prompt` is fetched (selecting
|
||||
* an argument-less prompt auto-fetches), and the messages render.
|
||||
* panel renders its contents. Templates are asserted present (the RFC 6570
|
||||
* fixture, #1919), which is what proves `resources/templates/list` reached
|
||||
* the screen — that list is fetched by a *different* call than
|
||||
* `resources/list` and can fail on its own.
|
||||
* 3. **Prompts** — the list renders, `simple_prompt` is fetched (selecting an
|
||||
* argument-less prompt auto-fetches), and the messages panel renders.
|
||||
*
|
||||
* The read/get stages assert the RPC status **and** the rendered panel, not the
|
||||
* status alone. Each `data-*-status` flips the moment its call resolves, so a
|
||||
* smoke stopping there would stay green with the panel deleted — which is the
|
||||
* opposite of what a connected-flow smoke is for.
|
||||
*
|
||||
* Deliberately **one script over one browser launch** rather than one per tab:
|
||||
* launching the browser and booting the server dominate the cost, and
|
||||
|
|
@ -79,11 +84,8 @@
|
|||
import { resolve } from "node:path";
|
||||
import { startProdWebServer } from "./lib/prod-web-server.mjs";
|
||||
import { stopChild } from "./lib/child-cleanup.mjs";
|
||||
import {
|
||||
attachPageDiagnostics,
|
||||
loadBrowser,
|
||||
resolveBrowserName,
|
||||
} from "./lib/headless-browser.mjs";
|
||||
import { attachPageDiagnostics, loadBrowser } from "./lib/headless-browser.mjs";
|
||||
import { resolveRequestedBrowser } from "./install-smoke-browser.mjs";
|
||||
import {
|
||||
buildConnectDeepLink,
|
||||
connectViaDeepLink,
|
||||
|
|
@ -92,14 +94,20 @@ import { startMcpAppServer } from "./lib/mcp-app-flow.mjs";
|
|||
|
||||
const repoRoot = resolve(import.meta.dirname, "..");
|
||||
|
||||
// Resolved before anything starts, so an unsupported SMOKE_BROWSER fails
|
||||
// immediately rather than after a web server and an MCP server are up. Every
|
||||
// message carries the engine so a failure names which run it came from — even
|
||||
// though this smoke is Chromium-only in every gate, it can still be pointed
|
||||
// elsewhere by hand.
|
||||
// The engine comes from an explicit ARGUMENT first, and only then from
|
||||
// `SMOKE_BROWSER`. That precedence is the load-bearing part: `smoke:web:tabs`
|
||||
// passes `chromium` to both this and the installer, so an ambient
|
||||
// `SMOKE_BROWSER=firefox npm run ci` cannot silently redirect a gated tier that
|
||||
// claims to be Chromium-only — the same invariant `run-engine-smokes.test.mjs`
|
||||
// pins for the engine tiers (Copilot, #2148). Running another engine by hand is
|
||||
// still one argument away: `node scripts/smoke-web-tabs.mjs firefox`.
|
||||
//
|
||||
// Resolved before anything starts, so an unsupported value fails immediately
|
||||
// rather than after a web server and an MCP server are up. Every message
|
||||
// carries the engine, so a failure names which run it came from.
|
||||
let BROWSER;
|
||||
try {
|
||||
BROWSER = resolveBrowserName();
|
||||
BROWSER = resolveRequestedBrowser(process.argv.slice(2), process.env);
|
||||
} catch (err) {
|
||||
console.error(
|
||||
`smoke:web:tabs FAILED — ${err instanceof Error ? err.message : String(err)}`,
|
||||
|
|
@ -279,10 +287,16 @@ try {
|
|||
.getByRole("button", { name: RESOURCE, exact: true })
|
||||
.click({ timeout: 30_000 });
|
||||
await waitForStage(page, {
|
||||
selector: '[data-testid="resources-screen"][data-read-status="ok"]',
|
||||
// Both the RPC status AND the rendered preview. The status flips the
|
||||
// moment `resources/read` resolves, so stopping there would stay green
|
||||
// with the preview panel removed entirely (Copilot, #2148) — and #2148
|
||||
// asks for the content to *render*, not merely for the call to return.
|
||||
selector:
|
||||
'[data-testid="resources-screen"][data-read-status="ok"]' +
|
||||
' [data-testid="resource-preview"]',
|
||||
screen: '[data-testid="resources-screen"]',
|
||||
attrs: ["data-read-status"],
|
||||
what: `reading \`${RESOURCE}\` did not succeed`,
|
||||
what: `reading \`${RESOURCE}\` did not render a preview`,
|
||||
});
|
||||
|
||||
// ── Prompts ────────────────────────────────────────────────────────────
|
||||
|
|
@ -299,10 +313,14 @@ try {
|
|||
.getByRole("button", { name: new RegExp(`^${PROMPT}`) })
|
||||
.click({ timeout: 30_000 });
|
||||
await waitForStage(page, {
|
||||
selector: '[data-testid="prompts-screen"][data-get-status="ok"]',
|
||||
// As with the resource preview: the messages panel must have rendered,
|
||||
// not merely `prompts/get` returned (Copilot, #2148).
|
||||
selector:
|
||||
'[data-testid="prompts-screen"][data-get-status="ok"]' +
|
||||
' [data-testid="prompt-messages"]',
|
||||
screen: '[data-testid="prompts-screen"]',
|
||||
attrs: ["data-get-status"],
|
||||
what: `getting \`${PROMPT}\` did not succeed`,
|
||||
what: `getting \`${PROMPT}\` did not render its messages`,
|
||||
});
|
||||
};
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue