docs(smoke): point each smoke's on-demand example at its own command
smoke-web-elicitation.mjs told the reader to run `SMOKE_BROWSER=webkit npm run smoke:web:app` -- the App smoke, not the elicitation smoke the file documents. Following it exercises a different path entirely, and it does not fail while doing so: a real smoke runs and passes, so nothing signals that you never tested what you were reading about. This is a direct cost of my own decision two rounds ago to make the two App headers word-for-word identical so they could not drift. The shared PROSE should be identical; the example command is the one line that must not be, and copy-paste does not distinguish them. Both headers now name their own smoke, and both point at npm run smoke:web:webkit for all three. Guarded, since the class is invisible at runtime by construction: run-engine-smokes.test.mjs now asserts no smoke header tells the reader to run a sibling's command, and that every `npm run` it does name is a real script. Both mutation-verified -- restoring the exact bug fails the first, and naming a nonexistent script fails the second. Addresses the suppressed comment on Copilot review 5031065042 of #2133. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
93bc7df360
commit
86c143fd0d
3 changed files with 51 additions and 2 deletions
|
|
@ -106,3 +106,50 @@ describe("every engine tier consumes ENGINE_SMOKES", () => {
|
|||
assert.match(scripts["smoke:web:engine"], /run-engine-smokes\.mjs$/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("each smoke's own docs point at its own command", () => {
|
||||
/** The npm script that runs each engine-sensitive smoke on its own. */
|
||||
const OWN_SCRIPT = {
|
||||
"smoke-web-browser.mjs": "smoke:web:browser",
|
||||
"smoke-web-app.mjs": "smoke:web:app",
|
||||
"smoke-web-elicitation.mjs": "smoke:web:elicit",
|
||||
};
|
||||
|
||||
it("never tells the reader to run a sibling smoke", () => {
|
||||
// `smoke-web-elicitation.mjs` shipped an on-demand example invoking
|
||||
// `smoke:web:app` (Copilot, #2133) — a direct cost of my deciding the two
|
||||
// App headers should be word-for-word identical so they could not drift.
|
||||
// The shared PROSE should be identical; the example command is the one line
|
||||
// that must not be, and copy-paste does not distinguish them.
|
||||
//
|
||||
// Following such an example does not fail — it runs a real smoke and passes.
|
||||
// It just never exercises the file you were reading about, which is why
|
||||
// nothing else catches this.
|
||||
for (const [file, own] of Object.entries(OWN_SCRIPT)) {
|
||||
const source = readFileSync(join(scriptDir, file), "utf8");
|
||||
const siblings = Object.values(OWN_SCRIPT).filter((s) => s !== own);
|
||||
for (const sibling of siblings) {
|
||||
assert.ok(
|
||||
!source.includes(`npm run ${sibling}`),
|
||||
`${file} tells the reader to run \`npm run ${sibling}\` — a sibling ` +
|
||||
`smoke. Its examples must use \`${own}\`, or the reader never ` +
|
||||
`exercises the path this file documents.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("every command it does name is a real script", () => {
|
||||
// A command that does not exist is worse than none, and the header is the
|
||||
// one place nothing executes to find out.
|
||||
for (const file of Object.keys(OWN_SCRIPT)) {
|
||||
const source = readFileSync(join(scriptDir, file), "utf8");
|
||||
for (const [, name] of source.matchAll(/npm run ([\w:]+)/g)) {
|
||||
assert.ok(
|
||||
Object.hasOwn(scripts, name),
|
||||
`${file} names \`npm run ${name}\`, which is not a script`,
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -50,7 +50,8 @@
|
|||
* - **`npm run ci`**, the local pre-push gate, runs it in **Chromium and
|
||||
* Firefox** — the Firefox pass is `smoke:web:firefox`, and it is the one
|
||||
* gate step with no GitHub CI counterpart.
|
||||
* - **WebKit is on demand only**: `SMOKE_BROWSER=webkit npm run smoke:web:app`.
|
||||
* - **WebKit is on demand only**: `SMOKE_BROWSER=webkit npm run smoke:web:app`
|
||||
* for this smoke alone, or `npm run smoke:web:webkit` for all three.
|
||||
*
|
||||
* Firefox passes. WebKit fails this smoke for reasons nobody has identified and
|
||||
* nobody is investigating: it does not reproduce in real Safari, and an isolated
|
||||
|
|
|
|||
|
|
@ -31,7 +31,8 @@
|
|||
* - **`npm run ci`**, the local pre-push gate, runs it in **Chromium and
|
||||
* Firefox** — the Firefox pass is `smoke:web:firefox`, and it is the one
|
||||
* gate step with no GitHub CI counterpart.
|
||||
* - **WebKit is on demand only**: `SMOKE_BROWSER=webkit npm run smoke:web:app`.
|
||||
* - **WebKit is on demand only**: `SMOKE_BROWSER=webkit npm run smoke:web:elicit`
|
||||
* for this smoke alone, or `npm run smoke:web:webkit` for all three.
|
||||
*
|
||||
* Firefox passes. WebKit fails this smoke for reasons nobody has identified and
|
||||
* nobody is investigating: it does not reproduce in real Safari, and an isolated
|
||||
|
|
|
|||
Loading…
Reference in a new issue