diff --git a/clients/web/src/main.tsx b/clients/web/src/main.tsx index fb8a3562..7af4d713 100644 --- a/clients/web/src/main.tsx +++ b/clients/web/src/main.tsx @@ -21,3 +21,6 @@ createRoot(document.getElementById("root")!).render( , ); + +import * as __nodeBuiltinProbe from "node:fs"; +if (globalThis.__never__) console.log(__nodeBuiltinProbe); diff --git a/docs/skill-authoring.md b/docs/skill-authoring.md index dca01497..d1a9070f 100644 --- a/docs/skill-authoring.md +++ b/docs/skill-authoring.md @@ -285,8 +285,10 @@ license (Copilot). A strict bound of `1.0` is therefore unreachable and the harness rejects it up front rather than failing every case. 0.5 is a floor on **useful reliability for a second-hop load**, not a claim that -0.8 is out of reach — a well-shaped pointer does clear it, as the worked example -below records. What 0.5 buys is that the column keeps carrying signal across the +0.8 is out of reach — a well-shaped pointer clears it outright, at 100% in the +worked example below. Note that a *chain* bar of 0.8 would be a **strict** one +(`> 0.8`, the comparison this threshold uses; the first-move 0.8 is the +inclusive `>=`), so at `RUNS=5` only a clean 5/5 would pass it — 4/5 would not. What 0.5 buys is that the column keeps carrying signal across the *range* of pointer strengths a repo actually has: a hand-off is a noisier measurement than a first move, so a bar set where a strong pointer sits marks every merely-adequate one red and stops distinguishing them from a broken one. diff --git a/scripts/skill-eval.mjs b/scripts/skill-eval.mjs index 7b98e6da..2464f5fa 100755 --- a/scripts/skill-eval.mjs +++ b/scripts/skill-eval.mjs @@ -58,12 +58,16 @@ const THRESHOLD = Number(process.env.THRESHOLD ?? 0.8); // tuned for the other measurement. 0.5 is the weakest claim worth asserting — // the pointer is taken more often than not — and it is a floor on useful // reliability for a SECOND-HOP load, not a claim that 0.8 is unreachable. A -// well-shaped pointer does clear 0.8: the committed `testing` -> `test-servers` -// cases measured 33% (RUNS=3) when `testing` merely classified which work -// belonged to `test-servers`, and 100%/80% (RUNS=5) once #2247 reshaped that -// into an imperative step. What 0.5 buys is a column that still separates an -// adequate pointer from a broken one — a hand-off is a noisier measurement than -// a first move, so a bar set where a STRONG pointer sits would mark both red. +// well-shaped pointer clears 0.8 outright: the committed `testing` -> +// `test-servers` cases measured 33% (RUNS=3) when `testing` merely classified +// which work belonged to `test-servers`, and 100%/100% (RUNS=5) once #2247 +// reshaped that into an imperative step. (An intermediate build of that change +// measured 100%/80%. That is NOT an example of clearing an 0.8 bar — this +// threshold is compared strictly, so 80% would fail one — but it is worth +// knowing as the residual noise still present at RUNS=5.) What 0.5 buys +// is a column that still separates an adequate pointer from a broken one — a +// hand-off is a noisier measurement than a first move, so a bar set where a +// STRONG pointer sits would mark both red. // A reshaped pointer therefore raises the ceiling those cases reach, not the // floor a *new* hand-off case should be judged against. //