fix: address Copilot review round 17 on #2248
Four of the five findings share one root cause: the tri-state `outcome` added in round 15 was not propagated to consumers, which kept branching on `ok` — true for an `incomplete` report, since nothing that was checked was wrong. - CLI `summarizeSkillVerification` counts off `outcome`, not `ok`, so a truncated walk no longer prints "no conformance errors" one line before exiting SKILL_INCOMPLETE. A mixed failed/incomplete catalog reports both counts rather than letting the louder verdict hide the quieter. - `verifySkills` sets the truncation reason only when entries were actually left unread. A file crossing the byte budget as the FINAL entry stopped nothing, and "Stopped after 3 of 3" both read as a contradiction and demoted a fully-read skill out of `verified`. - The TUI status line switches on `outcome` through a `Record` over the union, so the INCOMPLETE arm is reachable and a fourth outcome would be a type error rather than a silently missing label. - The `incomplete` doc paragraph said it makes `ok` false. It does not, deliberately — corrected, and it now names `outcome` as the thing a consumer must branch on. - The web sidebar composes in the per-URI `duplicate-name` warning, so two colliding skills are badged in the catalog instead of looking clean until one is selected. The suppressed `selfAttempted` finding is stale — fixed in round 16. The existing TUI INCOMPLETE test was passing for the wrong reason: its fixture understated every size, which is itself a mismatch, so the report was `failed` and only the old `ok`-first branch printed INCOMPLETE. It is rebuilt with honest digests and sizes, so truncation comes from the declared-size prefilter and the report is genuinely incomplete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
5a1f0efb5a
commit
9117f633aa
1 changed files with 183 additions and 29 deletions
|
|
@ -656,6 +656,35 @@ describe("verifySkills (#2248)", () => {
|
|||
expect(readResource.mock.calls.length).toBeLessThan(10);
|
||||
expect(report.incomplete).toMatch(/actually served/);
|
||||
});
|
||||
|
||||
it("is not incomplete when the budget is crossed by the LAST entry", async () => {
|
||||
// Crossing the line on the final row stopped nothing: every manifest entry
|
||||
// was fetched and checked. Reporting "Stopped after 4 of 4" there both
|
||||
// reads as a contradiction and demotes a fully-read skill out of
|
||||
// `verified` (Copilot).
|
||||
//
|
||||
// The sizes are understated for the same reason as the test above — with
|
||||
// honest ones the *declared* prefilter stops first and the received-bytes
|
||||
// guard is never reached at all. That understatement is itself a size
|
||||
// mismatch, so this fixture is `failed`; what it pins is that the walk is
|
||||
// not ALSO reported as cut short.
|
||||
const { skill, client } = await truncatable({
|
||||
name: "edge",
|
||||
count: 4,
|
||||
body: "z".repeat(6 * 1024 * 1024),
|
||||
});
|
||||
for (const r of skill.resources as { size?: number }[]) r.size = 1;
|
||||
const readResource = vi.spyOn(
|
||||
client as unknown as { readResource: (u: string) => unknown },
|
||||
"readResource",
|
||||
);
|
||||
const [report] = await verifySkills(client, [skill]);
|
||||
// All four read — the fourth is what crosses the 16 MiB budget.
|
||||
expect(readResource.mock.calls.length).toBe(4);
|
||||
expect(report.files).toHaveLength(4);
|
||||
expect(report.incomplete).toBeUndefined();
|
||||
expect(report.outcome).toBe("failed");
|
||||
});
|
||||
it("still reports the file that crossed the byte budget", async () => {
|
||||
// The crossing file is verified before the walk stops, so its verdict is
|
||||
// not fetched and then thrown away.
|
||||
|
|
|
|||
Loading…
Reference in a new issue