fix: address Copilot review round 17 on #2248

Four of the five findings share one root cause: the tri-state `outcome`
added in round 15 was not propagated to consumers, which kept branching
on `ok` — true for an `incomplete` report, since nothing that was checked
was wrong.

- CLI `summarizeSkillVerification` counts off `outcome`, not `ok`, so a
  truncated walk no longer prints "no conformance errors" one line before
  exiting SKILL_INCOMPLETE. A mixed failed/incomplete catalog reports
  both counts rather than letting the louder verdict hide the quieter.
- `verifySkills` sets the truncation reason only when entries were
  actually left unread. A file crossing the byte budget as the FINAL
  entry stopped nothing, and "Stopped after 3 of 3" both read as a
  contradiction and demoted a fully-read skill out of `verified`.
- The TUI status line switches on `outcome` through a `Record` over the
  union, so the INCOMPLETE arm is reachable and a fourth outcome would
  be a type error rather than a silently missing label.
- The `incomplete` doc paragraph said it makes `ok` false. It does not,
  deliberately — corrected, and it now names `outcome` as the thing a
  consumer must branch on.
- The web sidebar composes in the per-URI `duplicate-name` warning, so
  two colliding skills are badged in the catalog instead of looking
  clean until one is selected.

The suppressed `selfAttempted` finding is stale — fixed in round 16.

The existing TUI INCOMPLETE test was passing for the wrong reason: its
fixture understated every size, which is itself a mismatch, so the report
was `failed` and only the old `ok`-first branch printed INCOMPLETE. It is
rebuilt with honest digests and sizes, so truncation comes from the
declared-size prefilter and the report is genuinely incomplete.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-07 23:12:28 -04:00
commit 9117f633aa

View file

@ -94,14 +94,17 @@ export interface SkillVerifyReport {
/**
* Why the manifest could not be checked in full, or `undefined` when it was.
*
* Set when the read bounds truncated the manifest. It is reported separately
* from `ok` being false so a consumer can tell "this skill is wrong" from
* "this skill was not fully checked" — but it does make `ok` false, because
* the alternative is worse: entries past the cap are never fetched, and
* `resource-limit-exceeded` is only a WARNING, so a manifest whose 513th
* file was tampered with reported `ok: true` and the CLI said the skill
* verified (Copilot). A bound that turns a denial of service into a false
* pass has traded down.
* Set when the read bounds truncated the manifest, and **only** when entries
* were actually left unread — a file that crosses the byte budget as the
* last entry checked nothing short, so it is not incomplete.
*
* ⚠️ It does **not** make {@link ok} false. `ok` keeps the narrow meaning of
* "nothing that was checked is wrong", and a truncated walk checked nothing
* that was wrong. The signal a consumer must branch on is {@link outcome}
* being `"incomplete"`, never `!ok` — a manifest whose unread 513th file was
* tampered with reports `ok: true`, and a consumer that prints "verified" on
* `ok` alone turns a denial of service into a false pass (Copilot). Both the
* CLI summary and the TUI status line had exactly that bug.
*/
incomplete?: string;
/**
@ -306,7 +309,7 @@ export async function verifySkills(
// an equivalent form, so a raw string test would disagree with it and read
// the same file a second time.
let selfAttempted = false;
for (const resource of manifest) {
for (const [index, resource] of manifest.entries()) {
if (skillUriIdentity(resource.uri) === entryIdentity) {
// Marked before the read, not after: a row the walk reached but could
// not read has still been attempted, and its failure is recorded here
@ -357,7 +360,15 @@ export async function verifySkills(
// prevent and is not something this API exposes.
receivedBytes += bytes.byteLength;
if (receivedBytes > SKILL_MAX_TOTAL_BYTES) {
incomplete = `Stopped after ${files.length} of ${declared.length} manifest entries: the files actually served exceed the ${SKILL_MAX_TOTAL_BYTES}-byte interoperability limit, whatever sizes the manifest declared.`;
// ⚠️ Only *incomplete* when the budget actually cost a read. Crossing
// the line on the final entry stopped nothing — every manifest row was
// fetched and checked — and reporting "Stopped after 3 of 3" there
// both reads as a contradiction and demotes a fully-checked skill out
// of `verified` (Copilot). The prefilter's own reason, if it dropped
// entries before the walk, is already set and is not overwritten.
if (index < manifest.length - 1) {
incomplete = `Stopped after ${index + 1} of ${declared.length} manifest entries: the files actually served exceed the ${SKILL_MAX_TOTAL_BYTES}-byte interoperability limit, whatever sizes the manifest declared.`;
}
break;
}
}