fix: address Copilot review round 2 on #2248

Four findings, two of them silent holes in the verification itself.

**A blob-served SKILL.md skipped the frontmatter check entirely.** The entry's
own file was captured as `contents.text`, so a server returning the markdown
as a base64 `blob` — a legal `resources/read` shape this module already
decodes for the digest — never reached the comparison, and the report still
said `ok`. A MUST that quietly did not run. The file is now held as bytes and
the text derived from them, which also guarantees the digest and the
frontmatter describe one snapshot.

**A dynamic skill whose SKILL.md could not be read reported `ok: true`.** It
has no manifest rows, so `files` stayed empty, and its only static finding is
a warning — a verification that could not be performed was reported as one
that passed. The fallback read now records a `read-error` for all three ways
it can fail. It is gated on `manifestListsSelf` (by normalized identity) so a
self-entry the manifest listed and failed to read is not read, or reported,
twice.

**A served `.nan` compared equal to a listed `null`.** YAML expresses
non-finite numbers, JSON does not, and `JSON.stringify` turns all of them
into `null` — so the canonical comparison reported a real mismatch as
agreement. Verified against the parser before fixing. They now canonicalize
to a form no JSON scalar can equal, and the value is named in the finding.

Also: the self-entry match is by normalized identity rather than raw string,
which stops a second read and stops this function disagreeing with
`checkSkillConformance`; and `SkillsTab.tsx` gains the file header AGENTS.md
requires.

The four `err instanceof Error` ternaries are one `reasonOf` helper now —
extracted because the coverage gate found every one of their non-Error arms
uncovered, and one honestly-tested branch beats four ignores.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ND46Ci64DC3tzBy9aG81Nc
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-09-07 19:24:30 -04:00
commit 033600e4ca
2 changed files with 288 additions and 13 deletions

View file

@ -561,6 +561,16 @@ function canonicalEntry(entry: SkillEntry): string {
/** Object keys sorted recursively; array ORDER is preserved throughout. */
function canonicalize(value: unknown): unknown {
// YAML can express `.nan` and `.inf`; JSON cannot. `JSON.stringify` turns
// every one of them into `null`, so without this a served `x: .nan` would
// compare EQUAL to a listing declaring `x: null` — a mismatch silently
// reported as agreement (Copilot). The listing side arrived over JSON-RPC and
// can never hold a non-finite number, so one appearing here is always a real
// difference. Rendered as an object, which cannot equal any JSON scalar, and
// which names the value in the finding rather than hiding it.
if (typeof value === "number" && !Number.isFinite(value)) {
return { "#non-finite": String(value) };
}
if (Array.isArray(value)) return value.map(canonicalize);
if (value === null || typeof value !== "object") return value;
return sortKeys(value as Record<string, unknown>);
@ -639,6 +649,20 @@ export function textToBytes(text: string): Uint8Array {
return new TextEncoder().encode(text);
}
/**
* A skill file's bytes as UTF-8 text — the inverse of {@link textToBytes}.
*
* Deliberately **non-fatal**: a `SKILL.md` that is not valid UTF-8 decodes with
* replacement characters rather than throwing. That is the more useful failure,
* because the frontmatter comparison then reports a concrete difference between
* what the listing claimed and what the file actually holds, instead of
* collapsing into "could not decode" and skipping the check the SEP makes
* mandatory.
*/
export function bytesToText(bytes: Uint8Array): string {
return new TextDecoder().decode(bytes);
}
/**
* Raw bytes of a `resources/read` blob content block (standard base64).
* Uses `atob`, which Node ≥22 and every browser provide, so this stays

View file

@ -28,6 +28,7 @@ import { AuthRecoveryRequiredError } from "../auth/challenge.js";
import type { InspectorClientProtocol } from "./inspectorClientProtocol.js";
import type { RequestMetadata } from "./types.js";
import {
bytesToText,
checkSkillConformance,
checkSkillFrontmatterMatch,
checkSkillNameCollisions,
@ -79,6 +80,18 @@ export interface SkillVerifyReport {
ok: boolean;
}
/**
* The reason string for a failed read.
*
* One helper rather than the same ternary at each of the four call sites: a
* rejection is not required to be an `Error` — a `throw "string"` anywhere in a
* transport or its dependencies reaches here — and reading `.message` off one
* would put `undefined` where the diagnosis belongs.
*/
function reasonOf(err: unknown): string {
return err instanceof Error ? err.message : String(err);
}
/** Result shape of one `resources/read`, narrowed to what a digest needs. */
interface ReadContents {
text?: string;
@ -154,11 +167,27 @@ export async function verifySkills(
// for the frontmatter cross-check. Reading it twice would double the load
// on the server and, worse, could compare a digest against one snapshot
// and frontmatter against another.
let entryText: string | undefined;
//
// Held as BYTES, not text. Taking `contents.text` skipped the whole
// frontmatter comparison whenever a server returned the markdown as a
// base64 `blob` — which is a legal `resources/read` shape, and which this
// module already decodes for the digest — so a mandatory check silently did
// not run while the report still said `ok` (Copilot). Deriving the text from
// the same verified bytes also guarantees the digest and the frontmatter
// describe one snapshot.
let entryBytes: Uint8Array | undefined;
const files: SkillFileReport[] = [];
const manifest =
entry.resources === DYNAMIC_RESOURCES ? [] : entry.resources;
const entryIdentity = skillUriIdentity(entry.uri);
// Compared by NORMALIZED identity, like every other URI comparison here —
// `checkSkillConformance` already accepts a manifest self-entry written in
// an equivalent form, so a raw string test would disagree with it and read
// the same file a second time.
const manifestListsSelf = manifest.some(
(resource) => skillUriIdentity(resource.uri) === entryIdentity,
);
for (const resource of manifest) {
let contents: ReadContents | undefined;
try {
@ -169,7 +198,7 @@ export async function verifySkills(
files.push({
uri: resource.uri,
status: "read-error",
reason: err instanceof Error ? err.message : String(err),
reason: reasonOf(err),
});
continue;
}
@ -182,9 +211,6 @@ export async function verifySkills(
});
continue;
}
if (resource.uri === entry.uri && typeof contents.text === "string") {
entryText = contents.text;
}
let bytes: Uint8Array;
try {
bytes = skillFileBytes(contents);
@ -192,10 +218,11 @@ export async function verifySkills(
files.push({
uri: resource.uri,
status: "read-error",
reason: err instanceof Error ? err.message : String(err),
reason: reasonOf(err),
});
continue;
}
if (skillUriIdentity(resource.uri) === entryIdentity) entryBytes = bytes;
const verification = await verifySkillResource(resource, bytes);
files.push({ uri: resource.uri, ...verification });
}
@ -203,21 +230,44 @@ export async function verifySkills(
// A `"dynamic"` skill has no manifest, so the loop above read nothing —
// but its SKILL.md is still served and still has to match the frontmatter
// the listing advertised. That obligation is not waived by the file set
// being unenumerable; only integrity is.
if (entryText === undefined) {
// being unenumerable; only integrity is. The same applies to a skill whose
// manifest omits its own file.
//
// Gated on `manifestListsSelf` rather than on `entryBytes`, so a self-entry
// the loop already tried and FAILED to read is not read a second time — its
// failure is recorded there.
if (!manifestListsSelf) {
// Recorded as a file result, not swallowed. Because a dynamic skill has
// no manifest rows, `files` would otherwise stay empty and its only static
// finding is a warning — so an unreadable SKILL.md returned `ok: true`
// for a skill whose mandatory frontmatter check never ran (Copilot).
const fail = (reason: string) =>
files.push({ uri: entry.uri, status: "read-error", reason });
try {
const invocation = await client.readResource(entry.uri, metadata);
const contents = contentsFor(invocation.result, entry.uri);
if (typeof contents?.text === "string") entryText = contents.text;
if (!contents) {
fail(
"resources/read returned no content block for this skill's own SKILL.md, so its frontmatter cannot be checked against the listing.",
);
} else {
try {
entryBytes = skillFileBytes(contents);
} catch (err) {
fail(reasonOf(err));
}
}
} catch (err) {
// Left undefined: the frontmatter check is skipped below. When the
// manifest listed this file the failure is already reported there, and
// when it did not, `manifest-missing-self` is the finding that matters.
// An expired authorization is not that case — see the note above.
// An expired authorization is the one error that is not this file's
// problem — see the note on the function.
if (err instanceof AuthRecoveryRequiredError) throw err;
fail(reasonOf(err));
}
}
const entryText =
entryBytes === undefined ? undefined : bytesToText(entryBytes);
const collision = collisions.get(skillUriIdentity(entry.uri));
const conformance = [
...checkSkillConformance(entry),