Commit graph inspector/docs/test-servers.md
Author SHA1 Message Date
cliffhall
4e676a0a54 fix: address Copilot review round 6 on #2251
Most of this round re-reported code the previous commit already changed
(it generated 0 new comments). Two findings were genuinely open.

- SkillsScreen: give "View SKILL.md" and "Fetch with skills/get" the same
  attempt ordering `verifyRow` got in round 3. Both were still
  last-completion-wins within one manifest: a double click whose older
  request settled last replaced the newer result. One shared `isStale`
  helper now covers all three actions — a result is dropped when its
  manifest was invalidated OR a later click already wrote.
- test-servers/skills.ts + docs: stop calling all three edge cases
  "non-conforming". `resources: "dynamic"` is a LEGAL wire form for
  generated content — it is unverifiable, not invalid, which is exactly
  why the checker reports it as a warning. The prose contradicted the
  code it was describing. Now: one legal-but-unverifiable case and two
  actual violations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 23:21:51 -04:00
cliffhall
754ce5bd95 fix: address Copilot review round 5 on #2251
Two of these reverse round 4, which asked for the opposite. Where the
readings conflict I took the permissive one: a wrong tightening rejects
conforming servers, a wrong loosening only fails to report something.

- skills.ts: stop requiring the `skill:` scheme. SEP-2640 says a server
  SHOULD use `skill://` and explicitly allows a domain-native scheme, so
  round 4's requirement handed a conforming `github://…` skill a false
  `malformed-uri` and skipped its name and root checks. Containment is
  scheme-independent — it compares against THIS entry's own root — so the
  traversal fix that requirement came packaged with is unaffected.
- skills.ts: the 512-entry and 16 MiB limits are warnings. Both are
  SHOULD NOTs for a server and MAYs for a host, so calling them errors
  contradicted the "error = a MUST was broken" contract round 4
  established and told authors a permitted skill was invalid.
- skills.ts: `skillEntriesMatch` compares canonically — keys sorted,
  manifest sorted by URI. `JSON.stringify` treated key order and manifest
  order as differences, though neither carries meaning.
- SkillsScreen: a differing `skills/get` is "a different snapshot"
  (yellow), not a disagreement (red). The SEP defines it as a fresh
  point-in-time read, so a skill that changed since the listing
  legitimately differs.
- useServerCommands: route `onGetSkill` through the command-scoped OAuth
  recovery like every other server command. An expired authorization
  rendered an error and stopped there — no reauthorization, no retry.
- test-servers/skills.ts: stamp the modern base result envelope
  (`resultType` / `ttlMs` / `cacheScope`) on every skills result. The SDK
  stamps it only for methods in its own codec, and `skills/*` are
  consumer-owned, so a 2026-era connection was receiving results without
  it. Unconditional rather than per era: the modern leg builds a server
  per request, so there is no era to branch on at registration time, and
  on legacy they are three members no codec inspects.
- composable-test-server: remove the `skills.directoryRead` option. It was
  publicly settable and advertised a method nothing serves, producing the
  exact false capability the fixture exists to help catch. `skills` is now
  a plain boolean; the flag returns in phase 3 (#2248) with its handler.

Still not changed: whether a modern `skills/list` RESULT SCHEMA must
require that envelope. Stamping it server-side is free; requiring it
client-side would reject servers on a reading nobody has checked against
the normative text. #2248 owns settling it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 23:11:46 -04:00
cliffhall
da8cc166ce fix: address Copilot review round 1 on #2251
- skills.ts: report a manifest that omits the skill's own SKILL.md
  (an empty list included), duplicate URIs, entries outside the skill
  root, and a missing size. `Conforms` was reachable for manifests that
  break invariants SEP-2640 states.
- skills.ts: cross-check the declared byte length before hashing. A size
  that disagrees fails verification on its own — the digest is taken over
  the bytes the server served, so agreeing with it says nothing about
  whether the manifest describes them.
- skills.ts: copy the view instead of slicing its backing store in
  `sha256Digest`. `SharedArrayBuffer.prototype.slice()` returns another
  SharedArrayBuffer, which `crypto.subtle.digest` rejects — the cast
  claimed to handle the exact input that would have thrown. No cast now.
- skills.ts: state the one obligation NOT checked here — that an entry's
  frontmatter matches the fetched SKILL.md's. The digest cannot cover it,
  and closing it needs a YAML parser, so it is tracked on #2248.
- managedSkillsState: cap the walk at LIST_MAX_PAGES. The repeated-cursor
  guard only catches a server stuck on one cursor; endlessly unique ones
  walked forever. Raises rather than truncating, like the salvage walk.
- managedSkillsState: gate every write on a session generation, so a walk
  that resolves after a disconnect or destroy cannot repopulate a cleared
  store or deliver the previous session's skills into the next.
- inspectorClient: send a cursor when it is `!== undefined`, not when it
  is truthy. An opaque cursor may be `""`, and dropping it re-requested
  page one — which the store then reported as a repeated-cursor failure.
- SkillsScreen: key verdict invalidation on the manifest (URI + digests +
  sizes), not the URI alone. A Refresh that changed the manifest left a
  green badge attached to a digest nothing had checked.
- SkillsScreen: epoch-guard every read continuation, so a fetch that
  resolves after the selection moved on cannot write into the new one.
- SkillsScreen: bound "Verify all" to 4 concurrent reads. A conforming
  manifest may hold 512 files.
- skillsSchemas: drop the guessed `resources/directory/read` result
  schema. Nothing calls it, so an unverified shape could sit wrong
  indefinitely; phase 3 adds it against the normative text.
- skills-http.json: declare the extension bare. It advertised
  `directoryRead: true` with no handler, so Connection Info reported
  "Supported" for a method that answers -32601.
- types.ts: restore the Tasks doc comment the Skills interface displaced.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 21:23:58 -04:00
cliffhall
63623b754b feat: support the Skills extension (SEP-2640) with digest verification
Adds detection, enumeration, and conformance checking for the Skills
extension (`io.modelcontextprotocol/skills`) — phases 1 and 2 of #2234,
which is everything its Acceptance list names.

Skills is a *server*-declared extension, read off the connecting server's
`capabilities.extensions`, so it is deliberately absent from
`ADVERTISABLE_EXTENSIONS` — that registry is what the Inspector advertises
and the user toggles, and an entry there would be a meaningless toggle.

`skills/list` and `skills/get` are consumer-owned extension methods that
neither era codec defines, so the SDK's era gate skips them and they go out
as ordinary `client.request` calls with explicit result schemas. The
raw-wire channel modern `tasks/*` needs is not used, and the Skills tab is
not era-gated: a legacy-era server that declares the extension is serving it.

The checks are the point, not the list view. `core/mcp/skills.ts` reports
each obligation SEP-2640 states — the name/path invariant, the digest
format, the 512-entry and 16 MiB limits, and `resources: "dynamic"`, which
means integrity cannot be verified at all. Digest verification hashes the
fetched bytes with WebCrypto and returns a mismatch with both digests
attached rather than throwing, because showing a mismatch loudly is the
whole value proposition.

Files are fetched on demand: SEP-2640 is explicit that a `resources/read`
of a `SKILL.md` is not a load and confers no standing, so none of the SEP's
host machinery is implemented.

The `skills-http` fixture serves four skills over two pages, three of them
deliberately non-conforming — without those the verification code is
untestable.

Phase 3 (CLI, TUI, `resources/directory/read`, paged mode) is #2248.

Closes #2234

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kfrH3uRNUM4JAk1reKtyY
Signed-off-by: cliffhall <cliff@futurescale.com>
2026-09-04 20:45:15 -04:00
cliffhall
a51e62dfa1 chore: address Copilot review round 1
- skill-manifest: the closing `---` fence must occupy a whole line. `---oops`
  was accepted as a terminator, silently truncating the frontmatter and passing
  validation — the exact failure the guard exists to catch. Regression cases
  added (prefix, trailing whitespace, EOF, CRLF).
- skill-eval: reject a nonzero `claude -p` exit instead of resolving it as an
  empty observation. An auth error or rate limit was scoring as "no skill
  invoked", which passes every negative case and reads as a trigger miss on
  every positive one.
- skill-eval: extract `collectSkillInvocations` / `sampleHit` behind a `main()`
  guard and cover them in `skill-eval.test.mjs` — the stream parsing and the
  process-outcome handling are unreachable from an eval run's happy path.
- docs/test-servers: annotate the era on every catalogue row. Six legacy rows
  carried no annotation under a "unless noted, Modern" default, so the doc sent
  readers to the wrong era for the feature they were trying to see.
- docs/quality-gate: `validate` runs four guards, not three; add the
  `verify:skills` row and the two new `test:scripts` suites; fix the `#publishing`
  fragment left dangling by the split.
- issue-create / pr-flow: delegate the card steps to `/board-ops` rather than
  restating its option IDs, which are explicitly unstable.
- issue-triage: the audit counted presence, not cardinality — an issue carrying
  both `v1` and `v2` passed a check that claims "exactly one". Same for the five
  type labels. Also scope the two milestone checks to board #28.
- AGENTS.md: codify the v1 milestone exception. Every milestone is a v2 release
  bucket, so a v1 issue has none to take and the Incoming-milestone invariant
  cannot apply to board #11.
- clients/launcher/README: its pointer promised an explanation of why the cli/tui
  manifests ship, which the extracted guide does not make.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 00:52:45 -04:00
cliffhall
73447c72d1 chore: extract AGENTS.md recipes into agent skills
Closes #2163

`AGENTS.md` was one 1,223-line contract loaded in full on every turn, whether
the task was a one-line CSS fix or a release cut. This splits it: rules stay
resident, procedures move to `.claude/skills/`.

- Ten skills under `.claude/skills/`, seven `disable-model-invocation: true`
  (invoked by name only, so they cost no listing context and have no trigger
  question) and three model-invoked with committed eval cases.
- `AGENTS.md` 1,223 → 380 lines: rules, a skills index, and a project tree
  reduced to one line per entry. The removed recipes are not duplicated.
- `README.md` 811 → 105 lines, split into `docs/{architecture,quality-gate,
  test-servers,publishing,docker}.md`, and dropped from `CLAUDE.md`'s auto-load.
- `.github/copilot-instructions.md` and its mirroring rule removed — Copilot
  reads `AGENTS.md`.
- `npm run verify:skills` (in `validate`, so in `local:gate` and CI) parses each
  SKILL.md the way Claude Code does, requires an explicit invocation mode and
  eval cases for model-invoked skills, and budgets the listing.
- `npm run skills:eval` runs the committed trigger cases headless.

Signed-off-by: cliffhall <cliff@futurescale.com>
2026-08-29 00:36:37 -04:00