chore: address Copilot review round 1
- skill-manifest: the closing `---` fence must occupy a whole line. `---oops` was accepted as a terminator, silently truncating the frontmatter and passing validation — the exact failure the guard exists to catch. Regression cases added (prefix, trailing whitespace, EOF, CRLF). - skill-eval: reject a nonzero `claude -p` exit instead of resolving it as an empty observation. An auth error or rate limit was scoring as "no skill invoked", which passes every negative case and reads as a trigger miss on every positive one. - skill-eval: extract `collectSkillInvocations` / `sampleHit` behind a `main()` guard and cover them in `skill-eval.test.mjs` — the stream parsing and the process-outcome handling are unreachable from an eval run's happy path. - docs/test-servers: annotate the era on every catalogue row. Six legacy rows carried no annotation under a "unless noted, Modern" default, so the doc sent readers to the wrong era for the feature they were trying to see. - docs/quality-gate: `validate` runs four guards, not three; add the `verify:skills` row and the two new `test:scripts` suites; fix the `#publishing` fragment left dangling by the split. - issue-create / pr-flow: delegate the card steps to `/board-ops` rather than restating its option IDs, which are explicitly unstable. - issue-triage: the audit counted presence, not cardinality — an issue carrying both `v1` and `v2` passed a check that claims "exactly one". Same for the five type labels. Also scope the two milestone checks to board #28. - AGENTS.md: codify the v1 milestone exception. Every milestone is a v2 release bucket, so a v1 issue has none to take and the Incoming-milestone invariant cannot apply to board #11. - clients/launcher/README: its pointer promised an explanation of why the cli/tui manifests ship, which the extracted guide does not make. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall
parent
73447c72d1
commit
a51e62dfa1
4 changed files with 248 additions and 83 deletions
|
|
@ -89,18 +89,16 @@ in **Todo** with its milestone already set — not in Incoming, which is the que
|
|||
for issues nobody has evaluated yet. Work you are starting immediately goes
|
||||
straight to **In Progress**.
|
||||
|
||||
```sh
|
||||
ITEM_ID=$(gh project item-add 28 --owner modelcontextprotocol --url <issue-url> --format json --jq '.id')
|
||||
# Status → Todo
|
||||
gh project item-edit --project-id PVT_kwDOCt2Azc4BJVxt --id "$ITEM_ID" \
|
||||
--field-id PVTSSF_lADOCt2Azc4BJVxtzg5iI8c --single-select-option-id fbdaf21e
|
||||
# Priority → Medium (score it with the rubric in /issue-triage; don't eyeball it)
|
||||
gh project item-edit --project-id PVT_kwDOCt2Azc4BJVxt --id "$ITEM_ID" \
|
||||
--field-id PVTSSF_lADOCt2Azc4BJVxtzg5iJE4 --single-select-option-id da944a9c
|
||||
```
|
||||
Run the **add-card recipe in `/board-ops`** with **Status → Todo** and the
|
||||
Priority you scored. The project, field and option IDs live there and only
|
||||
there: an option ID is regenerated whenever its field's option list is edited,
|
||||
so a second copy here would go stale silently and break issue creation even
|
||||
after `board-ops` was fixed.
|
||||
|
||||
Full ID tables, the v1 board's recipe, and the option-deletion hazard are in
|
||||
`/board-ops`. The priority rubric is in `/issue-triage`.
|
||||
For a **v1** issue, the same recipe against board #11 — no Priority, that board
|
||||
has no such field.
|
||||
|
||||
The priority rubric is in `/issue-triage`.
|
||||
|
||||
## Note on issues that arrive from elsewhere
|
||||
|
||||
|
|
|
|||
|
|
@ -200,11 +200,11 @@ count means the board contradicts a rule, not that the rule needs revisiting.
|
|||
| Double-boarded | An issue has a card on **one** board, the one matching its version label | Delete the wrong-board card |
|
||||
| Non-Issue items | **Only issues go on a board** — never PRs, never drafts | Delete the item |
|
||||
| No Status | Every card carries a Status | Set one — `Incoming` if unmilestoned, else by where it actually is |
|
||||
| `Incoming` **with** a milestone | Incoming ⇔ no milestone | Approval was never recorded: move to **Todo**, or clear the milestone |
|
||||
| Past Incoming **without** a milestone | Everything past Incoming ⇔ milestoned | Claims an approval nobody made: milestone it, or move back to Incoming |
|
||||
| `Incoming` **with** a milestone (#28) | Incoming ⇔ no milestone | Approval was never recorded: move to **Todo**, or clear the milestone |
|
||||
| Past Incoming **without** a milestone (#28) | Everything past Incoming ⇔ milestoned | Claims an approval nobody made: milestone it, or move back to Incoming |
|
||||
| Wrong board for label | `v1` → #11, `v2` → #28 | Move the card to the right board |
|
||||
| No version label | Every issue carries exactly one of `v1`/`v2` | Apply it |
|
||||
| No type label | Every issue carries one of the five types | Classify it |
|
||||
| Not exactly one version label | Every issue carries **exactly one** of `v1`/`v2` | Apply the missing one, or remove the extra |
|
||||
| Not exactly one type label | Every issue carries **exactly one** of the five types | Classify it, or remove the extra |
|
||||
| No Priority (#28) | Every board item is prioritized | Score it with the rubric |
|
||||
| Closed, not shipped, still carded | **Done means the work shipped** | Delete the card |
|
||||
|
||||
|
|
@ -231,17 +231,21 @@ jq -nr --slurpfile o "$D/i.json" --slurpfile a "$D/b28.json" --slurpfile b "$D/b
|
|||
"double-boarded": [$B28[].n | select(. as $n | [$B11[].n]|index($n))],
|
||||
"non-Issue on a board": [(own($a)[], own($b)[]) | select(.content.type!="Issue") | .content.number],
|
||||
"no Status": [($B28[], $B11[]) | select(.s==null) | .n],
|
||||
"Incoming w/ milestone": [($B28[], $B11[]) | select(.s=="Incoming" and ms(.n)!=null) | .n],
|
||||
"past Incoming, no ms": [($B28[], $B11[]) | select(.s!=null and .s!="Incoming" and .s!="Done"
|
||||
and isopen(.n) and ms(.n)==null) | .n],
|
||||
"Incoming w/ milestone": [$B28[] | select(.s=="Incoming" and ms(.n)!=null) | .n],
|
||||
"past Incoming, no ms": [$B28[] | select(.s!=null and .s!="Incoming" and .s!="Done"
|
||||
and isopen(.n) and ms(.n)==null) | .n],
|
||||
"v1 label on #28": [$B28[] | select(isopen(.n) and (lab(.n)|index("v1"))) | .n],
|
||||
"v2 label on #11": [$B11[] | select(isopen(.n) and (lab(.n)|index("v2"))) | .n],
|
||||
"open, no version label":[$o[0][] | select(.state=="OPEN")
|
||||
| select(([.labels[].name]|index("v1") or index("v2"))|not) | .number],
|
||||
"open, no type label": [$o[0][] | select(.state=="OPEN")
|
||||
| select(([.labels[].name] | index("bug") or index("enhancement")
|
||||
or index("documentation") or index("chore")
|
||||
or index("question"))|not) | .number],
|
||||
"open, not exactly 1 version label":
|
||||
[$o[0][] | select(.state=="OPEN")
|
||||
| select(([.labels[].name] | map(select(IN("v1","v2"))) | length) != 1)
|
||||
| .number],
|
||||
"open, not exactly 1 type label":
|
||||
[$o[0][] | select(.state=="OPEN")
|
||||
| select(([.labels[].name]
|
||||
| map(select(IN("bug","enhancement","documentation","chore","question")))
|
||||
| length) != 1)
|
||||
| .number],
|
||||
"#28 open, no Priority": [$B28[] | select(.p==null and isopen(.n)) | .n],
|
||||
"closed unshipped, still carded":
|
||||
[($B28[], $B11[]) | select(I(.n)!=null and (isopen(.n)|not)
|
||||
|
|
@ -256,6 +260,14 @@ Two things the queries must account for, both learned the hard way:
|
|||
`modelcontextprotocol/servers` issue. Without the `.content.repository` filter
|
||||
it reads as a statusless-card defect, and "fixing" it would mean editing
|
||||
another repo's tracking.
|
||||
- **The two milestone checks are #28-only.** Every milestone in this repo is a
|
||||
v2 release bucket, so a `v1` issue has none it could take — running the
|
||||
Incoming⇔milestone invariant over board #11 would flag every card on it for a
|
||||
state it cannot reach.
|
||||
- **Count the labels; don't test for presence.** The invariant is *exactly
|
||||
one*, so a predicate that only asks "is any version label present" passes an
|
||||
issue carrying **both** `v1` and `v2` — which belongs to two lines at once
|
||||
and shows up in both filtered queries. Same for the five type labels (Copilot).
|
||||
- **Only open issues have a `$I` entry.** A closed issue still has a card
|
||||
(correctly, in `Done`), so a check that treats "no milestone found" as a
|
||||
violation must gate on `isopen(.n)` or it flags every closed card.
|
||||
|
|
|
|||
|
|
@ -173,7 +173,9 @@ Poll for the review with a `startswith` match — the review login carries a
|
|||
## 9. Merge and close out
|
||||
|
||||
**On merge of a v2 PR, manually close its issue and move the board item to
|
||||
Done** (`259d6aab`), since auto-close won't fire on `v2/main`.
|
||||
Done**, since auto-close won't fire on `v2/main`. Use the move-a-card recipe in
|
||||
`/board-ops` — the option IDs are unstable, so this file names the column and
|
||||
nothing else.
|
||||
|
||||
`Done` is only for work that **shipped** — a merged PR, or a parent whose last
|
||||
sub-issue closed. Anything else (duplicate, won't fix, not planned, obsolete)
|
||||
|
|
|
|||
|
|
@ -43,15 +43,28 @@ the cache.
|
|||
|
||||
## Run one
|
||||
|
||||
Two processes: the test server, then the Inspector.
|
||||
|
||||
```sh
|
||||
# From the repo root, with a built launcher:
|
||||
node clients/launcher/build/index.js --web
|
||||
# then in the Inspector, add the server the config announces
|
||||
# 1. The server, from the repo root, with the config you picked:
|
||||
node test-servers/build/server-composable.js --config test-servers/configs/<name>.json
|
||||
```
|
||||
|
||||
The server prints its URL on **stderr**, and the bound port is **not
|
||||
necessarily** the config's — `findAvailablePort()` walks upward when the
|
||||
configured port is taken. Parse the announced URL rather than assuming.
|
||||
```sh
|
||||
# 2. The Inspector, in another terminal (needs a built launcher — `npm run build`):
|
||||
node clients/launcher/build/index.js --web
|
||||
```
|
||||
|
||||
Then add the server in the Inspector using the URL the first process announced.
|
||||
|
||||
Two mechanics that bite:
|
||||
|
||||
- **The server announces its URL on _stderr_**, not stdout (`console.error` in
|
||||
`server-composable.ts`). Watching stdout alone looks like a server that never
|
||||
started.
|
||||
- **The bound port is not necessarily the config's.** `createTestServerHttp`
|
||||
resolves through `findAvailablePort()`, which walks upward when the configured
|
||||
port is taken — so read the announced URL rather than assuming.
|
||||
|
||||
## Pick the right protocol era
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue