conformance/src/checks
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Felix Weinberger da56f6631c
Allow expected-failures entries to name a single check (#406)
A baseline entry names a scenario, but a scenario is many checks:
server-stateless is 29. Excusing the one or two a spec change breaks
means baselining the whole scenario, which stops enforcing the rest.

An entry may now be '<scenario>:<check-id>', in which case every failing
check in that scenario is judged on its own. Bare scenario entries are
unchanged, and YAML parses '- a:b' as a plain string, so every existing
baseline keeps working untouched.

loadExpectedFailures now owns the entry grammar and returns parsed
BaselineEntry values, so evaluateBaseline only matches and cannot throw.
Rejecting a bad entry and interpreting a good one no longer live in
separate layers.

A check id matches all of its occurrences, since ids repeat within a run;
they collapse to one verdict, most-severe first. Absent and SKIPPED are
tolerated, matching how the runner already treats them as green.

Move collapseDuplicateChecks to src/checks/collapse.ts with its tests:
expected-failures.ts needs the reducer and cannot import dpop.ts's auth
server stack. No behavior change.

Reject two configs that previously coerced to an entry matching nothing:
a mapping ('- scenario: check-id', with a space) and an empty list item.
2026-07-20 17:20:32 +02:00
..
checks.test.ts feat: support "draft" as a first-class spec version target (#255) 2026-04-30 11:37:32 +01:00
client.ts feat: support "draft" as a first-class spec version target (#255) 2026-04-30 11:37:32 +01:00
collapse.test.ts Allow expected-failures entries to name a single check (#406) 2026-07-20 17:20:32 +02:00
collapse.ts Allow expected-failures entries to name a single check (#406) 2026-07-20 17:20:32 +02:00
index.ts [auth] Implement CIMD conformance test (#37) 2025-11-20 21:00:13 +00:00
server.ts [compliance tests] initial server scenarios (#4) 2025-11-04 15:08:20 +00:00