da56f6631cA baseline entry names a scenario, but a scenario is many checks:
server-stateless is 29. Excusing the one or two a spec change breaks
means baselining the whole scenario, which stops enforcing the rest.
An entry may now be '<scenario>:<check-id>', in which case every failing
check in that scenario is judged on its own. Bare scenario entries are
unchanged, and YAML parses '- a:b' as a plain string, so every existing
baseline keeps working untouched.
loadExpectedFailures now owns the entry grammar and returns parsed
BaselineEntry values, so evaluateBaseline only matches and cannot throw.
Rejecting a bad entry and interpreting a good one no longer live in
separate layers.
A check id matches all of its occurrences, since ids repeat within a run;
they collapse to one verdict, most-severe first. Absent and SKIPPED are
tolerated, matching how the runner already treats them as green.
Move collapseDuplicateChecks to src/checks/collapse.ts with its tests:
expected-failures.ts needs the reducer and cannot import dpop.ts's auth
server stack. No behavior change.
Reject two configs that previously coerced to an entry matching nothing:
a mapping ('- scenario: check-id', with a space) and an empty list item.
35 lines
1.5 KiB
TypeScript
35 lines
1.5 KiB
TypeScript
import type { CheckStatus, ConformanceCheck } from '../types';
|
|
|
|
/**
|
|
* Collapse duplicate non-INFO check IDs to a single entry, preferring the
|
|
* MOST-SEVERE occurrence (FAILURE > WARNING > SUCCESS > any other status, e.g.
|
|
* SKIPPED) so a real failure is never masked. Equal-severity ties keep the LAST
|
|
* occurrence. Per-request INFO log entries are always kept.
|
|
*
|
|
* A check ID is not unique within a run: scenarios re-emit a shared ID when a
|
|
* flow repeats. The RFC 9449 §8/§9 nonce round-trip re-POSTs /token (challenge
|
|
* → retry), so the shared token-flow checks (`token-request`, `pkce-*`) are
|
|
* appended twice; `sep-2575-http-server-meta-invalid-400` is emitted once per
|
|
* iteration of the `_meta` test-case loop. Collapsing reports each ID once
|
|
* without hiding a failure recorded on any occurrence, which is what lets an
|
|
* expected-failures baseline address a check by ID.
|
|
*/
|
|
export function collapseDuplicateChecks(
|
|
checks: ConformanceCheck[]
|
|
): ConformanceCheck[] {
|
|
const severity = (s: CheckStatus): number =>
|
|
s === 'FAILURE' ? 3 : s === 'WARNING' ? 2 : s === 'SUCCESS' ? 1 : 0;
|
|
// Winning index per non-INFO id: highest severity, ties → last occurrence.
|
|
const winner = new Map<string, number>();
|
|
checks.forEach((c, i) => {
|
|
if (c.status === 'INFO') return;
|
|
const cur = winner.get(c.id);
|
|
if (
|
|
cur === undefined ||
|
|
severity(c.status) >= severity(checks[cur].status)
|
|
) {
|
|
winner.set(c.id, i);
|
|
}
|
|
});
|
|
return checks.filter((c, i) => c.status === 'INFO' || winner.get(c.id) === i);
|
|
}
|