fix(runner): bound server scenarios with a per-scenario timeout (#454)

The client runner has had a 30s per-scenario timeout since it was written,
but `runServerConformanceTest` awaited `scenario.run(ctx)` with no bound. A
server under test that accepts the TCP connection and then never answers an
initialize POST, a tools call, or an SSE read stalled the entire suite run,
not just its own scenario.

#316 bounded the session-termination DELETEs in teardown, but that is a
call-site fix: covering the rest that way needs the same treatment on every
raw fetch in every scenario. Bounding at the runner makes those per-call
bounds defense-in-depth instead of the only protection.

`--timeout` now applies to server mode too, with the same 30s default as
client mode. On expiry the scenario reports a `scenario-timeout` FAILURE
naming the elapsed bound, so a wedged server reads as a failing scenario
rather than a hung process.

The timed-out scenario promise is left pending deliberately — a read blocked
on a socket has no cancellation channel — with its rejection swallowed so a
late failure can't surface against whichever scenario is running by then.

Fixes #427

Co-authored-by: Avi Seth <aviseth@users.noreply.github.com>
This commit is contained in:
Avi Seth 2026-09-06 21:14:58 +02:00 • committed by GitHub
commit 88ac3e595f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -27,12 +27,59 @@ function formatMarkdown(text: string): string {
);
}
/**
* Bound `scenario.run` so a server that accepts connections but never answers
* fails one scenario instead of stalling the whole suite.
*
* The losing promise is left pending on purpose: a scenario blocked on a socket
* read has no cancellation channel, so there is nothing to await. Its rejection
* is swallowed to keep a late failure from surfacing as an unhandled rejection
* against whichever scenario happens to be running by then.
*/
async function runScenarioBounded(
run: Promise<ConformanceCheck[]>,
scenarioName: string,
timeout: number
): Promise<ConformanceCheck[]> {
const timedOut = Symbol('timed-out');
let timeoutHandle: NodeJS.Timeout | undefined;
run.catch(() => {});
const result = await Promise.race([
run,
new Promise<typeof timedOut>((resolve) => {
timeoutHandle = setTimeout(() => resolve(timedOut), timeout);
})
]);
clearTimeout(timeoutHandle);
if (result !== timedOut) {
return result;
}
console.log(`\nScenario timed out after ${timeout}ms`);
return [
{
id: 'scenario-timeout',
name: 'Scenario completes within the timeout',
description:
'The scenario must finish within the configured timeout. A server that ' +
'accepts the connection but never responds leaves it running forever.',
status: 'FAILURE',
timestamp: new Date().toISOString(),
errorMessage: `Scenario '${scenarioName}' did not complete within ${timeout}ms. The server under test accepted the connection but did not finish the exchange.`
}
];
}
export async function runServerConformanceTest(
serverUrl: string,
scenarioName: string,
outputDir?: string,
specVersion?: SpecVersion,
force = false
force = false,
timeout: number = 30000
): Promise<{
checks: ConformanceCheck[];
resultDir?: string;
@ -98,7 +145,11 @@ export async function runServerConformanceTest(
connect: (opts) => connectFor(resolvedSpecVersion)(serverUrl, opts)
};
resetWireValidation();
const checks = await scenario.run(ctx);
const checks = await runScenarioBounded(
scenario.run(ctx),
scenarioName,
timeout
);
checks.push(...wireSchemaChecks(resolvedSpecVersion));
if (resultDir) {