fix(runner): bound server scenarios with a per-scenario timeout (#454)
The client runner has had a 30s per-scenario timeout since it was written, but `runServerConformanceTest` awaited `scenario.run(ctx)` with no bound. A server under test that accepts the TCP connection and then never answers an initialize POST, a tools call, or an SSE read stalled the entire suite run, not just its own scenario. #316 bounded the session-termination DELETEs in teardown, but that is a call-site fix: covering the rest that way needs the same treatment on every raw fetch in every scenario. Bounding at the runner makes those per-call bounds defense-in-depth instead of the only protection. `--timeout` now applies to server mode too, with the same 30s default as client mode. On expiry the scenario reports a `scenario-timeout` FAILURE naming the elapsed bound, so a wedged server reads as a failing scenario rather than a hung process. The timed-out scenario promise is left pending deliberately — a read blocked on a socket has no cancellation channel — with its rejection swallowed so a late failure can't surface against whichever scenario is running by then. Fixes #427 Co-authored-by: Avi Seth <aviseth@users.noreply.github.com>
This commit is contained in:
parent
31c5e7aa55
commit
88ac3e595f
1 changed files with 105 additions and 4 deletions
|
|
@ -27,12 +27,59 @@ function formatMarkdown(text: string): string {
|
|||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Bound `scenario.run` so a server that accepts connections but never answers
|
||||
* fails one scenario instead of stalling the whole suite.
|
||||
*
|
||||
* The losing promise is left pending on purpose: a scenario blocked on a socket
|
||||
* read has no cancellation channel, so there is nothing to await. Its rejection
|
||||
* is swallowed to keep a late failure from surfacing as an unhandled rejection
|
||||
* against whichever scenario happens to be running by then.
|
||||
*/
|
||||
async function runScenarioBounded(
|
||||
run: Promise<ConformanceCheck[]>,
|
||||
scenarioName: string,
|
||||
timeout: number
|
||||
): Promise<ConformanceCheck[]> {
|
||||
const timedOut = Symbol('timed-out');
|
||||
let timeoutHandle: NodeJS.Timeout | undefined;
|
||||
|
||||
run.catch(() => {});
|
||||
|
||||
const result = await Promise.race([
|
||||
run,
|
||||
new Promise<typeof timedOut>((resolve) => {
|
||||
timeoutHandle = setTimeout(() => resolve(timedOut), timeout);
|
||||
})
|
||||
]);
|
||||
clearTimeout(timeoutHandle);
|
||||
|
||||
if (result !== timedOut) {
|
||||
return result;
|
||||
}
|
||||
|
||||
console.log(`\nScenario timed out after ${timeout}ms`);
|
||||
return [
|
||||
{
|
||||
id: 'scenario-timeout',
|
||||
name: 'Scenario completes within the timeout',
|
||||
description:
|
||||
'The scenario must finish within the configured timeout. A server that ' +
|
||||
'accepts the connection but never responds leaves it running forever.',
|
||||
status: 'FAILURE',
|
||||
timestamp: new Date().toISOString(),
|
||||
errorMessage: `Scenario '${scenarioName}' did not complete within ${timeout}ms. The server under test accepted the connection but did not finish the exchange.`
|
||||
}
|
||||
];
|
||||
}
|
||||
|
||||
export async function runServerConformanceTest(
|
||||
serverUrl: string,
|
||||
scenarioName: string,
|
||||
outputDir?: string,
|
||||
specVersion?: SpecVersion,
|
||||
force = false
|
||||
force = false,
|
||||
timeout: number = 30000
|
||||
): Promise<{
|
||||
checks: ConformanceCheck[];
|
||||
resultDir?: string;
|
||||
|
|
@ -98,7 +145,11 @@ export async function runServerConformanceTest(
|
|||
connect: (opts) => connectFor(resolvedSpecVersion)(serverUrl, opts)
|
||||
};
|
||||
resetWireValidation();
|
||||
const checks = await scenario.run(ctx);
|
||||
const checks = await runScenarioBounded(
|
||||
scenario.run(ctx),
|
||||
scenarioName,
|
||||
timeout
|
||||
);
|
||||
checks.push(...wireSchemaChecks(resolvedSpecVersion));
|
||||
|
||||
if (resultDir) {
|
||||
|
|
|
|||
Loading…
Reference in a new issue