modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

chore(scripts): derive dep-lockstep candidates from what enters each tsc program

Browse source 
`verify:dep-lockstep` derived its candidate set from the packages the shared
first-party sources named *directly*, so a package whose declarations reach a
program only through another package's `.d.ts` was invisible to it — it could
skew across installs and recreate the two-copy blow-up #1896 was about while
the guard stayed green. `@modelcontextprotocol/sdk` was the live example: never
written in first-party code, yet 16 of its `.d.ts` files land in clients/web's
test program.

Replace the derivation with the measured one: list every client tsconfig
project with `tsc --listFilesOnly`, map each resolved `node_modules` file to
its owning install and package, and keep the packages that reach ONE program
from TWO installs. That is exactly the set that can put two structurally
distinct copies of a type in front of one checker. The rest of the guard is
unchanged — top-level lockfile comparison, deny-by-default, `TOLERATED_SKEW` as
a name allowlist scoped to within-a-major.

The listing machinery moves to `scripts/lib/tsc-program.mjs`, shared with
`verify:typecheck-coverage`, so the two guards can't disagree about what a
program contains. It is deliberately not cached to disk between the two
processes: a stale fingerprint would make a guard measure a program that no
longer exists and pass on a real miss. The second pass costs ~14s.

`TOLERATED_SKEW` is now empty. `jose` and `@modelcontextprotocol/ext-apps` are
root-only since #1970 and cannot skew; `react` (whose types are `@types/react`,
resolved from one install) and `hono` never reach one program from two
installs. Each would be a rationale for a skew that cannot occur.

Closes #1965

Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-16 12:17:15 -04:00
commit 7f9be24bf3
1 changed files with 1069 additions and 933 deletions
Download patch file Download diff file Expand all files Collapse all files

250
scripts/verify-dep-lockstep.main.test.mjs
View file

@ -1,14 +1,21 @@
// End-to-end tests for the dep-lockstep guard's executable path (Copilot,
// #1962). The sibling tests cover the pure helpers; nothing exercised `main()`,
// so a regression in source enumeration, install discovery, lockfile loading,
// so a regression in candidate derivation, install discovery, lockfile loading,
// the sibling-guard vouch, or the nonzero exit on real skew would have left the
// whole suite green.
//
// Each case builds a throwaway repo — the guard derives its root from its own
// file location, so the script is copied into the fixture rather than pointed
// at one — `git add`s it (the enumeration is `git ls-files`, which reads the
// index), and runs the guard as a subprocess to assert the exit status and
// message. Run via `npm run test:scripts`.
// at one — `git add`s it (install discovery is by manifest, but the sibling
// `verify:format-coverage` vouch and git-based tooling expect a repo), and runs
// the guard as a subprocess to assert the exit status and message.
//
// Since #1965 the fixture must contain a REAL tsc program: the candidate set is
// derived from what `tsc --listFilesOnly` resolves, so each fixture ships two
// installs holding the same tiny stub packages, one client tsconfig whose
// program spans both (its own `src` resolves from the client install, the shared
// `core/` it includes resolves from the root — the actual v2 shape), and a
// symlinked real `typescript` to run it. Run via `npm run test:scripts`.
import { test } from "node:test";
import assert from "node:assert/strict";
@ -30,7 +37,7 @@ import { fileURLToPath } from "node:url";
const scriptsDir = path.dirname(fileURLToPath(import.meta.url));
const repoRoot = path.join(scriptsDir, "..");
/** The real typescript install, symlinked into each fixture so the guard resolves it. */
/** The real typescript install, symlinked into each fixture so `npx tsc` runs. */
const typescriptDir = path.dirname(
createRequire(path.join(repoRoot, "clients", "web", "package.json")).resolve(
"typescript/package.json",
@ -52,11 +59,24 @@ const lock = (deps) => ({
});
/**
* Build a fixture repo: shared sources importing `zod` and `express`, a root
* install and one client install, and the guard itself. `rootDeps`/`webDeps`
* decide whether the two installs agree.
* Build a fixture repo whose one client program spans two installs.
*
* `outer` is named by first-party code in both trees; `inner` is named ONLY by
* `outer`'s own `.d.ts`, so it reaches the program transitively — the
* `@modelcontextprotocol/sdk` shape #1965 is about. Both stubs are installed
* under the root AND under `clients/web`, so each program holds two copies of
* each and both are candidates. `solo` is installed at the root only and
* imported only by the shared tree, so it can never be a candidate however its
* lockfile entries read.
*/
function makeFixture({ rootDeps, webDeps, scripts, rawWebLock }) {
function makeFixture({
rootDeps,
webDeps,
scripts,
webScripts,
rawWebLock,
webTsconfig,
} = {}) {
// realpath matters: on macOS `tmpdir()` is `/var/...`, a symlink to
// `/private/var/...`. The guard only runs `main()` when `import.meta.url`
// (always the resolved path) matches `process.argv[1]`, so launching it via
@ -81,34 +101,91 @@ function makeFixture({ rootDeps, webDeps, scripts, rawWebLock }) {
"verify:dep-lockstep": "node scripts/verify-dep-lockstep.mjs",
},
});
write("core/client.ts", 'import { z } from "zod";\nexport const s = z;\n');
// The shared tree, compiled into the client's program but resolving its own
// imports from the ROOT install — the reason a package can appear twice.
write(
"test-servers/src/server.ts",
'import express from "express";\nexport const app = express;\n',
"core/client.ts",
'import type { Outer } from "outer";\nimport type { Solo } from "solo";\nexport type FromCore = Outer | Solo;\n',
);
// The client's own source, resolving from the CLIENT install.
write(
"vitest.shared.mts",
'import path from "node:path";\nexport default path;\n',
"clients/web/src/main.ts",
'import type { Outer } from "outer";\nexport type FromWeb = Outer;\n',
);
write("package-lock.json", lock(rootDeps));
write("clients/web/package.json", { name: "web" });
write("clients/web/package-lock.json", rawWebLock ?? lock(webDeps));
write("clients/web/package.json", {
name: "web",
scripts: webScripts ?? { typecheck: "tsc --noEmit -p tsconfig.json" },
});
write(
"clients/web/tsconfig.json",
webTsconfig ?? {
compilerOptions: {
noEmit: true,
module: "esnext",
target: "esnext",
moduleResolution: "bundler",
types: [],
},
include: ["src/**/*.ts", "../../core/**/*.ts"],
},
);
// Stub installs. `outer` pulls `inner` in through its own declarations. The
// installed version must match the lockfile's: TypeScript keys its
// package-identity dedup on name@version, so a stub that lied about its
// version would make the program under test differ from the one the lockfile
// describes.
const root = rootDeps ?? ALIGNED;
const web = webDeps ?? ALIGNED;
const stub = (installRoot, versions, name, body) => {
if (!versions[name]) return;
write(`${installRoot}node_modules/${name}/package.json`, {
name,
version: versions[name],
types: "index.d.ts",
});
write(`${installRoot}node_modules/${name}/index.d.ts`, body);
};
for (const [installRoot, versions] of [
["", root],
["clients/web/", web],
]) {
stub(
installRoot,
versions,
"outer",
'import type { Inner } from "inner";\nexport type Outer = Inner;\n',
);
stub(installRoot, versions, "inner", "export type Inner = number;\n");
}
stub("", root, "solo", "export type Solo = string;\n");
write("package-lock.json", lock(root));
write("clients/web/package-lock.json", rawWebLock ?? lock(web));
// The guard resolves its repo root from its own location, so it has to live
// inside the fixture; `lib/npm-scripts.mjs` comes along as its import.
// inside the fixture; its `lib/` imports come along.
mkdirSync(path.join(dir, "scripts", "lib"), { recursive: true });
for (const rel of [
"verify-dep-lockstep.mjs",
path.join("lib", "npm-scripts.mjs"),
path.join("lib", "tsc-program.mjs"),
])
cpSync(path.join(scriptsDir, rel), path.join(dir, "scripts", rel));
mkdirSync(path.join(dir, "node_modules"), { recursive: true });
// A real typescript, plus the `.bin` entry `npx --no-install tsc` resolves by
// walking up from the client dir.
mkdirSync(path.join(dir, "node_modules", ".bin"), { recursive: true });
symlinkSync(
typescriptDir,
path.join(dir, "node_modules", "typescript"),
"dir",
);
symlinkSync(
path.join(typescriptDir, "bin", "tsc"),
path.join(dir, "node_modules", ".bin", "tsc"),
);
execFileSync("git", ["init", "-q"], { cwd: dir });
execFileSync("git", ["add", "-A"], { cwd: dir });
@ -134,69 +211,110 @@ function withFixture(options, fn) {
}
}
const ALIGNED = { zod: "4.4.3", express: "5.2.1" };
const ALIGNED = { outer: "1.2.3", inner: "4.5.6", solo: "7.8.9" };
test("main: exits 0 when every install agrees", () => {
withFixture({ rootDeps: ALIGNED, webDeps: ALIGNED }, (dir) => {
withFixture({}, (dir) => {
const { status, out } = runGuard(dir);
assert.equal(status, 0, out);
assert.match(out, /verify:dep-lockstep — OK/);
// Both shared trees and the named shared file contributed their imports.
assert.match(out, /2 install-crossing dependencies/);
// `outer` alone: both installs hold it and both files land in the program.
// `solo` is installed at the root only, so it is not install-crossing — and
// `inner`, at the same version in both installs, is collapsed by
// TypeScript's package-identity redirect (see the note on `crossInstallPackages`
// in lib/tsc-program.mjs): only one copy is ever loaded, so there is nothing
// for the checker to relate. The next test is the version that matters.
assert.match(out, /1 install-crossing dependencies/);
assert.match(out, /1 tsc programs/);
});
});
test("main: exits 1 and names the skewed package and every holder", () => {
withFixture(
{ rootDeps: { ...ALIGNED, zod: "4.3.6" }, webDeps: ALIGNED },
{ rootDeps: { ...ALIGNED, outer: "1.0.0" }, webDeps: ALIGNED },
(dir) => {
const { status, out } = runGuard(dir);
assert.equal(status, 1, out);
assert.match(out, /\bzod\b/);
assert.match(out, /4\.3\.6\s+\(\.\)/);
assert.match(out, /4\.4\.3\s+\(clients\/web\)/);
// `express` agrees, so it must not be reported.
assert.doesNotMatch(out, /^\s+express$/m);
assert.match(out, /\bouter\b/);
assert.match(out, /1\.0\.0\s+\(\.\)/);
assert.match(out, /1\.2\.3\s+\(clients\/web\)/);
// The program that saw both copies is named, so the claim is checkable.
assert.match(out, /seen in clients\/web\/tsconfig\.json/);
// `inner` agrees, so it must not be reported.
assert.doesNotMatch(out, /^\s+inner$/m);
},
);
});
test("main: a package imported only by test-servers/src is still checked", () => {
// Guards the enumeration of the *second* shared tree: if only `core/` were
// scanned, this skew would pass.
test("main: a package reached only through another package's .d.ts is caught (#1965)", () => {
// `inner` is never written in first-party code — it enters the program solely
// through `outer`'s declarations. The derivation this replaced read the
// shared sources' own imports and could not see it, which is how
// `@modelcontextprotocol/sdk` sat skewed across two installs while the guard
// stayed green. Its parent skews too, which is what puts both copies of the
// parent in the program to resolve from — the real #1965 shape, where
// `ext-apps` was split 1.7.4/1.7.5 alongside the SDK.
withFixture(
{ rootDeps: { ...ALIGNED, express: "5.0.0" }, webDeps: ALIGNED },
{
rootDeps: { ...ALIGNED, outer: "1.0.0", inner: "4.0.0" },
webDeps: ALIGNED,
},
(dir) => {
const { status, out } = runGuard(dir);
assert.equal(status, 1, out);
assert.match(out, /\bexpress\b/);
assert.match(out, /^\s+inner$/m);
assert.match(out, /4\.0\.0\s+\(\.\)/);
assert.match(out, /4\.5\.6\s+\(clients\/web\)/);
},
);
});
test("main: exits 1 when a configured shared source matches no file", () => {
withFixture({ rootDeps: ALIGNED, webDeps: ALIGNED }, (dir) => {
// Drop `core/` from the index — the other sources keep the file count
// nonzero, which is exactly what the old aggregate check missed.
execFileSync("git", ["rm", "-r", "-q", "--cached", "core"], { cwd: dir });
test("main: a package that reaches the program from ONE install is not a candidate", () => {
// `solo` is installed at the root only, so no program ever holds two copies of
// it — its lockfile entries can disagree without TypeScript ever relating two
// declarations. Failing here would be the old derivation's false positive
// (which is what made an allowlist of inert names necessary).
withFixture(
{ rootDeps: { ...ALIGNED, solo: "7.0.0" }, webDeps: ALIGNED },
(dir) => {
const { status, out } = runGuard(dir);
assert.equal(status, 0, out);
assert.doesNotMatch(out, /\bsolo\b/);
},
);
});
test("main: exits 1 when a client names no tsconfig project", () => {
// No `typecheck` script and no `tsconfig.json` references: none of that
// client's programs is measured, so a skew reaching only them would pass.
withFixture({ webScripts: { build: "vite build" } }, (dir) => {
rmSync(path.join(dir, "clients", "web", "tsconfig.json"));
const { status, out } = runGuard(dir);
assert.equal(status, 1, out);
assert.match(out, /matched no tracked file/);
assert.match(out, /^\s+core$/m);
assert.match(out, /names no tsconfig project/);
});
});
test("main: exits 1 when a program cannot be listed, rather than measuring nothing", () => {
// A broken tsconfig makes `tsc --listFilesOnly` resolve nothing. Reading that
// as "no candidates here" is the gate failing open.
withFixture({ webTsconfig: "{ this is not json" }, (dir) => {
const { status, out } = runGuard(dir);
assert.equal(status, 1, out);
assert.match(out, /exited non-zero/);
});
});
test("main: exits 1 on a lockfile it cannot read, rather than failing open (Copilot, #1962)", () => {
// A v1 lockfile has `dependencies` and no `packages` table. Treating it as an
// install that simply holds nothing would leave the *root's* zod unopposed
// install that simply holds nothing would leave the *root's* copy unopposed
// and the skew below reported as aligned — the gate failing open.
withFixture(
{
rootDeps: { ...ALIGNED, zod: "4.3.6" },
webDeps: ALIGNED,
rootDeps: { ...ALIGNED, outer: "1.0.0" },
rawWebLock: {
lockfileVersion: 1,
dependencies: { zod: { version: "4.4.3" } },
dependencies: { outer: { version: "1.2.3" } },
},
},
(dir) => {
@ -213,11 +331,9 @@ test("main: exits 1 on a lockfile with a `packages` table but no root entry", ()
// `""` root npm always writes is not a lockfile this guard can trust.
withFixture(
{
rootDeps: ALIGNED,
webDeps: ALIGNED,
rawWebLock: {
lockfileVersion: 3,
packages: { "node_modules/zod": { version: "4.4.3" } },
packages: { "node_modules/outer": { version: "1.2.3" } },
},
},
(dir) => {
@ -232,22 +348,19 @@ test("main: exits 1 on an install whose lockfile is missing (Copilot, #1962)", (
// Enrolment is by `package.json`, so a missing lockfile is loud rather than a
// silently absent row. The fixture IS skewed, so dropping the install instead
// would leave the remaining holder unopposed and report aligned.
withFixture(
{ rootDeps: { ...ALIGNED, zod: "4.3.6" }, webDeps: ALIGNED },
(dir) => {
rmSync(path.join(dir, "clients", "web", "package-lock.json"));
const { status, out } = runGuard(dir);
assert.equal(status, 1, out);
assert.match(out, /no lockfile/);
assert.match(out, /clients\/web\/package-lock\.json/);
},
);
withFixture({ rootDeps: { ...ALIGNED, outer: "1.0.0" } }, (dir) => {
rmSync(path.join(dir, "clients", "web", "package-lock.json"));
const { status, out } = runGuard(dir);
assert.equal(status, 1, out);
assert.match(out, /no lockfile/);
assert.match(out, /clients\/web\/package-lock\.json/);
});
});
test("main: exits 1 when the ROOT lockfile is missing", () => {
// The worst variant: the root is the install every shared source resolves
// from, so omitting it could let the guard pass on client locks alone.
withFixture({ rootDeps: ALIGNED, webDeps: ALIGNED }, (dir) => {
withFixture({}, (dir) => {
rmSync(path.join(dir, "package-lock.json"));
const { status, out } = runGuard(dir);
assert.equal(status, 1, out);
@ -257,8 +370,9 @@ test("main: exits 1 when the ROOT lockfile is missing", () => {
});
test("main: a clients/ dir with no package.json is not an install", () => {
// Enrolling a stray directory would demand a lockfile it should never have.
withFixture({ rootDeps: ALIGNED, webDeps: ALIGNED }, (dir) => {
// Enrolling a stray directory would demand a lockfile it should never have,
// and a tsconfig project it has no way to name.
withFixture({}, (dir) => {
mkdirSync(path.join(dir, "clients", "scratch"), { recursive: true });
writeFileSync(
path.join(dir, "clients", "scratch", "notes.md"),
@ -269,11 +383,21 @@ test("main: a clients/ dir with no package.json is not an install", () => {
});
});
test("main: exits 1 when there is no client program to measure at all", () => {
// A moved `clients/` dir leaves the derivation with nothing to look at, and an
// empty candidate set from a broken enumeration reads exactly like a clean
// bill of health.
withFixture({}, (dir) => {
rmSync(path.join(dir, "clients"), { recursive: true, force: true });
const { status, out } = runGuard(dir);
assert.equal(status, 1, out);
assert.match(out, /found no client program to measure/);
});
});
test("main: exits 1 when the root validate no longer runs the sibling guard", () => {
withFixture(
{
rootDeps: ALIGNED,
webDeps: ALIGNED,
// `verify:format-coverage` dropped from the chain: the vouch must fail
// even though the dependency versions themselves are fine.
scripts: {