modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

feat(auth): give the secrets file a real cross-process lock (#2082)

Browse source 
#1950 shipped FileSecretStore without cross-process mutual exclusion, by
decision: an earlier revision hand-rolled a `mkdir` election with an owner
stamp, a heartbeat and a stale-takeover, and three consecutive review rounds
found a real race in it. The last one is not closable with what Node exposes
— claiming a stale lock atomically needs compare-and-swap on a directory
entry (`renameat2(RENAME_EXCHANGE)`) — so it was replaced with optimistic
verify-and-retry and the residual documented.

#2082 settles that as "borrow, don't hand-roll". `proper-lockfile` is what
npm itself locks with, and stale-takeover is precisely the problem it has
already solved: it re-stats the lock directory after claiming it and gives
the lock up when the mtime is not the one it wrote, so the loser of a
takeover race releases instead of proceeding.

- core/auth/node/file-lock.ts: `withSecretFileLock`. `realpath: false` so a
  file can be locked into existence (the very first `set` has no
  `secrets.json`, and the library's default resolves through `fs.realpath`);
  a warning in place of the library's `onCompromised`, which throws from a
  timer and would take the session down; and a degrade-with-one-warning path
  rather than a throw when no lock can be taken — this store exists for boxes
  missing the usual mechanism (#1848, #1905) and must not gain a new way to
  be unavailable.
- FileSecretStore.mutate holds it across the whole read-modify-write. The
  optimistic verify stays underneath and is not redundant: a lock is advisory
  between the processes that take it, so the verify covers a writer outside
  this codebase and covers the degrade path. The in-process queue stays too,
  and gains a second job — proper-lockfile is not reentrant, so serializing
  per path keeps ELOCKED meaning "another process".
- absorbFileSecretsIntoKeyring takes the same lock around orphan adoption and
  the atomic claim, behind a lock-free `readdir` fast path so the common
  startup (keychain available, no file ever written) neither creates a lock
  directory nor warns about one it could not create.

proper-lockfile is a root `dependency` per the placement rule, and is named
in all three bundler `external` lists: tsup externalizes what the *client's*
manifest declares, so a root-only CJS package was being inlined into the ESM
bundles, leaving esbuild's `Dynamic require of "path" is not supported` shim
that killed `--cli` at import time. That rule was undocumented; it is now in
AGENTS.md beside the placement rule that creates it, and mirrored into
.github/copilot-instructions.md.

Tests: 7 new in file-lock.test.ts driving a real second process (the existing
suite structurally cannot — `serialize` orders in-process callers before the
lock sees them), plus two for the migration fast path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-23 09:32:41 -04:00
commit 04dbc36544
3 changed files with 718 additions and 90 deletions
Download patch file Download diff file Expand all files Collapse all files

183
core/auth/node/file-lock.ts Normal file
View file

@ -0,0 +1,183 @@
/**
* Cross-process mutual exclusion for the secrets file (#2082).
*
* **Why a library and not a hand-rolled election.** #1950 shipped without a
* lock on purpose. An earlier revision of it *did* take one — a `mkdir`
* election with an owner stamp, a heartbeat and a stale-takeover — and three
* consecutive review rounds found a real race in it. The last one is not
* closable with what Node exposes: claiming a stale lock atomically needs
* compare-and-swap on a directory entry (`renameat2(RENAME_EXCHANGE)`), and
* without it a waiter that loses the race can move the winner's *fresh* lock
* aside and enter alongside it.
*
* So the choice #2082 settles is not "lock versus no lock" but "hand-rolled
* versus borrowed". `proper-lockfile` is the borrowed one: it is what npm
* itself uses, and stale-takeover is precisely the problem it has already
* solved — it re-`stat`s the lock directory after claiming it and gives the
* lock up if the mtime is not the one it wrote, so a loser of the takeover
* race releases rather than proceeding.
*
* **What this does not remove.** The optimistic verify in
* {@link FileSecretStore.mutate} stays. A lock is an advisory convention
* between participants that take it, so it orders Inspector against
* Inspector and says nothing about an editor, a backup restore, or an
* Inspector old enough to predate this file. The verify is also what covers
* {@link withSecretFileLock} *declining* — see below.
*
* **Degrading rather than failing is deliberate.** The store's whole reason
* for existing is a box where the usual mechanism is unavailable (no
* keychain, #1848/#1905), so it must not acquire a *new* way to be
* unavailable. A directory that cannot hold a lock file — a read-only
* `$HOME`, a filesystem without `mkdir` semantics, a container mount owned
* by another uid — would otherwise turn every `set` into a hard failure on
* exactly the deployments this store was written for. So a lock that cannot
* be taken runs the body anyway, with the #1950 optimistic behaviour
* underneath it, and says so once.
*/
import * as path from "node:path";
// CJS-only package. A default import is the shape that survives every
// bundler this repo runs core/ through (tsup for cli/tui, vite's SSR/node
// graph for the web runner); named imports off a CJS module depend on
// lexer detection that esbuild and rollup disagree about.
import properLockfile from "proper-lockfile";
/**
* How long a lock may go untouched before another process may claim it.
*
* `proper-lockfile` refreshes the lock's mtime at `stale / 2` for as long as
* the holder is alive, so this is not "how long a mutation may take" — it is
* how long after a holder *dies* the file stays unwritable. 10s is the
* library's own default and the value npm ships with; a mutation is a read,
* an scrypt derivation and an atomic write, so the margin is enormous.
*/
const STALE_MS = 10_000;
/**
* How long a waiter will keep trying before giving up.
*
* The retry schedule below tops out around 3s of waiting. That is long
* enough for any real mutation to finish (see above) and short enough that a
* pathological case surfaces as a slow save rather than a hung one.
*/
const RETRY = {
retries: 8,
factor: 2,
minTimeout: 20,
maxTimeout: 1_000,
} as const;
/** Emitted once per process, not once per call — see {@link warnOnce}. */
const warned = new Set<string>();
/**
* Say why locking is unavailable here, once per reason per process.
*
* Once per *reason* rather than once overall: "the directory is read-only"
* and "the lock is held by something that never releases it" are different
* problems with different fixes, and collapsing them would print whichever
* happened first and hide the other for the life of the process. Keyed on
* the message, which already encodes the reason.
*/
function warnOnce(message: string): void {
if (warned.has(message)) return;
warned.add(message);
console.warn(`[mcp-inspector] ${message}`);
}
/** Test seam: forget which warnings have been emitted. */
export function resetFileLockWarnings(): void {
warned.clear();
}
/**
* Did `proper-lockfile` decline because someone else holds the lock?
*
* A bare cast rather than a guarded narrowing: the only caller is the `catch`
* around `properLockfile.lock`, and the library rejects with a real `Error`
* carrying a `code` on every path. Guarding would add branches nothing can
* exercise, which is a worse trade than an assertion whose one caller is two
* lines away.
*/
const isHeldElsewhere = (err: unknown): boolean =>
(err as NodeJS.ErrnoException).code === "ELOCKED";
/**
* The message from whatever was thrown.
*
* Same reasoning as above for the non-`Error` arm — `proper-lockfile` does not
* reject with one — except that this is used where a thrown non-`Error` would
* otherwise be reported as `[object Object]`, so the fallback earns its place
* even though nothing can provoke it.
*/
/* v8 ignore next 2 -- @preserve: the non-Error arm is unreachable via
proper-lockfile, which rejects only with Errors. */
const describeError = (err: unknown): string =>
err instanceof Error ? err.message : String(err);
/**
* Run `fn` holding an exclusive cross-process lock on `filePath`.
*
* The lock is `<filePath>.lock`, a directory beside the secrets file rather
* than inside it — `proper-lockfile` never opens or truncates the file it
* guards, so a lock that outlives its holder can only ever block a write,
* never damage one.
*
* `realpath: false` is load-bearing: by default the library resolves the
* target through `fs.realpath`, which fails `ENOENT` on a secrets file that
* does not exist yet — i.e. on the very first `set`, the one call that has
* nothing to fall back on. Resolving the path lexically instead lets a file
* be locked into existence. The cost is that two paths reaching one file
* through different symlinks take different locks; the store resolves its
* path once at construction and every caller goes through it, so that is a
* shape this codebase does not produce.
*
* Returns whatever `fn` returns. `fn` runs exactly once either way — the
* lock's absence changes the guarantee, never whether the work happens.
*/
export async function withSecretFileLock<T>(
filePath: string,
fn: () => Promise<T>,
): Promise<T> {
const target = path.resolve(filePath);
let release: (() => Promise<void>) | undefined;
try {
release = await properLockfile.lock(target, {
realpath: false,
stale: STALE_MS,
retries: RETRY,
// The library's default `onCompromised` *throws* — from a timer, with
// no caller on the stack, so it lands as an uncaught exception and
// takes the process down. A compromised lock means someone declared
// ours stale and took it; the write in flight is at risk, which is
// worth saying and is not worth killing an Inspector session over.
onCompromised: (err) =>
warnOnce(
`The lock on the secrets file at ${target} was taken over by another process while a write was in progress (${err.message}). If a secret you just saved is missing, save it again.`,
),
});
} catch (err) {
warnOnce(
isHeldElsewhere(err)
? `Another process has held the secrets file at ${target} for longer than this write was willing to wait, so the write went ahead without the lock. If two Inspectors are saving secrets at once, one of them may not be saved.`
: `Could not take a lock on the secrets file at ${target} (${describeError(err)}), so writes to it are not protected against another process writing at the same moment.`,
);
return fn();
}
try {
return await fn();
} finally {
try {
await release();
} catch (err) {
// The body already ran and its result is being returned; a release
// that failed means the lock was taken from us (declared stale while
// we held it) or the directory went away. Neither is worth turning a
// successful save into a failure, but a silent catch would leave a
// lock nobody can explain, so say it.
warnOnce(
`Could not release the lock on the secrets file at ${target} (${describeError(err)}). It expires on its own after ${STALE_MS / 1000}s.`,
);
}
}
}

91
core/auth/node/file-secret-store.ts
View file

@ -62,6 +62,7 @@ import * as crypto from "node:crypto";
import * as fs from "node:fs/promises";
import * as path from "node:path";
import { readStoreFile, writeStoreFile } from "../../storage/store-io.js";
import { withSecretFileLock } from "./file-lock.js";
import {
SecretStoreUnavailableError,
type SecretBulkRequest,
@ -597,30 +598,33 @@ export class FileSecretStore implements SecretStore {
}
/**
* Run `fn` with the file to itself across *processes* as well.
* Apply a mutation to the file, under a cross-process lock, and confirm it
* survived.
*
* The in-process queue is necessary and not sufficient: a durable secrets
* file is shared state, and a second Inspector on the same box — a CLI run
* beside a web session is the ordinary case, not a contrived one — reads
* the same "before" map and then atomically replaces the file, dropping
* whatever the first process had just added. Both writes report success;
* one secret is simply gone.
* **Two mechanisms, and they answer different questions** — this is not
* belt-and-braces.
*
* Apply a mutation to the file, and confirm it survived.
* The lock ({@link withSecretFileLock}, #2082) is the one that provides
* mutual exclusion: a durable secrets file is shared state, and a second
* Inspector on the same box — a CLI run beside a web session is the
* ordinary case, not a contrived one — otherwise reads the same "before"
* map and then atomically replaces the file, dropping whatever the first
* process had just added. Both writes report success; one secret is simply
* gone. Held across the read *and* the write, so no other holder can
* observe or replace the map in between.
*
* **There is no cross-process lock.** There was one — a `mkdir` election
* with an owner stamp, a heartbeat and a stale-takeover — and three
* consecutive review rounds found a real race in it. The last one is
* unfixable with what Node exposes: claiming a stale lock needs
* compare-and-swap on a directory entry (`renameat2(RENAME_EXCHANGE)`),
* and without it a waiter that loses the race can still move the winner's
* *fresh* lock aside and enter alongside it.
* The verify covers what a lock structurally cannot. A lock is an advisory
* convention between the processes that take it, so it says nothing about
* a writer that does not: an editor, a restored backup, a `jq` one-liner,
* or an Inspector predating #2082. It is also the fallback when the lock
* is unavailable at all — `withSecretFileLock` runs the body anyway on a
* read-only or lock-hostile directory rather than failing a `set` on
* exactly the deployments this store exists for (#1848, #1905).
*
* So this does the opposite: it lets writers collide and makes the loser
* notice. Read `M0`, apply the mutation to get `M1`, write it, then read
* back `M2`. If `M2` equals `M1` nothing interleaved. If it does not,
* someone wrote between our write and our read — so re-apply onto what
* they left and try again.
* So: read `M0`, apply the mutation to get `M1`, write it, then read back
* `M2`. If `M2` equals `M1` nothing interleaved. If it does not, someone
* wrote between our write and our read — so re-apply onto what they left
* and try again.
*
* The comparison is over the **whole map**, not just the entry we touched.
* Checking only our own key would pass in precisely the case that loses
@ -629,32 +633,29 @@ export class FileSecretStore implements SecretStore {
* repairs it — A writes `MA`, B writes `MB` over it, B verifies `MB`
* correctly, A verifies, sees `M2 !== MA`, re-applies onto `MB`.
*
* **This is not mutual exclusion, and the gap is wider than a crash.**
* The verify only catches a clobber that has already landed. Order the
* same two writers as write-A, verify-A, write-B, verify-B and both
* succeed while A's entry is gone: A's verify ran before B's write, so
* there was nothing yet to see, and B did nothing wrong. Nothing detects
* it afterwards. A crash between write and verify is one instance of the
* same shape, not the whole of it — an earlier version of this comment
* said otherwise, which understated it.
*
* What that buys, stated without overselling: two processes must write the
* same file within the window between one's write and its read-back, and
* the loss is one secret that reported success. The lock this replaced
* lost updates in a *wider* set of interleavings, with every participant
* alive, and could not be closed without a primitive Node does not expose
* (`renameat2(RENAME_EXCHANGE)`); this costs ~220 fewer lines and converges
* in every interleaving where the clobber lands before the verify. If the
* residual matters for a deployment, the answer is a real lock — an
* OS-backed one from a dedicated library — not another hand-rolled
* election.
* **What is left, stated without overselling.** Against an unlocked writer
* the verify is not mutual exclusion: it only catches a clobber that has
* already landed, so ordering the two as write-A, verify-A, write-B,
* verify-B loses A's entry with both reporting success. That residual is
* now confined to a writer outside this codebase, or to a directory where
* no lock could be taken — and the latter announces itself, once, on the
* console.
*
* Reads deliberately do not participate: `writeStoreFile` is atomic
* (write-temp-then-rename), so a reader sees either the old file or the
* new one, never a torn one.
* new one, never a torn one. Taking the lock for them would serialize
* every `GET /api/servers` behind whatever else holds it, to prevent
* nothing.
*/
private async mutate(
apply: (map: Record<string, string>) => Record<string, string> | null,
): Promise<void> {
await withSecretFileLock(this.filePath, () => this.mutateLocked(apply));
}
/** {@link mutate}'s body, with the lock already held. */
private async mutateLocked(
apply: (map: Record<string, string>) => Record<string, string> | null,
): Promise<void> {
for (let attempt = 0; attempt < MAX_WRITE_ATTEMPTS; attempt++) {
let current: Record<string, string>;
@ -713,9 +714,15 @@ export class FileSecretStore implements SecretStore {
* Measured, not theorised — two instances racing `set` lost an entry on
* the first run of the convergence test below.
*
* So the in-process case is made correct by construction here, and the
* verify-and-retry in {@link mutate} covers what this cannot see: a
* So the in-process case is made correct by construction here, and
* {@link mutate}'s cross-process lock covers what this cannot see: a
* second Inspector process.
*
* The queue is *also* what keeps that lock usable. `proper-lockfile` is
* not reentrant — a second `lock()` on a path this process already holds
* fails `ELOCKED`, which is indistinguishable from a genuine remote
* holder. Serializing here means the lock is only ever contended between
* processes, which is the only contention it is asked to arbitrate.
*/
private serialize<T>(fn: () => Promise<T>): Promise<T> {
const key = path.resolve(this.filePath);

134
core/auth/node/secret-store-selection.ts
View file

@ -54,6 +54,7 @@ import {
readSecretFilePermissions,
tightenSecretFilePermissions,
} from "./file-secret-store.js";
import { withSecretFileLock } from "./file-lock.js";
import {
KeyringSecretStore,
parseAccount,
@ -532,51 +533,76 @@ export async function absorbFileSecretsIntoKeyring(
): Promise<void> {
const filePath = defaultSecretFilePath();
// A crash between the claim and the delete leaves only
// `secrets.json.migrating-<pid>`. Checking the canonical path alone then
// reports "nothing to migrate", the keychain is selected, and every stored
// credential silently disappears — the claim protecting the delete having
// introduced a way to lose everything. Adopt any orphan first.
await recoverOrphanedSnapshots(filePath);
if (!fsSync.existsSync(filePath)) return;
// **Claim the file atomically before reading it.** Comparing its contents
// immediately before `rm` narrowed the window and could not close it: a
// writer that completes a `set` after the comparison and before the delete
// has its write verified, reports success, and then loses it — a *later*
// successful write destroyed, which is worse than the optimistic-write
// residual and, unlike that one, fixable here.
// Cheap, lock-free "is there anything at all to do". Taking the lock first
// would mean every startup on the overwhelmingly common path — a keychain
// is available and no file was ever written — creates and removes a lock
// directory, and on a box whose storage dir does not exist yet the lock
// cannot be created at all, so `withSecretFileLock` would warn about
// unprotected writes on every single run with nothing to protect.
//
// `rename` is atomic and leaves the live path free. Everything after this
// point operates on a snapshot nobody else can reach, and a writer that
// recreates `secrets.json` in the meantime is simply untouched — its file
// is a different one, and the next run migrates it.
//
// The staged name carries the pid so two Inspectors starting together
// cannot claim the same destination; whichever wins the rename does the
// migration and the loser sees ENOENT and returns.
// A per-attempt nonce, not just the pid. `recoverOrphanedSnapshots`
// deliberately leaves an orphan in place when a live `secrets.json` also
// exists — and a pid-only name is reusable across restarts (pid 1 on every
// container start), so the next claim would `rename` straight over that
// orphan and permanently discard secrets it may uniquely hold.
const staged = `${filePath}.migrating-${process.pid}-${randomUUID()}`;
try {
await fs.rename(filePath, staged);
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
if (code !== "ENOENT") {
// Only ENOENT means someone else claimed it. Anything else — EACCES on
// the directory, EROFS — leaves the file exactly where it is, and
// returning quietly would select the keychain while file-backed
// secrets sit there unreadable by anything, with nothing said.
console.warn(
`\n[mcp-inspector] Could not claim the secrets file at ${filePath} for migration into the OS keychain (${code ?? "unknown error"}), so it has been left in place. Its secrets are not visible to this session.`,
);
// Racy by construction, and that is fine: it can only be wrong by saying
// "nothing here" about a file created a moment later, which is a file the
// next run migrates — the same outcome as a writer that recreates the path
// after the claim below. Everything that *acts* re-checks under the lock.
if (!(await anythingToMigrate(filePath))) return;
// Orphan adoption and the claim below both move the live path around, so
// they run under the same cross-process lock a `set` takes (#2082) —
// otherwise a concurrent writer's atomic rename can land between the two
// and be adopted, claimed, or clobbered depending on the interleaving.
// The lock is released before the hand-off: that part reads a snapshot
// nobody else can reach, and holding it across a keychain round-trip per
// secret would block every writer for the duration of a migration.
const claimed = await withSecretFileLock(filePath, async () => {
// A crash between the claim and the delete leaves only
// `secrets.json.migrating-<pid>`. Checking the canonical path alone then
// reports "nothing to migrate", the keychain is selected, and every stored
// credential silently disappears — the claim protecting the delete having
// introduced a way to lose everything. Adopt any orphan first.
await recoverOrphanedSnapshots(filePath);
if (!fsSync.existsSync(filePath)) return null;
// **Claim the file atomically before reading it.** Comparing its contents
// immediately before `rm` narrowed the window and could not close it: a
// writer that completes a `set` after the comparison and before the delete
// has its write verified, reports success, and then loses it — a *later*
// successful write destroyed, which is worse than the optimistic-write
// residual and, unlike that one, fixable here.
//
// `rename` is atomic and leaves the live path free. Everything after this
// point operates on a snapshot nobody else can reach, and a writer that
// recreates `secrets.json` in the meantime is simply untouched — its file
// is a different one, and the next run migrates it.
//
// The staged name carries the pid so two Inspectors starting together
// cannot claim the same destination; whichever wins the rename does the
// migration and the loser sees ENOENT and returns.
// A per-attempt nonce, not just the pid. `recoverOrphanedSnapshots`
// deliberately leaves an orphan in place when a live `secrets.json` also
// exists — and a pid-only name is reusable across restarts (pid 1 on every
// container start), so the next claim would `rename` straight over that
// orphan and permanently discard secrets it may uniquely hold.
const staged = `${filePath}.migrating-${process.pid}-${randomUUID()}`;
try {
await fs.rename(filePath, staged);
} catch (err) {
const code = (err as NodeJS.ErrnoException).code;
if (code !== "ENOENT") {
// Only ENOENT means someone else claimed it. Anything else — EACCES on
// the directory, EROFS — leaves the file exactly where it is, and
// returning quietly would select the keychain while file-backed
// secrets sit there unreadable by anything, with nothing said.
console.warn(
`\n[mcp-inspector] Could not claim the secrets file at ${filePath} for migration into the OS keychain (${code ?? "unknown error"}), so it has been left in place. Its secrets are not visible to this session.`,
);
}
return null;
}
return;
}
return staged;
});
if (claimed === null) return;
const staged = claimed;
const file = new FileSecretStore({ filePath: staged });
// True only when every value reached the keychain *and* there was
@ -625,6 +651,28 @@ export async function absorbFileSecretsIntoKeyring(
}
}
/**
* Is there a `secrets.json`, or a snapshot orphaned by an interrupted
* migration, worth taking the lock for?
*
* One `readdir` rather than a `stat` of the canonical path: an orphan is the
* case where the live file is *absent* and there is still everything to
* migrate, so checking only `secrets.json` would skip the recovery that
* exists because skipping it loses every stored credential.
*/
async function anythingToMigrate(filePath: string): Promise<boolean> {
const base = path.basename(filePath);
try {
return (await fs.readdir(path.dirname(filePath))).some(
(name) => name === base || name.startsWith(`${base}.migrating-`),
);
} catch {
// No storage directory yet — the first run on a fresh install, and the
// path this check exists to keep quiet.
return false;
}
}
/**
* Adopt a snapshot left behind by a process that died mid-migration.
*