modelcontextprotocol/inspector
Watch
1
0
Fork
You've already forked inspector
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

fix(auth): stop degrading on ELOCKED, and correct the claim about the lock (#2082)

Browse source 
Copilot review round 2, both comments upheld.

**ELOCKED must not degrade.** Exhausting the retries on a held lock means the
lock is *working* and something else demonstrably holds it — so running the
mutation anyway entered the exact interleaving the lock exists to prevent,
and entered it knowing a concurrent writer was there. `set` now fails
instead. Two supporting changes:

- The retry budget went from ~3.3s to ~15s so it outlasts the 10s stale
  window. Refusing is only defensible because a *crashed* holder resolves by
  takeover first; with the shorter budget, one Inspector killed mid-save
  would have failed every later save on the box until someone deleted the
  lock by hand. Pinned by a new test that stages a dead holder's lock
  directly (mkdir + backdated mtime) and asserts the save succeeds.
- Acquire is now probe-then-wait. `proper-lockfile` drives its whole acquire
  through `retry`, which re-attempts on *any* error — so with a 20-retry
  budget a read-only `$HOME` would have spent the full 15s re-issuing an
  identical failing `mkdir` on every save before degrading. One retry-less
  probe separates "held" (worth waiting out) from "unavailable" (not).

**The comment overclaimed what proper-lockfile provides, and the reviewer was
right about the mechanism.** Verified against lib/lockfile.js@4.1.2: on
EEXIST it stats, and if stale it rmdirs and re-mkdirs *without* checking the
directory it removed is the one it found stale — so a slow waiter can still
delete a fast waiter's fresh lock and both proceed. That is the same race the
hand-rolled version could not close, and it is not closable without
renameat2(RENAME_EXCHANGE).

What the library actually adds is that the loser is *detected*: its refresh
tick compares the lock's mtime against the value recorded at acquire, so a
compromised holder is told rather than proceeding silently. And what is
genuinely exclusive is the case that matters — mkdir is atomic and a live
holder refreshes its mtime, so its lock never goes stale and two running
Inspectors are serialized. The residual window opens only after a holder dies
without releasing, and the optimistic verify still covers it.

file-lock.ts, README.md, specification/v2_servers_file.md and AGENTS.md all
carried the overclaim; all four now state the narrower, true version.

Also fixes a test that was quietly asserting the opposite of its name: its
holder used stale: 60_000 while the waiter used 10_000, and staleness is
judged by the *waiter's* threshold — so the holder was declared stale after
10s and the save succeeded by takeover.

Not done: a two-process stale-takeover race test. It would be testing the
library rather than this code, and is inherently nondeterministic; the honest
response to that finding was to stop claiming the race is closed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H
Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
cliffhall 2026-08-23 10:08:12 -04:00
commit dcd70e2f11
5 changed files with 200 additions and 77 deletions
Download patch file Download diff file Expand all files Collapse all files

14
AGENTS.md
View file

@ -71,11 +71,15 @@ v2/main/
│ │ │ # file it cannot decrypt rather than destroying it),
│ │ │ # file-lock.ts (withSecretFileLock: the cross-process
│ │ │ # mutual exclusion #2082 settled on — proper-lockfile,
│ │ │ # borrowed rather than hand-rolled, because stale-takeover
│ │ │ # is what three review rounds of a mkdir election failed to
│ │ │ # get right; DEGRADES rather than throws when no lock can
│ │ │ # be taken, since this store exists for boxes missing the
│ │ │ # usual mechanism and must not gain a new way to fail),
│ │ │ # borrowed rather than hand-rolled. Read its header before
│ │ │ # citing it: it makes two LIVE Inspectors exclusive, and
│ │ │ # does NOT make stale takeover single-winner — it makes the
│ │ │ # loser detectable (ECOMPROMISED), which is the honest claim.
│ │ │ # DEGRADES when no lock CAN be taken (read-only $HOME etc),
│ │ │ # since this store exists for boxes missing the usual
│ │ │ # mechanism; but THROWS on ELOCKED — a lock held by a live
│ │ │ # writer is evidence the lock works, not licence to bypass
│ │ │ # it — after waiting past the stale window),
│ │ │ # and
│ │ │ # secret-store-selection.ts (the POLICY: explicit
│ │ │ # MCP_INSPECTOR_SECRET_STORE wins, else probe the keychain,

6
README.md
View file

@ -504,7 +504,11 @@ The Inspector writes the file `0600` and re-tightens it at startup if something
**Two Inspectors, one file.** Within a process, mutations are serialized per file path, so a web session's own concurrent saves cannot lose each other. Across processes — a CLI run beside a web session — each mutation takes an exclusive lock on `secrets.json.lock` for the whole read-modify-write, using [`proper-lockfile`](https://github.com/moxystudio/node-proper-lockfile) (the same library npm itself locks with). The lock expires 10 seconds after its holder stops refreshing it, so an Inspector that is killed mid-save does not leave the file unwritable.
Underneath the lock, each mutation still reads the file, applies its change, writes, then reads back and compares the whole map; if something wrote in between it re-applies onto what was left and retries, failing loudly after five lost rounds rather than returning as though the value were saved. That check is not redundant with the lock — a lock only orders the writers that *take* it, so it covers an editor, a restored backup, or an Inspector older than this release.
Two running Inspectors are therefore genuinely serialized. What a lock file cannot make single-winner is the *takeover of a lock whose holder died* — that needs a compare-and-swap on a directory entry (`renameat2`) which Node does not expose, and it is what an earlier hand-rolled attempt failed three review rounds on. `proper-lockfile` does not close that race either; what it adds is that the loser is **told** (it compares the lock's mtime against the value it recorded, so a holder whose lock was replaced is marked compromised and warns). The window opens only after a holder dies without releasing.
Which is why, underneath the lock, each mutation still reads the file, applies its change, writes, then reads back and compares the whole map; if something wrote in between it re-applies onto what was left and retries, failing loudly after five lost rounds rather than returning as though the value were saved. That check is what still catches a clobber inside that window — and it covers what no lock can, since a lock only orders the writers that *take* it: an editor, a restored backup, or an Inspector older than this release.
If another process holds the lock and will not let go, the save **fails** rather than going ahead unlocked — waiting past the stale window first, so a crashed Inspector resolves itself rather than failing everyone else's saves. Writing alongside a writer you can see is the one case where degrading would lose the secret it was trying to protect.
It is also what covers the lock being unavailable. This store exists for boxes where the usual mechanism isn't there, so a directory that can't hold a lock file — a read-only `$HOME`, a mount owned by another uid — makes the save proceed unlocked with a warning, rather than turning every `set` into a failure on exactly the deployments the store was written for.

123
clients/web/src/test/integration/auth/node/file-lock.test.ts
View file

@ -19,6 +19,7 @@ import {
} from "vitest";
import { execFile } from "node:child_process";
import { createRequire } from "node:module";
import { existsSync } from "node:fs";
import * as fs from "node:fs/promises";
import * as os from "node:os";
import * as path from "node:path";
@ -28,6 +29,7 @@ import {
resetFileLockWarnings,
} from "@inspector/core/auth/node/file-lock.js";
import { FileSecretStore } from "@inspector/core/auth/node/file-secret-store.js";
import { SecretStoreUnavailableError } from "@inspector/core/auth/node/secret-store.js";
const run = promisify(execFile);
const require_ = createRequire(import.meta.url);
@ -261,53 +263,100 @@ describe("withSecretFileLock degrades rather than failing", () => {
expect(warn).toHaveBeenCalledTimes(1);
});
it("still saves the secret when no lock can be taken", async () => {
// The degrade has to be end-to-end, not just in the helper: `set` on a
// lock-hostile directory must persist, falling back to the #1950
// optimistic behaviour.
const target = filePath();
const lockPath = `${target}.lock`;
// Occupy the lock's own path with a *file*, so `mkdir` fails EEXIST
// forever and no takeover can succeed — a permanent, non-ELOCKED
// failure rather than contention.
await fs.writeFile(lockPath, "not a lock directory", "utf-8");
const store = new FileSecretStore({ filePath: target });
await store.set("srv", "env:MINE", "1");
const reader = new FileSecretStore({ filePath: target });
expect(await reader.get("srv", "env:MINE")).toBe("1");
expect(warnings()).toMatch(
/Could not take a lock|has held the secrets file/,
);
}, 20_000);
it("gives up waiting on a holder that never releases, and proceeds", async () => {
// Holding it from *this* process: `proper-lockfile` is not reentrant, so
// a second `lock()` on the same path fails `ELOCKED` exactly as a remote
// holder's would — the difference the in-process queue exists to keep
// out of the lock's way.
it("refuses the save rather than writing alongside a live holder", async () => {
// `ELOCKED` is evidence the lock is *working*, so degrading here would
// enter the exact interleaving the lock exists to prevent — and enter it
// knowing another writer is there. Held from this process, which is
// indistinguishable to `proper-lockfile` from a remote holder (it is not
// reentrant); the in-process queue is what keeps that out of the way in
// production.
//
// The wait is real: the retry budget deliberately outlasts the 10s stale
// window so a *crashed* holder resolves by takeover instead of failing
// everyone else's saves. This holder is alive and refreshing, so it never
// goes stale and the budget is spent in full.
const target = filePath();
const lockfile = require_(LOCKFILE_MODULE) as {
lock: (f: string, o: object) => Promise<() => Promise<void>>;
};
// The **same** `stale` production uses, and that is not incidental:
// `isLockStale` is evaluated against the *waiter's* threshold while the
// holder refreshes on its own `stale / 2`. A holder configured looser
// (say 60s) refreshes every 30s and is therefore declared stale by a
// 10s waiter after 10s — the waiter takes over and the save succeeds,
// quietly testing the opposite of what this test claims.
const release = await lockfile.lock(target, {
realpath: false,
stale: 60_000,
stale: 10_000,
});
let ran = false;
await withSecretFileLock(target, async () => {
ran = true;
});
const store = new FileSecretStore({ filePath: target });
// One call, both assertions off the same rejection: each attempt spends
// the full retry budget, so a second would double the test's runtime to
// re-prove the same thing.
const err = await store
.set("srv", "env:MINE", "1")
.catch((e: unknown) => e);
expect(err).toBeInstanceOf(SecretStoreUnavailableError);
expect((err as Error).message).toMatch(/was not saved/);
await release();
// Proceeding is the lesser evil: refusing would lose the value outright,
// whereas proceeding falls back to the residual #1950 behaviour — and it
// says which of the two happened.
expect(ran).toBe(true);
expect(warnings()).toContain("without the lock");
}, 20_000);
// Nothing was written behind the holder's back.
expect(existsSync(target)).toBe(false);
}, 90_000);
it("takes over the lock of a holder that died, rather than failing the save", async () => {
// The invariant behind refusing on `ELOCKED`: refusing is only defensible
// because a *crashed* holder resolves on its own first. `RETRY` therefore
// has to outlast `STALE_MS` — if the budget were the shorter of the two,
// one Inspector killed mid-save would make every later save on the box
// fail until someone deleted the lock by hand.
//
// A dead holder is exactly a lock directory nobody is refreshing, so it
// is staged directly: no child to race, and no dependence on how quickly
// a killed process is reaped.
const target = filePath();
const lockPath = `${target}.lock`;
await fs.mkdir(path.dirname(target), { recursive: true });
await fs.mkdir(lockPath);
const longDead = new Date(Date.now() - 60_000);
await fs.utimes(lockPath, longDead, longDead);
const store = new FileSecretStore({ filePath: target });
await store.set("srv", "env:AFTER_CRASH", "1");
const reader = new FileSecretStore({ filePath: target });
expect(await reader.get("srv", "env:AFTER_CRASH")).toBe("1");
// Took the lock over — did not fall through to an unlocked write.
expect(warnings()).toBe("");
}, 60_000);
it("stays silent per the delete contract when the lock is held", async () => {
// `delete` reports nothing by contract — only `set` hard-fails — so the
// refusal above must not turn a delete into a throw.
const target = filePath();
const store = new FileSecretStore({ filePath: target });
await store.set("srv", "env:A", "1");
const lockfile = require_(LOCKFILE_MODULE) as {
lock: (f: string, o: object) => Promise<() => Promise<void>>;
};
// The **same** `stale` production uses, and that is not incidental:
// `isLockStale` is evaluated against the *waiter's* threshold while the
// holder refreshes on its own `stale / 2`. A holder configured looser
// (say 60s) refreshes every 30s and is therefore declared stale by a
// 10s waiter after 10s — the waiter takes over and the save succeeds,
// quietly testing the opposite of what this test claims.
const release = await lockfile.lock(target, {
realpath: false,
stale: 10_000,
});
await expect(store.delete("srv", "env:A")).resolves.toBeUndefined();
await release();
// …and the entry it could not delete is still there, not half-removed.
expect(await store.get("srv", "env:A")).toBe("1");
}, 90_000);
});
describe("withSecretFileLock reports what it cannot clean up", () => {

142
core/auth/node/file-lock.ts
View file

@ -11,18 +11,38 @@
* aside and enter alongside it.
*
* So the choice #2082 settles is not "lock versus no lock" but "hand-rolled
* versus borrowed". `proper-lockfile` is the borrowed one: it is what npm
* itself uses, and stale-takeover is precisely the problem it has already
* solved — it re-`stat`s the lock directory after claiming it and gives the
* lock up if the mtime is not the one it wrote, so a loser of the takeover
* race releases rather than proceeding.
* versus borrowed". `proper-lockfile` is the borrowed one — what npm itself
* locks with — and it is worth being exact about what it does and does not
* buy, because the temptation is to overclaim it.
*
* **What this does not remove.** The optimistic verify in
* {@link FileSecretStore.mutate} stays. A lock is an advisory convention
* between participants that take it, so it orders Inspector against
* Inspector and says nothing about an editor, a backup restore, or an
* Inspector old enough to predate this file. The verify is also what covers
* {@link withSecretFileLock} *declining* — see below.
* **What it makes exclusive.** `mkdir` is atomic, and a live holder refreshes
* the lock's mtime at `stale / 2` for as long as it lives, so its lock never
* becomes eligible for takeover. Two running Inspectors are therefore
* genuinely serialized: one holds, the other waits. That is the case #1950
* lost updates in, and it is closed.
*
* **What it does not.** Stale takeover is still not single-winner. Reading
* `lib/lockfile.js@4.1.2`: on `EEXIST` it `stat`s, and if stale it `rmdir`s
* and re-`mkdir`s — without checking that the directory it removed is the
* one it found stale. So a slow waiter can delete a fast waiter's *fresh*
* lock and claim a replacement, and both proceed. That is the identical race
* the hand-rolled version could not close, and it cannot be closed with what
* Node exposes (`renameat2(RENAME_EXCHANGE)`).
*
* What the library adds over the hand-rolled one is that the loser finds
* out. Its refresh tick compares the lock's mtime against the value it
* recorded at acquire, so a holder whose directory was replaced is marked
* `ECOMPROMISED` and told — see `onCompromised` below. The window is also
* narrow and conditional: it opens only after a holder *dies without
* releasing*, since nothing else lets a lock go stale.
*
* **Which is why the optimistic verify in {@link FileSecretStore.mutate}
* stays, and is not belt-and-braces.** It is what still catches a clobber
* inside that window. It also covers what no lock can: an advisory
* convention orders Inspector against Inspector and says nothing about an
* editor, a backup restore, or an Inspector old enough to predate this file
* — and it covers {@link withSecretFileLock} being unable to lock at all,
* see below.
*
* **Degrading rather than failing is deliberate.** The store's whole reason
* for existing is a box where the usual mechanism is unavailable (no
@ -42,6 +62,7 @@ import * as path from "node:path";
// graph for the web runner); named imports off a CJS module depend on
// lexer detection that esbuild and rollup disagree about.
import properLockfile from "proper-lockfile";
import { SecretStoreUnavailableError } from "./secret-store.js";
/**
* How long a lock may go untouched before another process may claim it.
@ -55,14 +76,21 @@ import properLockfile from "proper-lockfile";
const STALE_MS = 10_000;
/**
* How long a waiter will keep trying before giving up.
* How long a waiter keeps trying before giving up.
*
* The retry schedule below tops out around 3s of waiting. That is long
* enough for any real mutation to finish (see above) and short enough that a
* pathological case surfaces as a slow save rather than a hung one.
* This schedule sums to roughly 15s, and the number that matters is that it
* is comfortably **longer than {@link STALE_MS}**. A waiter that gives up
* first would abandon the save while the lock still belonged to a process
* that had already died — the takeover that resolves it becomes possible
* only once the lock goes stale, so a budget under 10s would turn a crashed
* Inspector into a failed save for every other one on the box.
*
* An uncontended acquire is one `mkdir` and pays none of this; the first
* retries are tens of milliseconds, so ordinary contention (a mutation is a
* read, an scrypt derivation and an atomic write) resolves imperceptibly.
*/
const RETRY = {
retries: 8,
retries: 20,
factor: 2,
minTimeout: 20,
maxTimeout: 1_000,
@ -117,12 +145,7 @@ const describeError = (err: unknown): string =>
err instanceof Error ? err.message : String(err);
/**
* Run `fn` holding an exclusive cross-process lock on `filePath`.
*
* The lock is `<filePath>.lock`, a directory beside the secrets file rather
* than inside it — `proper-lockfile` never opens or truncates the file it
* guards, so a lock that outlives its holder can only ever block a write,
* never damage one.
* One `proper-lockfile` acquire against `target`, with the given retry policy.
*
* `realpath: false` is load-bearing: by default the library resolves the
* target through `fs.realpath`, which fails `ENOENT` on a secrets file that
@ -132,6 +155,36 @@ const describeError = (err: unknown): string =>
* through different symlinks take different locks; the store resolves its
* path once at construction and every caller goes through it, so that is a
* shape this codebase does not produce.
*/
function acquire(
target: string,
retries: number | typeof RETRY,
): Promise<() => Promise<void>> {
return properLockfile.lock(target, {
realpath: false,
stale: STALE_MS,
retries,
// The library's default `onCompromised` *throws* — from a timer, with no
// caller on the stack, so it lands as an uncaught exception and takes the
// process down. This is also the library's *only* signal for the
// stale-takeover race described at the top of this file — someone
// declared our lock stale and replaced it — so it is the one place a user
// learns the guarantee was lost. Worth saying; not worth killing an
// Inspector session over.
onCompromised: (err) =>
warnOnce(
`The lock on the secrets file at ${target} was taken over by another process while a write was in progress (${err.message}). If a secret you just saved is missing, save it again.`,
),
});
}
/**
* Run `fn` holding an exclusive cross-process lock on `filePath`.
*
* The lock is `<filePath>.lock`, a directory beside the secrets file rather
* than inside it — `proper-lockfile` never opens or truncates the file it
* guards, so a lock that outlives its holder can only ever block a write,
* never damage one.
*
* Returns whatever `fn` returns. `fn` runs exactly once either way — the
* lock's absence changes the guarantee, never whether the work happens.
@ -156,25 +209,38 @@ export async function withSecretFileLock<T>(
await fs.mkdir(path.dirname(target), { recursive: true }).catch(() => {});
let release: (() => Promise<void>) | undefined;
try {
release = await properLockfile.lock(target, {
realpath: false,
stale: STALE_MS,
retries: RETRY,
// The library's default `onCompromised` *throws* — from a timer, with
// no caller on the stack, so it lands as an uncaught exception and
// takes the process down. A compromised lock means someone declared
// ours stale and took it; the write in flight is at risk, which is
// worth saying and is not worth killing an Inspector session over.
onCompromised: (err) =>
warnOnce(
`The lock on the secrets file at ${target} was taken over by another process while a write was in progress (${err.message}). If a secret you just saved is missing, save it again.`,
),
release = await acquire(target, 0).catch((err: unknown) => {
// **Retries are for contention, and only for contention.**
// `proper-lockfile` drives its whole acquire through `retry`, which
// re-attempts on *any* error — so a read-only `$HOME` would spend the
// full ~15s budget re-issuing an `mkdir` that fails identically every
// time, on every save, before degrading. Probing once with no retries
// separates the two answers at the cost of one syscall: `ELOCKED` is
// worth waiting out, an infrastructure failure is not.
if (!isHeldElsewhere(err)) throw err;
return acquire(target, RETRY);
});
} catch (err) {
// **`ELOCKED` is not a reason to degrade — it is the opposite.** It means
// the lock is working and something else demonstrably holds it right now,
// so running the body anyway would write alongside a *known* concurrent
// writer: the precise interleaving this exists to prevent, entered
// deliberately. Having already waited past the stale window (see
// {@link RETRY}), a holder still there is not one that crashed; it is one
// that is stuck. Refusing loses nothing — `set` reports it and the user
// retries — whereas proceeding can lose a secret while reporting success.
if (isHeldElsewhere(err)) {
throw new SecretStoreUnavailableError(
`Could not save to the secrets file at ${target}: another process has held the lock on it for over ${Math.round((RETRY.retries * RETRY.maxTimeout) / 1000)} seconds. Its secrets are intact; the value you just entered was not saved. If no other Inspector is running, remove ${target}.lock and try again.`,
);
}
// Everything else is the lock being *unavailable* rather than held — a
// read-only `$HOME`, a mount owned by another uid, a filesystem without
// `mkdir` semantics. There the choice is between degrading and refusing
// every save on a box that has no other way to keep a secret, and this
// store exists for exactly those boxes (#1848, #1905).
warnOnce(
isHeldElsewhere(err)
? `Another process has held the secrets file at ${target} for longer than this write was willing to wait, so the write went ahead without the lock. If two Inspectors are saving secrets at once, one of them may not be saved.`
: `Could not take a lock on the secrets file at ${target} (${describeError(err)}), so writes to it are not protected against another process writing at the same moment.`,
`Could not take a lock on the secrets file at ${target} (${describeError(err)}), so writes to it are not protected against another process writing at the same moment.`,
);
return fn();
}

2
specification/v2_servers_file.md
View file

@ -261,7 +261,7 @@ Each server entry may carry these Inspector-extension fields at the top level:
- **Secret store selection (#1950)**: #1356 assumed a keychain. On a host without one — the published container (no D-Bus session, #1848), Android/Termux (no prebuilt binary, #1905), a minimal Linux install — `set` hard-failed, so those users could not persist an OAuth client secret or a stdio `env:` value **at all**. #1950 adds the two missing implementations plus the policy that picks one, and the surfaces that say which was picked.
- **Selection**: `MCP_INSPECTOR_SECRET_STORE=keyring|file|memory` wins outright. Otherwise the keychain is _probed_ — an `AsyncEntry` construction plus a real read, because the container that motivated #1848 imports the package fine and only fails when it reaches for a Secret Service that isn't there. A read and not a write, deliberately: probing by writing would deposit a value in the user's login keyring at every startup for a store they may never use. If the probe fails the fallback is **`memory`** in a container whose secrets directory is not on a mount, and **`file`** otherwise — so mounting the volume the README already recommends for the catalog flips the same run to durable storage with no configuration, and an unmounted container gets the honest answer rather than a file `docker run --rm` will discard. Resolution is cached per process so the startup banner, `GET /api/config`, and the store doing the writing cannot disagree.
- **`FileSecretStore` at rest**: one JSON document at `~/.mcp-inspector/secrets.json` (or `MCP_INSPECTOR_SECRET_FILE`, else under `MCP_STORAGE_DIR`), written `0600` and re-tightened at startup. With `MCP_INSPECTOR_SECRET_KEY` set it is AES-256-GCM with a scrypt-derived key and a per-write random salt; without it the values are in the clear, and _that_ is what the loud banner and the warning-toned modal footer are about. The **whole map is encrypted as a unit**, not value-by-value, so the account names (`serverId:field`) are hidden too — a per-value scheme would leave a readable index of which servers you hold a client secret for. Upgrading is lazy: setting the passphrase later re-encrypts on the next write rather than rewriting the file during a run that may never touch a secret, and the descriptor reports `pendingEncryption` until it does. A file that can no longer be decrypted reads as empty but **refuses to be written**, because replacing a file of still-valid secrets to satisfy an additive request destroys data.
- **Concurrency**: within a process, mutations are serialized per **resolved file path** (not per store instance — two `FileSecretStore`s on one file are ordinary, since the resolved store holds one and the keychain hand-off builds another). Across processes, each mutation holds an exclusive `proper-lockfile` lock on `secrets.json.lock` for the whole read-modify-write (#2082). The in-process queue is what keeps that usable: `proper-lockfile` is not reentrant, so a second `lock()` from the same process fails `ELOCKED` and would be indistinguishable from a genuine remote holder. An earlier iteration hand-rolled the lock — a `mkdir` election with an owner stamp, heartbeat and stale-takeover — and three review rounds each found a real race, the last not closable with what Node exposes (claiming a stale lock needs compare-and-swap on a directory entry, `renameat2(RENAME_EXCHANGE)`). #2082 settled that as "borrow, don't hand-roll": stale-takeover is exactly the problem the library has already solved, re-`stat`ing the lock after claiming it so the loser of a takeover race releases instead of proceeding. **The optimistic verify stays underneath it**, and is not redundant: read `M0`, apply, write `M1`, read back `M2`, re-apply onto whatever a concurrent writer left if they differ, bounded, with `set` throwing on non-convergence rather than returning as though the value were saved. The comparison is over the **whole map** — checking only your own entry passes in exactly the case that loses data, because yours is present and the other writer's is gone. A lock is advisory between the processes that take it, so the verify is what covers a writer outside this codebase (an editor, a restored backup, an Inspector predating #2082) and what covers the lock being *unavailable*: `withSecretFileLock` runs the body anyway, warning once, on a directory that cannot hold a lock file — this store exists for boxes where the usual mechanism is missing (#1848, #1905) and must not acquire a new way to be unavailable. Reads take no lock; `writeStoreFile` is atomic, so a reader sees the old file or the new one, never a torn one.
- **Concurrency**: within a process, mutations are serialized per **resolved file path** (not per store instance — two `FileSecretStore`s on one file are ordinary, since the resolved store holds one and the keychain hand-off builds another). Across processes, each mutation holds an exclusive `proper-lockfile` lock on `secrets.json.lock` for the whole read-modify-write (#2082). The in-process queue is what keeps that usable: `proper-lockfile` is not reentrant, so a second `lock()` from the same process fails `ELOCKED` and would be indistinguishable from a genuine remote holder. An earlier iteration hand-rolled the lock — a `mkdir` election with an owner stamp, heartbeat and stale-takeover — and three review rounds each found a real race, the last not closable with what Node exposes (claiming a stale lock needs compare-and-swap on a directory entry, `renameat2(RENAME_EXCHANGE)`). #2082 settled that as "borrow, don't hand-roll" — but the borrowed lock is not claimed to close that race, because it does not. `proper-lockfile@4.1.2` `rmdir`s a stale lock and re-`mkdir`s without checking the directory it removed is the one it found stale, so a slow waiter can still delete a fast waiter's fresh lock; what it adds over the hand-rolled version is *detection* — its refresh tick compares the lock's mtime against the value recorded at acquire, so a compromised holder is told (`ECOMPROMISED`, surfaced as a warning) instead of proceeding silently. What **is** exclusive is the case that matters: `mkdir` is atomic and a live holder refreshes its mtime, so its lock never goes stale and two running Inspectors are genuinely serialized. The residual window opens only after a holder dies without releasing. A waiter also waits past the stale window before giving up (so a crashed holder resolves by takeover rather than failing everyone else's saves), and a lock still held after that makes `set` **fail** rather than write alongside a visible concurrent writer — `ELOCKED` is evidence the lock is working, not a reason to bypass it. **The optimistic verify stays underneath it**, and is not redundant: read `M0`, apply, write `M1`, read back `M2`, re-apply onto whatever a concurrent writer left if they differ, bounded, with `set` throwing on non-convergence rather than returning as though the value were saved. The comparison is over the **whole map** — checking only your own entry passes in exactly the case that loses data, because yours is present and the other writer's is gone. A lock is advisory between the processes that take it, so the verify is what covers a writer outside this codebase (an editor, a restored backup, an Inspector predating #2082) and what covers the lock being *unavailable*: `withSecretFileLock` runs the body anyway, warning once, on a directory that cannot hold a lock file — this store exists for boxes where the usual mechanism is missing (#1848, #1905) and must not acquire a new way to be unavailable. Reads take no lock; `writeStoreFile` is atomic, so a reader sees the old file or the new one, never a torn one.
- **Strict reads (`getStrict`)**: `get` is tolerant by contract, and that tolerance is wrong for exactly one caller — a migration that treats `null` as proof of absence and then _writes_. A transient read failure would look like "nothing there", and the write would replace a newer stored value with an older on-disk copy, inverting the keychain-wins rule the migration is built on. `getStrict` throws instead, and both plaintext migrations plus the keychain hand-off use it. Two traps this hit on the way in, both worth remembering: the seam must be forwarded by `DeferredSecretStore` (the store production actually uses, so without forwarding the strictness existed only in tests that injected a concrete store), and it must be implemented by _every_ store rather than falling back to `get` for the one it was introduced for.
- **Bulk reads (`getMany`)**: rehydration asked field by field, and `FileSecretStore.get` reads and decrypts the entire file per call. scrypt at `N=16384` measures ~23ms, so an encrypted catalog spent ~450ms of pure key derivation on every `GET /api/servers` — a visible stall rather than a micro-optimization. The seam takes a **list of `{ serverId, fields }` across servers**, and both rehydration callers pass the whole catalog in one request, so `FileSecretStore` decrypts once per rehydration; stores for which per-field reads are already cheap (the keychain) fall back to parallel `get`. The cross-server shape is the point: a per-server version shipped first and left a 20-server catalog paying 20 serialized derivations, because both callers iterate servers — the same stall reached one server at a time instead of one field at a time.
- **Durability gate on migration**: the plaintext-stripping migrations only delete the disk copy once the value is somewhere that outlives the process (`isDurable`). Against a session-scoped store, stripping would trade a secret that survives restarts for one that dies with the process — and it runs on an ordinary `GET`, so merely opening the app would destroy it.