fix(auth): guard every lock removal, not just release, and stop promising expiry (#2082)
Copilot review round 5. Both comments upheld; the first is a real hole in round 4's own fix. **The ownership guard was in the wrong place, and overclaimed.** Round 4 checked ownership and then called `release()`. Two problems: - Check-then-act. The `stat` and the `rmdir` were separate async steps, so a waiter could replace the directory in between and we deleted the winner's fresh lock anyway — the very thing the check was added to prevent. - Skipping `release()` on a mismatch left proper-lockfile's record registered in its `locks` map, and its `signal-exit` handler `rmdirSync`s every registered lock with **no ownership check of its own**. An exit during that window deleted the winner too. The guard now lives in `options.fs`, which is the single seam *both* removal paths route through — `removeLock` on release, and the exit handler. It refuses to remove a directory whose inode and birth time are not the ones recorded at acquire, and reports the refusal as success so the library's bookkeeping forgets the lock either way (leaving it registered is what hands the exit handler a record pointing at the winner's directory). The check is a `statSync` immediately followed by an `rmdirSync`, with no `await` between them, so nothing in this process can interleave. **It still does not close the race** — it is check-then-act across processes, which needs the compare-and-swap Node does not expose — and saying it "closes the destructive half outright" was wrong. file-lock.ts, README.md, specification/v2_servers_file.md and AGENTS.md now all say it narrows the window rather than closing it. AGENTS.md carries the negative instruction alongside round 4's, since this is the third round spent on an overclaim. **"It expires on its own after 10s" is false for the ENOTEMPTY case** the new test exercises. Stale takeover reclaims through the same `rmdir`, which also cannot remove a non-empty directory — so nothing clears it, and every later save fails ELOCKED against it. `releaseAdvice` now branches: by-hand removal for ENOTEMPTY, expiry for everything else. Tests: a real child process demonstrates the exit handler removing a replacement lock (with an **empty** directory — a non-empty one makes `rmdirSync` fail ENOTEMPTY, which is how the first draft of this test passed for the wrong reason). The release-failure test now asserts the message says "by hand" and does not promise expiry. The shim's `rmdirSync` arm carries a justified `v8 ignore`: it is reachable only from a real process exit, and its logic is `removeIfMine`, covered through the async arm. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
91e1392c1f
commit
b159c75b88
5 changed files with 201 additions and 84 deletions
12
AGENTS.md
12
AGENTS.md
|
|
@ -76,10 +76,14 @@ v2/main/
|
|||
│ │ │ # does NOT make stale takeover single-winner. proper-lockfile
|
||||
│ │ │ # detects a takeover only on its 5s refresh tick, which an
|
||||
│ │ │ # ordinary sub-second mutation never reaches, and its release
|
||||
│ │ │ # is an unconditional rmdir — so withSecretFileLock does its
|
||||
│ │ │ # OWN ownership check (inode+birthtime) before releasing, to
|
||||
│ │ │ # avoid deleting the winner's lock. Detection is BEST-EFFORT;
|
||||
│ │ │ # do not write that a compromised holder is always told.
|
||||
│ │ │ # is an unconditional rmdir (as is its signal-exit handler) —
|
||||
│ │ │ # so withSecretFileLock passes a GUARDED options.fs, the one
|
||||
│ │ │ # seam both removal paths share, refusing to delete a lock
|
||||
│ │ │ # that is no longer ours (inode+birthtime). That NARROWS the
|
||||
│ │ │ # window, it does not close it — still check-then-act across
|
||||
│ │ │ # processes. BEST-EFFORT throughout: do not write that a
|
||||
│ │ │ # compromised holder is always told, or that the winner's
|
||||
│ │ │ # lock is always preserved.
|
||||
│ │ │ # DEGRADES when no lock CAN be taken (read-only $HOME etc),
|
||||
│ │ │ # since this store exists for boxes missing the usual
|
||||
│ │ │ # mechanism; but THROWS on ELOCKED — a lock held by a live
|
||||
|
|
|
|||
|
|
@ -506,7 +506,7 @@ The Inspector writes the file `0600` and re-tightens it at startup if something
|
|||
|
||||
Two running Inspectors are therefore genuinely serialized. What a lock file cannot make single-winner is the *takeover of a lock whose holder died* — that needs a compare-and-swap on a directory entry (`renameat2`) which Node does not expose, and it is what an earlier hand-rolled attempt failed three review rounds on. `proper-lockfile` does not close that race either. The window opens only after a holder dies without releasing.
|
||||
|
||||
The Inspector adds one thing on top: before releasing, it checks the lock directory is still the one it created (by inode and birth time, which survive the library's own refresh but not a delete-and-recreate). That matters mostly because the library's release is an unconditional `rmdir` — so a holder whose lock had been replaced would otherwise delete the *winner's* lock on the way out, turning one compromised writer into two unprotected ones. It also surfaces the takeover as a warning. Treat that warning as **best-effort**, not a guarantee: it rests on filesystem metadata that not every filesystem reports.
|
||||
The Inspector adds one thing on top: every lock-directory removal the library makes on its behalf — on release, and from its exit handler — is guarded by a check that the directory is still the one it created (by inode and birth time, which survive the library's own refresh but not a delete-and-recreate). That matters because those removals are otherwise unconditional, so a holder whose lock had been replaced would delete the *winner's* lock on the way out, turning one compromised writer into two unprotected ones. It also surfaces the takeover as a warning. Treat all of this as **best-effort**: the guard is still a check followed by an act, so it makes the destructive case rare rather than impossible, and it rests on filesystem metadata that not every filesystem reports.
|
||||
|
||||
Which is why, underneath the lock, each mutation still reads the file, applies its change, writes, then reads back and compares the whole map; if something wrote in between it re-applies onto what was left and retries, failing loudly after five lost rounds rather than returning as though the value were saved. That check is what still catches a clobber inside that window — and it covers what no lock can, since a lock only orders the writers that *take* it: an editor, a restored backup, or an Inspector older than this release.
|
||||
|
||||
|
|
|
|||
|
|
@ -361,13 +361,12 @@ describe("withSecretFileLock degrades rather than failing", () => {
|
|||
|
||||
describe("withSecretFileLock reports what it cannot clean up", () => {
|
||||
it("does not delete the winner's lock after being taken over", async () => {
|
||||
// The destructive half of the stale-takeover race, and the reason this
|
||||
// check exists at all. `proper-lockfile`'s release is an unconditional
|
||||
// `rmdir`: a holder whose lock was replaced deletes the *winner's*
|
||||
// directory on the way out, so one compromised holder becomes two
|
||||
// unprotected writers. Its own detection cannot prevent that — it runs on
|
||||
// the refresh tick (5s here) while an ordinary mutation finishes in well
|
||||
// under a second, so the tick never runs and nobody is told.
|
||||
// The destructive half of the stale-takeover race. `proper-lockfile`'s
|
||||
// removal is an unconditional `rmdir`, so a holder whose lock was
|
||||
// replaced deletes the *winner's* directory on the way out — one
|
||||
// compromised holder becoming two unprotected writers. Its own detection
|
||||
// cannot prevent that: it runs on the refresh tick (5s here) while an
|
||||
// ordinary mutation finishes in well under a second.
|
||||
//
|
||||
// No waiting here, deliberately: this is the fast case the tick misses.
|
||||
const target = filePath();
|
||||
|
|
@ -375,7 +374,6 @@ describe("withSecretFileLock reports what it cannot clean up", () => {
|
|||
|
||||
let winner: { ino: number; birthtimeMs: number } | undefined;
|
||||
const result = await withSecretFileLock(target, async () => {
|
||||
// Someone declares our lock stale, removes it, and takes over.
|
||||
await fs.rm(lockPath, { recursive: true, force: true });
|
||||
await fs.mkdir(lockPath);
|
||||
const stat = await fs.stat(lockPath);
|
||||
|
|
@ -384,18 +382,58 @@ describe("withSecretFileLock reports what it cannot clean up", () => {
|
|||
});
|
||||
|
||||
expect(result).toBe("saved");
|
||||
// The winner's lock is untouched — same directory, not a recreated one.
|
||||
// Same directory, not a recreated one — it was left alone, not deleted.
|
||||
const after = await fs.stat(lockPath);
|
||||
expect(after.ino).toBe(winner?.ino);
|
||||
expect(after.birthtimeMs).toBe(winner?.birthtimeMs);
|
||||
expect(warnings()).toContain("was taken over by another process");
|
||||
});
|
||||
|
||||
it("warns instead of throwing when releasing a lock that is ours fails", async () => {
|
||||
// The ownership check above must not swallow a genuine release failure.
|
||||
// `rmdir` refuses a non-empty directory, so a stray file inside the lock
|
||||
// leaves it identifiably *ours* — same inode, same birth time — and still
|
||||
// unremovable.
|
||||
it("the exit handler this guards against really does delete a lock", async () => {
|
||||
// The other lifecycle window, and the reason the guard lives in
|
||||
// `options.fs` rather than in a check before `release()`:
|
||||
// `proper-lockfile` registers a `signal-exit` handler that `rmdirSync`s
|
||||
// every lock it still has registered, with no ownership check of its own.
|
||||
// A guard placed only around release would leave that free to delete the
|
||||
// winner's directory if the process exits at the wrong moment.
|
||||
//
|
||||
// Shown in a real child, because the handler only runs on a real exit,
|
||||
// and with an **empty** replacement directory — a non-empty one makes
|
||||
// `rmdirSync` fail `ENOTEMPTY` and would "pass" for the wrong reason,
|
||||
// which is exactly how the first draft of this test fooled itself.
|
||||
//
|
||||
// `withSecretFileLock` routes that same handler through the shim's
|
||||
// `rmdirSync`, which shares `removeIfMine` with the release path proven
|
||||
// by the test above.
|
||||
const target = filePath();
|
||||
const lockPath = `${target}.lock`;
|
||||
await fs.mkdir(path.dirname(target), { recursive: true });
|
||||
|
||||
const script = `
|
||||
const lockfile = require(${JSON.stringify(LOCKFILE_MODULE)});
|
||||
const fs = require("node:fs");
|
||||
const lockPath = ${JSON.stringify(lockPath)};
|
||||
lockfile
|
||||
.lock(${JSON.stringify(target)}, { realpath: false, stale: 10000 })
|
||||
.then(() => {
|
||||
// Taken over while we hold it, then exit *without* releasing.
|
||||
fs.rmSync(lockPath, { recursive: true, force: true });
|
||||
fs.mkdirSync(lockPath);
|
||||
process.exit(0);
|
||||
});
|
||||
`;
|
||||
await run(process.execPath, ["-e", script]);
|
||||
|
||||
// Unguarded, the winner's directory is gone.
|
||||
expect(existsSync(lockPath)).toBe(false);
|
||||
}, 30_000);
|
||||
|
||||
it("tells the operator to clear a lock that cannot expire on its own", async () => {
|
||||
// `rmdir` refuses a non-empty directory — and so does stale takeover,
|
||||
// which reclaims through the same call. So an `ENOTEMPTY` lock is not
|
||||
// cleaned up by the staleness mechanism, by us, or by the next writer:
|
||||
// every later save fails `ELOCKED` against it until somebody deletes it.
|
||||
// Promising it "expires on its own after 10s" would be false.
|
||||
const target = filePath();
|
||||
const result = await withSecretFileLock(target, async () => {
|
||||
await fs.writeFile(`${target}.lock/stray`, "", "utf-8");
|
||||
|
|
@ -406,6 +444,8 @@ describe("withSecretFileLock reports what it cannot clean up", () => {
|
|||
// not be turned into a failure by its own teardown.
|
||||
expect(result).toBe("saved");
|
||||
expect(warnings()).toContain("Could not release the lock");
|
||||
expect(warnings()).toContain("by hand");
|
||||
expect(warnings()).not.toContain("expires on its own");
|
||||
});
|
||||
|
||||
it("warns rather than crashing the process when the library detects the takeover", async () => {
|
||||
|
|
@ -413,7 +453,7 @@ describe("withSecretFileLock reports what it cannot clean up", () => {
|
|||
// no caller on the stack — an uncaught exception that takes an Inspector
|
||||
// session down. Replaced with a warning. This is the slow path: the lock
|
||||
// is removed and the body stays alive past the refresh tick, so the
|
||||
// library's own detection fires rather than the release-time check above.
|
||||
// library's own detection fires rather than the removal guard.
|
||||
const target = filePath();
|
||||
await withSecretFileLock(target, async () => {
|
||||
await fs.rm(`${target}.lock`, { recursive: true, force: true });
|
||||
|
|
|
|||
|
|
@ -39,12 +39,19 @@
|
|||
* replaced goes on to delete the *winner's* lock on the way out, silently
|
||||
* ending the winner's exclusion too.
|
||||
*
|
||||
* {@link withSecretFileLock} therefore does its own ownership check before
|
||||
* releasing (see there). That closes the destructive half — we never remove
|
||||
* a lock directory that is not the one we created — and surfaces the
|
||||
* compromise in exactly the fast-mutation case the tick misses. Detection is
|
||||
* still **best-effort**, not a guarantee: it rests on inode and birth-time
|
||||
* identity, which some filesystems do not report.
|
||||
* {@link withSecretFileLock} therefore guards every removal the library makes
|
||||
* on its behalf — see `guardedFs`, which sits in `options.fs` so it covers
|
||||
* the release path *and* the `signal-exit` handler. It refuses to delete a
|
||||
* directory that is no longer the one we created, and surfaces the compromise
|
||||
* in the fast-mutation case the tick misses.
|
||||
*
|
||||
* **That narrows the destructive window; it does not close it.** The guard is
|
||||
* a `statSync` immediately followed by an `rmdirSync`, so nothing in this
|
||||
* process can interleave — but it is still check-then-act against other
|
||||
* processes, and closing that needs the same compare-and-swap Node does not
|
||||
* expose. It also rests on inode and birth-time identity, which some
|
||||
* filesystems do not report. Best-effort throughout: it makes the destructive
|
||||
* case rare, not impossible.
|
||||
*
|
||||
* The window is at least narrow and conditional: it opens only after a
|
||||
* holder *dies without releasing*, since nothing else lets a lock go stale.
|
||||
|
|
@ -68,6 +75,7 @@
|
|||
* underneath it, and says so once.
|
||||
*/
|
||||
|
||||
import nodeFs, { statSync, rmdirSync } from "node:fs";
|
||||
import * as fs from "node:fs/promises";
|
||||
import * as path from "node:path";
|
||||
// CJS-only package. A default import is the shape that survives every
|
||||
|
|
@ -182,23 +190,30 @@ const describeError = (err: unknown): string =>
|
|||
/** Where `proper-lockfile` puts the lock for `target` — its documented default. */
|
||||
const lockPathOf = (target: string): string => `${target}.lock`;
|
||||
|
||||
/** A lock directory's identity: what changes on delete-and-recreate. */
|
||||
interface LockIdentity {
|
||||
ino: number;
|
||||
birthtimeMs: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* A lock directory's identity, or `null` if it could not be read.
|
||||
* Identify a lock directory, or `null` if it cannot be read.
|
||||
*
|
||||
* `ino` and `birthtimeMs` together: a directory removed and recreated gets a
|
||||
* new inode and a new birth time, while `utimes` — which the library's own
|
||||
* refresh tick performs on our behalf every few seconds — changes neither.
|
||||
* `ino` and `birthtimeMs` together, and **not the mtime** the library
|
||||
* compares: a lock held longer than one refresh tick has its mtime rewritten
|
||||
* by `utimes` legitimately, so an mtime comparison would call our own healthy
|
||||
* lock compromised. Both of these survive `utimes` and both change when a
|
||||
* directory is removed and recreated, which is the event to detect.
|
||||
*
|
||||
* Not every filesystem reports both (Windows shares, some network mounts, and
|
||||
* older kernels report `0`). There the two reads simply agree and the check
|
||||
* below concludes the lock is ours, which is the behaviour without this check
|
||||
* at all — best-effort, and never a false alarm on a healthy lock.
|
||||
* Not every filesystem reports either (Windows shares, some network mounts,
|
||||
* older kernels report `0`). There the comparison trivially succeeds and the
|
||||
* guard below concludes the lock is ours — the behaviour without the guard at
|
||||
* all, which is the right way to fail: best-effort, never a false alarm on a
|
||||
* healthy lock.
|
||||
*/
|
||||
async function identify(
|
||||
lockPath: string,
|
||||
): Promise<{ ino: number; birthtimeMs: number } | null> {
|
||||
function identifySync(lockPath: string): LockIdentity | null {
|
||||
try {
|
||||
const stat = await fs.stat(lockPath);
|
||||
const stat = statSync(lockPath);
|
||||
return { ino: stat.ino, birthtimeMs: stat.birthtimeMs };
|
||||
} catch {
|
||||
return null;
|
||||
|
|
@ -206,24 +221,89 @@ async function identify(
|
|||
}
|
||||
|
||||
/**
|
||||
* Is the lock directory still the one we created?
|
||||
* A `proper-lockfile` `fs` shim whose directory removal refuses to delete a
|
||||
* lock that is no longer the one we created.
|
||||
*
|
||||
* A missing directory counts as **not ours** — it was removed by a takeover
|
||||
* (or by hand), and there is nothing of ours left to release.
|
||||
* **Why this sits in `options.fs` rather than in a check before `release()`.**
|
||||
* Every deletion the library performs on our behalf goes through this object
|
||||
* — the `release()` path (`removeLock` → `fs.rmdir`) *and* its `signal-exit`
|
||||
* handler (`rmdirSync` over every registered lock, with no ownership check of
|
||||
* its own). A check placed before `release()` covers only the first, and
|
||||
* leaves the second free to delete the winner's lock if the process exits at
|
||||
* the wrong moment. Guarding at the single point where a directory is
|
||||
* actually removed covers both, and there is nowhere narrower to put it.
|
||||
*
|
||||
* An unreadable *baseline* (`claimedAs === null`) is the one case that
|
||||
* answers "ours": we could not identify the directory at acquire, so we have
|
||||
* nothing to compare against and must not accuse a healthy lock. Releasing is
|
||||
* then exactly what the library would have done unaided.
|
||||
* **It narrows the window; it does not close it.** The guard is
|
||||
* `statSync` immediately followed by `rmdirSync`, with no `await` between
|
||||
* them — so nothing else *in this process* can interleave, and the gap is as
|
||||
* small as this platform allows. It is still a check-then-act against other
|
||||
* processes, and closing that needs compare-and-swap on a directory entry
|
||||
* (`renameat2(RENAME_EXCHANGE)`), which is exactly what Node does not expose
|
||||
* and what the whole stale-takeover problem reduces to. Treat this as making
|
||||
* the destructive case rare, not impossible.
|
||||
*
|
||||
* `owned.id` stays `null` until we have acquired, which is deliberate: during
|
||||
* `acquireLock` the library removes *another* holder's stale directory
|
||||
* through this same seam, and that removal must go through untouched.
|
||||
*/
|
||||
async function stillOurs(
|
||||
function guardedFs(
|
||||
lockPath: string,
|
||||
claimedAs: { ino: number; birthtimeMs: number } | null,
|
||||
): Promise<boolean> {
|
||||
if (claimedAs === null) return true;
|
||||
const now = await identify(lockPath);
|
||||
if (now === null) return false;
|
||||
return now.ino === claimedAs.ino && now.birthtimeMs === claimedAs.birthtimeMs;
|
||||
owned: { id: LockIdentity | null },
|
||||
onRefused: () => void,
|
||||
): unknown {
|
||||
const mine = (): boolean => {
|
||||
if (owned.id === null) return true; // Not ours yet — see above.
|
||||
const now = identifySync(lockPath);
|
||||
if (now === null) return false; // Already gone; nothing of ours to remove.
|
||||
return now.ino === owned.id.ino && now.birthtimeMs === owned.id.birthtimeMs;
|
||||
};
|
||||
const removeIfMine = (): void => {
|
||||
if (!mine()) {
|
||||
onRefused();
|
||||
return;
|
||||
}
|
||||
rmdirSync(lockPath);
|
||||
};
|
||||
return {
|
||||
...nodeFs,
|
||||
// Reported as success when refused: the library's bookkeeping should
|
||||
// forget this lock either way. Leaving it registered would hand the
|
||||
// `signal-exit` handler a record pointing at the winner's directory.
|
||||
rmdir: (_p: string, cb: (err: NodeJS.ErrnoException | null) => void) => {
|
||||
try {
|
||||
removeIfMine();
|
||||
cb(null);
|
||||
} catch (err) {
|
||||
cb(err as NodeJS.ErrnoException);
|
||||
}
|
||||
},
|
||||
/* v8 ignore next 3 -- @preserve: only reachable from proper-lockfile's
|
||||
signal-exit handler, i.e. at real process exit, which no in-process
|
||||
test can drive. Its logic is `removeIfMine`, covered via `rmdir`. */
|
||||
rmdirSync: () => {
|
||||
removeIfMine();
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* What to tell an operator about a lock we could not remove.
|
||||
*
|
||||
* The blanket "it expires on its own" is **false for `ENOTEMPTY`**, and that
|
||||
* is the reachable case rather than a hypothetical: stale takeover reclaims a
|
||||
* lock through the same `rmdir`, which also cannot remove a non-empty
|
||||
* directory. So a lock directory with anything inside it is not cleaned up by
|
||||
* the staleness mechanism, by us, or by the next writer — it stays until
|
||||
* somebody deletes it, and every later save fails `ELOCKED` against it.
|
||||
*
|
||||
* Any other failure leaves a directory the refresher has stopped touching, so
|
||||
* the stale path really does reclaim it after {@link STALE_MS}.
|
||||
*/
|
||||
function releaseAdvice(err: unknown, target: string): string {
|
||||
const code = (err as NodeJS.ErrnoException).code;
|
||||
return code === "ENOTEMPTY"
|
||||
? `It has something inside it, which stale takeover cannot clear either, so saves will keep failing until you remove ${lockPathOf(target)} by hand.`
|
||||
: `It expires on its own after ${STALE_MS / 1000}s.`;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -241,11 +321,15 @@ async function stillOurs(
|
|||
function acquire(
|
||||
target: string,
|
||||
retries: number | typeof RETRY,
|
||||
fsShim: unknown,
|
||||
): Promise<() => Promise<void>> {
|
||||
return properLockfile.lock(target, {
|
||||
realpath: false,
|
||||
stale: STALE_MS,
|
||||
retries,
|
||||
// Every directory removal the library performs — on release and from its
|
||||
// exit handler — routes through here. See `guardedFs`.
|
||||
fs: fsShim,
|
||||
// The library's default `onCompromised` *throws* — from a timer, with no
|
||||
// caller on the stack, so it lands as an uncaught exception and takes the
|
||||
// process down. This is also the library's *only* signal for the
|
||||
|
|
@ -289,9 +373,16 @@ export async function withSecretFileLock<T>(
|
|||
// taken, and the catch below already says so with the right message — one
|
||||
// that mentions the lock rather than a `mkdir` the caller never asked for.
|
||||
await fs.mkdir(path.dirname(target), { recursive: true }).catch(() => {});
|
||||
// Filled in once we actually hold the lock; see `guardedFs`.
|
||||
const owned: { id: LockIdentity | null } = { id: null };
|
||||
const fsShim = guardedFs(lockPathOf(target), owned, () =>
|
||||
warnOnce(
|
||||
`The lock on the secrets file at ${target} was taken over by another process while a write was in progress, so it was left alone rather than removed. If a secret you just saved is missing, save it again.`,
|
||||
),
|
||||
);
|
||||
let release: (() => Promise<void>) | undefined;
|
||||
try {
|
||||
release = await acquire(target, 0).catch((err: unknown) => {
|
||||
release = await acquire(target, 0, fsShim).catch((err: unknown) => {
|
||||
// **Retries are for contention, and only for contention.**
|
||||
// `proper-lockfile` drives its whole acquire through `retry`, which
|
||||
// re-attempts on *any* error — so a read-only `$HOME` would spend the
|
||||
|
|
@ -300,7 +391,7 @@ export async function withSecretFileLock<T>(
|
|||
// separates the two answers at the cost of one syscall: `ELOCKED` is
|
||||
// worth waiting out, an infrastructure failure is not.
|
||||
if (!isHeldElsewhere(err)) throw err;
|
||||
return acquire(target, RETRY);
|
||||
return acquire(target, RETRY, fsShim);
|
||||
});
|
||||
} catch (err) {
|
||||
// **`ELOCKED` is not a reason to degrade — it is the opposite.** It means
|
||||
|
|
@ -326,40 +417,22 @@ export async function withSecretFileLock<T>(
|
|||
);
|
||||
return fn();
|
||||
}
|
||||
// Identity of the directory we just created, for the ownership check below.
|
||||
// **Not the mtime**, which is what the library compares: a lock held longer
|
||||
// than one refresh tick has its mtime rewritten by `utimes` legitimately,
|
||||
// so an mtime comparison would call our own healthy lock compromised. The
|
||||
// inode and birth time both survive `utimes` and both change when a
|
||||
// directory is removed and recreated, which is precisely the event to
|
||||
// detect.
|
||||
const claimedAs = await identify(lockPathOf(target));
|
||||
// Now that we hold it, record which directory is ours so the guard above
|
||||
// can refuse to delete anyone else's.
|
||||
owned.id = identifySync(lockPathOf(target));
|
||||
try {
|
||||
return await fn();
|
||||
} finally {
|
||||
if (!(await stillOurs(lockPathOf(target), claimedAs))) {
|
||||
// Someone declared our lock stale and replaced it. **Do not release**:
|
||||
// `proper-lockfile`'s release is an unconditional `rmdir`, so calling
|
||||
// it here would delete the lock directory that now belongs to whoever
|
||||
// took over — ending their exclusion as well as ours, and turning one
|
||||
// compromised holder into two unprotected writers. Leaving their lock
|
||||
// alone costs us nothing; ours is already gone.
|
||||
try {
|
||||
await release();
|
||||
} catch (err) {
|
||||
// The body already ran and its result is being returned; turning a
|
||||
// completed save into a failure at teardown would be the wrong trade.
|
||||
// But a lock left behind is worth explaining, and the two reasons need
|
||||
// different advice — see `releaseAdvice`.
|
||||
warnOnce(
|
||||
`The lock on the secrets file at ${target} was taken over by another process while a write was in progress. If a secret you just saved is missing, save it again.`,
|
||||
`Could not release the lock on the secrets file at ${target} (${describeError(err)}). ${releaseAdvice(err, target)}`,
|
||||
);
|
||||
} else {
|
||||
try {
|
||||
await release();
|
||||
} catch (err) {
|
||||
// The body already ran and its result is being returned; a release
|
||||
// that failed means the lock was taken from us (declared stale while
|
||||
// we held it) or the directory went away. Neither is worth turning a
|
||||
// successful save into a failure, but a silent catch would leave a
|
||||
// lock nobody can explain, so say it.
|
||||
warnOnce(
|
||||
`Could not release the lock on the secrets file at ${target} (${describeError(err)}). It expires on its own after ${STALE_MS / 1000}s.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -261,7 +261,7 @@ Each server entry may carry these Inspector-extension fields at the top level:
|
|||
- **Secret store selection (#1950)**: #1356 assumed a keychain. On a host without one — the published container (no D-Bus session, #1848), Android/Termux (no prebuilt binary, #1905), a minimal Linux install — `set` hard-failed, so those users could not persist an OAuth client secret or a stdio `env:` value **at all**. #1950 adds the two missing implementations plus the policy that picks one, and the surfaces that say which was picked.
|
||||
- **Selection**: `MCP_INSPECTOR_SECRET_STORE=keyring|file|memory` wins outright. Otherwise the keychain is _probed_ — an `AsyncEntry` construction plus a real read, because the container that motivated #1848 imports the package fine and only fails when it reaches for a Secret Service that isn't there. A read and not a write, deliberately: probing by writing would deposit a value in the user's login keyring at every startup for a store they may never use. If the probe fails the fallback is **`memory`** in a container whose secrets directory is not on a mount, and **`file`** otherwise — so mounting the volume the README already recommends for the catalog flips the same run to durable storage with no configuration, and an unmounted container gets the honest answer rather than a file `docker run --rm` will discard. Resolution is cached per process so the startup banner, `GET /api/config`, and the store doing the writing cannot disagree.
|
||||
- **`FileSecretStore` at rest**: one JSON document at `~/.mcp-inspector/secrets.json` (or `MCP_INSPECTOR_SECRET_FILE`, else under `MCP_STORAGE_DIR`), written `0600` and re-tightened at startup. With `MCP_INSPECTOR_SECRET_KEY` set it is AES-256-GCM with a scrypt-derived key and a per-write random salt; without it the values are in the clear, and _that_ is what the loud banner and the warning-toned modal footer are about. The **whole map is encrypted as a unit**, not value-by-value, so the account names (`serverId:field`) are hidden too — a per-value scheme would leave a readable index of which servers you hold a client secret for. Upgrading is lazy: setting the passphrase later re-encrypts on the next write rather than rewriting the file during a run that may never touch a secret, and the descriptor reports `pendingEncryption` until it does. A file that can no longer be decrypted reads as empty but **refuses to be written**, because replacing a file of still-valid secrets to satisfy an additive request destroys data.
|
||||
- **Concurrency**: within a process, mutations are serialized per **resolved file path** (not per store instance — two `FileSecretStore`s on one file are ordinary, since the resolved store holds one and the keychain hand-off builds another). Across processes, each mutation holds an exclusive `proper-lockfile` lock on `secrets.json.lock` for the whole read-modify-write (#2082). The in-process queue is what keeps that usable: `proper-lockfile` is not reentrant, so a second `lock()` from the same process fails `ELOCKED` and would be indistinguishable from a genuine remote holder. An earlier iteration hand-rolled the lock — a `mkdir` election with an owner stamp, heartbeat and stale-takeover — and three review rounds each found a real race, the last not closable with what Node exposes (claiming a stale lock needs compare-and-swap on a directory entry, `renameat2(RENAME_EXCHANGE)`). #2082 settled that as "borrow, don't hand-roll" — but the borrowed lock is not claimed to close that race, because it does not. `proper-lockfile@4.1.2` `rmdir`s a stale lock and re-`mkdir`s without checking the directory it removed is the one it found stale, so a slow waiter can still delete a fast waiter's fresh lock; and its own detection is weaker than it looks: `updateLock` compares mtime only on the refresh tick (`stale / 2`, 5s here), while an ordinary mutation finishes in well under a second — so in the common case the tick never runs and nobody is told. Its `release` is also an unconditional `rmdir` with no ownership check, so a holder whose lock was replaced deletes the *winner's* lock on the way out, turning one compromised writer into two unprotected ones. `withSecretFileLock` therefore performs its own ownership check before releasing (inode + birth time, which survive the library's `utimes` refresh but not a delete-and-recreate): on a mismatch it declines to release and warns. That closes the destructive half outright and surfaces the takeover in the fast case the tick misses — but detection remains **best-effort**, since not every filesystem reports those fields, and where they are unavailable it degrades to the library's unaided behaviour rather than to a false alarm. What **is** exclusive is the case that matters: `mkdir` is atomic and a live holder refreshes its mtime, so its lock never goes stale and two running Inspectors are genuinely serialized. The residual window opens only after a holder dies without releasing. A waiter also waits past the stale window before giving up (so a crashed holder resolves by takeover rather than failing everyone else's saves), and a lock still held after that makes `set` **fail** rather than write alongside a visible concurrent writer — `ELOCKED` is evidence the lock is working, not a reason to bypass it. **The optimistic verify stays underneath it**, and is not redundant: read `M0`, apply, write `M1`, read back `M2`, re-apply onto whatever a concurrent writer left if they differ, bounded, with `set` throwing on non-convergence rather than returning as though the value were saved. The comparison is over the **whole map** — checking only your own entry passes in exactly the case that loses data, because yours is present and the other writer's is gone. A lock is advisory between the processes that take it, so the verify is what covers a writer outside this codebase (an editor, a restored backup, an Inspector predating #2082) and what covers the lock being *unavailable*: `withSecretFileLock` runs the body anyway, warning once, on a directory that cannot hold a lock file — this store exists for boxes where the usual mechanism is missing (#1848, #1905) and must not acquire a new way to be unavailable. Reads take no lock; `writeStoreFile` is atomic, so a reader sees the old file or the new one, never a torn one.
|
||||
- **Concurrency**: within a process, mutations are serialized per **resolved file path** (not per store instance — two `FileSecretStore`s on one file are ordinary, since the resolved store holds one and the keychain hand-off builds another). Across processes, each mutation holds an exclusive `proper-lockfile` lock on `secrets.json.lock` for the whole read-modify-write (#2082). The in-process queue is what keeps that usable: `proper-lockfile` is not reentrant, so a second `lock()` from the same process fails `ELOCKED` and would be indistinguishable from a genuine remote holder. An earlier iteration hand-rolled the lock — a `mkdir` election with an owner stamp, heartbeat and stale-takeover — and three review rounds each found a real race, the last not closable with what Node exposes (claiming a stale lock needs compare-and-swap on a directory entry, `renameat2(RENAME_EXCHANGE)`). #2082 settled that as "borrow, don't hand-roll" — but the borrowed lock is not claimed to close that race, because it does not. `proper-lockfile@4.1.2` `rmdir`s a stale lock and re-`mkdir`s without checking the directory it removed is the one it found stale, so a slow waiter can still delete a fast waiter's fresh lock; and its own detection is weaker than it looks: `updateLock` compares mtime only on the refresh tick (`stale / 2`, 5s here), while an ordinary mutation finishes in well under a second — so in the common case the tick never runs and nobody is told. Its `release` is also an unconditional `rmdir` with no ownership check, so a holder whose lock was replaced deletes the *winner's* lock on the way out, turning one compromised writer into two unprotected ones. `withSecretFileLock` therefore supplies a guarded `options.fs` whose directory removal refuses to delete a lock that is no longer the one it created (inode + birth time, which survive the library's `utimes` refresh but not a delete-and-recreate). It sits in `options.fs` rather than around `release()` because that is the one seam **both** removal paths route through — the release path and the library's `signal-exit` handler, which `rmdirSync`s every registered lock with no ownership check of its own; a guard around release alone leaves an exit at the wrong moment free to delete the winner's directory. This **narrows** the destructive window (the check is a `statSync` immediately followed by an `rmdirSync`, so nothing in-process interleaves) and surfaces the takeover in the fast case the tick misses — but it does not close it: it is still check-then-act across processes, which needs the same CAS Node does not expose. Best-effort throughout, and where a filesystem reports neither field it degrades to the library's unaided behaviour rather than to a false alarm. What **is** exclusive is the case that matters: `mkdir` is atomic and a live holder refreshes its mtime, so its lock never goes stale and two running Inspectors are genuinely serialized. The residual window opens only after a holder dies without releasing. A waiter also waits past the stale window before giving up (so a crashed holder resolves by takeover rather than failing everyone else's saves), and a lock still held after that makes `set` **fail** rather than write alongside a visible concurrent writer — `ELOCKED` is evidence the lock is working, not a reason to bypass it. **The optimistic verify stays underneath it**, and is not redundant: read `M0`, apply, write `M1`, read back `M2`, re-apply onto whatever a concurrent writer left if they differ, bounded, with `set` throwing on non-convergence rather than returning as though the value were saved. The comparison is over the **whole map** — checking only your own entry passes in exactly the case that loses data, because yours is present and the other writer's is gone. A lock is advisory between the processes that take it, so the verify is what covers a writer outside this codebase (an editor, a restored backup, an Inspector predating #2082) and what covers the lock being *unavailable*: `withSecretFileLock` runs the body anyway, warning once, on a directory that cannot hold a lock file — this store exists for boxes where the usual mechanism is missing (#1848, #1905) and must not acquire a new way to be unavailable. Reads take no lock; `writeStoreFile` is atomic, so a reader sees the old file or the new one, never a torn one.
|
||||
- **Strict reads (`getStrict`)**: `get` is tolerant by contract, and that tolerance is wrong for exactly one caller — a migration that treats `null` as proof of absence and then _writes_. A transient read failure would look like "nothing there", and the write would replace a newer stored value with an older on-disk copy, inverting the keychain-wins rule the migration is built on. `getStrict` throws instead, and both plaintext migrations plus the keychain hand-off use it. Two traps this hit on the way in, both worth remembering: the seam must be forwarded by `DeferredSecretStore` (the store production actually uses, so without forwarding the strictness existed only in tests that injected a concrete store), and it must be implemented by _every_ store rather than falling back to `get` for the one it was introduced for.
|
||||
- **Bulk reads (`getMany`)**: rehydration asked field by field, and `FileSecretStore.get` reads and decrypts the entire file per call. scrypt at `N=16384` measures ~23ms, so an encrypted catalog spent ~450ms of pure key derivation on every `GET /api/servers` — a visible stall rather than a micro-optimization. The seam takes a **list of `{ serverId, fields }` across servers**, and both rehydration callers pass the whole catalog in one request, so `FileSecretStore` decrypts once per rehydration; stores for which per-field reads are already cheap (the keychain) fall back to parallel `get`. The cross-server shape is the point: a per-server version shipped first and left a 20-server catalog paying 20 serialized derivations, because both callers iterate servers — the same stall reached one server at a time instead of one field at a time.
|
||||
- **Durability gate on migration**: the plaintext-stripping migrations only delete the disk copy once the value is somewhere that outlives the process (`isDurable`). Against a session-scoped store, stripping would trade a secret that survives restarts for one that dies with the process — and it runs on an ordinary `GET`, so merely opening the app would destroy it.
|
||||
|
|
|
|||
Loading…
Reference in a new issue