fix(auth): replace the secrets-file lock with optimistic verify-and-retry, and address review round 9
Two changes, interleaved in the same files because both are about the same read-modify-write path. ## The lock is gone (maintainer decision on the round-5/7/8 finding) Three consecutive review rounds found a real race in the `mkdir` lock, and the last one is not patchable with what Node exposes: claiming a stale lock needs compare-and-swap on a directory entry (`renameat2(RENAME_EXCHANGE)`). Without it, `canSteal` inspects one lock but `rename` acts on whatever is at that path *later*, so a waiter that lost the first race can move the winner's fresh lock aside and enter alongside it. Rather than patch it a fourth time, writers now collide and the loser notices. `mutate` reads `M0`, applies the mutation to get `M1`, writes it, then reads back `M2`; if they differ, someone wrote in between, so it re-applies onto what they left and retries, bounded. The comparison is over the **whole map**, and that is the load-bearing detail. Checking only the entry we touched passes in exactly the case that loses data — ours is present, and it is the *other* writer's that is gone. With A and B from the same `M0`: A writes `MA`, B writes `MB` and clobbers A; B verifies and correctly sees `MB`; A verifies, sees `M2 !== MA`, re-applies onto `MB`, writes again. Both survive, and the writer that was clobbered is the one that repairs it. Two things this deliberately does not claim: - It is **not mutual exclusion**. A process that crashes between its write and its verifying read cannot notice it was clobbered, and nothing else will. That window is strictly narrower than the lock's — which lost updates with every participant alive — and needs no primitive Node lacks. Stated in the code rather than left to be rediscovered in review. - Non-convergence **fails loudly**. After five lost rounds `set` throws rather than returning, because returning would reintroduce the silent loss with extra steps. `delete` stays silent, per the interface contract. `serialize` is now keyed on the resolved path **process-wide** rather than per instance. Two `FileSecretStore`s on one file are ordinary — the resolved store holds one and `absorbFileSecretsIntoKeyring` builds another — and a per-instance queue does not order them against each other at all. The new convergence test caught this on its first run: two instances racing `set` lost an entry. In-process is now correct by construction; the verify covers what it cannot see, a second Inspector process. The hand-off's read → copy → delete was the lock's other user. Its delete is now conditional: the file is re-read immediately before removal and compared against what was copied, and it declines to delete when they differ. Leaving the file costs nothing (keychain-wins makes the next run idempotent) while deleting on a stale read costs a secret. ## Round 9 (two inline, two suppressed) - **`KeyringSecretStore.getStrict` let raw binding errors escape.** Both plaintext migrations keep their source file only when they catch `SecretStoreUnavailableError`, so a transient keychain failure 500'd `GET /api/servers` instead of abandoning the migration. Wrapped, as `set` already does. - **`FileSecretStore` had no strict read at all**, so `secretStoreGetStrict` fell back to the tolerant `get` and mapped every read failure to `null` — leaving the exact inversion the seam was introduced to prevent open on the store it was introduced for. - **`readOnDiskEncryption` reported "encrypted" for an envelope it cannot open.** Naming the cipher is not the same as being openable: a file with no `kdf`/`data`, or a `data` that is not `iv.tag.ciphertext`, produced the quiet reassuring footer for a file whose next save was guaranteed to fail. - **`/api/config` kept a stale descriptor when the resolver returned `undefined`**, because the conditional spread left the startup value standing. Assigned unconditionally now; `c.json` omits the key. Full `npm run ci` green. Signed-off-by: cliffhall <cliff@futurescale.com>
This commit is contained in:
parent
37faf1be88
commit
339aea3729
3 changed files with 528 additions and 751 deletions
|
|
@ -8,16 +8,13 @@
|
|||
* agree with any of those without them being true.
|
||||
*/
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
|
||||
import { existsSync } from "node:fs";
|
||||
import * as crypto from "node:crypto";
|
||||
import * as fs from "node:fs/promises";
|
||||
import * as os from "node:os";
|
||||
import * as path from "node:path";
|
||||
import {
|
||||
acquireLock,
|
||||
FileSecretStore,
|
||||
readSecretFilePermissions,
|
||||
SecretFileLockTimeoutError,
|
||||
SecretFileKeyMismatchError,
|
||||
tightenSecretFilePermissions,
|
||||
} from "@inspector/core/auth/node/file-secret-store.js";
|
||||
|
|
@ -876,380 +873,188 @@ describe("payload shape validation", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("acquireLock", () => {
|
||||
it("excludes a second holder while the first is live", async () => {
|
||||
const target = path.join(tmpDir, "locked.json");
|
||||
const release = await acquireLock(target);
|
||||
let secondTaken = false;
|
||||
const second = acquireLock(target).then((r) => {
|
||||
secondTaken = true;
|
||||
return r;
|
||||
describe("getStrict (round 9)", () => {
|
||||
it("returns the value like get does when the file is readable", async () => {
|
||||
const store = new FileSecretStore({ filePath: filePath() });
|
||||
await store.set("srv", "env:A", "1");
|
||||
expect(await store.getStrict("srv", "env:A")).toBe("1");
|
||||
expect(await store.getStrict("srv", "env:MISSING")).toBe(null);
|
||||
});
|
||||
|
||||
it("throws where get would answer null, so a migration cannot write over a newer value", async () => {
|
||||
// The whole point of the seam: `get` maps an unreadable store to `null`,
|
||||
// and both plaintext migrations *write* on `null` and then delete the
|
||||
// disk copy. Falling back to `get` here left that inversion open on the
|
||||
// very store the seam was added for.
|
||||
const writer = new FileSecretStore({
|
||||
filePath: filePath(),
|
||||
passphrase: "right-key",
|
||||
});
|
||||
// Still held: the waiter must not have been let through.
|
||||
await new Promise((r) => setTimeout(r, 60));
|
||||
expect(secondTaken).toBe(false);
|
||||
await release();
|
||||
await (
|
||||
await second
|
||||
)();
|
||||
expect(secondTaken).toBe(true);
|
||||
});
|
||||
await writer.set("srv", "env:A", "1");
|
||||
|
||||
it("steals a lock left behind by a killed process", async () => {
|
||||
// A container killed with SIGKILL leaves the directory. A lock only a
|
||||
// clean exit can release would turn that into permanently broken writes —
|
||||
// strictly worse than the race it prevents.
|
||||
const target = path.join(tmpDir, "stale.json");
|
||||
const lockDir = `${target}.lock`;
|
||||
await fs.mkdir(lockDir);
|
||||
const old = new Date(Date.now() - 60_000);
|
||||
await fs.utimes(lockDir, old, old);
|
||||
const release = await acquireLock(target);
|
||||
await release();
|
||||
});
|
||||
|
||||
it("creates the parent directory so a first write is locked too", async () => {
|
||||
// The first run has no secrets directory yet — `writeStoreFile` creates
|
||||
// it later. Without creating it here, mkdir failed ENOENT and the lock
|
||||
// was skipped, so two processes' *first* writes could both proceed
|
||||
// unlocked and overwrite each other. First writes are exactly when two
|
||||
// processes are most likely to start together.
|
||||
const target = path.join(tmpDir, "no-such-dir", "secrets.json");
|
||||
const release = await acquireLock(target, { pollMs: 10 });
|
||||
// A real lock was taken: a second acquisition must not be handed out.
|
||||
let secondTaken = false;
|
||||
// Held, not floated: an acquisition still running when the test ends
|
||||
// would race `afterEach` removing the temp directory.
|
||||
const second = acquireLock(target, { pollMs: 10 }).then((r) => {
|
||||
secondTaken = true;
|
||||
return r;
|
||||
const wrongKey = new FileSecretStore({
|
||||
filePath: filePath(),
|
||||
passphrase: "wrong-key",
|
||||
});
|
||||
await new Promise((r) => setTimeout(r, 60));
|
||||
expect(secondTaken).toBe(false);
|
||||
await release();
|
||||
await (
|
||||
await second
|
||||
)();
|
||||
});
|
||||
|
||||
it("releases only a lock it still owns", async () => {
|
||||
// A holder whose work outran the stale threshold must not delete the
|
||||
// next owner's lock on its way out — that would drop a third writer into
|
||||
// a section two processes believed they held.
|
||||
const target = path.join(tmpDir, "owned.json");
|
||||
const lockDir = `${target}.lock`;
|
||||
const release = await acquireLock(target);
|
||||
// Simulate a steal: someone judged us stale, removed our lock, and took
|
||||
// their own.
|
||||
await fs.rm(lockDir, { recursive: true, force: true });
|
||||
await fs.mkdir(lockDir);
|
||||
await fs.writeFile(path.join(lockDir, "owner"), "someone-else", "utf-8");
|
||||
await release();
|
||||
// Their lock is untouched.
|
||||
expect(await fs.readFile(path.join(lockDir, "owner"), "utf-8")).toBe(
|
||||
"someone-else",
|
||||
expect(await wrongKey.get("srv", "env:A")).toBe(null);
|
||||
await expect(wrongKey.getStrict("srv", "env:A")).rejects.toBeInstanceOf(
|
||||
SecretFileKeyMismatchError,
|
||||
);
|
||||
await fs.rm(lockDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("fails the mutation rather than writing past a live holder", async () => {
|
||||
// Falling through to an unlocked write under contention would be the
|
||||
// worst possible moment for it: another writer is provably active. `set`
|
||||
// is allowed to hard-fail, so this surfaces as the documented 503.
|
||||
//
|
||||
// The holder heartbeats faster than it goes stale, so the waiter can
|
||||
// never mistake it for dead and must time out instead.
|
||||
const target = path.join(tmpDir, "contended.json");
|
||||
const timings = {
|
||||
staleMs: 400,
|
||||
timeoutMs: 250,
|
||||
heartbeatMs: 50,
|
||||
pollMs: 10,
|
||||
};
|
||||
const release = await acquireLock(target, timings);
|
||||
try {
|
||||
await expect(acquireLock(target, timings)).rejects.toBeInstanceOf(
|
||||
SecretFileLockTimeoutError,
|
||||
);
|
||||
} finally {
|
||||
await release();
|
||||
}
|
||||
});
|
||||
|
||||
it("does not steal from a holder that is merely slow", async () => {
|
||||
// The heartbeat is what separates "dead" from "slow". Without it a
|
||||
// holder on a slow filesystem loses its lock mid-section and the lost
|
||||
// update comes straight back — so the waiter here must time out rather
|
||||
// than acquire, even though it waits well past the stale threshold.
|
||||
const target = path.join(tmpDir, "slow.json");
|
||||
const release = await acquireLock(target, {
|
||||
staleMs: 120,
|
||||
timeoutMs: 60_000,
|
||||
heartbeatMs: 20,
|
||||
pollMs: 10,
|
||||
it("wraps a filesystem failure as SecretStoreUnavailableError", async () => {
|
||||
// Not a typed refusal — a raw ENOTDIR — which must still arrive as
|
||||
// something the migrations' catch recognises.
|
||||
const blocker = path.join(tmpDir, "blocker");
|
||||
await fs.writeFile(blocker, "x", "utf-8");
|
||||
const store = new FileSecretStore({
|
||||
filePath: path.join(blocker, "secrets.json"),
|
||||
});
|
||||
try {
|
||||
let stolen = false;
|
||||
// Held and settled below rather than floated, so the waiter cannot
|
||||
// still be running when `afterEach` removes the temp directory.
|
||||
const waiter = acquireLock(target, {
|
||||
staleMs: 120,
|
||||
timeoutMs: 400,
|
||||
pollMs: 10,
|
||||
})
|
||||
.then(async (r) => {
|
||||
stolen = true;
|
||||
await r();
|
||||
})
|
||||
.catch(() => {
|
||||
// Timed out, which is the expected outcome.
|
||||
});
|
||||
await waiter;
|
||||
expect(stolen).toBe(false);
|
||||
} finally {
|
||||
await release();
|
||||
}
|
||||
});
|
||||
|
||||
it("does not steal a stale lock whose owner process is still alive", async () => {
|
||||
// An old mtime means the holder has not heartbeated lately, and there are
|
||||
// two very different reasons: it died, or it is alive but not running —
|
||||
// SIGSTOPed, event-loop-blocked, or on a machine that suspended. Taking
|
||||
// the lock on elapsed time alone would pull it out from under a live
|
||||
// critical section, which is the lost update the lock exists to prevent.
|
||||
const target = path.join(tmpDir, "suspended.json");
|
||||
const lockDir = `${target}.lock`;
|
||||
await fs.mkdir(lockDir, { recursive: true });
|
||||
// Stamp it with *this* process, which is definitionally alive, and age it
|
||||
// well past stale — the shape of a suspended holder.
|
||||
await fs.writeFile(
|
||||
path.join(lockDir, "owner"),
|
||||
JSON.stringify({
|
||||
token: "someone",
|
||||
pid: process.pid,
|
||||
host: os.hostname(),
|
||||
}),
|
||||
"utf-8",
|
||||
await expect(store.getStrict("srv", "env:A")).rejects.toBeInstanceOf(
|
||||
SecretStoreUnavailableError,
|
||||
);
|
||||
const old = new Date(Date.now() - 60_000);
|
||||
await fs.utimes(lockDir, old, old);
|
||||
|
||||
await expect(
|
||||
acquireLock(target, { staleMs: 50, timeoutMs: 150, pollMs: 10 }),
|
||||
).rejects.toBeInstanceOf(SecretFileLockTimeoutError);
|
||||
// Their lock survives the attempt.
|
||||
expect(existsSync(lockDir)).toBe(true);
|
||||
await fs.rm(lockDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("steals a stale lock whose owner process is gone", async () => {
|
||||
// The case the stealing exists for. A pid that no longer resolves is a
|
||||
// dead holder, and refusing to break its lock would turn one killed
|
||||
// container into permanently broken secret writes.
|
||||
const target = path.join(tmpDir, "dead.json");
|
||||
const lockDir = `${target}.lock`;
|
||||
await fs.mkdir(lockDir, { recursive: true });
|
||||
await fs.writeFile(
|
||||
path.join(lockDir, "owner"),
|
||||
// A pid that cannot be running: `kill(0)` answers ESRCH.
|
||||
JSON.stringify({ token: "gone", pid: 2 ** 31 - 1, host: os.hostname() }),
|
||||
"utf-8",
|
||||
);
|
||||
const old = new Date(Date.now() - 60_000);
|
||||
await fs.utimes(lockDir, old, old);
|
||||
|
||||
const release = await acquireLock(target, {
|
||||
staleMs: 50,
|
||||
timeoutMs: 2_000,
|
||||
pollMs: 10,
|
||||
});
|
||||
await release();
|
||||
});
|
||||
});
|
||||
|
||||
describe("readOnDiskEncryption rejects an envelope it could not open", () => {
|
||||
// Naming the cipher is not the same as being openable, and reporting
|
||||
// "encrypted" for a file whose next save is guaranteed to fail is the
|
||||
// quiet, reassuring footer arriving for the worst case.
|
||||
it.each([
|
||||
["not JSON at all", "{ not json"],
|
||||
["JSON that is not an object", '"just-a-string"'],
|
||||
["an object with no token", JSON.stringify({ pid: 1, host: "h" })],
|
||||
])("steals a stale lock whose owner stamp is %s", async (_label, stamp) => {
|
||||
// An unusable stamp cannot answer the liveness question, so the mtime is
|
||||
// all there is — and stealing on it is right, because the alternative is
|
||||
// a lock nothing can ever break.
|
||||
const target = path.join(tmpDir, `stamp-${_label.length}.json`);
|
||||
const lockDir = `${target}.lock`;
|
||||
await fs.mkdir(lockDir, { recursive: true });
|
||||
await fs.writeFile(path.join(lockDir, "owner"), stamp, "utf-8");
|
||||
const old = new Date(Date.now() - 60_000);
|
||||
await fs.utimes(lockDir, old, old);
|
||||
|
||||
const release = await acquireLock(target, {
|
||||
staleMs: 50,
|
||||
timeoutMs: 2_000,
|
||||
pollMs: 10,
|
||||
[
|
||||
"no kdf or data",
|
||||
{ version: 1, encryption: "aes-256-gcm" },
|
||||
"encrypted envelope is missing its kdf or data",
|
||||
],
|
||||
[
|
||||
"data that is not iv.tag.ciphertext",
|
||||
{
|
||||
version: 1,
|
||||
encryption: "aes-256-gcm",
|
||||
kdf: { algorithm: "scrypt", salt: "AAAA", N: 16384, r: 8, p: 1 },
|
||||
data: "only-one-part",
|
||||
},
|
||||
"encrypted payload is not iv.tag.ciphertext",
|
||||
],
|
||||
])("reports unreadable for %s", async (_label, envelope, detail) => {
|
||||
await fs.writeFile(filePath(), JSON.stringify(envelope), "utf-8");
|
||||
const store = new FileSecretStore({ filePath: filePath() });
|
||||
expect(await store.readOnDiskEncryption()).toEqual({
|
||||
state: "unreadable",
|
||||
detail,
|
||||
});
|
||||
await release();
|
||||
});
|
||||
});
|
||||
|
||||
describe("concurrent writers (no lock, optimistic verify-and-retry)", () => {
|
||||
// The lock these replace was removed after three review rounds each found
|
||||
// a real race in it; the last needed a compare-and-swap on a directory
|
||||
// entry that Node does not expose. The store now lets writers collide and
|
||||
// makes the loser notice — so these assert convergence, which is the
|
||||
// property that actually matters, rather than mutual exclusion, which it
|
||||
// no longer claims.
|
||||
|
||||
it("two stores racing set on one file keep both entries", async () => {
|
||||
// The classic lost update: both read the same map, both write. Whichever
|
||||
// lands second clobbers the other's entry — and the clobbered writer's
|
||||
// verifying read is what catches it and re-applies.
|
||||
const a = new FileSecretStore({ filePath: filePath() });
|
||||
const b = new FileSecretStore({ filePath: filePath() });
|
||||
await Promise.all([a.set("srv", "env:A", "1"), b.set("srv", "env:B", "2")]);
|
||||
|
||||
const reader = new FileSecretStore({ filePath: filePath() });
|
||||
expect(await reader.get("srv", "env:A")).toBe("1");
|
||||
expect(await reader.get("srv", "env:B")).toBe("2");
|
||||
});
|
||||
|
||||
it("steals a stale lock whose stamp carries no usable pid", async () => {
|
||||
// A stamp with a non-numeric pid names no process to ask about, so it
|
||||
// cannot vouch for the holder being alive.
|
||||
const target = path.join(tmpDir, "nopid.json");
|
||||
const lockDir = `${target}.lock`;
|
||||
await fs.mkdir(lockDir, { recursive: true });
|
||||
await fs.writeFile(
|
||||
path.join(lockDir, "owner"),
|
||||
JSON.stringify({ token: "t", pid: "not-a-number", host: os.hostname() }),
|
||||
"utf-8",
|
||||
it("keeps every entry when many independent stores write at once", async () => {
|
||||
// Independent instances, so the in-process queue does not serialize them
|
||||
// for us — each is its own writer with its own view of the file.
|
||||
const stores = Array.from(
|
||||
{ length: 8 },
|
||||
() => new FileSecretStore({ filePath: filePath() }),
|
||||
);
|
||||
const old = new Date(Date.now() - 60_000);
|
||||
await fs.utimes(lockDir, old, old);
|
||||
await Promise.all(stores.map((s, i) => s.set("srv", `env:K${i}`, `v${i}`)));
|
||||
|
||||
const release = await acquireLock(target, {
|
||||
staleMs: 50,
|
||||
timeoutMs: 2_000,
|
||||
pollMs: 10,
|
||||
});
|
||||
await release();
|
||||
});
|
||||
|
||||
it("steals a stale lock stamped by another machine", async () => {
|
||||
// A shared filesystem: we cannot ask this host about that host's pid, so
|
||||
// the mtime is all there is. Refusing would make a lock nothing can ever
|
||||
// break.
|
||||
const target = path.join(tmpDir, "remote.json");
|
||||
const lockDir = `${target}.lock`;
|
||||
await fs.mkdir(lockDir, { recursive: true });
|
||||
await fs.writeFile(
|
||||
path.join(lockDir, "owner"),
|
||||
JSON.stringify({
|
||||
token: "elsewhere",
|
||||
pid: process.pid,
|
||||
host: "some-other-host",
|
||||
}),
|
||||
"utf-8",
|
||||
);
|
||||
const old = new Date(Date.now() - 60_000);
|
||||
await fs.utimes(lockDir, old, old);
|
||||
|
||||
const release = await acquireLock(target, {
|
||||
staleMs: 50,
|
||||
timeoutMs: 2_000,
|
||||
pollMs: 10,
|
||||
});
|
||||
await release();
|
||||
});
|
||||
|
||||
it("stops heartbeating a lock that is no longer ours", async () => {
|
||||
// After a steal, the directory belongs to someone else. Continuing to
|
||||
// touch it would keep *their* lock looking alive after they died, which
|
||||
// turns a stale lock into an unbreakable one.
|
||||
const target = path.join(tmpDir, "stolen.json");
|
||||
const lockDir = `${target}.lock`;
|
||||
const release = await acquireLock(target, { heartbeatMs: 10 });
|
||||
// Someone judges us dead and takes it.
|
||||
await fs.writeFile(
|
||||
path.join(lockDir, "owner"),
|
||||
JSON.stringify({ token: "thief", pid: process.pid, host: os.hostname() }),
|
||||
"utf-8",
|
||||
);
|
||||
const afterSteal = new Date(Date.now() - 30_000);
|
||||
await fs.utimes(lockDir, afterSteal, afterSteal);
|
||||
// Give the heartbeat several chances to (wrongly) refresh it.
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
const mtime = (await fs.stat(lockDir)).mtimeMs;
|
||||
expect(Date.now() - mtime).toBeGreaterThan(1_000);
|
||||
// And our release leaves their lock alone, since the token no longer
|
||||
// matches.
|
||||
await release();
|
||||
expect(existsSync(lockDir)).toBe(true);
|
||||
await fs.rm(lockDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("elects a single winner when two waiters find the same stale lock", async () => {
|
||||
// `rm`-then-`mkdir` is not an election: both waiters judge the lock
|
||||
// stale, the first removes it and takes a fresh one, and the second then
|
||||
// removes *that* — putting both inside the section at once, which is the
|
||||
// lost update the takeover exists to prevent. The rename is atomic, so
|
||||
// exactly one waiter can claim a given stale lock.
|
||||
const target = path.join(tmpDir, "elected.json");
|
||||
const lockDir = `${target}.lock`;
|
||||
await fs.mkdir(lockDir, { recursive: true });
|
||||
await fs.writeFile(
|
||||
path.join(lockDir, "owner"),
|
||||
// A pid that cannot be running, so both waiters agree it is stale.
|
||||
JSON.stringify({ token: "dead", pid: 2 ** 31 - 1, host: os.hostname() }),
|
||||
"utf-8",
|
||||
);
|
||||
const old = new Date(Date.now() - 60_000);
|
||||
await fs.utimes(lockDir, old, old);
|
||||
|
||||
let inside = 0;
|
||||
let maxInside = 0;
|
||||
const timings = { staleMs: 50, timeoutMs: 10_000, pollMs: 5 };
|
||||
const run = async () => {
|
||||
const release = await acquireLock(target, timings);
|
||||
inside += 1;
|
||||
maxInside = Math.max(maxInside, inside);
|
||||
await new Promise((r) => setTimeout(r, 40));
|
||||
inside -= 1;
|
||||
await release();
|
||||
};
|
||||
await Promise.all([run(), run()]);
|
||||
// Never two holders at once.
|
||||
expect(maxInside).toBe(1);
|
||||
});
|
||||
|
||||
it("cleans up and reports when it cannot stamp ownership", async () => {
|
||||
// mkdir succeeded, the owner write did not. Returning a no-op release
|
||||
// here would enter the critical section unlocked *and* leave the
|
||||
// directory behind, so every later writer queues behind an orphan nobody
|
||||
// can deliberately release.
|
||||
const target = path.join(tmpDir, "unstampable.json");
|
||||
vi.resetModules();
|
||||
vi.doMock("node:fs/promises", async () => {
|
||||
const actual =
|
||||
await vi.importActual<typeof import("node:fs/promises")>(
|
||||
"node:fs/promises",
|
||||
);
|
||||
return {
|
||||
...actual,
|
||||
default: actual,
|
||||
writeFile: vi.fn((p: string, ...rest: unknown[]) =>
|
||||
String(p).endsWith(`${path.sep}owner`)
|
||||
? Promise.reject(new Error("ENOSPC: no space left on device"))
|
||||
: (actual.writeFile as (...a: unknown[]) => Promise<void>)(
|
||||
p,
|
||||
...rest,
|
||||
),
|
||||
),
|
||||
};
|
||||
});
|
||||
try {
|
||||
const mod =
|
||||
await import("@inspector/core/auth/node/file-secret-store.js");
|
||||
await expect(mod.acquireLock(target)).rejects.toThrow(
|
||||
/Could not initialize the lock/,
|
||||
);
|
||||
// And nothing is left obstructing the next writer.
|
||||
expect(existsSync(`${target}.lock`)).toBe(false);
|
||||
} finally {
|
||||
vi.doUnmock("node:fs/promises");
|
||||
vi.resetModules();
|
||||
const reader = new FileSecretStore({ filePath: filePath() });
|
||||
for (let i = 0; i < stores.length; i++) {
|
||||
expect(await reader.get("srv", `env:K${i}`)).toBe(`v${i}`);
|
||||
}
|
||||
});
|
||||
|
||||
it("serializes concurrent writers through two independent store instances", async () => {
|
||||
// Two `FileSecretStore` objects have separate in-process queues, so this
|
||||
// is the cross-process race in miniature: without the file lock the second
|
||||
// write drops the first one's entry.
|
||||
const target = path.join(tmpDir, "shared.json");
|
||||
const a = new FileSecretStore({ filePath: target });
|
||||
const b = new FileSecretStore({ filePath: target });
|
||||
it("converges when the racing writers are encrypted", async () => {
|
||||
// Same property with a scrypt derivation inside every attempt, which is
|
||||
// where the window is widest.
|
||||
const opts = { filePath: filePath(), passphrase: "hunter2" };
|
||||
const a = new FileSecretStore(opts);
|
||||
const b = new FileSecretStore(opts);
|
||||
await Promise.all([a.set("srv", "env:A", "1"), b.set("srv", "env:B", "2")]);
|
||||
|
||||
const reader = new FileSecretStore(opts);
|
||||
expect(await reader.get("srv", "env:A")).toBe("1");
|
||||
expect(await reader.get("srv", "env:B")).toBe("2");
|
||||
});
|
||||
|
||||
it("a racing delete and set both land", async () => {
|
||||
const store = new FileSecretStore({ filePath: filePath() });
|
||||
await store.set("srv", "env:OLD", "x");
|
||||
|
||||
const a = new FileSecretStore({ filePath: filePath() });
|
||||
const b = new FileSecretStore({ filePath: filePath() });
|
||||
await Promise.all([
|
||||
a.set("alpha", "env:A", "1"),
|
||||
b.set("alpha", "env:B", "2"),
|
||||
a.delete("srv", "env:OLD"),
|
||||
b.set("srv", "env:NEW", "y"),
|
||||
]);
|
||||
expect(await a.get("alpha", "env:A")).toBe("1");
|
||||
expect(await a.get("alpha", "env:B")).toBe("2");
|
||||
|
||||
const reader = new FileSecretStore({ filePath: filePath() });
|
||||
expect(await reader.get("srv", "env:OLD")).toBe(null);
|
||||
expect(await reader.get("srv", "env:NEW")).toBe("y");
|
||||
});
|
||||
|
||||
// A writer that never wins. Simulated by replacing the file *behind* the
|
||||
// store immediately after each write — with a raw `fs` write rather than a
|
||||
// second store, because every store on this path now shares one in-process
|
||||
// queue and writing through one from inside a queued section would
|
||||
// deadlock on it.
|
||||
const clobberAfterEveryWrite = (store: FileSecretStore): void => {
|
||||
const inner = store as unknown as {
|
||||
writeMap: (m: Record<string, string>) => Promise<void>;
|
||||
};
|
||||
const realWrite = inner.writeMap.bind(store);
|
||||
let n = 0;
|
||||
inner.writeMap = async (map: Record<string, string>) => {
|
||||
await realWrite(map);
|
||||
await fs.writeFile(
|
||||
filePath(),
|
||||
JSON.stringify({
|
||||
version: 1,
|
||||
encryption: "none",
|
||||
secrets: { "srv:env:INTRUDER": `v${n++}` },
|
||||
}),
|
||||
"utf-8",
|
||||
);
|
||||
};
|
||||
};
|
||||
|
||||
it("reports non-convergence instead of silently dropping the value", async () => {
|
||||
// Returning after N attempts would reintroduce the silent loss with
|
||||
// extra steps, so this must reject — and say which value was not saved.
|
||||
const store = new FileSecretStore({ filePath: filePath() });
|
||||
clobberAfterEveryWrite(store);
|
||||
|
||||
await expect(store.set("srv", "env:MINE", "1")).rejects.toThrow(
|
||||
/kept overwriting it/,
|
||||
);
|
||||
});
|
||||
|
||||
it("a non-convergent delete stays silent, per the interface contract", async () => {
|
||||
// `delete` reports nothing by contract — only `set` hard-fails — so a
|
||||
// delete that cannot converge must still resolve rather than throw.
|
||||
const store = new FileSecretStore({ filePath: filePath() });
|
||||
await store.set("srv", "env:A", "1");
|
||||
clobberAfterEveryWrite(store);
|
||||
|
||||
await expect(store.delete("srv", "env:A")).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -772,35 +772,58 @@ describe("absorbFileSecretsIntoKeyring", () => {
|
|||
);
|
||||
});
|
||||
|
||||
it("holds the file lock for the whole hand-off", async () => {
|
||||
// Read → copy → delete must be one transaction. Another Inspector
|
||||
// completing a `set` between the read and the delete would have its
|
||||
// brand-new secret removed without ever being copied — a write that
|
||||
// reported success and then vanished.
|
||||
it("declines to delete a file that changed while the hand-off ran", async () => {
|
||||
// Read → copy → delete is no longer one transaction: the store dropped
|
||||
// its cross-process lock in favour of optimistic writes, so another
|
||||
// Inspector can complete a `set` between the read and the delete. That
|
||||
// entry has not been copied, so deleting the file would discard a write
|
||||
// that reported success. The hand-off re-reads immediately before the
|
||||
// delete and backs off when the contents moved.
|
||||
const filePath = await seedFile({ "srv:oauthClientSecret": "from-file" });
|
||||
process.env.MCP_INSPECTOR_SECRET_FILE = filePath;
|
||||
const warn = vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||
const mod = await loadWithProbe(true);
|
||||
|
||||
// Stand in for the other Inspector: write to the file *during* the copy,
|
||||
// which is exactly the window the old lock covered.
|
||||
const keyring = new InMemorySecretStore();
|
||||
const { FileSecretStore } =
|
||||
await import("@inspector/core/auth/node/file-secret-store.js");
|
||||
const original = keyring.set.bind(keyring);
|
||||
let intruded = false;
|
||||
keyring.set = async (id: string, field: string, value: string) => {
|
||||
await original(id, field, value);
|
||||
if (!intruded) {
|
||||
intruded = true;
|
||||
await new FileSecretStore({ filePath }).set("other", "env:LATE", "z");
|
||||
}
|
||||
};
|
||||
|
||||
await mod.absorbFileSecretsIntoKeyring(keyring);
|
||||
|
||||
// The file survives, and so does the entry that arrived late.
|
||||
expect(existsSync(filePath)).toBe(true);
|
||||
expect(
|
||||
await new FileSecretStore({ filePath }).get("other", "env:LATE"),
|
||||
).toBe("z");
|
||||
expect(warn.mock.calls.flat().join("\n")).toMatch(
|
||||
/changed while that ran, so it has been left in place/,
|
||||
);
|
||||
});
|
||||
|
||||
it("deletes the file when nothing changed under it", async () => {
|
||||
// The ordinary path: no concurrent writer, so the comparison matches and
|
||||
// the now-redundant file goes away.
|
||||
const filePath = await seedFile({ "srv:oauthClientSecret": "from-file" });
|
||||
process.env.MCP_INSPECTOR_SECRET_FILE = filePath;
|
||||
vi.spyOn(console, "warn").mockImplementation(() => {});
|
||||
const mod = await loadWithProbe(true);
|
||||
|
||||
const { acquireLock } =
|
||||
await import("@inspector/core/auth/node/file-secret-store.js");
|
||||
// Hold the lock as the "other process" would.
|
||||
const release = await acquireLock(filePath, { heartbeatMs: 20 });
|
||||
let finished = false;
|
||||
const handOff = mod
|
||||
.absorbFileSecretsIntoKeyring(new InMemorySecretStore())
|
||||
.then(() => {
|
||||
finished = true;
|
||||
});
|
||||
const keyring = new InMemorySecretStore();
|
||||
await mod.absorbFileSecretsIntoKeyring(keyring);
|
||||
|
||||
await new Promise((r) => setTimeout(r, 120));
|
||||
// Blocked on us, not steaming ahead and deleting the file.
|
||||
expect(finished).toBe(false);
|
||||
expect(existsSync(filePath)).toBe(true);
|
||||
|
||||
await release();
|
||||
await handOff;
|
||||
expect(finished).toBe(true);
|
||||
expect(existsSync(filePath)).toBe(false);
|
||||
expect(await keyring.get("srv", "oauthClientSecret")).toBe("from-file");
|
||||
});
|
||||
|
||||
it("runs the hand-off for an explicitly configured keyring too", async () => {
|
||||
|
|
|
|||
|
|
@ -378,6 +378,43 @@ describe("KeyringSecretStore (mocked native bindings)", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("getStrict wraps its failures (round 9)", () => {
|
||||
// "Strict" must mean the caller *learns* the read failed, in a form it
|
||||
// can act on. Both plaintext migrations keep their source file only when
|
||||
// they catch `SecretStoreUnavailableError`; a raw binding error escaping
|
||||
// here sails past that and 500s `GET /api/servers` instead of abandoning
|
||||
// the migration.
|
||||
it("returns the value when the keychain answers", async () => {
|
||||
await store.set("alpha", SECRET_FIELD_OAUTH_CLIENT_SECRET, "shh");
|
||||
expect(
|
||||
await store.getStrict("alpha", SECRET_FIELD_OAUTH_CLIENT_SECRET),
|
||||
).toBe("shh");
|
||||
});
|
||||
|
||||
it("throws the typed error when the read itself fails", async () => {
|
||||
keyringMocks.failures.getThrows = true;
|
||||
await expect(
|
||||
store.getStrict("alpha", SECRET_FIELD_OAUTH_CLIENT_SECRET),
|
||||
).rejects.toBeInstanceOf(KeychainUnavailableError);
|
||||
});
|
||||
|
||||
it("throws the typed error when AsyncEntry construction fails", async () => {
|
||||
// #1848's shape: the platform store is unreachable, and it is the
|
||||
// constructor that says so.
|
||||
keyringMocks.failures.constructorThrows = true;
|
||||
await expect(
|
||||
store.getStrict("alpha", SECRET_FIELD_OAUTH_CLIENT_SECRET),
|
||||
).rejects.toBeInstanceOf(KeychainUnavailableError);
|
||||
});
|
||||
|
||||
it("still answers null for a genuinely absent entry", async () => {
|
||||
// Strictness is about the *store* failing, not about absence.
|
||||
expect(
|
||||
await store.getStrict("alpha", SECRET_FIELD_OAUTH_CLIENT_SECRET),
|
||||
).toBe(null);
|
||||
});
|
||||
});
|
||||
|
||||
describe("probeKeyringAvailable (#1950)", () => {
|
||||
// The input to the automatic-fallback policy. It has to answer "is the
|
||||
// keychain *usable*", not "did the package import" — the container that
|
||||
|
|
|
|||
Loading…
Reference in a new issue