fix(auth): lock the snapshot during hand-off, keep quiet on ERELEASED (#2082)

Copilot review round 7 — two findings, both upheld.

**An in-progress migration snapshot could be adopted by another startup.**
The fast path counts every `*.migrating-*` as work and
`recoverOrphanedSnapshots` adopts them — including one whose owner is still
reading it. It link/unlinks the snapshot back to the live path and re-claims
it under a new name, the owner's hand-off then fails `ENOENT`, and if the
adopting process exits before copying, that healthy session starts with none
of those secrets, recoverable only on some later run.

Fixed the second way the reviewer suggested, not the first. Holding the main
lock across the hand-off also closes it and was tried — it broke "claims the
file atomically, so a later write is not deleted", because it blocks every
ordinary writer for the whole migration and fails their save past the retry
budget. That is a worse regression than the race, since #1950 guarantees a
write completing after the claim survives. The existing test caught it.

So the hand-off takes a lock on the **snapshot** instead, and
`recoverOrphanedSnapshots` skips an orphan whose lock is held. Liveness comes
from the lock rather than from the pid already in the filename: a pid both
outlives its process and recurs — pid 1 on every container start, which
#1950 documents as the reason the name carries a nonce — whereas a lock
expires by itself if its owner dies, so a genuinely abandoned snapshot
becomes adoptable with nothing to clean up. `isFileLockHeld` answers `false`
when it cannot tell, since the alternative to a wrong `false` is never
recovering an abandoned file.

**A compromised lock arrives as `ERELEASED`, and we told the operator to
delete the lock.** `setLockAsCompromised` marks the lock released and drops
its registry entry *before* calling `onCompromised`, so `release()` answers
`ERELEASED` without touching the filesystem — and the directory at that path
is by then the **winner's live lock**. "Remove it by hand" would destroy the
exclusion of a process that did nothing wrong. Handled separately and
silently; `onCompromised` has already reported what happened. A conditional
rather than an early return, since a `return` inside `finally` discards
whatever the body was returning or throwing.

Both new tests verified by mutation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JTHVxSu8AUgHRLvo1ntZ8H
Signed-off-by: cliffhall <cliff@futurescale.com>

core/auth/node/file-lock.ts

@@ -367,6 +367,30 @@ function acquire(
   });
 }
 
+/**
+ * Is someone holding the lock on `filePath` right now?
+ *
+ * Liveness, not ownership: `check` reports a *stale* lock as unheld, so a
+ * holder that died stops counting on its own after {@link STALE_MS} with no
+ * bookkeeping to clean up. That is what makes this usable as an
+ * "is this in progress" test — a pid stamp cannot say it, since a pid both
+ * outlives its process and recurs (pid 1 on every container start).
+ *
+ * Answers `false` when it cannot tell. The callers use this to decide whether
+ * to *leave something alone*, and the alternative to a wrong `false` is
+ * refusing to ever recover an abandoned file.
+ */
+export async function isFileLockHeld(filePath: string): Promise<boolean> {
+  try {
+    return await properLockfile.check(path.resolve(filePath), {
+      realpath: false,
+      stale: STALE_MS,
+    });
+  } catch {
+    return false;
+  }
+}
+
 /**
  * Run `fn` holding an exclusive cross-process lock on `filePath`.
  *
@@ -449,15 +473,29 @@ export async function withSecretFileLock<T>(
       // The body already ran and its result is being returned; turning a
       // completed save into a failure at teardown would be the wrong trade.
       // But a lock left behind is worth explaining.
-      warnOnce(
-        // No branch on the error code, deliberately. Our own guard `stat`s
-        // before removing, so anything reaching here is an `rmdir` that was
-        // refused — and **stale takeover reclaims through that same `rmdir`**,
-        // so whatever blocked ours blocks that too. `ENOTEMPTY` is the
-        // reachable one; `EACCES`, `EPERM` and `EROFS` behave identically.
-        // Promising expiry for "everything else" was wrong for all of them.
-        `Could not release the lock on the secrets file at ${target} (${describeError(err)}). Stale takeover reclaims a lock through the same directory removal, so whatever blocked this blocks that too: saves will keep failing until you remove ${lockPathOf(target)} by hand.`,
-      );
+      //
+      // **`ERELEASED` is not a lock left behind.** When the library's refresh
+      // tick detects a takeover it marks the lock released and drops its
+      // registry entry *before* calling `onCompromised`, so our `release()`
+      // returns `ERELEASED` without touching the filesystem. The directory
+      // sitting there is then the **winner's live lock** — and the message
+      // below would tell the operator to delete it, destroying the exclusion
+      // of a process that did nothing wrong. `onCompromised` has already said
+      // what happened, so there is nothing to add.
+      //
+      // A conditional rather than an early `return`: a `return` inside
+      // `finally` discards whatever the body was returning or throwing.
+      if ((err as NodeJS.ErrnoException).code !== "ERELEASED") {
+        warnOnce(
+          // No branch on the *removal* error code, deliberately. Our own
+          // guard `stat`s before removing, so anything else reaching here is
+          // an `rmdir` that was refused — and **stale takeover reclaims
+          // through that same `rmdir`**, so whatever blocked ours blocks that
+          // too. `ENOTEMPTY` is the reachable one; `EACCES`, `EPERM` and
+          // `EROFS` behave identically. Promising expiry was wrong for all.
+          `Could not release the lock on the secrets file at ${target} (${describeError(err)}). Stale takeover reclaims a lock through the same directory removal, so whatever blocked this blocks that too: saves will keep failing until you remove ${lockPathOf(target)} by hand.`,
+        );
+      }
     }
   }
 }