peergos-making-yourself-audit-ready-with-verae-datacubes/source/iceberg-archive.rst

Write-once Iceberg archive
==========================

The deletion problem
--------------------

Encryption at rest stops a disk operator from **reading**.
Hash verification stops a replica from **substituting**.
Timestamped receipts stop a custodian from **back-dating**.
None of those, by themselves, stop a person with legitimate
administrative power from **deleting**.

Deletion is the failure mode that turns a recordkeeping
program into a fine. Off-channel messaging cases at
broker-dealers were, at bottom, failures to **produce**. A
record that existed and was then removed, "cleaned up," or
rotated out of the only copy is, to an examiner, a record
that cannot be produced.

Compliance therefore needs a tier where **ordinary
administrative power does not include delete**.

What "write-once Iceberg" means in this solution
------------------------------------------------

DataCubes --- the encrypted, hash-chained containers --- are
**archived** into an **external, write-once file-storage
solution** built on the **Iceberg** table/file model (a
layout of immutable data files plus a metadata log that
records new snapshots rather than rewriting old ones).

Properties the solution is specified to have:

* **Write once.** A committed archive object is not updated
  in place. A correction is a **new** object with a **new**
  receipt. The original remains.
* **External.** The archive is not the same disk as the
  live Peergos instance. Compromising the live server should
  not confer delete on the archive.
* **Outside any single party's deletion control.** The
  customer operator, Verae, and the hosting vendor are each
  insufficient, by themselves, to erase a committed archive
  file. Practical implementations of that requirement use
  some combination of object-lock / WORM flags, dual-control
  hold, independent cloud or tape accounts, and contractual
  prohibition on early delete --- plus the technical fact
  that Iceberg snapshots are new files, not overwrites.
* **For compliance reasons.** This tier exists because
  retention schedules, legal holds, and examiner production
  require it --- not because it is a convenient backup.

How cubes move into the archive
-------------------------------

A live DataCube is an operational object: it receives new
writes, it is replicated as encrypted IPFS blocks, it is
readable by holders of the right capabilities. On a schedule
and on events (legal hold, period close, exam notice), the
server solution **exports** a cube snapshot:

1. freeze a CID set and cube-chain head;
2. confirm Peergos hash verification of every block in the
   set;
3. register that snapshot's hash with Verae timestamping
   (first registration of *this* snapshot);
4. write the encrypted snapshot into Iceberg as a new
   immutable data file (or set of files) under a snapshot
   id that never reuses a previous id;
5. record, in admin-history, that the archive job ran, who
   authorized it, and which receipt was issued.

A restore from the archive is the same tamper-evident path
as a restore from IPFS: re-fetch, re-verify hashes, decrypt
only with customer keys.

What "outside the control of any party for deletion" is
and is not
-------------------------------------------------------

It is **not** a claim that physics forbids destruction of
every copy in a fire, a court order, or a coordinated
malicious act by every holder of every credential. No
storage system honestly claims that.

It **is** a claim that the **ordinary** delete path --- the
button, the ``rm``, the lifecycle rule, the "empty trash,"
the support ticket to "please remove that customer folder"
--- is not available to any one party acting alone against a
committed archive object. Dual control, object lock, and
separation of the archive account from the live-server
account are the organizational complements to the file
format.

The organization's retention policy still has to name:

* how long a class of record stays in write-once;
* who can place a legal hold that **extends** that period;
* who, under **what dual control**, may allow a record to
  age out **after** the policy period, if aging out is
  legally permitted at all.

Software can refuse a delete API. It cannot write the
retention schedule. Counsel and the records officer do that.

Relation to classical WORM
--------------------------

Broker-dealer Rule 17a-4 and similar texts speak of
non-rewriteable, non-erasable media, originally meaning
optical WORM, later allowing disk with object-lock semantics
and a designated third party. Verae's public comparison is
that conventional WORM is **immutable by policy** (a vendor
setting), whereas Verae **seals** a record with
cryptographic proof. The Iceberg write-once tier is where
those two ideas meet: the file is not overwritten **and**
the snapshot hash is receipted. An examiner can check the
receipt even if they do not trust the vendor's "WORM was
on" screenshot.

What this does, and does not, satisfy
-------------------------------------

This is the **technical portion** of retention and
production: committed cubes are not under a single delete
key, they are hash-checkable, and they carry a time of
archival.

It does not satisfy a retention **policy** that was never
written, a legal hold that was never placed, or a production
request that the organization answers from a laptop copy
instead of from the archive. Those are procedures.