Split NATS modules: splitter, WORM bloom archive, aggregator, docs-master

Each module has SUMMARY.md and NATS.md (who sends/receives, subject addresses, payload). Tests: splitter 3, worm 3, aggregator 2. Forgejo repos pushed separately; this commit keeps them in the monorepo.
2026-09-11 12:33:40 -04:00 · 2026-09-11 12:33:40 -04:00 · 8f9707a3b7
commit 8f9707a3b7
parent ca693366dc
39 changed files with 893 additions and 0 deletions
--- a/docs/02-architecture/modules-and-nats.md
+++ b/docs/02-architecture/modules-and-nats.md
@ -0,0 +1,52 @@
+# Modules and NATS message contracts
+
+Master map of every service in the Zapier ↔ middleware ↔ chain ↔ WORM-archive path.
+
+Zapier cloud **never** connects to NATS. Only middleware workers and archives do.
+
+## Who talks to whom (HTTPS vs NATS)
+
+```text
+[Zapier Platform app]  HTTPS  [zappier-edge]  HTTPS  [middleware-http]
+                                                              |
+                         NATS JetStream (NS1 loopback :4222)  |
+          +------------------+------------------+-------------+
+          |                  |                  |
+   [job-poller]      [webhook-deliver]   [archive-aggregator]
+          |                                     |
+          | HTTPS                               | NATS query/reply
+   [verae-chain-client]                         |
+                                          [archive-worm × N]
+                                          bloom miss = silence
+```
+
+## Module catalog
+
+| Module | Repo (Forgejo) | Runtime | Listens | Sends |
+|--------|----------------|---------|---------|-------|
+| zapier-platform-app | `verae-zapier-app` | Zapier cloud | User Zap steps | HTTPS to zappier-edge |
+| zappier-edge | `zappier-edge` | Public HTTPS :3000 | Zapier, portal, admin | HTTPS to middleware |
+| middleware-http | `verae-middleware` | Public HTTPS :3100 | zappier-edge | NATS jobs.watch; HTTP to chain; wait on jobs.events |
+| job-poller | `verae-job-poller` | Worker | `verae.zapier.jobs.watch` | `verae.zapier.jobs.events`; HTTP GET chain status |
+| webhook-deliver | `verae-webhook-deliver` | Worker | `verae.zapier.webhooks.deliver` + events | HTTPS POST Zapier REST Hook |
+| request-splitter | `verae-request-splitter` | In middleware | HTTP body / multipart | chain hash; `verae.archive.put` |
+| archive-aggregator | `verae-archive-aggregator` | Worker / in wait | `includeAttached` on wait | `verae.archive.query`; reads `verae.archive.reply.<id>` |
+| archive-worm | `verae-archive-worm` | N copies | `verae.archive.query`, `verae.archive.put` | `verae.archive.reply.<id>` if bloom hits |
+| verae-chain-client | `verae-chain-client` | Library | — | HTTPS `api.veraetime.net` or MOCK |
+| docs-master | `zapier-docs-master` | Static | — | published on zapier.georgelambert.org |
+
+## Addresses (subjects)
+
+| Address | Kind | Payload (required fields) |
+|---------|------|---------------------------|
+| `verae.zapier.jobs.watch` | JetStream work queue | `tenantId`, `jobId`, `tokenRef`, `enqueuedAt`, `attempt`, `maxAttempts`, `intervalMs`, `traceId` |
+| `verae.zapier.jobs.events` | JetStream events | `event` (`timestamp.completed\|failed\|timeout`), `tenantId`, `jobId`, `status`, `traceId`, `emittedAt` |
+| `verae.zapier.webhooks.deliver` | JetStream work queue | `hookId`, `tenantId`, `targetUrl`, `event`, `payload`, `attempt`, `traceId` |
+| `verae.zapier.usage` | optional | `tenantId`, `action`, `amount`, `at` |
+| `verae.archive.put` | JetStream | `sha256`, `tenantId`, `kind` (`publicMeta\|privateMeta\|file\|tree`), `record`, `traceId` |
+| `verae.archive.query` | pub to **all** archives | `correlationId`, `sha256`, `tenantId`, `kinds[]`, `traceId` |
+| `verae.archive.reply.<correlationId>` | replies | `archiveId`, `sha256`, `records[]`, `traceId` |
+
+## Completed job JSON (back to Zapier)
+
+See `docs/02-architecture/archive-nats.md`. Seal receipts from chain; extra receipts and `files[]` from archive aggregation when `includeAttached` is true.
--- a/packages/docs-master/MESSAGE-FLOWS.md
+++ b/packages/docs-master/MESSAGE-FLOWS.md
@ -0,0 +1,29 @@
+# Message flows
+
+## 1. Create timestamp (async)
+
+1. Zapier → `POST https://zappier-edge/v1/timestamp` (`x-api-key`)
+2. zappier-edge meters → `POST middleware/zapier/v1/timestamp`
+3. splitter: chain sha256; optional `verae.archive.put`
+4. middleware → chain create or existing lookup
+5. middleware publishes `verae.zapier.jobs.watch` `{ jobId, tenantId, … }`
+6. HTTP 202 `{ jobId }` back to Zapier
+7. job-poller consumes watch, GET chain status
+8. On terminal: `verae.zapier.jobs.events`
+9. webhook-deliver POSTs Zapier REST Hook if subscribed
+
+## 2. Wait
+
+Same as (1) but HTTP holds until `jobs.events` or `WAIT_TIMEOUT_MS` → `{ status: pending, jobId }`.
+
+## 3. Hash already registered
+
+Step 4 returns original jobId + original seal. No second chain write. `receipts[0].kind = seal`. Later archive attaches become `receipts[1…]` if `includeAttached`.
+
+## 4. includeAttached
+
+After seal is known, aggregator publishes `verae.archive.query`. Each WORM: bloom miss = no packet; hit = `verae.archive.reply.<correlationId>`. Aggregator merges into wait JSON.
+
+## 5. Multipart attachments
+
+Splitter emits one `archive.put` per file (`kind: file`, `contentSha256`). Chain never stores bytes.
--- a/packages/docs-master/README.md
+++ b/packages/docs-master/README.md
@ -0,0 +1,21 @@
+# zapier-docs-master
+
+Summaries and NATS interaction map for every Verae Time × Zapier module.
+
+**Live HTML:** https://zapier.georgelambert.org/
+
+**Monorepo:** https://git.georgelambert.org/marchon/master-zapier-plan-draft
+
+## Modules
+
+| Module | Summary | NATS in | NATS out |
+|--------|---------|---------|----------|
+| zappier-edge | Metered HTTPS for Zapier | — | — (HTTPS to middleware) |
+| verae-middleware | Job id + wait HTTP | jobs.events | jobs.watch, archive.query via aggregator |
+| verae-activate | Pushable Zapier app | — | — |
+| verae-zapier-app | Full Zapier nouns | — | — |
+| request-splitter | Hash vs files | — | archive.put |
+| archive-worm | Bloom WORM node | archive.query, archive.put | archive.reply.* (hit only) |
+| archive-aggregator | Merge archive replies | archive.reply.* | archive.query |
+
+Full contracts: `../docs/02-architecture/modules-and-nats.md` and `archive-nats.md` in the monorepo / on the website.
--- a/packages/docs-master/archive-nats.md
+++ b/packages/docs-master/archive-nats.md
@ -0,0 +1,90 @@
+# Archive NATS: jobs, multipart split, hash receipts, WORM bloom fan-out
+
+Zapier never speaks NATS. HTTPS stops at zappier → middleware. Middleware owns jobs, splitting, chain lookup, and archive aggregation.
+
+## End-to-end
+
+```text
+Zapier  --HTTPS-->  zappier (meter, x-api-key)
+                 --HTTPS-->  middleware /zapier/v1/timestamp[/wait]
+                              1. split multipart
+                                 chain: SHA256 only (+ optional public-meta digest)
+                                 archives: public JSON, encrypted private JSON, files
+                              2. publish jobs.watch → return jobId (202)
+                              3. waiters subscribe jobs.events
+                              4. lookup SHA256 on chain (mock or Verae)
+                                 already sealed → original receipt, no new seal
+                                 later attach records → extra receipts
+                              5. if includeAttached:
+                                 publish verae.archive.query
+                                 WORM nodes: bloom miss = silence
+                                 bloom hit = reply
+                                 aggregate until WAIT_ARCHIVE_MS
+                              6. jobs.events completed JSON → waiter / REST Hook
+```
+
+Blockchain stores **hash + time + block + certificate**. Public metadata, encrypted metadata, and file bytes live on **WORM archives**.
+
+## Splitter
+
+`splitRequest(body | multipart)`:
+
+| Field | Destination |
+|-------|-------------|
+| `data` / `sha256` | Chain register or lookup |
+| `publicMetadata` | Archive put (clear) |
+| `privateMetadata` | Archive put (ciphertext) |
+| `files[]` | Archive put; chain gets content hashes + ids |
+| `includeAttached` | Whether wait path queries archives |
+
+## Hash already registered
+
+Return original `jobId` and original seal. Do not write a second chain timestamp.
+
+If later attach jobs exist for that hash, `receipts` is an array: seal first, then attachment receipts in time order.
+
+## Subjects
+
+| Subject | Publisher | Subscriber |
+|---------|-----------|------------|
+| `verae.zapier.jobs.watch` | HTTP edge | job poller |
+| `verae.zapier.jobs.events` | poller | waiter, webhook router |
+| `verae.archive.put` | splitter | archive that owns the shard |
+| `verae.archive.query` | aggregator | **every** archive (not a shared queue group) |
+| `verae.archive.reply.<correlationId>` | archive on bloom hit | aggregator |
+
+Query payload: `{ correlationId, sha256, tenantId, kinds[] }`.  
+Reply payload: `{ archiveId, sha256, records[] }`.
+
+Bloom miss → no reply. Aggregator timeout → complete with whatever arrived.
+
+## WORM archives
+
+Each process holds append-only records + a bloom of SHA256 keys it stores. False positives OK; false negatives must be rare. Bloom is not an ACL — on hit, still check tenant/share.
+
+Harness: three mock archives with overlapping hashes.
+
+## Completed job JSON (wait / hook)
+
+```json
+{
+  "jobId": "…",
+  "status": "completed",
+  "sha256": "…",
+  "receipts": [
+    { "kind": "seal", "timestamp": "…", "certificate": "…", "blockIndex": 42 },
+    { "kind": "metadata-attach", "attachedAt": "…", "publicMetadata": {} }
+  ],
+  "files": [{ "id": "…", "sha256": "…", "archiveId": "archive-b" }],
+  "archivesQueried": true,
+  "archiveReplies": 2
+}
+```
+
+Flag off or all blooms miss → `files` empty, extra receipts omitted.
+
+## Security
+
+- Zapier never connects to NATS or archives.
+- Private metadata only on authenticated archive replies.
+- NS1 NATS stays loopback; use `scripts/nats-tunnel.sh`.
--- a/packages/docs-master/composition.md
+++ b/packages/docs-master/composition.md
@ -0,0 +1,33 @@
+# Composition: zappier commercial edge + Verae adapter
+
+```text
+Users → Zapier UI
+Zapier cloud runs packages/verae-zapier
+  --HTTPS, x-api-key--> packages/zappier /v1/*
+                          (meter, quote, 401/403, usage)
+                          --internal HTTPS--> packages/verae-zapier-middleware /zapier/v1/*
+                                --sync--> api.veraetime.net  or MOCK_VERAE
+                                --NATS--> workers --> Verae + Zapier REST Hooks
+
+Humans → zappier /portal   signup, API key, usage, reloads, invoices
+Ops    → zappier /admin    rate card, tiers, customers, PO invoices
+```
+
+## Ownership
+
+| Concern | Package |
+|---------|---------|
+| API keys, custom pricing, Stripe, invoices, portal | `packages/zappier` |
+| Timestamp/verify/status, NATS, REST Hooks, mock Verae | `packages/verae-zapier-middleware` |
+| Zapier Platform nouns (creates/searches/triggers) | `packages/verae-zapier` |
+| Official SDK reference | `vendor/zapier-platform` |
+
+Zapier never talks to NATS or `api.veraetime.net`.
+
+Do not reimplement Stripe, invoices, or the rate-card UI inside the Verae middleware.
+
+Public keys are zappier `x-api-key` (issued at portal signup). Middleware `zmw_` / `PLAN_LIMITS` stay internal until composition PR 4 removes them from the public path.
+
+Demo zappier routes `/v1/transform` and `/v1/storage` are a metering sandbox, not Verae timestamping or encrypted LTS.
+
+Payment invoices (zappier) are not certified timestamp receipts (Verae feature i).
--- a/packages/docs-master/modules-and-nats.md
+++ b/packages/docs-master/modules-and-nats.md
@ -0,0 +1,52 @@
+# Modules and NATS message contracts
+
+Master map of every service in the Zapier ↔ middleware ↔ chain ↔ WORM-archive path.
+
+Zapier cloud **never** connects to NATS. Only middleware workers and archives do.
+
+## Who talks to whom (HTTPS vs NATS)
+
+```text
+[Zapier Platform app]  HTTPS  [zappier-edge]  HTTPS  [middleware-http]
+                                                              |
+                         NATS JetStream (NS1 loopback :4222)  |
+          +------------------+------------------+-------------+
+          |                  |                  |
+   [job-poller]      [webhook-deliver]   [archive-aggregator]
+          |                                     |
+          | HTTPS                               | NATS query/reply
+   [verae-chain-client]                         |
+                                          [archive-worm × N]
+                                          bloom miss = silence
+```
+
+## Module catalog
+
+| Module | Repo (Forgejo) | Runtime | Listens | Sends |
+|--------|----------------|---------|---------|-------|
+| zapier-platform-app | `verae-zapier-app` | Zapier cloud | User Zap steps | HTTPS to zappier-edge |
+| zappier-edge | `zappier-edge` | Public HTTPS :3000 | Zapier, portal, admin | HTTPS to middleware |
+| middleware-http | `verae-middleware` | Public HTTPS :3100 | zappier-edge | NATS jobs.watch; HTTP to chain; wait on jobs.events |
+| job-poller | `verae-job-poller` | Worker | `verae.zapier.jobs.watch` | `verae.zapier.jobs.events`; HTTP GET chain status |
+| webhook-deliver | `verae-webhook-deliver` | Worker | `verae.zapier.webhooks.deliver` + events | HTTPS POST Zapier REST Hook |
+| request-splitter | `verae-request-splitter` | In middleware | HTTP body / multipart | chain hash; `verae.archive.put` |
+| archive-aggregator | `verae-archive-aggregator` | Worker / in wait | `includeAttached` on wait | `verae.archive.query`; reads `verae.archive.reply.<id>` |
+| archive-worm | `verae-archive-worm` | N copies | `verae.archive.query`, `verae.archive.put` | `verae.archive.reply.<id>` if bloom hits |
+| verae-chain-client | `verae-chain-client` | Library | — | HTTPS `api.veraetime.net` or MOCK |
+| docs-master | `zapier-docs-master` | Static | — | published on zapier.georgelambert.org |
+
+## Addresses (subjects)
+
+| Address | Kind | Payload (required fields) |
+|---------|------|---------------------------|
+| `verae.zapier.jobs.watch` | JetStream work queue | `tenantId`, `jobId`, `tokenRef`, `enqueuedAt`, `attempt`, `maxAttempts`, `intervalMs`, `traceId` |
+| `verae.zapier.jobs.events` | JetStream events | `event` (`timestamp.completed\|failed\|timeout`), `tenantId`, `jobId`, `status`, `traceId`, `emittedAt` |
+| `verae.zapier.webhooks.deliver` | JetStream work queue | `hookId`, `tenantId`, `targetUrl`, `event`, `payload`, `attempt`, `traceId` |
+| `verae.zapier.usage` | optional | `tenantId`, `action`, `amount`, `at` |
+| `verae.archive.put` | JetStream | `sha256`, `tenantId`, `kind` (`publicMeta\|privateMeta\|file\|tree`), `record`, `traceId` |
+| `verae.archive.query` | pub to **all** archives | `correlationId`, `sha256`, `tenantId`, `kinds[]`, `traceId` |
+| `verae.archive.reply.<correlationId>` | replies | `archiveId`, `sha256`, `records[]`, `traceId` |
+
+## Completed job JSON (back to Zapier)
+
+See `docs/02-architecture/archive-nats.md`. Seal receipts from chain; extra receipts and `files[]` from archive aggregation when `includeAttached` is true.
--- a/packages/docs-master/modules/verae-activate/SUMMARY.md
+++ b/packages/docs-master/modules/verae-activate/SUMMARY.md
@ -0,0 +1,11 @@
+# verae-activate (Zapier Platform app)
+
+**Job:** Private Zapier integration you push first. Actions run in Zapier cloud.
+
+**Expects:** User connection (API key optional).
+
+**Sends:** nothing if fields blank (Add Numbers / Echo / SHA256 / Create Timestamp local mock). If `api_base` + `api_key`, HTTPS to zappier-edge.
+
+**Does not** speak NATS.
+
+**Test:** `npm test`; `zapier-platform validate`.
--- a/packages/docs-master/modules/verae-archive-aggregator/NATS.md
+++ b/packages/docs-master/modules/verae-archive-aggregator/NATS.md
@ -0,0 +1,8 @@
+# NATS — verae-archive-aggregator
+
+| Direction | Address | Peer | Body |
+|-----------|---------|------|------|
+| OUT | `verae.archive.query` | all WORM | `{ correlationId, sha256, tenantId, kinds[], traceId }` |
+| IN | `verae.archive.reply.<correlationId>` | WORM hits | `{ archiveId, sha256, records[] }` |
+
+Stop collecting at `WAIT_ARCHIVE_MS`. Silent archives are listed in `silent[]`, not errors.
--- a/packages/docs-master/modules/verae-archive-aggregator/SUMMARY.md
+++ b/packages/docs-master/modules/verae-archive-aggregator/SUMMARY.md
@ -0,0 +1,11 @@
+# verae-archive-aggregator
+
+**Job:** If `includeAttached`, fan out archive query and merge replies into the job JSON.
+
+**Expects:** wait-path flag from middleware-http; replies on `verae.archive.reply.<correlationId>`.
+
+**Sends:** `verae.archive.query` to all WORM nodes.
+
+**Timeout:** missing archives omit slices; chain receipt still succeeds.
+
+**Test:** `npm test` — one silent bloom miss + one hit; merge seal + metadata-attach.
--- a/packages/docs-master/modules/verae-archive-worm/NATS.md
+++ b/packages/docs-master/modules/verae-archive-worm/NATS.md
@ -0,0 +1,9 @@
+# NATS — verae-archive-worm
+
+| Direction | Address | From / to | Body |
+|-----------|---------|-----------|------|
+| IN | `verae.archive.put` | splitter | `{ sha256, tenantId, kind, record, traceId }` |
+| IN | `verae.archive.query` | aggregator (broadcast) | `{ correlationId, sha256, tenantId, kinds[], traceId }` |
+| OUT | `verae.archive.reply.<correlationId>` | aggregator | `{ archiveId, sha256, records[], traceId }` **only if bloom.mightHave(sha256)** |
+
+Subscribe to query **without** a shared queue group so every archive sees every query.
--- a/packages/docs-master/modules/verae-archive-worm/SUMMARY.md
+++ b/packages/docs-master/modules/verae-archive-worm/SUMMARY.md
@ -0,0 +1,14 @@
+# verae-archive-worm
+
+**Job:** Append-only WORM store of metadata and files. Bloom filter: miss → stay silent on NATS.
+
+**Expects:**
+- `verae.archive.put` — append record, add sha256 to bloom
+- `verae.archive.query` — `{ correlationId, sha256, tenantId, kinds[] }` (every node sees this; **not** a shared queue group)
+
+**Sends:**
+- `verae.archive.reply.<correlationId>` **only on bloom hit** — `{ archiveId, sha256, records[] }`
+
+**Does not** write to the blockchain.
+
+**Test:** `npm test` — miss is `null`; put then query returns records.
--- a/packages/docs-master/modules/verae-request-splitter/NATS.md
+++ b/packages/docs-master/modules/verae-request-splitter/NATS.md
@ -0,0 +1,8 @@
+# NATS — verae-request-splitter
+
+| Direction | Address | To | Body |
+|-----------|---------|-----|------|
+| OUT | `verae.archive.put` | each WORM | one message per publicMeta / privateMeta / file |
+| OUT | (in-process) chain | middleware | `{ sha256, hashAlg, data? }` |
+
+No inbound NATS in the in-process embedding.
--- a/packages/docs-master/modules/verae-request-splitter/SUMMARY.md
+++ b/packages/docs-master/modules/verae-request-splitter/SUMMARY.md
@ -0,0 +1,13 @@
+# verae-request-splitter
+
+**Job:** Peel a request into (1) chain hash and (2) off-chain archive puts.
+
+**Expects messages from:** middleware-http (in-process call today; later NATS `verae.splitter.in` if split out).
+
+**Sends messages to:**
+- Chain path: `{ sha256, hashAlg, data? }`
+- `verae.archive.put` for each `publicMeta` / `privateMeta` / `file`
+
+**Does not talk to Zapier or NATS job poller.**
+
+**Test:** `npm test` — hash-only has zero puts; two files + public meta → 3 puts.
--- a/packages/docs-master/modules/verae-zapier-middleware/NATS.md
+++ b/packages/docs-master/modules/verae-zapier-middleware/NATS.md
@ -0,0 +1,9 @@
+# NATS — verae-middleware
+
+| Direction | Address | Peer | Body |
+|-----------|---------|------|------|
+| OUT | `verae.zapier.jobs.watch` | job-poller | job watch payload |
+| IN | `verae.zapier.jobs.events` | waiter | terminal status |
+| OUT | (via aggregator) `verae.archive.query` | WORM | attached lookup |
+
+HTTP remains the only Zapier-facing surface.
--- a/packages/docs-master/modules/verae-zapier-middleware/SUMMARY.md
+++ b/packages/docs-master/modules/verae-zapier-middleware/SUMMARY.md
@ -0,0 +1,15 @@
+# verae-middleware (HTTP edge)
+
+**Job:** Public `/zapier/v1/*`. Auth, entitlements, timestamp/wait, hash lookup, objects, receipts. Publishes NATS jobs; waits on events.
+
+**Expects HTTPS from:** zappier-edge (or Zapier if pointed here).
+
+**Sends:**
+- HTTPS to verae-chain-client (`/api/timestamp`, `/api/status`, `/api/verify`)
+- `verae.zapier.jobs.watch` after create
+- waits `verae.zapier.jobs.events`
+- splitter + aggregator when attachments / `includeAttached`
+
+**Returns to requestor:** 202 `{ jobId }` or wait JSON (completed / pending+jobId).
+
+**NATS:** never exposed to Zapier.
--- a/packages/docs-master/modules/verae-zapier/SUMMARY.md
+++ b/packages/docs-master/modules/verae-zapier/SUMMARY.md
@ -0,0 +1,9 @@
+# verae-zapier-app
+
+**Job:** Full Zapier CLI app: timestamp, wait, verify, batch, job status, hash lookup, REST Hook.
+
+**Expects:** `zmw_` middleware key (today) or zappier key after composition.
+
+**Sends:** HTTPS to middleware `/zapier/v1/*`.
+
+**Does not** speak NATS.
--- a/packages/docs-master/modules/zappier/SUMMARY.md
+++ b/packages/docs-master/modules/zappier/SUMMARY.md
@ -0,0 +1,11 @@
+# zappier-edge
+
+**Job:** Metered commercial API, portal, admin, Stripe. Zapier’s only public HTTPS hop.
+
+**Expects:** Zapier Platform app with `x-api-key`.
+
+**Sends:** HTTPS to middleware `/zapier/v1/*` (or local mock `/v1/timestamp`, `/v1/add`, `/v1/hashes`, `/v1/receipts`).
+
+**Does not** speak NATS or `api.veraetime.net`.
+
+**Test:** `npm test` (Jest, 177+).
--- a/packages/verae-activate/SUMMARY.md
+++ b/packages/verae-activate/SUMMARY.md
@ -0,0 +1,11 @@
+# verae-activate (Zapier Platform app)
+
+**Job:** Private Zapier integration you push first. Actions run in Zapier cloud.
+
+**Expects:** User connection (API key optional).
+
+**Sends:** nothing if fields blank (Add Numbers / Echo / SHA256 / Create Timestamp local mock). If `api_base` + `api_key`, HTTPS to zappier-edge.
+
+**Does not** speak NATS.
+
+**Test:** `npm test`; `zapier-platform validate`.
--- a/packages/verae-archive-aggregator/NATS.md
+++ b/packages/verae-archive-aggregator/NATS.md
@ -0,0 +1,8 @@
+# NATS — verae-archive-aggregator
+
+| Direction | Address | Peer | Body |
+|-----------|---------|------|------|
+| OUT | `verae.archive.query` | all WORM | `{ correlationId, sha256, tenantId, kinds[], traceId }` |
+| IN | `verae.archive.reply.<correlationId>` | WORM hits | `{ archiveId, sha256, records[] }` |
+
+Stop collecting at `WAIT_ARCHIVE_MS`. Silent archives are listed in `silent[]`, not errors.
--- a/packages/verae-archive-aggregator/SUMMARY.md
+++ b/packages/verae-archive-aggregator/SUMMARY.md
@ -0,0 +1,11 @@
+# verae-archive-aggregator
+
+**Job:** If `includeAttached`, fan out archive query and merge replies into the job JSON.
+
+**Expects:** wait-path flag from middleware-http; replies on `verae.archive.reply.<correlationId>`.
+
+**Sends:** `verae.archive.query` to all WORM nodes.
+
+**Timeout:** missing archives omit slices; chain receipt still succeeds.
+
+**Test:** `npm test` — one silent bloom miss + one hit; merge seal + metadata-attach.
--- a/packages/verae-archive-aggregator/package.json
+++ b/packages/verae-archive-aggregator/package.json
@ -0,0 +1,8 @@
+{
+  "name": "verae-archive-aggregator",
+  "version": "0.1.0",
+  "type": "module",
+  "description": "Aggregate bloom-filtered WORM archive replies into job JSON",
+  "scripts": { "test": "node --test test/*.test.js" },
+  "engines": { "node": ">=20" }
+}
--- a/packages/verae-archive-aggregator/src/aggregate.js
+++ b/packages/verae-archive-aggregator/src/aggregate.js
@ -0,0 +1,53 @@
+/**
+ * Fan-out archive query and merge replies until timeout.
+ * @module aggregate
+ */
+
+/**
+ * @param {object} opts
+ * @param {string} opts.sha256
+ * @param {string} opts.tenantId
+ * @param {string[]} [opts.kinds]
+ * @param {Array<{ archiveId: string, query: (sha: string) => object[]|null }>} opts.archives
+ * @param {number} [opts.timeoutMs=50] unused in in-process mock (archives are sync)
+ * @returns {{ archivesQueried: boolean, archiveReplies: number, records: object[], silent: string[] }}
+ */
+export function aggregateAttached({ sha256, tenantId, kinds = ['publicMeta', 'privateMeta', 'file'], archives }) {
+  const records = [];
+  const silent = [];
+  for (const arch of archives) {
+    const hit = arch.query(sha256);
+    if (hit == null) {
+      silent.push(arch.archiveId);
+      continue;
+    }
+    for (const rec of hit) {
+      if (!kinds.length || kinds.includes(rec.kind)) records.push(rec);
+    }
+  }
+  return {
+    sha256,
+    tenantId,
+    archivesQueried: true,
+    archiveReplies: archives.length - silent.length,
+    silent,
+    records,
+  };
+}
+
+/**
+ * Merge chain receipts with archive records.
+ * @param {object[]} chainReceipts
+ * @param {object[]} archiveRecords
+ */
+export function mergeReceipts(chainReceipts, archiveRecords) {
+  const extra = archiveRecords
+    .filter((r) => r.kind === 'publicMeta' || r.kind === 'privateMeta' || r.kind === 'file')
+    .map((r) => ({
+      kind: r.kind === 'file' ? 'file-attach' : 'metadata-attach',
+      archiveId: r.archiveId,
+      attachedAt: r.storedAt,
+      ...r.record,
+    }));
+  return [...chainReceipts, ...extra];
+}
--- a/packages/verae-archive-aggregator/test/aggregate.test.js
+++ b/packages/verae-archive-aggregator/test/aggregate.test.js
@ -0,0 +1,28 @@
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { WormArchive } from '../../verae-archive-worm/src/archive.js';
+import { aggregateAttached, mergeReceipts } from '../src/aggregate.js';
+
+describe('aggregateAttached', () => {
+  it('silent miss + one hit', () => {
+    const a = new WormArchive('archive-a');
+    const b = new WormArchive('archive-b');
+    const sha = 'c'.repeat(64);
+    b.put({ sha256: sha, kind: 'publicMeta', record: { publicMetadata: { k: 1 } } });
+    const out = aggregateAttached({ sha256: sha, tenantId: 't1', archives: [a, b] });
+    assert.equal(out.archiveReplies, 1);
+    assert.deepEqual(out.silent, ['archive-a']);
+    assert.equal(out.records[0].record.publicMetadata.k, 1);
+  });
+
+  it('merges seal + attach receipts', () => {
+    const seal = [{ kind: 'seal', timestamp: '2026-01-01T00:00:00Z', certificate: 'x' }];
+    const recs = [
+      { kind: 'publicMeta', archiveId: 'archive-b', storedAt: '2026-01-02T00:00:00Z', record: { publicMetadata: { n: 2 } } },
+    ];
+    const merged = mergeReceipts(seal, recs);
+    assert.equal(merged.length, 2);
+    assert.equal(merged[0].kind, 'seal');
+    assert.equal(merged[1].kind, 'metadata-attach');
+  });
+});
--- a/packages/verae-archive-worm/NATS.md
+++ b/packages/verae-archive-worm/NATS.md
@ -0,0 +1,9 @@
+# NATS — verae-archive-worm
+
+| Direction | Address | From / to | Body |
+|-----------|---------|-----------|------|
+| IN | `verae.archive.put` | splitter | `{ sha256, tenantId, kind, record, traceId }` |
+| IN | `verae.archive.query` | aggregator (broadcast) | `{ correlationId, sha256, tenantId, kinds[], traceId }` |
+| OUT | `verae.archive.reply.<correlationId>` | aggregator | `{ archiveId, sha256, records[], traceId }` **only if bloom.mightHave(sha256)** |
+
+Subscribe to query **without** a shared queue group so every archive sees every query.
--- a/packages/verae-archive-worm/SUMMARY.md
+++ b/packages/verae-archive-worm/SUMMARY.md
@ -0,0 +1,14 @@
+# verae-archive-worm
+
+**Job:** Append-only WORM store of metadata and files. Bloom filter: miss → stay silent on NATS.
+
+**Expects:**
+- `verae.archive.put` — append record, add sha256 to bloom
+- `verae.archive.query` — `{ correlationId, sha256, tenantId, kinds[] }` (every node sees this; **not** a shared queue group)
+
+**Sends:**
+- `verae.archive.reply.<correlationId>` **only on bloom hit** — `{ archiveId, sha256, records[] }`
+
+**Does not** write to the blockchain.
+
+**Test:** `npm test` — miss is `null`; put then query returns records.
--- a/packages/verae-archive-worm/package.json
+++ b/packages/verae-archive-worm/package.json
@ -0,0 +1,8 @@
+{
+  "name": "verae-archive-worm",
+  "version": "0.1.0",
+  "type": "module",
+  "description": "Bloom-filtered WORM archive node for off-chain metadata and files",
+  "scripts": { "test": "node --test test/*.test.js" },
+  "engines": { "node": ">=20" }
+}
--- a/packages/verae-archive-worm/src/archive.js
+++ b/packages/verae-archive-worm/src/archive.js
@ -0,0 +1,40 @@
+/**
+ * In-process WORM archive: append-only records + bloom.
+ * @module archive
+ */
+
+import { Bloom } from './bloom.js';
+
+export class WormArchive {
+  /**
+   * @param {string} archiveId
+   */
+  constructor(archiveId) {
+    this.archiveId = archiveId;
+    this.bloom = new Bloom();
+    /** @type {Map<string, object[]>} */
+    this.byHash = new Map();
+  }
+
+  /**
+   * @param {{ sha256: string, kind: string, record: object }} put
+   */
+  put(put) {
+    const key = String(put.sha256).toLowerCase();
+    const list = this.byHash.get(key) ?? [];
+    list.push({ kind: put.kind, record: put.record, archiveId: this.archiveId, storedAt: new Date().toISOString() });
+    this.byHash.set(key, list);
+    this.bloom.add(key);
+  }
+
+  /**
+   * Bloom miss → null (caller must stay silent). Hit → records (may be empty if false positive).
+   * @param {string} sha256
+   * @returns {object[]|null}
+   */
+  query(sha256) {
+    const key = String(sha256).toLowerCase();
+    if (!this.bloom.mightHave(key)) return null;
+    return this.byHash.get(key) ?? [];
+  }
+}
--- a/packages/verae-archive-worm/src/bloom.js
+++ b/packages/verae-archive-worm/src/bloom.js
@ -0,0 +1,46 @@
+/**
+ * Tiny Bloom filter for SHA256 membership (false positives ok).
+ * @module bloom
+ */
+
+import { createHash } from 'node:crypto';
+
+export class Bloom {
+  /**
+   * @param {number} [m=16384] bit count
+   * @param {number} [k=4] hash rounds
+   */
+  constructor(m = 16384, k = 4) {
+    this.m = m;
+    this.k = k;
+    this.bits = Buffer.alloc(Math.ceil(m / 8));
+  }
+
+  /**
+   * @param {string} s
+   * @returns {number[]}
+   */
+  hashes(s) {
+    const digest = createHash('sha256').update(String(s)).digest();
+    const out = [];
+    for (let i = 0; i < this.k; i += 1) {
+      const n = digest.readUInt32BE((i * 4) % 28);
+      out.push(n % this.m);
+    }
+    return out;
+  }
+
+  add(s) {
+    for (const i of this.hashes(s)) {
+      this.bits[i >> 3] |= 1 << (i & 7);
+    }
+  }
+
+  /**
+   * @param {string} s
+   * @returns {boolean} maybe (never a guaranteed yes)
+   */
+  mightHave(s) {
+    return this.hashes(s).every((i) => (this.bits[i >> 3] & (1 << (i & 7))) !== 0);
+  }
+}
--- a/packages/verae-archive-worm/test/archive.test.js
+++ b/packages/verae-archive-worm/test/archive.test.js
@ -0,0 +1,28 @@
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { WormArchive } from '../src/archive.js';
+import { Bloom } from '../src/bloom.js';
+
+describe('WormArchive + Bloom', () => {
+  it('miss stays silent (null)', () => {
+    const a = new WormArchive('archive-a');
+    assert.equal(a.query('a'.repeat(64)), null);
+  });
+
+  it('put then query returns records', () => {
+    const a = new WormArchive('archive-b');
+    const sha = 'b'.repeat(64);
+    a.put({ sha256: sha, kind: 'publicMeta', record: { publicMetadata: { t: 1 } } });
+    const hit = a.query(sha);
+    assert.ok(Array.isArray(hit));
+    assert.equal(hit[0].kind, 'publicMeta');
+    assert.equal(hit[0].archiveId, 'archive-b');
+  });
+
+  it('bloom reports maybe after add', () => {
+    const b = new Bloom();
+    assert.equal(b.mightHave('hello'), false);
+    b.add('hello');
+    assert.equal(b.mightHave('hello'), true);
+  });
+});
--- a/packages/verae-request-splitter/NATS.md
+++ b/packages/verae-request-splitter/NATS.md
@ -0,0 +1,8 @@
+# NATS — verae-request-splitter
+
+| Direction | Address | To | Body |
+|-----------|---------|-----|------|
+| OUT | `verae.archive.put` | each WORM | one message per publicMeta / privateMeta / file |
+| OUT | (in-process) chain | middleware | `{ sha256, hashAlg, data? }` |
+
+No inbound NATS in the in-process embedding.
--- a/packages/verae-request-splitter/SUMMARY.md
+++ b/packages/verae-request-splitter/SUMMARY.md
@ -0,0 +1,13 @@
+# verae-request-splitter
+
+**Job:** Peel a request into (1) chain hash and (2) off-chain archive puts.
+
+**Expects messages from:** middleware-http (in-process call today; later NATS `verae.splitter.in` if split out).
+
+**Sends messages to:**
+- Chain path: `{ sha256, hashAlg, data? }`
+- `verae.archive.put` for each `publicMeta` / `privateMeta` / `file`
+
+**Does not talk to Zapier or NATS job poller.**
+
+**Test:** `npm test` — hash-only has zero puts; two files + public meta → 3 puts.
--- a/packages/verae-request-splitter/package.json
+++ b/packages/verae-request-splitter/package.json
@ -0,0 +1,10 @@
+{
+  "name": "verae-request-splitter",
+  "version": "0.1.0",
+  "type": "module",
+  "description": "Split hash/sign vs attachments for chain vs WORM archives",
+  "scripts": {
+    "test": "node --test test/*.test.js"
+  },
+  "engines": { "node": ">=20" }
+}
--- a/packages/verae-request-splitter/src/splitRequest.js
+++ b/packages/verae-request-splitter/src/splitRequest.js
@ -0,0 +1,71 @@
+/**
+ * Split a Zapier/middleware request into chain vs archive parts.
+ * @module splitRequest
+ */
+
+import { createHash } from 'node:crypto';
+
+/**
+ * @param {string} data
+ * @returns {string}
+ */
+export function sha256Hex(data) {
+  return createHash('sha256').update(String(data), 'utf8').digest('hex');
+}
+
+/**
+ * @param {object} body
+ * @returns {{
+ *   chain: { sha256: string, hashAlg: string, data?: string },
+ *   archivePuts: Array<{ kind: string, sha256: string, record: object }>,
+ *   includeAttached: boolean
+ * }}
+ */
+export function splitRequest(body = {}) {
+  const sha256 = (body.sha256 || (body.data != null ? sha256Hex(body.data) : '')).toLowerCase();
+  if (!sha256) {
+    throw new Error('data or sha256 is required');
+  }
+  const includeAttached = Boolean(body.includeAttached);
+  const archivePuts = [];
+  if (body.publicMetadata && Object.keys(body.publicMetadata).length) {
+    archivePuts.push({
+      kind: 'publicMeta',
+      sha256,
+      record: { publicMetadata: body.publicMetadata },
+    });
+  }
+  if (body.privateMetadata && Object.keys(body.privateMetadata).length) {
+    archivePuts.push({
+      kind: 'privateMeta',
+      sha256,
+      record: { privateMetadata: body.privateMetadata },
+    });
+  }
+  const files = Array.isArray(body.files) ? body.files : [];
+  for (const f of files) {
+    const bytes = f.contentBase64
+      ? Buffer.from(String(f.contentBase64), 'base64')
+      : Buffer.from(String(f.data ?? ''), 'utf8');
+    const fileHash = sha256Hex(bytes);
+    archivePuts.push({
+      kind: 'file',
+      sha256,
+      record: {
+        path: f.path || f.filename || 'file.bin',
+        size: bytes.length,
+        contentSha256: fileHash,
+        contentBase64: bytes.toString('base64'),
+      },
+    });
+  }
+  return {
+    chain: {
+      sha256,
+      hashAlg: body.hashAlg || 'SHA256',
+      data: body.data,
+    },
+    archivePuts,
+    includeAttached,
+  };
+}
--- a/packages/verae-request-splitter/test/splitRequest.test.js
+++ b/packages/verae-request-splitter/test/splitRequest.test.js
@ -0,0 +1,32 @@
+import { describe, it } from 'node:test';
+import assert from 'node:assert/strict';
+import { splitRequest, sha256Hex } from '../src/splitRequest.js';
+
+describe('splitRequest', () => {
+  it('hash-only has no archive puts', () => {
+    const out = splitRequest({ data: 'hello' });
+    assert.equal(out.chain.sha256, sha256Hex('hello'));
+    assert.equal(out.archivePuts.length, 0);
+    assert.equal(out.includeAttached, false);
+  });
+
+  it('splits public metadata and two files onto archive puts', () => {
+    const out = splitRequest({
+      data: 'hello',
+      publicMetadata: { title: 't' },
+      files: [
+        { path: 'a.txt', data: 'A' },
+        { path: 'b.txt', data: 'B' },
+      ],
+      includeAttached: true,
+    });
+    assert.equal(out.archivePuts.length, 3);
+    assert.equal(out.archivePuts.filter((p) => p.kind === 'file').length, 2);
+    assert.equal(out.includeAttached, true);
+    assert.equal(out.chain.sha256, sha256Hex('hello'));
+  });
+
+  it('throws without data or sha256', () => {
+    assert.throws(() => splitRequest({}), /data or sha256/);
+  });
+});
--- a/packages/verae-zapier-middleware/NATS.md
+++ b/packages/verae-zapier-middleware/NATS.md
@ -0,0 +1,9 @@
+# NATS — verae-middleware
+
+| Direction | Address | Peer | Body |
+|-----------|---------|------|------|
+| OUT | `verae.zapier.jobs.watch` | job-poller | job watch payload |
+| IN | `verae.zapier.jobs.events` | waiter | terminal status |
+| OUT | (via aggregator) `verae.archive.query` | WORM | attached lookup |
+
+HTTP remains the only Zapier-facing surface.
--- a/packages/verae-zapier-middleware/SUMMARY.md
+++ b/packages/verae-zapier-middleware/SUMMARY.md
@ -0,0 +1,15 @@
+# verae-middleware (HTTP edge)
+
+**Job:** Public `/zapier/v1/*`. Auth, entitlements, timestamp/wait, hash lookup, objects, receipts. Publishes NATS jobs; waits on events.
+
+**Expects HTTPS from:** zappier-edge (or Zapier if pointed here).
+
+**Sends:**
+- HTTPS to verae-chain-client (`/api/timestamp`, `/api/status`, `/api/verify`)
+- `verae.zapier.jobs.watch` after create
+- waits `verae.zapier.jobs.events`
+- splitter + aggregator when attachments / `includeAttached`
+
+**Returns to requestor:** 202 `{ jobId }` or wait JSON (completed / pending+jobId).
+
+**NATS:** never exposed to Zapier.
--- a/packages/verae-zapier/SUMMARY.md
+++ b/packages/verae-zapier/SUMMARY.md
@ -0,0 +1,9 @@
+# verae-zapier-app
+
+**Job:** Full Zapier CLI app: timestamp, wait, verify, batch, job status, hash lookup, REST Hook.
+
+**Expects:** `zmw_` middleware key (today) or zappier key after composition.
+
+**Sends:** HTTPS to middleware `/zapier/v1/*`.
+
+**Does not** speak NATS.
--- a/packages/zappier/SUMMARY.md
+++ b/packages/zappier/SUMMARY.md
@ -0,0 +1,11 @@
+# zappier-edge
+
+**Job:** Metered commercial API, portal, admin, Stripe. Zapier’s only public HTTPS hop.
+
+**Expects:** Zapier Platform app with `x-api-key`.
+
+**Sends:** HTTPS to middleware `/zapier/v1/*` (or local mock `/v1/timestamp`, `/v1/add`, `/v1/hashes`, `/v1/receipts`).
+
+**Does not** speak NATS or `api.veraetime.net`.
+
+**Test:** `npm test` (Jest, 177+).
--- a/scripts/push-module-repos.sh
+++ b/scripts/push-module-repos.sh
@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+# Create Forgejo repos and push each module (no node_modules).
+set -euo pipefail
+ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+TOKEN_FILE="${FORGEJO_TOKEN_FILE:-/tmp/forgejo-zapier.token}"
+TOKEN="$(cat "$TOKEN_FILE")"
+API="https://git.georgelambert.org/api/v1"
+GITSSH="ssh://git@git.georgelambert.org:2223/marchon"
+export GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=accept-new -p 2223"
+
+create() {
+  local name="$1" desc="$2"
+  curl -sS -X POST "$API/user/repos" \
+    -H "Authorization: token $TOKEN" \
+    -H "Content-Type: application/json" \
+    -d "{\"name\":\"$name\",\"description\":\"$desc\",\"private\":false,\"auto_init\":false}" \
+    >/tmp/forgejo-mod.json || true
+  echo "repo $name"
+}
+
+push_dir() {
+  local src="$1" name="$2"
+  local tmp
+  tmp="$(mktemp -d)"
+  rsync -a --exclude node_modules --exclude dist --exclude data --exclude '*.db' --exclude .git \
+    "$src/" "$tmp/"
+  git -C "$tmp" init -q -b main
+  git -C "$tmp" add -A
+  git -C "$tmp" -c user.email=marchon@gmail.com -c user.name="George Lambert" \
+    commit -q -m "Initial import of $name from zapier monorepo"
+  git -C "$tmp" remote add origin "$GITSSH/${name}.git"
+  git -C "$tmp" push -u origin main
+  git -C "$tmp" push origin main:master || true
+  rm -rf "$tmp"
+  echo "pushed $name"
+}
+
+create zappier-edge "Metered commercial HTTPS edge (portal, admin, Stripe)"
+create verae-middleware "Zapier-facing HTTP middleware + NATS workers"
+create verae-zapier-app "Zapier Platform CLI app (timestamp/verify/hooks)"
+create verae-activate "Activate-now Zapier app (Add Numbers, Echo, SHA256)"
+create verae-request-splitter "Split hash vs attachments for chain vs WORM"
+create verae-archive-worm "Bloom-filtered WORM archive node"
+create verae-archive-aggregator "Aggregate archive replies into job JSON"
+create zapier-docs-master "Master summaries, NATS contracts, message flows"
+
+push_dir "$ROOT/packages/zappier" zappier-edge
+push_dir "$ROOT/packages/verae-zapier-middleware" verae-middleware
+push_dir "$ROOT/packages/verae-zapier" verae-zapier-app
+push_dir "$ROOT/packages/verae-activate" verae-activate
+push_dir "$ROOT/packages/verae-request-splitter" verae-request-splitter
+push_dir "$ROOT/packages/verae-archive-worm" verae-archive-worm
+push_dir "$ROOT/packages/verae-archive-aggregator" verae-archive-aggregator
+push_dir "$ROOT/packages/docs-master" zapier-docs-master
+
+echo ALL_MODULE_REPOS_PUSHED