Split NATS modules: splitter, WORM bloom archive, aggregator, docs-master

Each module has SUMMARY.md and NATS.md (who sends/receives, subject
addresses, payload). Tests: splitter 3, worm 3, aggregator 2.
Forgejo repos pushed separately; this commit keeps them in the monorepo.

packages/docs-master/MESSAGE-FLOWS.md

@@ -0,0 +1,29 @@
+# Message flows
+
+## 1. Create timestamp (async)
+
+1. Zapier → `POST https://zappier-edge/v1/timestamp` (`x-api-key`)
+2. zappier-edge meters → `POST middleware/zapier/v1/timestamp`
+3. splitter: chain sha256; optional `verae.archive.put`
+4. middleware → chain create or existing lookup
+5. middleware publishes `verae.zapier.jobs.watch` `{ jobId, tenantId, … }`
+6. HTTP 202 `{ jobId }` back to Zapier
+7. job-poller consumes watch, GET chain status
+8. On terminal: `verae.zapier.jobs.events`
+9. webhook-deliver POSTs Zapier REST Hook if subscribed
+
+## 2. Wait
+
+Same as (1) but HTTP holds until `jobs.events` or `WAIT_TIMEOUT_MS` → `{ status: pending, jobId }`.
+
+## 3. Hash already registered
+
+Step 4 returns original jobId + original seal. No second chain write. `receipts[0].kind = seal`. Later archive attaches become `receipts[1…]` if `includeAttached`.
+
+## 4. includeAttached
+
+After seal is known, aggregator publishes `verae.archive.query`. Each WORM: bloom miss = no packet; hit = `verae.archive.reply.<correlationId>`. Aggregator merges into wait JSON.
+
+## 5. Multipart attachments
+
+Splitter emits one `archive.put` per file (`kind: file`, `contentSha256`). Chain never stores bytes.

packages/docs-master/README.md

@@ -0,0 +1,21 @@
+# zapier-docs-master
+
+Summaries and NATS interaction map for every Verae Time × Zapier module.
+
+**Live HTML:** https://zapier.georgelambert.org/
+
+**Monorepo:** https://git.georgelambert.org/marchon/master-zapier-plan-draft
+
+## Modules
+
+| Module | Summary | NATS in | NATS out |
+|--------|---------|---------|----------|
+| zappier-edge | Metered HTTPS for Zapier | — | — (HTTPS to middleware) |
+| verae-middleware | Job id + wait HTTP | jobs.events | jobs.watch, archive.query via aggregator |
+| verae-activate | Pushable Zapier app | — | — |
+| verae-zapier-app | Full Zapier nouns | — | — |
+| request-splitter | Hash vs files | — | archive.put |
+| archive-worm | Bloom WORM node | archive.query, archive.put | archive.reply.* (hit only) |
+| archive-aggregator | Merge archive replies | archive.reply.* | archive.query |
+
+Full contracts: `../docs/02-architecture/modules-and-nats.md` and `archive-nats.md` in the monorepo / on the website.

packages/docs-master/archive-nats.md

@@ -0,0 +1,90 @@
+# Archive NATS: jobs, multipart split, hash receipts, WORM bloom fan-out
+
+Zapier never speaks NATS. HTTPS stops at zappier → middleware. Middleware owns jobs, splitting, chain lookup, and archive aggregation.
+
+## End-to-end
+
+```text
+Zapier  --HTTPS-->  zappier (meter, x-api-key)
+                 --HTTPS-->  middleware /zapier/v1/timestamp[/wait]
+                              1. split multipart
+                                 chain: SHA256 only (+ optional public-meta digest)
+                                 archives: public JSON, encrypted private JSON, files
+                              2. publish jobs.watch → return jobId (202)
+                              3. waiters subscribe jobs.events
+                              4. lookup SHA256 on chain (mock or Verae)
+                                 already sealed → original receipt, no new seal
+                                 later attach records → extra receipts
+                              5. if includeAttached:
+                                 publish verae.archive.query
+                                 WORM nodes: bloom miss = silence
+                                 bloom hit = reply
+                                 aggregate until WAIT_ARCHIVE_MS
+                              6. jobs.events completed JSON → waiter / REST Hook
+```
+
+Blockchain stores **hash + time + block + certificate**. Public metadata, encrypted metadata, and file bytes live on **WORM archives**.
+
+## Splitter
+
+`splitRequest(body | multipart)`:
+
+| Field | Destination |
+|-------|-------------|
+| `data` / `sha256` | Chain register or lookup |
+| `publicMetadata` | Archive put (clear) |
+| `privateMetadata` | Archive put (ciphertext) |
+| `files[]` | Archive put; chain gets content hashes + ids |
+| `includeAttached` | Whether wait path queries archives |
+
+## Hash already registered
+
+Return original `jobId` and original seal. Do not write a second chain timestamp.
+
+If later attach jobs exist for that hash, `receipts` is an array: seal first, then attachment receipts in time order.
+
+## Subjects
+
+| Subject | Publisher | Subscriber |
+|---------|-----------|------------|
+| `verae.zapier.jobs.watch` | HTTP edge | job poller |
+| `verae.zapier.jobs.events` | poller | waiter, webhook router |
+| `verae.archive.put` | splitter | archive that owns the shard |
+| `verae.archive.query` | aggregator | **every** archive (not a shared queue group) |
+| `verae.archive.reply.<correlationId>` | archive on bloom hit | aggregator |
+
+Query payload: `{ correlationId, sha256, tenantId, kinds[] }`.  
+Reply payload: `{ archiveId, sha256, records[] }`.
+
+Bloom miss → no reply. Aggregator timeout → complete with whatever arrived.
+
+## WORM archives
+
+Each process holds append-only records + a bloom of SHA256 keys it stores. False positives OK; false negatives must be rare. Bloom is not an ACL — on hit, still check tenant/share.
+
+Harness: three mock archives with overlapping hashes.
+
+## Completed job JSON (wait / hook)
+
+```json
+{
+  "jobId": "…",
+  "status": "completed",
+  "sha256": "…",
+  "receipts": [
+    { "kind": "seal", "timestamp": "…", "certificate": "…", "blockIndex": 42 },
+    { "kind": "metadata-attach", "attachedAt": "…", "publicMetadata": {} }
+  ],
+  "files": [{ "id": "…", "sha256": "…", "archiveId": "archive-b" }],
+  "archivesQueried": true,
+  "archiveReplies": 2
+}
+```
+
+Flag off or all blooms miss → `files` empty, extra receipts omitted.
+
+## Security
+
+- Zapier never connects to NATS or archives.
+- Private metadata only on authenticated archive replies.
+- NS1 NATS stays loopback; use `scripts/nats-tunnel.sh`.

packages/docs-master/composition.md

@@ -0,0 +1,33 @@
+# Composition: zappier commercial edge + Verae adapter
+
+```text
+Users → Zapier UI
+Zapier cloud runs packages/verae-zapier
+  --HTTPS, x-api-key--> packages/zappier /v1/*
+                          (meter, quote, 401/403, usage)
+                          --internal HTTPS--> packages/verae-zapier-middleware /zapier/v1/*
+                                --sync--> api.veraetime.net  or MOCK_VERAE
+                                --NATS--> workers --> Verae + Zapier REST Hooks
+
+Humans → zappier /portal   signup, API key, usage, reloads, invoices
+Ops    → zappier /admin    rate card, tiers, customers, PO invoices
+```
+
+## Ownership
+
+| Concern | Package |
+|---------|---------|
+| API keys, custom pricing, Stripe, invoices, portal | `packages/zappier` |
+| Timestamp/verify/status, NATS, REST Hooks, mock Verae | `packages/verae-zapier-middleware` |
+| Zapier Platform nouns (creates/searches/triggers) | `packages/verae-zapier` |
+| Official SDK reference | `vendor/zapier-platform` |
+
+Zapier never talks to NATS or `api.veraetime.net`.
+
+Do not reimplement Stripe, invoices, or the rate-card UI inside the Verae middleware.
+
+Public keys are zappier `x-api-key` (issued at portal signup). Middleware `zmw_` / `PLAN_LIMITS` stay internal until composition PR 4 removes them from the public path.
+
+Demo zappier routes `/v1/transform` and `/v1/storage` are a metering sandbox, not Verae timestamping or encrypted LTS.
+
+Payment invoices (zappier) are not certified timestamp receipts (Verae feature i).

packages/docs-master/modules-and-nats.md

@@ -0,0 +1,52 @@
+# Modules and NATS message contracts
+
+Master map of every service in the Zapier ↔ middleware ↔ chain ↔ WORM-archive path.
+
+Zapier cloud **never** connects to NATS. Only middleware workers and archives do.
+
+## Who talks to whom (HTTPS vs NATS)
+
+```text
+[Zapier Platform app]  HTTPS  [zappier-edge]  HTTPS  [middleware-http]
+                                                              |
+                         NATS JetStream (NS1 loopback :4222)  |
+          +------------------+------------------+-------------+
+          |                  |                  |
+   [job-poller]      [webhook-deliver]   [archive-aggregator]
+          |                                     |
+          | HTTPS                               | NATS query/reply
+   [verae-chain-client]                         |
+                                          [archive-worm × N]
+                                          bloom miss = silence
+```
+
+## Module catalog
+
+| Module | Repo (Forgejo) | Runtime | Listens | Sends |
+|--------|----------------|---------|---------|-------|
+| zapier-platform-app | `verae-zapier-app` | Zapier cloud | User Zap steps | HTTPS to zappier-edge |
+| zappier-edge | `zappier-edge` | Public HTTPS :3000 | Zapier, portal, admin | HTTPS to middleware |
+| middleware-http | `verae-middleware` | Public HTTPS :3100 | zappier-edge | NATS jobs.watch; HTTP to chain; wait on jobs.events |
+| job-poller | `verae-job-poller` | Worker | `verae.zapier.jobs.watch` | `verae.zapier.jobs.events`; HTTP GET chain status |
+| webhook-deliver | `verae-webhook-deliver` | Worker | `verae.zapier.webhooks.deliver` + events | HTTPS POST Zapier REST Hook |
+| request-splitter | `verae-request-splitter` | In middleware | HTTP body / multipart | chain hash; `verae.archive.put` |
+| archive-aggregator | `verae-archive-aggregator` | Worker / in wait | `includeAttached` on wait | `verae.archive.query`; reads `verae.archive.reply.<id>` |
+| archive-worm | `verae-archive-worm` | N copies | `verae.archive.query`, `verae.archive.put` | `verae.archive.reply.<id>` if bloom hits |
+| verae-chain-client | `verae-chain-client` | Library | — | HTTPS `api.veraetime.net` or MOCK |
+| docs-master | `zapier-docs-master` | Static | — | published on zapier.georgelambert.org |
+
+## Addresses (subjects)
+
+| Address | Kind | Payload (required fields) |
+|---------|------|---------------------------|
+| `verae.zapier.jobs.watch` | JetStream work queue | `tenantId`, `jobId`, `tokenRef`, `enqueuedAt`, `attempt`, `maxAttempts`, `intervalMs`, `traceId` |
+| `verae.zapier.jobs.events` | JetStream events | `event` (`timestamp.completed\|failed\|timeout`), `tenantId`, `jobId`, `status`, `traceId`, `emittedAt` |
+| `verae.zapier.webhooks.deliver` | JetStream work queue | `hookId`, `tenantId`, `targetUrl`, `event`, `payload`, `attempt`, `traceId` |
+| `verae.zapier.usage` | optional | `tenantId`, `action`, `amount`, `at` |
+| `verae.archive.put` | JetStream | `sha256`, `tenantId`, `kind` (`publicMeta\|privateMeta\|file\|tree`), `record`, `traceId` |
+| `verae.archive.query` | pub to **all** archives | `correlationId`, `sha256`, `tenantId`, `kinds[]`, `traceId` |
+| `verae.archive.reply.<correlationId>` | replies | `archiveId`, `sha256`, `records[]`, `traceId` |
+
+## Completed job JSON (back to Zapier)
+
+See `docs/02-architecture/archive-nats.md`. Seal receipts from chain; extra receipts and `files[]` from archive aggregation when `includeAttached` is true.

packages/docs-master/modules/verae-activate/SUMMARY.md

@@ -0,0 +1,11 @@
+# verae-activate (Zapier Platform app)
+
+**Job:** Private Zapier integration you push first. Actions run in Zapier cloud.
+
+**Expects:** User connection (API key optional).
+
+**Sends:** nothing if fields blank (Add Numbers / Echo / SHA256 / Create Timestamp local mock). If `api_base` + `api_key`, HTTPS to zappier-edge.
+
+**Does not** speak NATS.
+
+**Test:** `npm test`; `zapier-platform validate`.

packages/docs-master/modules/verae-archive-aggregator/NATS.md

@@ -0,0 +1,8 @@
+# NATS — verae-archive-aggregator
+
+| Direction | Address | Peer | Body |
+|-----------|---------|------|------|
+| OUT | `verae.archive.query` | all WORM | `{ correlationId, sha256, tenantId, kinds[], traceId }` |
+| IN | `verae.archive.reply.<correlationId>` | WORM hits | `{ archiveId, sha256, records[] }` |
+
+Stop collecting at `WAIT_ARCHIVE_MS`. Silent archives are listed in `silent[]`, not errors.

packages/docs-master/modules/verae-archive-aggregator/SUMMARY.md

@@ -0,0 +1,11 @@
+# verae-archive-aggregator
+
+**Job:** If `includeAttached`, fan out archive query and merge replies into the job JSON.
+
+**Expects:** wait-path flag from middleware-http; replies on `verae.archive.reply.<correlationId>`.
+
+**Sends:** `verae.archive.query` to all WORM nodes.
+
+**Timeout:** missing archives omit slices; chain receipt still succeeds.
+
+**Test:** `npm test` — one silent bloom miss + one hit; merge seal + metadata-attach.

packages/docs-master/modules/verae-archive-worm/NATS.md

@@ -0,0 +1,9 @@
+# NATS — verae-archive-worm
+
+| Direction | Address | From / to | Body |
+|-----------|---------|-----------|------|
+| IN | `verae.archive.put` | splitter | `{ sha256, tenantId, kind, record, traceId }` |
+| IN | `verae.archive.query` | aggregator (broadcast) | `{ correlationId, sha256, tenantId, kinds[], traceId }` |
+| OUT | `verae.archive.reply.<correlationId>` | aggregator | `{ archiveId, sha256, records[], traceId }` **only if bloom.mightHave(sha256)** |
+
+Subscribe to query **without** a shared queue group so every archive sees every query.

packages/docs-master/modules/verae-archive-worm/SUMMARY.md

@@ -0,0 +1,14 @@
+# verae-archive-worm
+
+**Job:** Append-only WORM store of metadata and files. Bloom filter: miss → stay silent on NATS.
+
+**Expects:**
+- `verae.archive.put` — append record, add sha256 to bloom
+- `verae.archive.query` — `{ correlationId, sha256, tenantId, kinds[] }` (every node sees this; **not** a shared queue group)
+
+**Sends:**
+- `verae.archive.reply.<correlationId>` **only on bloom hit** — `{ archiveId, sha256, records[] }`
+
+**Does not** write to the blockchain.
+
+**Test:** `npm test` — miss is `null`; put then query returns records.

packages/docs-master/modules/verae-request-splitter/NATS.md

@@ -0,0 +1,8 @@
+# NATS — verae-request-splitter
+
+| Direction | Address | To | Body |
+|-----------|---------|-----|------|
+| OUT | `verae.archive.put` | each WORM | one message per publicMeta / privateMeta / file |
+| OUT | (in-process) chain | middleware | `{ sha256, hashAlg, data? }` |
+
+No inbound NATS in the in-process embedding.

packages/docs-master/modules/verae-request-splitter/SUMMARY.md

@@ -0,0 +1,13 @@
+# verae-request-splitter
+
+**Job:** Peel a request into (1) chain hash and (2) off-chain archive puts.
+
+**Expects messages from:** middleware-http (in-process call today; later NATS `verae.splitter.in` if split out).
+
+**Sends messages to:**
+- Chain path: `{ sha256, hashAlg, data? }`
+- `verae.archive.put` for each `publicMeta` / `privateMeta` / `file`
+
+**Does not talk to Zapier or NATS job poller.**
+
+**Test:** `npm test` — hash-only has zero puts; two files + public meta → 3 puts.

packages/docs-master/modules/verae-zapier-middleware/NATS.md

@@ -0,0 +1,9 @@
+# NATS — verae-middleware
+
+| Direction | Address | Peer | Body |
+|-----------|---------|------|------|
+| OUT | `verae.zapier.jobs.watch` | job-poller | job watch payload |
+| IN | `verae.zapier.jobs.events` | waiter | terminal status |
+| OUT | (via aggregator) `verae.archive.query` | WORM | attached lookup |
+
+HTTP remains the only Zapier-facing surface.

packages/docs-master/modules/verae-zapier-middleware/SUMMARY.md

@@ -0,0 +1,15 @@
+# verae-middleware (HTTP edge)
+
+**Job:** Public `/zapier/v1/*`. Auth, entitlements, timestamp/wait, hash lookup, objects, receipts. Publishes NATS jobs; waits on events.
+
+**Expects HTTPS from:** zappier-edge (or Zapier if pointed here).
+
+**Sends:**
+- HTTPS to verae-chain-client (`/api/timestamp`, `/api/status`, `/api/verify`)
+- `verae.zapier.jobs.watch` after create
+- waits `verae.zapier.jobs.events`
+- splitter + aggregator when attachments / `includeAttached`
+
+**Returns to requestor:** 202 `{ jobId }` or wait JSON (completed / pending+jobId).
+
+**NATS:** never exposed to Zapier.

packages/docs-master/modules/verae-zapier/SUMMARY.md

@@ -0,0 +1,9 @@
+# verae-zapier-app
+
+**Job:** Full Zapier CLI app: timestamp, wait, verify, batch, job status, hash lookup, REST Hook.
+
+**Expects:** `zmw_` middleware key (today) or zappier key after composition.
+
+**Sends:** HTTPS to middleware `/zapier/v1/*`.
+
+**Does not** speak NATS.

packages/docs-master/modules/zappier/SUMMARY.md

@@ -0,0 +1,11 @@
+# zappier-edge
+
+**Job:** Metered commercial API, portal, admin, Stripe. Zapier’s only public HTTPS hop.
+
+**Expects:** Zapier Platform app with `x-api-key`.
+
+**Sends:** HTTPS to middleware `/zapier/v1/*` (or local mock `/v1/timestamp`, `/v1/add`, `/v1/hashes`, `/v1/receipts`).
+
+**Does not** speak NATS or `api.veraetime.net`.
+
+**Test:** `npm test` (Jest, 177+).