Document every repo in Sphinx, catalog PDFs, and verae-ops
Some checks are pending
offline / test (push) Waiting to run

Add missing package READMEs, expand Sphinx to all runtime modules,
publish a LaTeX/PDF README book, and add verae-ops for Docker,
Proxmox, VMs, dedicated hardware, and service linking. Catalog
defaults to public PDF links for each module README and samples.
This commit is contained in:
George Lambert 2026-09-11 14:36:17 -04:00
parent 5ad3222def
commit e401635fd6
222 changed files with 4498 additions and 123 deletions

View file

@ -26,6 +26,7 @@ Summaries, NATS contracts, and message flows for every Verae Time × Zapier modu
| **overview** | https://git.georgelambert.org/marchon/overview |
| verae-nats-process | https://git.georgelambert.org/marchon/verae-nats-process |
| **zapier-docs-master** (this repo) | https://git.georgelambert.org/marchon/zapier-docs-master |
| **verae-ops** | https://git.georgelambert.org/marchon/verae-ops |
Clone (SSH port 2223):
@ -48,6 +49,7 @@ Clone (SSH port 2223):
| verae-fleet | Replica floors + monitor | — | — |
| overview | System map | — | — |
| nats-process | Template worker | example.process.in | example.process.out / reply.* |
| ops | Install / Docker / Proxmox / metal | — | — |
## Documents in this repo

View file

@ -0,0 +1,3 @@
# verae-ops NATS
Documentation only. Runtime NATS bind stays private. See https://zapier.georgelambert.org/packages/verae-ops/06-linking-services.pdf

View file

@ -0,0 +1,3 @@
# verae-ops
Install and maintenance documentation: Docker, Proxmox, VMs, dedicated hardware, linking services.

View file

@ -19,6 +19,7 @@ Each runtime piece is its **own git repo** on Forgejo (`git.georgelambert.org`,
| **verae-zapier-simulator** | `packages/verae-zapier-simulator` | Trace console before `zapier-platform push` |
| **zapier-user-docs** | `packages/zapier-user-docs` | Customer signup → register → lookup |
| **zapier-docs-master** | `packages/docs-master` | Per-module `SUMMARY.md` + `NATS.md` |
| **verae-ops** | `packages/verae-ops` | Docker, Proxmox, VMs, dedicated hardware, linking services |
Libraries that are **not** separate Forgejo apps today: `verae-chain-client` (inside middleware), job-poller and webhook-deliver (middleware workers, fleet-spawned).

View file

@ -12,7 +12,8 @@ Operator console (loopback): http://127.0.0.1:3850/ · [CONSOLE.pdf](https://zap
| [01-system.md](01-system.md) | What runs where, in plain language |
| [02-modules-and-repos.md](02-modules-and-repos.md) | Short names of every independent repo |
| [zapier-docs-master](https://git.georgelambert.org/marchon/zapier-docs-master) | One-line SUMMARY + NATS per module |
| [zapier-user-docs](https://git.georgelambert.org/marchon/zapier-user-docs) | Signup → register → lookup |
| [zapier-user-docs](https://zapier.georgelambert.org/user-docs/README.pdf) | Signup → register → lookup |
| [verae-ops](https://zapier.georgelambert.org/packages/verae-ops/README.pdf) | Docker / Proxmox / VM / metal install |
## 2. Architecture
@ -43,12 +44,15 @@ Operator console (loopback): http://127.0.0.1:3850/ · [CONSOLE.pdf](https://zap
| Tree-node lookup (users) | https://zapier.georgelambert.org/packages/zapier-user-docs/09-lookup-tree-nodes.pdf |
| Zapier developer setup | https://zapier.georgelambert.org/docs/04-activate/SETUP-ZAPIER-DEVELOPER.pdf |
| docs-master | https://zapier.georgelambert.org/docs-master/README.pdf |
| Sphinx HTML | https://zapier.georgelambert.org/sphinx/ |
| Sphinx LaTeX PDF | https://zapier.georgelambert.org/sphinx/verae-zapier-modules.pdf |
| Install (verae-ops) | https://zapier.georgelambert.org/packages/verae-ops/README.pdf |
## Independent git repositories
Prefix: `https://git.georgelambert.org/marchon/`
overview · verae-nats-process · master-zapier-plan-draft · zappier-edge · verae-middleware · verae-zapier-app · verae-activate · verae-request-splitter · verae-archive-worm · verae-archive-aggregator · verae-tree-node · verae-fleet · verae-zapier-simulator · zapier-user-docs · zapier-docs-master
overview · verae-nats-process · verae-ops · master-zapier-plan-draft · zappier-edge · verae-middleware · verae-zapier-app · verae-activate · verae-request-splitter · verae-archive-worm · verae-archive-aggregator · verae-tree-node · verae-fleet · verae-zapier-simulator · zapier-user-docs · zapier-docs-master
## Per-module contracts

View file

@ -30,6 +30,7 @@ Zapier → HTTPS `zappier-edge` → HTTPS `verae-middleware` → **NATS cluster*
| Repo | Role |
|------|------|
| [overview](https://git.georgelambert.org/marchon/overview) | This document |
| [verae-ops](https://git.georgelambert.org/marchon/verae-ops) | Docker, Proxmox, VMs, dedicated hardware |
| [verae-nats-process](https://git.georgelambert.org/marchon/verae-nats-process) | **Model** for a new addressed process |
| [master-zapier-plan-draft](https://git.georgelambert.org/marchon/master-zapier-plan-draft) | Monorepo snapshot |
| [zappier-edge](https://git.georgelambert.org/marchon/zappier-edge) | Metered public HTTPS |

View file

@ -6,7 +6,11 @@ This is the integration you **push tomorrow**. It has one action:
It runs **inside Zapier**. You do not need `api.veraetime.net`, NATS, or a public URL.
Step-by-step account setup: [../../docs/04-activate/SETUP-ZAPIER-DEVELOPER.md](../../docs/04-activate/SETUP-ZAPIER-DEVELOPER.md)
**Catalog:** https://zapier.georgelambert.org/packages/verae-activate/README.pdf
**Setup (PDF):** https://zapier.georgelambert.org/docs/04-activate/SETUP-ZAPIER-DEVELOPER.pdf
**Sample action:** `creates/add_numbers.js` — public sheet https://zapier.georgelambert.org/docs/modules/verae-activate/creates/add_numbers.pdf
Step-by-step account setup: [SETUP-ZAPIER-DEVELOPER](https://zapier.georgelambert.org/docs/04-activate/SETUP-ZAPIER-DEVELOPER.pdf)
```bash
cd packages/verae-activate

View file

@ -0,0 +1,29 @@
# verae-archive-aggregator
When a wait path sets `includeAttached`, this process broadcasts `verae.archive.query` and **merges** replies into the job JSON. Silent bloom misses omit slices; the chain receipt still succeeds.
**Forgejo:** https://git.georgelambert.org/marchon/verae-archive-aggregator
**Catalog:** https://zapier.georgelambert.org/packages/verae-archive-aggregator/README.pdf
**NATS:** https://zapier.georgelambert.org/docs-master/modules/verae-archive-aggregator/NATS.pdf
## Run
```bash
cd packages/verae-archive-aggregator
npm test
```
## Addresses
| Direction | Subject |
|-----------|---------|
| out | `verae.archive.query` `{ correlationId, sha256, tenantId, kinds[] }` |
| in | `verae.archive.reply.<correlationId>` (hits only) |
## Depends on
- middleware wait path (`includeAttached` / `includeTree`)
- one or more archive-worm and/or tree-node replicas
- NATS (private)
Timeout: missing archives omit their records; do not fail the seal.

View file

@ -0,0 +1,34 @@
# verae-archive-worm
Bloom-filtered append-only WORM node. Stores metadata and files **off chain**. A bloom **miss is silence** (no NATS reply).
**Forgejo:** https://git.georgelambert.org/marchon/verae-archive-worm
**Catalog:** https://zapier.georgelambert.org/packages/verae-archive-worm/README.pdf
**NATS:** https://zapier.georgelambert.org/docs-master/modules/verae-archive-worm/NATS.pdf
## Run
```bash
cd packages/verae-archive-worm
npm test
```
Fleet keeps **min 3** copies (`keepFloor`). Pause does not count as available.
## Addresses
| Subject | Role |
|---------|------|
| `verae.archive.put` | Append `{ sha256, kind, record }` and add the hash to the bloom |
| `verae.archive.query` | Broadcast (no queue group). Hit → `verae.archive.reply.<correlationId>` |
| miss | **no message** |
Kinds: `publicMeta`, `privateMeta`, `file`. Tree leaves live on **verae-tree-node** (`kind=tree`).
## Depends on
- NATS JetStream reachable on the **private** URL (`NATS_URL`)
- archive-aggregator (query fan-out / merge)
- fleet replica floor
Does not talk to Zapier or the chain.

View file

@ -39,4 +39,6 @@ Operator console (Fleet · Trace · Docs) at http://127.0.0.1:3850/ — [docs/CO
Zapier cloud apps are listed but **not spawned**. NATS on NS1 is **monitored only** (loopback `:4222`, never a public bind).
Install on Docker / Proxmox / metal: https://zapier.georgelambert.org/packages/verae-ops/README.pdf
Clone: `ssh://git@git.georgelambert.org:2223/marchon/verae-fleet.git`

View file

@ -4,7 +4,8 @@ Template for a new **addressed messaging process** on the central Verae NATS.IO
**Repo:** https://git.georgelambert.org/marchon/verae-nats-process
**Clone:** `ssh://git@git.georgelambert.org:2223/marchon/verae-nats-process.git`
**Overview:** https://git.georgelambert.org/marchon/overview
**Overview:** https://zapier.georgelambert.org/overview/README.pdf
**Routing:** https://zapier.georgelambert.org/packages/verae-nats-process/ROUTING.pdf
```bash
npm test
@ -17,4 +18,4 @@ npm start # /health on 127.0.0.1:13900
4. Update `ROUTING.md`; add a fleet service with `min`/`max`.
5. If Zapier needs the result, add HTTPS on **verae-middleware** only.
Zapier cloud must not subscribe. See [overview 06-address-routing](https://git.georgelambert.org/marchon/overview/src/branch/main/06-address-routing.md).
Zapier cloud must not subscribe. See [address routing](https://zapier.georgelambert.org/overview/06-address-routing.pdf).

View file

@ -0,0 +1,64 @@
# 1. Dependencies and data flow
```text
Humans / Zapier cloud
│ HTTPS x-api-key
zappier-edge :3000 (meter, portal, admin, Stripe)
│ HTTPS internal
middleware-http :3100 (/zapier/v1)
├── HTTPS → api.veraetime.net (or MOCK_VERAE)
└── NATS → private cluster
jobs.watch / jobs.events
webhooks.deliver
archive.put / query / reply.*
archive-worm × N tree-node × N
archive-aggregator job-poller
webhook-deliver
```
## Repositories (clone all that you run)
Prefix `ssh://git@git.georgelambert.org:2223/marchon/<name>.git` (SSH **2223**).
| You need | Repo | Catalog README |
|----------|------|----------------|
| Billing HTTPS | `zappier-edge` | https://zapier.georgelambert.org/packages/zappier/README.pdf |
| Zapier adapter | `verae-middleware` | https://zapier.georgelambert.org/packages/verae-zapier-middleware/README.pdf |
| Zapier app (cloud) | `verae-zapier-app` | https://zapier.georgelambert.org/packages/verae-zapier/README.pdf |
| WORM nodes | `verae-archive-worm` | https://zapier.georgelambert.org/packages/verae-archive-worm/README.pdf |
| Merkle leaves | `verae-tree-node` | https://zapier.georgelambert.org/packages/verae-tree-node/README.pdf |
| Archive merge | `verae-archive-aggregator` | https://zapier.georgelambert.org/packages/verae-archive-aggregator/README.pdf |
| Splitter | `verae-request-splitter` | https://zapier.georgelambert.org/packages/verae-request-splitter/README.pdf |
| Replica floors | `verae-fleet` | https://zapier.georgelambert.org/packages/verae-fleet/README.pdf |
| This install guide | `verae-ops` | https://zapier.georgelambert.org/packages/verae-ops/README.pdf |
Optional: `verae-activate` (tiny pushable app), `verae-zapier-simulator` (trace), `overview`, `zapier-docs-master`, `zapier-user-docs`, `verae-nats-process`.
## Runtime dependencies
| Service | Needs |
|---------|--------|
| zappier-edge | Node 20+, SQLite path or volume, `PORT`, optional Stripe keys |
| middleware | Node 22+, `NATS_URL` **or** `NATS_ENABLED=false`, chain URL or `MOCK_VERAE` |
| NATS | JetStream, **no public bind** |
| worm / tree-node | Node, `NATS_URL`, local state dir |
| fleet | Node, `machines.json` (SSH **paths**), loopback `:3850` |
| Zapier cloud app | `MIDDLEWARE_BASE_URL` = **public HTTPS of zappier-edge** (or middleware if you skip metering in lab) |
## Ports (lab defaults)
| Port | Bind | Process |
|------|------|---------|
| 3000 | public or docker publish | zappier-edge |
| 3100 | public or docker publish | middleware |
| 4222 | **private only** | NATS client |
| 6222 | **private only** | NATS cluster |
| 8222 | private / localhost | NATS monitor |
| 3847 | loopback | simulator |
| 3850 | loopback | operator console |
Chain: `https://api.veraetime.net` (prod) or `MOCK_VERAE=true` (lab).

View file

@ -0,0 +1,54 @@
# 2. Docker Compose
`docker-compose.yml` in this repo starts:
- NATS JetStream **3-node cluster** (`nats1``nats3`) on an internal network
- **zappier-edge** `:3000`
- **middleware** `:3100` with `NATS_URL=nats://nats1:4222` and `MOCK_VERAE=true` by default
- **archive-worm** ×3 and **tree-node** ×3 (fleet floors) as long-running Node workers
- **aggregator** + **job-poller** stand-ins via middleware workers when `NATS_ENABLED=true`
NATS ports are **not** published to the host by default (private `verae` network). HTTPS edges are.
## Prereqs
Docker Engine 24+ and Compose v2. Clone **this repo** plus the application repos as siblings, **or** set `MONOREPO` to a checkout of `master-zapier-plan-draft`.
```bash
export MONOREPO=/path/to/master-zapier-plan-draft # contains packages/
docker compose up --build
curl -fsS http://127.0.0.1:3000/health
curl -fsS http://127.0.0.1:3100/health
```
Without `MONOREPO`, Compose expects `../zappier`, `../verae-zapier-middleware`, … (independent clones next to `verae-ops`).
## Link Zapier (cloud) to the lab
1. Publish zappier-edge on a TLS hostname (Caddy/nginx in front of `:3000`).
2. In the Zapier app env: `MIDDLEWARE_BASE_URL=https://<zappier-host>` (edge meters, then proxies).
3. Sign up on `/portal`, copy `x-api-key`.
## Scale
```bash
docker compose up --scale archive-worm=3 --scale tree-node=3
```
Do not publish `4222`. To debug NATS from the host:
```bash
docker compose exec nats1 wget -qO- http://127.0.0.1:8222/healthz
```
## Operator console
Fleet is **not** inside Compose by default (it SSH-spawns). On the Docker host:
```bash
cd $MONOREPO/packages/verae-fleet
NATS_URL=nats://127.0.0.1:4222 # only if you published 4222 on localhost for lab
node src/cli.js serve # http://127.0.0.1:3850/ loopback
```
For Compose-only labs, skip fleet; replica counts come from `--scale`.

View file

@ -0,0 +1,60 @@
# 3. Proxmox (LXC and QEMU)
Target: one **NATS cluster VM/CT** (or three), one **HTTPS edge** VM, one or more **worker** VMs. Fleet SSH-spreads worm/tree-node onto worker hosts.
## Suggested guests
| Guest | OS | vCPU / RAM / disk | Role |
|-------|----|-------------------|------|
| `nats-a` `nats-b` `nats-c` | Debian 12 LXC unprivileged | 1 / 1G / 8G | JetStream, bind `127.0.0.1` **or** a vmbr **private** bridge only |
| `edge` | Debian 12 QEMU | 2 / 2G / 20G | Caddy + zappier-edge + middleware |
| `worker-1` … | Debian 12 LXC | 2 / 2G / 20G | fleet SSH target; worm + tree-node |
Use a **private bridge** (`vmbr1`, RFC1918) for NATS `4222`/`6222`. Public `vmbr0` only on `edge` (80/443).
## NATS on an LXC
```bash
apt-get update && apt-get install -y curl
# official nats-server binary
curl -sSL https://github.com/nats-io/nats-server/releases/latest/download/nats-server-v2.10.26-linux-amd64.tar.gz | tar -xz
install nats-server /usr/local/bin/
```
`/etc/nats/nats.conf`:
```
jetstream {}
listen: 10.10.10.11:4222 # private bridge, not 0.0.0.0 on public
cluster {
name: verae
listen: 10.10.10.11:6222
routes: [ nats-route://10.10.10.12:6222, nats-route://10.10.10.13:6222 ]
}
http: 127.0.0.1:8222
```
systemd: `ExecStart=/usr/local/bin/nats-server -c /etc/nats/nats.conf`.
## Edge VM
Install Node 22, clone `zappier-edge` and `verae-middleware`, Caddy reverse-proxy:
```
zappier.example.com {
reverse_proxy 127.0.0.1:3000
}
middleware.example.com {
reverse_proxy 127.0.0.1:3100
}
```
Middleware env: `NATS_URL=nats://10.10.10.11:4222,nats://10.10.10.12:4222,nats://10.10.10.13:4222`.
## Fleet on Proxmox
On the operator workstation or a jump CT (loopback `:3850`):
`machines.json` entries: `kind=ssh`, `user=root` or `marchon`, `host=10.10.10.2x`, `identityFile=~/.ssh/id_ed25519` (**path only**). Roles: `tree-node,archive-worm`. Capacity ≥ 3 per worker if you want floors on one host.
`ssh-check` from the operator console before enabling `keepFloor`.

View file

@ -0,0 +1,28 @@
# 4. Virtual servers (cloud / hypervisor)
Same topology as Proxmox, without the Proxmox UI: three small VMs or one VM for lab.
## Single VM lab
Install Docker, clone `verae-ops`, `docker compose up`. Put Caddy/nginx TLS in front of `:3000` and `:3100`. Do **not** open 4222 on the security group.
## Three-VM prod-shaped
| VM | Security group | Software |
|----|----------------|----------|
| nats | private SG only 4222/6222 between the three | nats-server -js cluster |
| edge | 80/443 public; 4222 egress to nats SG | zappier + middleware + Caddy |
| workers | private; SSH from operator IP | Node; fleet identity file |
Cloud-init example (edge):
```yaml
packages: [curl, git, ca-certificates]
runcmd:
- curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
- apt-get install -y nodejs
```
Set `MIDDLEWARE_BASE_URL` in Zapier to the **public zappier hostname**.
Snapshots: SQLite (`ZAPPIER_DB`) and worm/tree state directories. NATS JetStream store (`/var/lib/nats`).

View file

@ -0,0 +1,59 @@
# 5. Dedicated hardware
NS1-style: a box that already runs NATS on **127.0.0.1:4222** (never a public bind), plus optional extra hosts for SSH workers.
## This host (control + edges)
1. Node 22+, git (SSH port **2223** to Forgejo).
2. `nats-server -js` listening **127.0.0.1:4222** only (cluster later on a private NIC).
3. Clone `zappier-edge`, `verae-middleware`, `verae-fleet`.
4. systemd units (below) or `verae-fleet serve` which keepFloor-spawns workers.
5. Caddy/nginx for public HTTPS to `:3000` / `:3100`.
6. Operator console: `127.0.0.1:3850` — SSH tunnel if you are off-box:
`ssh -L 3850:127.0.0.1:3850 -L 4222:127.0.0.1:4222 user@ns1`
## systemd (middleware)
```
[Unit]
Description=verae-middleware
After=network.target nats.service
[Service]
WorkingDirectory=/opt/verae/verae-middleware
Environment=PORT=3100
Environment=NATS_ENABLED=true
Environment=NATS_URL=nats://127.0.0.1:4222
Environment=VERAE_API_BASE_URL=https://api.veraetime.net
ExecStart=/usr/bin/node src/index.js
Restart=on-failure
User=verae
[Install]
WantedBy=multi-user.target
```
zappier-edge similar with `PORT=3000` and `WorkingDirectory` of that clone.
## Extra machines
Add to fleet `machines.json`:
```json
{
"id": "ns1",
"kind": "ssh",
"user": "marchon",
"host": "70.88.205.138",
"port": 22,
"identityFile": "~/.ssh/id_ed25519",
"capacity": 8,
"roles": ["tree-node", "archive-worm"]
}
```
Never commit private key bytes. Optional `machines.secrets.json` is gitignored.
## NATS 3-server cluster on metal
Three boxes, private IPs, same `cluster_name`, routes to each other, `listen` on the private IP only. Clients (`NATS_URL`) list all three URLs. Target architecture: https://zapier.georgelambert.org/overview/03-nats-cluster.pdf

View file

@ -0,0 +1,57 @@
# 6. Linking dependent services
Every process learns peers from **environment variables** and **fleet service JSON**. There is no service mesh.
## Environment map
| Process | Variable | Points at |
|---------|----------|-----------|
| Zapier cloud app | `MIDDLEWARE_BASE_URL` | Public **zappier-edge** HTTPS (prod) or middleware (lab) |
| zappier-edge | `ZAPPIER_UPSTREAM` or internal proxy target | middleware `:3100` |
| middleware | `NATS_URL` | `nats://nats1:4222,nats://nats2:4222,nats://nats3:4222` |
| middleware | `VERAE_API_BASE_URL` | `https://api.veraetime.net` |
| middleware | `MOCK_VERAE` | `true` only in lab |
| worm / tree-node / aggregator / poller | `NATS_URL` | same cluster |
| fleet | `NATS_URL` (monitor only) | same; **publicBind=false** |
| fleet SSH | `identityFile` | path on the operator disk |
## HTTPS hops (must work)
```text
curl -H "x-api-key: $KEY" https://zappier.example.com/v1/status
# edge meters, then
curl http://127.0.0.1:3100/health
```
If Zapier gets 401/402, fix keys and the rate card on zappier-edge — not NATS.
## NATS hops (must stay private)
| Address | From | To |
|---------|------|----|
| `verae.zapier.jobs.watch` | middleware | job-poller (queue) |
| `verae.zapier.jobs.events` | poller | middleware wait |
| `verae.zapier.webhooks.deliver` | middleware | webhook-deliver |
| `verae.archive.put` | splitter | worm / tree-node |
| `verae.archive.query` | aggregator | **all** worm/tree (broadcast) |
| `verae.archive.reply.<id>` | worm/tree on **hit only** | aggregator |
Bloom **miss = silence**. Do not “fix” that with a negative reply.
## Fleet floors
https://zapier.georgelambert.org/docs/02-architecture/fleet.pdf
| Service | min (keepFloor) |
|---------|-----------------|
| archive-worm | 3 |
| tree-node | 3 |
| middleware-http | 1 |
| zappier-edge | 1 |
| aggregator / poller / webhook | 1 |
Link a new host: operator console → Add machine → `ssh-check` → reconcile.
## Adding an unplanned function
Copy `verae-nats-process`, pick `verae.<area>.<resource>.*`, add a fleet service, and if Zapier must see the result, add **one HTTPS route on middleware**. Do not teach Zapier NATS. https://zapier.georgelambert.org/overview/06-address-routing.pdf

View file

@ -0,0 +1,45 @@
# 7. Maintenance
## Health
| Check | Command |
|-------|---------|
| Edges | `curl -fsS https://zappier.example.com/health` and middleware `/health` |
| NATS | `curl -fsS http://127.0.0.1:8222/healthz` on each nats node (localhost) |
| Fleet | `node src/cli.js status` or http://127.0.0.1:3850/ (loopback) |
| Tree floor | status: tree-node available ≥ min, none of those paused |
Green / yellow / red on the operator console: https://zapier.georgelambert.org/packages/verae-fleet/docs/CONSOLE.pdf
## Restart
- systemd: `systemctl restart verae-middleware zappier-edge`
- Compose: `docker compose up -d --no-deps middleware`
- One replica: fleet `restart tree-node-1` (floor spawns a replacement if needed)
- Pause is **not** off — paused copies do not count toward `keepFloor`
## Backup
| Data | Where |
|------|--------|
| zappier SQLite | `ZAPPIER_DB` path / volume |
| worm / tree blobs | instance `data/` dirs |
| NATS JetStream | nats store dir |
| fleet overlay | `packages/verae-fleet/data/` (gitignored) |
Do not back up private keys into git. Restore `identityFile` paths on the operator host.
## Upgrade
1. `git pull` each independent repo (or refresh the monorepo snapshot).
2. `npm test` in that package.
3. Restart that unit only; NATS cluster stays up.
4. Zapier Platform app: `zapier-platform push` from `verae-zapier-app` when HTTPS contracts change.
## Logs
journald for systemd; `docker compose logs -f middleware nats1`. Redact API keys (middleware debug redaction).
## Docs
After doc edits: from the monorepo `npm run docs:site && npm run docs:deploy` so https://zapier.georgelambert.org/ stays current (PDF default, Markdown indexes at `/index-md.html`).

View file

@ -0,0 +1,10 @@
# NATS (ops)
Ops does not subscribe. Runtime processes use:
- `NATS_URL` — comma-separated client URLs on a **private** network
- JetStream required (`-js`)
- Cluster routes on port 6222
- **No public bind** of 4222
See [06-linking-services.md](06-linking-services.md) and https://zapier.georgelambert.org/overview/03-nats-cluster.pdf

View file

@ -0,0 +1,44 @@
# verae-ops — install, run, maintain
How to stand up Verae Time × Zapier on **Docker**, **Proxmox**, **generic VMs**, or **dedicated hardware**, and how to **link dependent services**.
**Forgejo:** https://git.georgelambert.org/marchon/verae-ops
**Clone:** `ssh://git@git.georgelambert.org:2223/marchon/verae-ops.git`
**Catalog:** https://zapier.georgelambert.org/packages/verae-ops/README.pdf
**System map:** https://zapier.georgelambert.org/overview/README.pdf
This is the operations repo. Application code lives in the other independent git repositories.
## Reading order
1. [01-dependencies.md](01-dependencies.md) — what must exist, who talks to whom
2. [02-docker.md](02-docker.md) — Compose (NATS 3-node cluster + HTTPS edges)
3. [03-proxmox.md](03-proxmox.md) — LXC / QEMU VMs
4. [04-virtual-servers.md](04-virtual-servers.md) — cloud or hypervisor VMs
5. [05-dedicated-hardware.md](05-dedicated-hardware.md) — bare metal (NS1-style)
6. [06-linking-services.md](06-linking-services.md) — env vars, URLs, replica floors
7. [07-maintenance.md](07-maintenance.md) — upgrade, backup, fleet, NATS
Public PDFs of the same files: `https://zapier.georgelambert.org/packages/verae-ops/<name>.pdf`.
## Non-negotiables
| Rule | Why |
|------|-----|
| Zapier cloud → **HTTPS only** (zappier-edge) | Never NATS, never `api.veraetime.net` |
| NATS binds **loopback or a private docker/VM net** | Not on the public NIC |
| Tree-node **min 3**, pause does not count | Bulk Merkle lookups |
| Private keys stay on disk; git stores **paths** | `machines.json` `identityFile` |
| Operator console is **loopback** `:3850` | Not a public site |
## Quick lab (one machine)
```bash
git clone ssh://git@git.georgelambert.org:2223/marchon/verae-ops.git
cd verae-ops
docker compose up --build
```
Then: zappier http://127.0.0.1:3000/ middleware http://127.0.0.1:3100/health NATS monitoring http://127.0.0.1:8222/
Production layout is three NATS nodes + fleet-spread workers; see [02-docker.md](02-docker.md) and [05-dedicated-hardware.md](05-dedicated-hardware.md).

View file

@ -0,0 +1,7 @@
# verae-ops
**Job:** Installation and maintenance: Docker Compose, Proxmox, virtual servers, dedicated hardware, and how to link dependent services.
**Expects:** clones of the runtime repos (or the monorepo snapshot) and a private NATS network.
**Sends:** nothing at runtime (documentation + compose file).

View file

@ -0,0 +1,115 @@
# Lab / small-prod. NATS stays on the internal network (not published).
# Usage:
# MONOREPO=/path/to/master-zapier-plan-draft docker compose up --build
# or clone sibling repos next to this directory.
x-node: &node
restart: unless-stopped
networks: [verae]
services:
nats1:
image: nats:2.10-alpine
command:
- "--js"
- "--http_port=8222"
- "--cluster=nats://0.0.0.0:6222"
- "--cluster_name=verae"
- "--routes=nats://nats2:6222,nats://nats3:6222"
- "--name=nats1"
networks: [verae]
restart: unless-stopped
nats2:
image: nats:2.10-alpine
command:
- "--js"
- "--cluster=nats://0.0.0.0:6222"
- "--cluster_name=verae"
- "--routes=nats://nats1:6222,nats://nats3:6222"
- "--name=nats2"
networks: [verae]
restart: unless-stopped
nats3:
image: nats:2.10-alpine
command:
- "--js"
- "--cluster=nats://0.0.0.0:6222"
- "--cluster_name=verae"
- "--routes=nats://nats1:6222,nats://nats2:6222"
- "--name=nats3"
networks: [verae]
restart: unless-stopped
zappier-edge:
<<: *node
build:
context: ${MONOREPO:-..}/packages/zappier
dockerfile: Dockerfile
environment:
PORT: "3000"
ZAPPIER_UPSTREAM: http://middleware:3100
ports:
- "3000:3000"
middleware:
<<: *node
build:
context: ${MONOREPO:-..}/packages/verae-zapier-middleware
dockerfile: Dockerfile
environment:
PORT: "3100"
MOCK_VERAE: "true"
NATS_ENABLED: "true"
NATS_URL: nats://nats1:4222
ports:
- "3100:3100"
depends_on: [nats1, nats2, nats3]
archive-worm:
<<: *node
image: node:22-alpine
working_dir: /opt/fleet
volumes:
- ${MONOREPO:-..}/packages/verae-fleet:/opt/fleet
- ${MONOREPO:-..}/packages/verae-archive-worm:/opt/worm
- worm-data:/data
command: ["node", "src/worker.js"]
environment:
NATS_URL: nats://nats1:4222
FLEET_ROLE: archive-worm
FLEET_SERVICE: archive-worm
FLEET_HEALTH_BIND: 0.0.0.0
FLEET_HEALTH_PORT: "13500"
FLEET_STATE_DIR: /data
deploy:
replicas: 3
tree-node:
<<: *node
image: node:22-alpine
working_dir: /opt/fleet
volumes:
- ${MONOREPO:-..}/packages/verae-fleet:/opt/fleet
- ${MONOREPO:-..}/packages/verae-tree-node:/opt/tree
- tree-data:/data
command: ["node", "src/worker.js"]
environment:
NATS_URL: nats://nats1:4222
FLEET_ROLE: tree-node
FLEET_SERVICE: tree-node
FLEET_HEALTH_BIND: 0.0.0.0
FLEET_HEALTH_PORT: "13600"
FLEET_STATE_DIR: /data
deploy:
replicas: 3
networks:
verae:
driver: bridge
internal: false
volumes:
worm-data:
tree-data:

View file

@ -0,0 +1,6 @@
{
"name": "verae-ops",
"version": "0.1.0",
"private": true,
"description": "Install, Docker, Proxmox, VM, dedicated hardware, and service linking for Verae Time × Zapier"
}

View file

@ -0,0 +1,32 @@
# verae-request-splitter
Splits a Zapier/middleware body into (1) the **chain hash** and (2) off-chain **archive puts**. File bytes never go on `api.veraetime.net`.
**Forgejo:** https://git.georgelambert.org/marchon/verae-request-splitter
**Catalog:** https://zapier.georgelambert.org/packages/verae-request-splitter/README.pdf
**NATS:** https://zapier.georgelambert.org/docs-master/modules/verae-request-splitter/NATS.pdf
## Run
```bash
cd packages/verae-request-splitter
npm test
```
Today middleware calls this **in-process**. A later split-out can subscribe `verae.splitter.in`.
## Outputs
| Path | Content |
|------|---------|
| Chain | `{ sha256, hashAlg }` only |
| `verae.archive.put` | each `publicMeta` / `privateMeta` / `file` |
| Batch | Merkle **root** on chain; leaves → tree-node puts (`kind=tree`) |
## Depends on
- middleware-http (caller)
- archive-worm / tree-node (puts)
- chain client (hash seal)
Does not talk to Zapier or the job poller.

View file

@ -1,7 +1,31 @@
# verae-tree-node
External tree-node role: Bloom-filtered WORM store for Merkle **leaf** receipts.
Bloom-filtered WORM store for Merkle **leaf** proofs. Only the Merkle **root** is sealed on the chain.
When Zapier submits a **batch**, middleware builds a Merkle tree of item SHA-256s, seals **only the root** on `api.veraetime.net`, and puts each leaf proof on a sharded tree node. A later lookup of a leaf that is not on the main chain fans out `verae.archive.query` with `kinds: ["tree"]`. Nodes that do not hold the leaf send nothing.
**Forgejo:** https://git.georgelambert.org/marchon/verae-tree-node
**Catalog:** https://zapier.georgelambert.org/packages/verae-tree-node/README.pdf
**User lookup guide:** https://zapier.georgelambert.org/user-docs/09-lookup-tree-nodes.pdf
When Zapier submits a **batch**, middleware builds a Merkle tree of item SHA-256s, seals **only the root** on `api.veraetime.net`, and puts each leaf proof on a sharded tree node. A later lookup of a leaf that is not on the main chain fans out `verae.archive.query` with `kinds: ["tree"]`. Nodes that do not hold the leaf send **nothing**.
## Run
```bash
cd packages/verae-tree-node
npm test
```
Fleet **keepFloor min=3**. Pause does not count toward the floor.
## Addresses
Same as archive-worm, with `kind=tree`: `verae.archive.put` / `query` / `reply.<id>` (hit only).
## Depends on
- NATS (private)
- archive-aggregator for bulk lookup
- fleet replica floor
- chain holds the root only
Clone: `ssh://git@git.georgelambert.org:2223/marchon/verae-tree-node.git`

View file

@ -0,0 +1,40 @@
# verae-middleware
Zapier-facing HTTPS adapter (`/zapier/v1/*`) plus NATS workers. Zapier never speaks NATS or `api.veraetime.net`; this process does.
**Forgejo:** https://git.georgelambert.org/marchon/verae-middleware
**Clone:** `ssh://git@git.georgelambert.org:2223/marchon/verae-middleware.git`
**Catalog README (PDF):** https://zapier.georgelambert.org/packages/verae-zapier-middleware/README.pdf
**NATS contract:** https://zapier.georgelambert.org/docs-master/modules/verae-zapier-middleware/NATS.pdf
**Install / Docker / Proxmox:** https://zapier.georgelambert.org/packages/verae-ops/README.pdf
## Run
```bash
cd packages/verae-zapier-middleware
npm install
MOCK_VERAE=true NATS_ENABLED=false npm test
PORT=3100 MOCK_VERAE=true npm start # /health, /zapier/v1
```
Docker image: `Dockerfile` (Node 22, port 3100). Full stack: `verae-ops` compose.
## Environment
| Variable | Default | Purpose |
|----------|---------|---------|
| `PORT` | `3100` | HTTP bind |
| `VERAE_API_BASE_URL` | `http://localhost:8080` | Chain HTTPS |
| `MOCK_VERAE` | `false` | In-process mock chain |
| `NATS_ENABLED` | `false` | JetStream vs in-process poller |
| `NATS_URL` | `nats://127.0.0.1:4222` | Cluster URL (loopback / docker DNS) |
| `WAIT_TIMEOUT_MS` | — | Sync wait on `jobs.events` |
## Depends on
- **Upstream HTTPS:** zappier-edge (`x-api-key` already checked) or a lab client
- **Chain:** `api.veraetime.net` or `MOCK_VERAE=true`
- **NATS:** `jobs.watch` out, `jobs.events` in; optional `archive.put` / aggregator wait
- **Optional:** request-splitter (in-process), archive-worm, tree-node, webhook-deliver
Does **not** bind NATS publicly. See [verae-ops linking](https://zapier.georgelambert.org/packages/verae-ops/06-linking-services.pdf).

View file

@ -27,4 +27,8 @@ Faults you can inject: edge 401/402, chain timeout, NATS watch/events drop, arch
The monitor panel flags broken assumptions (Zapier talking to NATS, file bytes on chain, bloom miss sending a reply) and suggests changes (tree-node search, wait+hook pairing, 402 upgrade URL, archive quorum).
**Catalog:** https://zapier.georgelambert.org/packages/verae-zapier-simulator/README.pdf
**Live sample UI (loopback):** http://127.0.0.1:3847/ — also inside the operator console Trace tab.
**Static copy:** https://zapier.georgelambert.org/simulator/
Clone: `ssh://git@git.georgelambert.org:2223/marchon/verae-zapier-simulator.git`

View file

@ -1,6 +1,10 @@
# verae-zapier
Zapier Platform CLI app for Verae. **Phase 11** in [../TODO.md](../TODO.md).
Zapier Platform CLI app for Verae. Runs on **Zapiers servers**. Calls only middleware HTTPS (`MIDDLEWARE_BASE_URL`), never NATS and never `api.veraetime.net`.
**Forgejo:** https://git.georgelambert.org/marchon/verae-zapier-app
**Catalog:** https://zapier.georgelambert.org/packages/verae-zapier/README.pdf
**Sphinx API sheets:** https://zapier.georgelambert.org/sphinx/index.html
## Role
@ -19,7 +23,7 @@ Runs on **Zapiers servers**. Calls only the middleware HTTPS API (`MIDDLEWARE
| `searches/job_status.js` | Lookup by jobId |
| `triggers/timestamp_completed.js` | REST Hook |
See [../docs/developer/modules/function-reference.md](../docs/developer/modules/function-reference.md) for I/O contracts.
I/O contracts: https://zapier.georgelambert.org/docs/developer/modules/function-reference.pdf
## Env

View file

@ -5,14 +5,19 @@ monthly credits, a usage ledger, Stripe metered billing, purchase-order
invoicing, a company admin console, a self-service customer portal, and a
Zapier integration.
## Documentation
**Forgejo:** https://git.georgelambert.org/marchon/zappier-edge
**Catalog README:** https://zapier.georgelambert.org/packages/zappier/README.pdf
- **[docs/USER-MANUAL.md](docs/USER-MANUAL.md)** — operations & usage manual
- **[docs/ACCOUNTING.md](docs/ACCOUNTING.md)** — company accounting: invoices, PO billing, reports, CSV
- **[docs/USER-MANAGEMENT.md](docs/USER-MANAGEMENT.md)** — pricing, customer types, customer accounts
- **[docs/CUSTOMER-PORTAL.md](docs/CUSTOMER-PORTAL.md)** — end-user portal: signup, 2FA, reloads, invoices
- **[docs/DEVELOPER.md](docs/DEVELOPER.md)** — full developer documentation
- **[docs/WALKTHROUGH.md](docs/WALKTHROUGH.md)** — original step-by-step pricing walkthrough
## Documentation (public PDFs)
- [USER-MANUAL](https://zapier.georgelambert.org/packages/zappier/docs/USER-MANUAL.pdf) — operations & usage
- [ACCOUNTING](https://zapier.georgelambert.org/packages/zappier/docs/ACCOUNTING.pdf) — invoices, PO billing, reports
- [USER-MANAGEMENT](https://zapier.georgelambert.org/packages/zappier/docs/USER-MANAGEMENT.pdf) — pricing, customer types
- [CUSTOMER-PORTAL](https://zapier.georgelambert.org/packages/zappier/docs/CUSTOMER-PORTAL.pdf) — signup, 2FA, reloads
- [DEVELOPER](https://zapier.georgelambert.org/packages/zappier/docs/DEVELOPER.pdf)
- [WALKTHROUGH](https://zapier.georgelambert.org/packages/zappier/docs/WALKTHROUGH.pdf)
- Markdown copies remain next to these files in `docs/`
- **Sample Zapier app:** `zapier-app/` — https://zapier.georgelambert.org/packages/zappier/zapier-app/README.pdf (if present) or the source tree in git
## Surfaces

View file

@ -0,0 +1,13 @@
# Sample Zapier Platform app (zappier-edge)
Minimal Zapier app that stores data/files through the **metered** zappier HTTPS API. This is a **sample**, not the Verae timestamp app (`verae-zapier-app`).
**Catalog:** https://zapier.georgelambert.org/packages/zappier/zapier-app/README.pdf
**Parent:** https://zapier.georgelambert.org/packages/zappier/README.pdf
```bash
cd packages/zappier/zapier-app
npm test
```
Point the app at the public zappier-edge base URL and an `x-api-key` from `/portal`.