master-zapier-plan-draft/packages/zapier-decisions/STATUS.md
George Lambert fef7590c27
Some checks are pending
offline / test (push) Waiting to run
Add a NATS cluster message-speed bench (throughput and delay).
Client LXC 510 against nats-a/b/c at several core and JetStream r=3 loads.
2026-09-12 00:56:48 -04:00

7 KiB
Raw Blame History

Current status — 2026-09-12

Verae Time × Zapier is a working lab on one Proxmox host (NS1.GEORGELAMBERT.ORG, 70.88.205.138). Public HTTPS doors, catalog, keep/fleet, and a test 3-node NATS cluster on separate LXC guests are up. The test environment points at that cluster. Zapier Platform push and live api.veraetime.net are still waiting on operator login / bind credentials.

Progress repo: this file in zapier-decisions.
Catalog: https://zapier.georgelambert.org/packages/zapier-decisions/STATUS.pdf
Monorepo: master-zapier-plan-draft (main / master).


Live right now

Layer State
Catalog https://zapier.georgelambert.org/ (PDF links path-relative; module/model books work)
Public doors portal, api, zap, iam, mw, fleet, git — last off-box GET 8/8 OK
Keep + fleet Running. systemd + linger enabled for next reboot (current processes were not double-started as systemd)
Test NATS 3-node JetStream on vmbr1: LXC 511/512/513 nats-a/b/c at 10.10.10.2123. Cluster name verae, 2 routes each.
Product streams ZAPIER_JOBS, ZAPIER_EVENTS, ZAPIER_WEBHOOKS, ZAPIER_USAGE, VERAE_ARCHIVEreplicas=3
Modules on that cluster Fleet, keep, middleware (natsConnected: true), billing, access planes. NATS_URL lists all three URLs.
Host loopback :4222 Still listening; clients no longer use it
Worker CT 510 verae-px-worker 10.10.10.20 on vmbr1. Keep worm+tree 200 on the cluster
Backup Timer verae-backup.timer enabled; snapshots under /SSD2/backups/verae/
Uptime Loop on px-worker (:3870/status), last tick failed: 0

Client URL (private vmbr1 only):

nats://10.10.10.21:4222,nats://10.10.10.22:4222,nats://10.10.10.23:4222

Repo: https://git.georgelambert.org/marchon/verae-nats-cluster

Cluster message speed (20260912T045131Z)

Measured from LXC 510 (not a nats-* server) against 10.10.10.2123. Full tables: BENCH.md / catalog BENCH.pdf. Re-run: bash scripts/bench.sh on NS1.

Kind Load Result
Core NATS 1p1s 50k×128 B ~1.20M msgs/s aggregate
Core NATS 8p8s 200k×128 B ~2.01M msgs/s aggregate
Core NATS 4p4s 50k×1 KiB ~630k msgs/s (~616 MB/s)
JetStream file r=3 14 pubs, 128 B ~16k durable pubs/s
JetStream file r=3 4 pubs, 1 KiB ~13.5k durable pubs/s
Ping (sequential RTT) 1k×128 B avg 0.307 ms, p99 0.734 ms
Flood (burst queueing) 8p 20k×128 B avg 454 ms

Product job/event/archive traffic is the JetStream row. Ping is one-message delay; flood is the mailbox catching up after a burst.


Built and checked in (Forgejo, each its own repo)

IAM, portal on access-web, keep/watch/guard, fleet floors, host-deps, catalog host-move docs, plus:

Repo Job
verae-uptime Off-box HTTPS GET of public doors
verae-backup SQLite / keep / worm-tree tar; restore dry-run default
verae-deploy Tagged checkout + npm ci on the target OS
verae-proxmox LXC 510 worker provision
verae-nats-cluster LXC 511513 JetStream cluster

Still open

Item Notes
Zapier Platform push Needs Zapier login (see below)
MOCK_VERAE=false Live api.veraetime.net + admin bind credentials (see below)
NATS nkeys/mTLS Cluster is up without accounts so existing NATS_URLs work. verae-nats-accounts remains the sketch. Flip only after every client has credentials. Isolation today is vmbr1, not nkeys.
lan-134 Still disabled (SSH not working)
Reboot test systemd should own keep/fleet after reboot; do not also nohup guard.sh
Hardware NATS Same three configs, three boxes, private NIC — change IPs in cluster.env

Once we have a Zapier login

Zapier cloud only talks HTTPS to our Zap plane (https://zap.zapier.georgelambert.org). No NATS, no api.veraetime.net.

  1. zapier login (or zapier-platform login).
  2. In packages/verae-zapier (and verae-activate if the tiny app is still wanted):
    • zapier-platform register (or attach an existing app id).
    • MIDDLEWARE_BASE_URL=https://zap.zapier.georgelambert.org (or https://mw.zapier.georgelambert.org if skipping the Zap plane).
    • zapier-platform build and validate.
  3. zapier-platform push as a private app.
  4. Invite the workspace; create a Zap:
    • Auth = portal API key (zmw_…), not a Verae JWT.
    • Test Create Timestamp and Wait, Create Timestamp (async jobId), Find Job Status, Timestamp Completed REST Hook.
  5. Fix anything validate / push flags (auth labels, samples, 402/403 mapping).
  6. When the contract is stable, bump the platform version and invite a real user.

Do not put NATS URLs or Verae passwords into Zapier.


Once MOCK_VERAE=false and live api.veraetime.net

Middleware already has the live HTTP client. Still needed:

  1. VERAE_API_BASE_URL=https://api.veraetime.net and working login (username/password → JWT). JWT stays in middleware (tokenRef); Zapier never sees it.
  2. Confirm OpenAPI vs our client:
    • POST /api/timestamp202 { jobId }
    • GET /api/status/{jobId} until completed / failed
    • POST /api/verify
    • Batch create/status if used
  3. Hash lookup is not on the live OpenAPIveraeClient.lookupHash returns 501 when not mocking. Central-chain “find by SHA-256” needs a Verae API gap fill or our own store.
  4. Map live status JSON into the receipt shape the Zap app expects (certificate, timestamp, blockIndex, sha256).
  5. Token refresh, 401 retry, and 402/403 mapping against real errors.
  6. One live seal in the lab, then REST Hook delivery to hooks.zapier.com.

Submit → jobId → wait → receipt

Yes — that is the designed path, and it already runs in mock.

  1. Zap (or curl) POST /zapier/v1/timestamp or /timestamp/wait.
  2. Middleware createTimestamp calls Verae POST /api/timestamp (or the mock) and gets jobId.
  3. It publishes verae.zapier.jobs.watch on NATS (the 3-node cluster).
  4. Wait: subscribe to verae.zapier.jobs.events for that jobId, or poll GET /api/status/{jobId}.
  5. Job-poller hits Verae status until completed.
  6. Event timestamp.completed carries status; webhook-deliver POSTs the Zapier REST Hook.
  7. Response includes certificate / timestamp / blockIndex (mock uses mock-cert-{jobId}, blockIndex: 42). Async create returns { jobId, sha256, existing } immediately; wait returns the finished status. Same SHA-256 is idempotent (existing: true).

If wait times out, middleware returns pending + jobId so the hook can finish the Zap. Live Verae must match that HTTP contract; we have not proven a real chain seal yet (MOCK_VERAE is still true).