marchon/dev-overview
Watch
1
0
Fork
You've already forked dev-overview
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
dev-overview/01-nats-routes.md

7.1 KiB
Raw Permalink Blame History

Designed vs running NATS routes

Designed = named in docs or in code.
Running = a process on NS1 is subscribed or publishing it today, and (for durable work) a JetStream stream exists.

Cluster name verae. Internal clients use the three LXC on vmbr1. Public edge is a leaf, not a second cluster.

Legend: JS = JetStream stream; RR = core request-reply; pub = core or JS fire-and-forget; Q = queue group.

Transport (TCP / leaf / public)

These are NATS server routes, not verae.* subjects.

Endpoint Designed Running Who
10.10.10.21–23:4222 client yes yes middleware, access-*, billing, worm/tree, identity, CS/sales/acct
10.10.10.21–23:6222 cluster routes yes yes (private vmbr1 only) nats-a/b/c to each other
10.10.10.21–23:7422 leaf hub yes (public-edge) yes public edge remotes
127.0.0.1:8222 monitor on each LXC yes yes operators on the guest
70.88.205.138:4222 public client yes (token) yes leaf edge; not Zapier
127.0.0.1:14422 WS on edge yes yes Caddy → wss://nats.zapier.georgelambert.org
127.0.0.1:8229 edge monitor yes yes host loopback
:6222 / :7422 / :8222 on vmbr0 no must stay closed —
Host leftover 127.0.0.1:4222 (docker-proxy) + 10.10.10.1:4222 socat lab leftover present do not point product NATS_URL here
MQTT :1883 study only removed —

Internal keep/fleet/middleware URL:

nats://10.10.10.21:4222,nats://10.10.10.22:4222,nats://10.10.10.23:4222

Public (token in /etc/nats/public-edge.token on NS1): nats://<token>@70.88.205.138:4222. Isolation on the hub is still $G + private LAN. nkeys/mTLS sketched in verae-nats-accounts, not flipped.

JetStream streams (running)

Created by verae-nats-cluster/scripts/ensure-streams.sh (middleware ensureStreams also creates the first three).

Stream Subjects Replicas Storage Who consumes
ZAPIER_JOBS verae.zapier.jobs.watch 3 file job-poller (in middleware process), Q job-poller
ZAPIER_EVENTS verae.zapier.jobs.events 3 file middleware wait + webhook router (event-webhook-router); optional exclusive jobs-events
ZAPIER_WEBHOOKS verae.zapier.webhooks.deliver 3 file webhook-deliver (in middleware), Q webhook-deliver
ZAPIER_USAGE verae.zapier.usage 3 file no durable consumer (USAGE_WRITER named only)
VERAE_ARCHIVE verae.archive.> 3 file worm + tree on LXC 510 (keep); aggregator not fleet-enabled

Billing is core request-reply / pub, not a JetStream stream.

Address inventory

Address Kind Designed Running Notes
verae.zapier.jobs.watch JS work Q job-poller yes yes middleware publishes; in-process poller consumes
verae.zapier.jobs.events JS events yes yes poller publishes; wait() subscribes; webhook worker routes
verae.zapier.webhooks.deliver JS work Q webhook-deliver yes yes same Node process as HTTP
verae.zapier.usage JS optional yes stream yes, consumer no middleware entitlement is local files
verae.internal.jobs.events core republish yes (jobs-events exclusive) partial only if JOBS_EVENTS_EXCLUSIVE=1
verae.billing.statement.get RR Q account-balance yes yes portal, CS, sales, accounting, access-web/api
verae.billing.balance.get RR yes yes account-balance
verae.billing.balance.adjust RR yes yes CS credits, portal reload
verae.billing.usage.recorded pub yes yes if edge NATS_URL set zappier/src/app.ts onUsage → natsPublish; account-balance subscribes
verae.billing.payment.recorded pub yes yes if NATS_URL portal reload (portal.ts)
verae.billing.credit.applied pub yes yes if NATS_URL admin/CS (admin.ts)
verae.billing.customer.put pub yes yes if NATS_URL ledger.ts; account-balance applies
verae.access.authz.check RR Q access-authz yes yes fleet keepFloor
verae.access.authz.deny pub audit yes yes (on deny) no SIEM sink
verae.access.web.in / verae.access.web.billing.* ingress yes yes mapped after authz, e.g. verae.access.web.billing.statement.get
verae.access.api.* ingress yes yes
verae.access.zapier.* ingress yes yes HTTPS plane; Zapier never speaks NATS
verae.access.leaf.in / verae.access.leaf.* ingress yes yes no billing
verae.access.staff.* ingress yes yes CS/sales/accounting
verae.identity.bind RR Q identity yes yes fleet
verae.identity.lookup RR Q identity yes yes
verae.archive.put JS (VERAE_ARCHIVE) yes partial splitter in-process in middleware; worm/tree on 510 consume
verae.archive.query broadcast (no Q) yes partial live aggregator hook TBD in middleware (simulator has it)
verae.archive.reply.<id> replies on bloom hit yes partial worm/tree on 510 can reply; aggregator not fleet-enabled
verae.splitter.in future yes (docs) no splitter stays a library
verae.example.process.in / .out / .reply.* template yes no verae-nats-process is a model
bench.* lab only n/a ephemeral speed studies; not product

Plane mapping (verae-access-authz): internal verae.billing.statement.get + plane web → ingress verae.access.web.billing.statement.get. Planes never publish internal subjects until authz.check returns allow.

Durable consumers (named in middleware)

Name Stream / subject Running
job-poller ZAPIER_JOBS / jobs.watch yes (in-process)
event-webhook-router ZAPIER_EVENTS / jobs.events yes (in-process)
webhook-deliver ZAPIER_WEBHOOKS / webhooks.deliver yes (in-process)
usage-writer ZAPIER_USAGE / verae.zapier.usage no
jobs-events-exclusive ZAPIER_EVENTS no unless JOBS_EVENTS_EXCLUSIVE=1

Fleet overlay vs in-process

packages/verae-fleet/overlays/ns1/fleet.json (2026-09-14):

Service fleet enabled Actual lab
middleware-http true running; starts job-poller + webhook workers in-process when NATS_ENABLED
job-poller false covered by middleware; do not spawn a second copy
webhook-deliver false same
archive-aggregator false not running as a fleet worker
archive-worm / tree-node false on host fleet running on LXC 510 via keep units.px-worker.json (1 worm + 1 tree)
account-balance, access-*, identity, jobs-events, CS/sales/acct, staff-iam/session, zappier-edge true keepFloor keepFloor
zapier-simulator true, keepFloor false optional
nats managed false 3-node cluster + leftover host loopback