History

George Lambert a73875f4ef Some checks are pending offline / test (push) Waiting to run Details Cut the test environment over to the 3-node Proxmox NATS cluster. Fleet/keep/middleware NATS_URL lists 10.10.10.21–23. Product streams are replicas=3. px-worker worm/tree run on that cluster. Backup timer and px-worker uptime watch are on. MOCK_VERAE and Zapier push unchanged.		2026-09-11 23:59:38 -04:00
..
conf	Add a three-node NATS JetStream cluster on distinct Proxmox LXC guests.	2026-09-11 23:47:42 -04:00
scripts	Cut the test environment over to the 3-node Proxmox NATS cluster.	2026-09-11 23:59:38 -04:00
systemd	Add a three-node NATS JetStream cluster on distinct Proxmox LXC guests.	2026-09-11 23:47:42 -04:00
client.env	Cut the test environment over to the 3-node Proxmox NATS cluster.	2026-09-11 23:59:38 -04:00
cluster.env	Add a three-node NATS JetStream cluster on distinct Proxmox LXC guests.	2026-09-11 23:47:42 -04:00
NATS.md	Add a three-node NATS JetStream cluster on distinct Proxmox LXC guests.	2026-09-11 23:47:42 -04:00
package.json	Add a three-node NATS JetStream cluster on distinct Proxmox LXC guests.	2026-09-11 23:47:42 -04:00
README.md	Cut the test environment over to the 3-node Proxmox NATS cluster.	2026-09-11 23:59:38 -04:00
SUMMARY.md	Add a three-node NATS JetStream cluster on distinct Proxmox LXC guests.	2026-09-11 23:47:42 -04:00

README.md

verae-nats-cluster

Three distinct Proxmox LXC guests running NATS JetStream as a cluster, on the private bridge only. This is the test stand before the same layout moves to separate hardware.

Forgejo: https://git.georgelambert.org/marchon/verae-nats-cluster
SSH: ssh://git@git.georgelambert.org:2223/marchon/verae-nats-cluster.git

Guest	VMID	`vmbr1` IP	Client	Cluster	Monitor
nats-a	511	10.10.10.21	`:4222`	`:6222`	`127.0.0.1:8222`
nats-b	512	10.10.10.22	`:4222`	`:6222`	`127.0.0.1:8222`
nats-c	513	10.10.10.23	`:4222`	`:6222`	`127.0.0.1:8222`

Cluster name: verae. Client URL:

nats://10.10.10.21:4222,nats://10.10.10.22:4222,nats://10.10.10.23:4222

Not on vmbr0. Not 0.0.0.0. Host lab NATS on 127.0.0.1:4222 is left running so current keep/fleet/middleware stay up until you cut over.

# on NS1 (the Proxmox host)
git clone ssh://git@git.georgelambert.org:2223/marchon/verae-nats-cluster.git
cd verae-nats-cluster
bash scripts/create-cluster.sh
bash scripts/status.sh
bash scripts/test.sh

Test-env cut-over (keep/fleet/middleware on NS1):

# 1) create product streams (replicas=3)
bash scripts/ensure-streams.sh
# 2) fleet overlay nats.url + services/*.json already list the three URLs
# 3) restart fleet and keep with NATS_URL from client.env

JETSTREAM_REPLICAS=3 on middleware. Host 127.0.0.1:4222 can stay up unused. Zapier cloud never talks to NATS.

nkeys/mTLS: verae-nats-accounts still has the INTERNAL/LEAF sketch. Do not enable accounts on this cluster until every client passes credentials in NATS_URL. Private vmbr1 is the current isolation.

Hardware move: same three configs, three boxes, private NIC only — change IPs in cluster.env and conf/nats.conf.tmpl.