offline / test (push) Waiting to run

Details

Add a three-node NATS JetStream cluster on distinct Proxmox LXC guests.

LXC 511–513 (nats-a/b/c on 10.10.10.21–23, vmbr1 only) form cluster
verae with replicas=3. Host loopback 127.0.0.1:4222 is unchanged.

2026-09-11 23:47:42 -04:00

2.3 KiB

Raw Blame History

3. Proxmox (NS1 is the hypervisor)

NS1.GEORGELAMBERT.ORG (70.88.205.138) is the Proxmox host. Control plane, Caddy, keep, and fleet run on the host. Extra worm/tree copies run in an unprivileged Ubuntu LXC on the private bridge.

Bridge	Address	Use
`vmbr0`	`70.88.205.138/28`	Public HTTPS / SSH. Never NATS.
`vmbr1`	`10.10.10.1/24`	Private. PKI `.11`, peergos-a `.12`, peergos-b `.13`, verae-px-worker `.20`

Guest we actually deploy


VMID	510
Name	`verae-px-worker`
Template	`ubuntu-24.04-standard_24.04-2_amd64.tar.zst` (`local`)
Disk	`SSD2` 16G
RAM / CPU	2G / 2
Roles	`tree-node`, `archive-worm`
Fleet id	`px-worker`
NATS	`nats://10.10.10.1:4222` (host `socat` → `127.0.0.1:4222`)

Repo: verae-proxmox

# on NS1
git clone ssh://git@git.georgelambert.org:2223/marchon/verae-proxmox.git
cd verae-proxmox
bash scripts/create-worker.sh
ssh -i ~/.ssh/id_ed25519 marchon@10.10.10.20 true

On the guest, bootstrap type proxmox-worker (verae-bootstrap). Then verae-fleet ssh-check px-worker and set "enabled": true in overlays/ns1/machines.json.

NATS for guests

Loopback NATS on the host stays 127.0.0.1:4222 (current lab bus). verae-nats-private-proxy.service binds only 10.10.10.1:4222 for the worker CT.

Three-node test cluster (distinct LXC, same chassis): verae-nats-cluster

Guest	VMID	IP
nats-a	511	10.10.10.21
nats-b	512	10.10.10.22
nats-c	513	10.10.10.23

bash scripts/create-cluster.sh then bash scripts/test.sh. Client URL lists all three. Do not listen on vmbr0. Same configs later move to separate hardware.

Suggested future guests (not created by create-worker.sh)

Guest	Role
`nats-b` `nats-c`	Extra JetStream nodes on `vmbr1` when leaving all-in-one
more `worker-*`	Same recipe as 510 with the next free `10.10.10.2x`

Edge / Caddy

Stay on the host (already live). Do not put zappier-edge in the worker CT.

Operator console: https://fleet.zapier.georgelambert.org/ · overlay packages/verae-fleet/overlays/ns1/.

2.3 KiB Raw Blame History