George Lambert
a73875f4ef
Some checks are pending
offline / test (push) Waiting to run
Fleet/keep/middleware NATS_URL lists 10.10.10.21–23. Product streams are replicas=3. px-worker worm/tree run on that cluster. Backup timer and px-worker uptime watch are on. MOCK_VERAE and Zapier push unchanged.
44 lines
1.8 KiB
Markdown
44 lines
1.8 KiB
Markdown
# verae-nats-cluster
|
|
|
|
Three **distinct Proxmox LXC** guests running **NATS JetStream** as a cluster, on the private bridge only. This is the test stand before the same layout moves to separate hardware.
|
|
|
|
**Forgejo:** https://git.georgelambert.org/marchon/verae-nats-cluster
|
|
**SSH:** `ssh://git@git.georgelambert.org:2223/marchon/verae-nats-cluster.git`
|
|
|
|
| Guest | VMID | `vmbr1` IP | Client | Cluster | Monitor |
|
|
|-------|------|------------|--------|---------|---------|
|
|
| nats-a | 511 | 10.10.10.21 | `:4222` | `:6222` | `127.0.0.1:8222` |
|
|
| nats-b | 512 | 10.10.10.22 | `:4222` | `:6222` | `127.0.0.1:8222` |
|
|
| nats-c | 513 | 10.10.10.23 | `:4222` | `:6222` | `127.0.0.1:8222` |
|
|
|
|
Cluster name: `verae`. Client URL:
|
|
|
|
```text
|
|
nats://10.10.10.21:4222,nats://10.10.10.22:4222,nats://10.10.10.23:4222
|
|
```
|
|
|
|
**Not** on `vmbr0`. **Not** `0.0.0.0`. Host lab NATS on `127.0.0.1:4222` is left running so current keep/fleet/middleware stay up until you cut over.
|
|
|
|
```bash
|
|
# on NS1 (the Proxmox host)
|
|
git clone ssh://git@git.georgelambert.org:2223/marchon/verae-nats-cluster.git
|
|
cd verae-nats-cluster
|
|
bash scripts/create-cluster.sh
|
|
bash scripts/status.sh
|
|
bash scripts/test.sh
|
|
```
|
|
|
|
Test-env cut-over (keep/fleet/middleware on NS1):
|
|
|
|
```bash
|
|
# 1) create product streams (replicas=3)
|
|
bash scripts/ensure-streams.sh
|
|
# 2) fleet overlay nats.url + services/*.json already list the three URLs
|
|
# 3) restart fleet and keep with NATS_URL from client.env
|
|
```
|
|
|
|
`JETSTREAM_REPLICAS=3` on middleware. Host `127.0.0.1:4222` can stay up unused. Zapier cloud never talks to NATS.
|
|
|
|
nkeys/mTLS: `verae-nats-accounts` still has the INTERNAL/LEAF sketch. Do **not** enable accounts on this cluster until every client passes credentials in `NATS_URL`. Private `vmbr1` is the current isolation.
|
|
|
|
Hardware move: same three configs, three boxes, private NIC only — change IPs in `cluster.env` and `conf/nats.conf.tmpl`.
|