marchon/master-zapier-plan-draft
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Add Proxmox worker CT, off-box watch, backup, and tagged deploy.
Some checks are pending
offline / test (push) Waiting to run
Details

Browse source 
NS1 is the Proxmox host. verae-proxmox creates LXC 510 (verae-px-worker
10.10.10.20 on vmbr1) with a private NATS proxy on 10.10.10.1:4222.
verae-uptime GET-watches public doors; verae-backup snapshots SQLite and
worm/tree data; verae-deploy does host-deps + checkout + npm ci.
Fleet overlays/ns1 are checked in (start.sh no longer rewrites JSON).
User systemd + linger for keep and fleet survive reboot.
This commit is contained in:
George Lambert 2026-09-11 23:35:44 -04:00
parent 2d51d7a0dd
commit 7a5e25639e
69 changed files with 1087 additions and 118 deletions
Download patch file Download diff file Expand all files Collapse all files

78
packages/verae-ops/03-proxmox.md
View file

@ -1,60 +1,50 @@
# 3. Proxmox (LXC and QEMU)
# 3. Proxmox (NS1 is the hypervisor)
Target: one **NATS cluster VM/CT** (or three), one **HTTPS edge** VM, one or more **worker** VMs. Fleet SSH-spreads worm/tree-node onto worker hosts.
**NS1.GEORGELAMBERT.ORG** (`70.88.205.138`) **is** the Proxmox host. Control plane, Caddy, keep, and fleet run on the host. Extra worm/tree copies run in an **unprivileged Ubuntu LXC** on the private bridge.
## Suggested guests
| Bridge | Address | Use |
|--------|---------|-----|
| `vmbr0` | `70.88.205.138/28` | Public HTTPS / SSH. **Never NATS.** |
| `vmbr1` | `10.10.10.1/24` | Private. PKI `.11`, peergos-a `.12`, peergos-b `.13`, **verae-px-worker `.20`** |
| Guest | OS | vCPU / RAM / disk | Role |
|-------|----|-------------------|------|
| `nats-a` `nats-b` `nats-c` | Debian 12 LXC unprivileged | 1 / 1G / 8G | JetStream, bind `127.0.0.1` **or** a vmbr **private** bridge only |
| `edge` | Debian 12 QEMU | 2 / 2G / 20G | Caddy + zappier-edge + middleware |
| `worker-1` … | Debian 12 LXC | 2 / 2G / 20G | fleet SSH target; worm + tree-node |
## Guest we actually deploy
Use a **private bridge** (`vmbr1`, RFC1918) for NATS `4222`/`6222`. Public `vmbr0` only on `edge` (80/443).
| | |
|--|--|
| VMID | **510** |
| Name | `verae-px-worker` |
| Template | `ubuntu-24.04-standard_24.04-2_amd64.tar.zst` (`local`) |
| Disk | `SSD2` 16G |
| RAM / CPU | 2G / 2 |
| Roles | `tree-node`, `archive-worm` |
| Fleet id | `px-worker` |
| NATS | `nats://10.10.10.1:4222` (host `socat` → `127.0.0.1:4222`) |
## NATS on an LXC
Repo: [verae-proxmox](https://git.georgelambert.org/marchon/verae-proxmox)
```bash
apt-get update && apt-get install -y curl
# official nats-server binary
curl -sSL https://github.com/nats-io/nats-server/releases/latest/download/nats-server-v2.10.26-linux-amd64.tar.gz | tar -xz
install nats-server /usr/local/bin/
# on NS1
git clone ssh://git@git.georgelambert.org:2223/marchon/verae-proxmox.git
cd verae-proxmox
bash scripts/create-worker.sh
ssh -i ~/.ssh/id_ed25519 marchon@10.10.10.20 true
```
`/etc/nats/nats.conf`:
On the guest, bootstrap type `proxmox-worker` ([verae-bootstrap](https://git.georgelambert.org/marchon/verae-bootstrap)). Then `verae-fleet ssh-check px-worker` and set `"enabled": true` in `overlays/ns1/machines.json`.
```
jetstream {}
listen: 10.10.10.11:4222 # private bridge, not 0.0.0.0 on public
cluster {
name: verae
listen: 10.10.10.11:6222
routes: [ nats-route://10.10.10.12:6222, nats-route://10.10.10.13:6222 ]
}
http: 127.0.0.1:8222
```
## NATS for guests
systemd: `ExecStart=/usr/local/bin/nats-server -c /etc/nats/nats.conf`.
Loopback NATS on the host stays `127.0.0.1:4222`. `verae-nats-private-proxy.service` binds **only** `10.10.10.1:4222`. Do not listen on `vmbr0`.
## Edge VM
## Suggested future guests (not created by create-worker.sh)
Install Node 22, clone `zappier-edge` and `verae-middleware`, Caddy reverse-proxy:
| Guest | Role |
|-------|------|
| `nats-b` `nats-c` | Extra JetStream nodes on `vmbr1` when leaving all-in-one |
| more `worker-*` | Same recipe as 510 with the next free `10.10.10.2x` |
```
zappier.example.com {
reverse_proxy 127.0.0.1:3000
}
middleware.example.com {
reverse_proxy 127.0.0.1:3100
}
```
## Edge / Caddy
Middleware env: `NATS_URL=nats://10.10.10.11:4222,nats://10.10.10.12:4222,nats://10.10.10.13:4222`.
Stay on the **host** (already live). Do not put zappier-edge in the worker CT.
## Fleet on Proxmox
On the operator workstation or a jump CT (loopback `:3850`):
`machines.json` entries: `kind=ssh`, `user=root` or `marchon`, `host=10.10.10.2x`, `identityFile=~/.ssh/id_ed25519` (**path only**). Roles: `tree-node,archive-worm`. Capacity ≥ 3 per worker if you want floors on one host.
`ssh-check` from the operator console before enabling `keepFloor`.
Operator console: `https://fleet.zapier.georgelambert.org/` · overlay `packages/verae-fleet/overlays/ns1/`.