Test env NATS cluster cut-over

This commit is contained in:
George Lambert 2026-09-11 23:59:56 -04:00
commit b1e9dfa39d
25 changed files with 833 additions and 0 deletions

View file

@ -0,0 +1,5 @@
# Server type: control-plane
HTTP control plane only: fleet, access doors, IAM, billing, middleware. Archive floors run on an `ns1-archive` host.
Public doors: access-web `:3021`, access-api `:3022`, access-leaf `:3023`, access-zapier `:3024`, access-staff `:3025`. Edge loopback `:3000`. Fleet `:3850`.

View file

@ -0,0 +1,23 @@
zappier-edge
verae-middleware
verae-request-splitter
verae-zapier-simulator
verae-fleet
zappier-account-balance
zappier-customer-service
zappier-sales-pricing
zappier-accounting-export
verae-access-authz
verae-access-web
verae-access-api
verae-access-leaf
verae-access-zapier
verae-access-staff
zappier-identity
verae-jobs-events
verae-staff-session
verae-staff-ui
verae-staff-iam
verae-keep
verae-ops
verae-bootstrap

View file

@ -0,0 +1,10 @@
# Server type: lan-worker
Extra SSH worker (example: `lan-134` at 70.88.205.134). Roles: `tree-node`, `archive-worm`.
Leave this type **disabled** in fleet `machines.json` until SSH works. The operator console shows a grey card for an intentionally off host.
```bash
bash scripts/fetch.sh lan-worker
# enable in fleet only after: ssh -i ~/.ssh/id_ed25519 marchon@70.88.205.134 true
```

View file

@ -0,0 +1,5 @@
verae-archive-worm
verae-tree-node
verae-fleet
verae-keep
verae-bootstrap

View file

@ -0,0 +1,9 @@
# Server type: nats-cluster
Three Proxmox LXC guests (`nats-a/b/c`) on `vmbr1`. Provision from the hypervisor with [verae-nats-cluster](https://git.georgelambert.org/marchon/verae-nats-cluster) `scripts/create-cluster.sh`.
Client URL (private only):
`nats://10.10.10.21:4222,nats://10.10.10.22:4222,nats://10.10.10.23:4222`
Host loopback `127.0.0.1:4222` stays the lab bus until you cut over.

View file

@ -0,0 +1,3 @@
verae-nats-cluster
verae-nats-accounts
verae-bootstrap

View file

@ -0,0 +1,29 @@
# Server type: ns1-all-in-one
Dedicated NS1 box (`70.88.205.138`): NATS loopback, archive workers, HTTP doors, fleet, IAM, keep.
## Layout
- NATS `127.0.0.1:4222` (already running on NS1)
- Archive workers: keep (`:3860`) — job-poller, webhook-deliver, aggregator, worm ×3, tree-node ×3
- HTTP: fleet `serve` on this host (`local` machine only)
- zappier-edge **`:13000` loopback** (host `:3000` is taken)
- Public portal: `:3021/portal/`
- Operator console: `:3850`
- IAM `:3028`, staff-session `:3027`
## Fetch
```bash
export VERAE_SRC=$HOME/verae-src
bash scripts/fetch.sh ns1-all-in-one
```
## Start (from a monorepo-shaped tree)
```bash
export VERAE_STACK=$HOME/verae-stack # packages/ sibling layout
bash types/ns1-all-in-one/start.sh
```
`start.sh` disables fleet spawn of archive workers (keep already owns those ports), points edge at `:13000`, starts `verae-fleet serve`.

View file

@ -0,0 +1,27 @@
zappier-edge
verae-middleware
verae-request-splitter
verae-archive-worm
verae-archive-aggregator
verae-tree-node
verae-zapier-simulator
verae-fleet
zappier-account-balance
zappier-customer-service
zappier-sales-pricing
zappier-accounting-export
verae-access-authz
verae-access-web
verae-access-api
verae-access-leaf
verae-access-zapier
verae-access-staff
zappier-identity
verae-jobs-events
verae-nats-accounts
verae-staff-session
verae-staff-ui
verae-staff-iam
verae-keep
verae-ops
verae-bootstrap

41
types/ns1-all-in-one/start.sh Executable file
View file

@ -0,0 +1,41 @@
#!/usr/bin/env bash
# Start HTTP control plane on NS1. Archive workers stay with verae-keep.
# Does not rewrite fleet.json / machines.json — use overlays/ns1/.
set -euo pipefail
STACK="${VERAE_STACK:-$HOME/verae-stack}"
FLEET="$STACK/packages/verae-fleet"
EDGE_PORT="${EDGE_PORT:-13000}"
export STAFF_IAM_URL="${STAFF_IAM_URL:-https://iam.zapier.georgelambert.org}"
export STAFF_COOKIE_DOMAIN="${STAFF_COOKIE_DOMAIN:-.zapier.georgelambert.org}"
export NATS_URL="${NATS_URL:-nats://10.10.10.21:4222,nats://10.10.10.22:4222,nats://10.10.10.23:4222}"
export JETSTREAM_REPLICAS="${JETSTREAM_REPLICAS:-3}"
export FLEET_ENABLE_LAN134="${FLEET_ENABLE_LAN134:-}"
export FLEET_FILE="${FLEET_FILE:-$FLEET/overlays/ns1/fleet.json}"
export FLEET_MACHINES="${FLEET_MACHINES:-$FLEET/overlays/ns1/machines.json}"
test -d "$FLEET" || { echo "missing $FLEET" >&2; exit 1; }
test -f "$FLEET_FILE" || { echo "missing $FLEET_FILE" >&2; exit 1; }
cd "$FLEET"
if curl -sf http://127.0.0.1:3850/health >/dev/null; then
echo "fleet already up"
exit 0
fi
nohup env \
STAFF_IAM_URL="$STAFF_IAM_URL" \
STAFF_COOKIE_DOMAIN="$STAFF_COOKIE_DOMAIN" \
NATS_URL="$NATS_URL" \
FLEET_FILE="$FLEET_FILE" \
FLEET_MACHINES="$FLEET_MACHINES" \
FLEET_ENABLE_LAN134="$FLEET_ENABLE_LAN134" \
node src/cli.js serve \
>/tmp/verae-fleet-serve.out 2>&1 &
echo "fleet pid $!"
for i in $(seq 1 40); do
if curl -sf http://127.0.0.1:3850/health >/dev/null; then
echo "fleet up (overlay $FLEET_FILE)"
exit 0
fi
sleep 0.5
done
echo "fleet did not become healthy; see /tmp/verae-fleet-serve.out" >&2
exit 1

View file

@ -0,0 +1,5 @@
# Server type: ns1-archive
NATS JetStream on loopback plus fleet/keep workers: tree-node ×3, archive-worm ×3, aggregator, job-poller, webhook-deliver.
Do not bind NATS on a public NIC. Keep (`:3860`) restarts workers unless the operator console paused or stopped them.

View file

@ -0,0 +1,8 @@
verae-archive-worm
verae-archive-aggregator
verae-tree-node
verae-fleet
verae-keep
verae-nats-accounts
verae-ops
verae-bootstrap

17
types/proxmox-worker/README.md Executable file
View file

@ -0,0 +1,17 @@
# Server type: proxmox-worker
Ubuntu LXC on **NS1 Proxmox** (`vmbr1` `10.10.10.20`, VMID 510). Extra `archive-worm` + `tree-node` copies so replica floors are not all on the host PID namespace.
Provision from the Proxmox host: [verae-proxmox](https://git.georgelambert.org/marchon/verae-proxmox) `scripts/create-worker.sh`.
```bash
export VERAE_SRC=$HOME/verae-src
export NATS_URL=nats://10.10.10.1:4222
bash scripts/host-deps.sh --type proxmox-worker
bash scripts/fetch.sh proxmox-worker
# keep extra copies:
KEEP_UNITS=$HOME/verae-src/verae-keep/units.px-worker.json \
bash $HOME/verae-src/verae-keep/scripts/guard.sh
```
Fleet machine id `px-worker`. Enable after `ssh-check`.

View file

@ -0,0 +1,6 @@
verae-archive-worm
verae-tree-node
verae-fleet
verae-keep
verae-bootstrap
verae-proxmox