marchon/master-zapier-plan-draft
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Add Proxmox worker CT, off-box watch, backup, and tagged deploy.
Some checks are pending
offline / test (push) Waiting to run
Details

Browse source 
NS1 is the Proxmox host. verae-proxmox creates LXC 510 (verae-px-worker
10.10.10.20 on vmbr1) with a private NATS proxy on 10.10.10.1:4222.
verae-uptime GET-watches public doors; verae-backup snapshots SQLite and
worm/tree data; verae-deploy does host-deps + checkout + npm ci.
Fleet overlays/ns1 are checked in (start.sh no longer rewrites JSON).
User systemd + linger for keep and fleet survive reboot.
This commit is contained in:
George Lambert 2026-09-11 23:35:44 -04:00
parent 2d51d7a0dd
commit 7a5e25639e
69 changed files with 1087 additions and 118 deletions
Download patch file Download diff file Expand all files Collapse all files

58
packages/verae-ops/07-maintenance.md
View file

@ -13,28 +13,62 @@ Green / yellow / red on the operator console: https://zapier.georgelambert.org/p
## Restart
- systemd: `systemctl restart verae-middleware zappier-edge`
- User systemd (survives reboot with linger):
- `bash packages/verae-keep/scripts/install-systemd.sh`
- `bash packages/verae-fleet/scripts/install-systemd.sh`
- `loginctl enable-linger $USER`
- `systemctl --user restart verae-keep-guard verae-fleet`
- Compose: `docker compose up -d --no-deps middleware`
- One replica: fleet `restart tree-node-1` (floor spawns a replacement if needed)
- Pause is **not** off — paused copies do not count toward `keepFloor`
- Do not run **both** `nohup guard.sh` and the systemd unit
## Backup
| Data | Where |
|------|--------|
| zappier SQLite | `ZAPPIER_DB` path / volume |
| worm / tree blobs | instance `data/` dirs |
| NATS JetStream | nats store dir |
| fleet overlay | `packages/verae-fleet/data/` (gitignored) |
Repo: [verae-backup](https://git.georgelambert.org/marchon/verae-backup)
Do not back up private keys into git. Restore `identityFile` paths on the operator host.
```bash
export BACKUP_ROOT=/SSD2/backups/verae
bash scripts/backup.sh
bash scripts/restore.sh --dry-run
```
| Data | Default path |
|------|----------------|
| zappier SQLite | `~/verae-stack/packages/zappier/zappier.db` |
| worm / tree blobs | `~/verae-fleet-runtime/data/` |
| keep state | `~/verae-keep/data` |
| NATS JetStream | `/var/lib/nats/jetstream` if present |
| IAM | `~/verae-stack/packages/verae-staff-iam/data` |
User timer: `verae-backup.timer` (03:17 UTC). Do not back up private keys into git. Restore `identityFile` paths on the operator host. After `--apply`, `curl` `/health` on edge and keep.
## Upgrade
1. `git pull` each independent repo (or refresh the monorepo snapshot).
2. `npm test` in that package.
3. Restart that unit only; NATS cluster stays up.
4. Zapier Platform app: `zapier-platform push` from `verae-zapier-app` when HTTPS contracts change.
Repo: [verae-deploy](https://git.georgelambert.org/marchon/verae-deploy)
```bash
export VERAE_SRC=$HOME/verae-src
export DEPLOY_TYPE=ns1-all-in-one
bash scripts/deploy.sh main # or a tag vYYYY-MM-DD
systemctl --user restart verae-keep-guard verae-fleet
```
1. `host-deps.sh` on **this** OS.
2. `git fetch` + checkout ref (not force-push).
3. `npm ci` / `npm rebuild` (never copy macOS `node_modules`).
4. Restart that unit only; NATS stays up.
5. Zapier Platform app: `zapier-platform push` from `verae-zapier-app` when HTTPS contracts change.
## Off-box watch
Repo: [verae-uptime](https://git.georgelambert.org/marchon/verae-uptime)
```bash
node src/watch.js --once # exit 2 if a public door failed
```
Run this on a **different** machine than NS1 (laptop cron is enough). `/status` on `:3870` in loop mode.
## Logs