Proxmox worker, uptime, backup, tagged deploy

This commit is contained in:
George Lambert 2026-09-11 23:36:39 -04:00
commit 08741f3a29
10 changed files with 181 additions and 0 deletions

115
LOG.md Normal file
View file

@ -0,0 +1,115 @@
# Action log
## 2026-09-11 — proxmox worker + deploy/monitor/backup
- NS1 **is** Proxmox (`vmbr0` public, `vmbr1` `10.10.10.0/24`). New repo **verae-proxmox** creates LXC **510** `verae-px-worker` at `10.10.10.20`.
- NATS for guests: socat `10.10.10.1:4222``127.0.0.1:4222` (never vmbr0).
- **verae-uptime** off-box HTTPS GET; **verae-backup** tar snapshots; **verae-deploy** tagged npm ci.
- Fleet `overlays/ns1/` checked in; `start.sh` no longer rewrites fleet/machines JSON.
- User systemd + linger for keep and fleet.
## 2026-09-11 — bootstrap
- Created this repo to record decisions while executing cleanup items 15 (and a second pass on 25).
- Assumption: single developer, lab only; breaking changes allowed.
## Step 1 — account-balance is the prepaid source of truth
- Persist books to `BOOKS_PATH` (atomic JSON).
- Mutations go through `balance.adjust` / `usage.recorded`; edge cache `balanceCents` is updated from the books reply only.
- Local cache mutate only when `NATS_URL` and `ACCOUNT_BALANCE_URL` are both unset (tests).
## Step 2 — identity join
- `zappier-identity` mailbox: `verae.identity.bind` / `verae.identity.lookup`.
- Portal + middleware tenants store `veraeUserId`; JWT never public.
## Step 3 — access planes are the public doors
- zappier-edge binds loopback (`127.0.0.1:3000`).
- Public: access-web `:3021`, access-api `:3022`, access-leaf `:3023`, access-zapier `:3024`, access-staff `:3025`.
## Step 4 — leaf hardening
- `verae-nats-accounts` holds nats.conf + subject policy.
- Leaf token required; billing subjects denied in policy file and access-leaf.
## Step 5 — fleet spawn
- Prefer local machines for HTTP `spawn.command` services.
- Disable lan-134 unless `FLEET_ENABLE_LAN134=1`.
- SSH spawn timeout 8s; failed hosts skipped.
## 2026-09-11 — catalog live doors
- Bottom of https://zapier.georgelambert.org/ lists each HTTPS door with description, screenshot, and Open link.
- UI-REVIEW.pdf includes the live-door atlas. Walkthrough §13.
## 2026-09-11 — Caddy host names live
- Installed `/etc/caddy/sites/zapier-services.georgelambert.org.caddy` on NS1. HTTPS: portal, api, leaf, zap, staff, iam, login, cs, sales, acct, mw, fleet.
- Not proxied: NATS, keep, workers, loopback edge.
## 2026-09-11 — host names
- Mapped every running service to **NS1.GEORGELAMBERT.ORG** (`70.88.205.138`).
- Suggested `*.zapier.georgelambert.org` names for public doors; only `zapier.georgelambert.org` (docs) and `git.georgelambert.org` exist in Caddy today.
## 2026-09-11 — docs sync, bootstrap, NS1 all-in-one
- Catalog and ops docs: public portal is access-web `:3021/portal/`; edge is loopback; IAM + keep listed.
- New `verae-bootstrap` repo: clone + npm for `ns1-all-in-one`, `control-plane`, `ns1-archive`, `lan-worker`.
- Fleet starts `staff-iam` and `staff-session`. NS1 all-in-one uses edge `:13000` because host `:3000` is taken; keep still owns archive workers. `better-sqlite3` must be rebuilt on Linux (`npm rebuild`).
## 2026-09-11 — verae-keep on NS1
- New repo `verae-keep`: keep `:3860` restarts crashed units; watch `:3861` restarts keep; `guard.sh` restarts watch.
- Honors admin pause/stop via intent files + `POST /intent`. Fleet writes `data/<instance>/intent.json` (local and SSH).
- Tested on 70.88.205.138: crash-restart, pause-hold, watch respawn. webhook-deliver brought back. NATS observed only.
## 2026-09-11 — IAM JSON doors closed
- CS/sales/accounting/staff JSON list and sales PUT pricing now go through IAM (401 JSON, 302 HTML).
- Fleet service env sets `STAFF_AUTH=1` with `STAFF_IAM_URL`.
- Re-walk: cs credit 200 agent=`cs`; cs export 403; operator Fleet POST 200; admin `/me` `permissions:['*']`.
## 2026-09-11 — IAM on lab, portal door, hardening
- Restarted fleet with `STAFF_IAM_URL`. Walk: cs credits (agent=`cs`) and is 403 on accounting export; operator fleet POST 200, cs 403; admin `/me` permissions `*`.
- Portal public door `verae-access-web` `:3021/portal/` (static + `/portal/api` → loopback edge).
- IAM sessions persist in JSON; login rate-limit per IP+username (8/10min); credits stamp IAM username as authz principal.
- lan-134 remains disabled unless `FLEET_ENABLE_LAN134=1`. Receipt PDF branded (indigo header + legal footer).
## 2026-09-11 — staff IAM
- New `verae-staff-iam` :3028 — users, roles, permissions, sessions, audit UI.
- Admin login uses IAM when `STAFF_IAM_URL` is set; tabs hide without permission.
- CS/sales/accounting/access-staff/fleet mutating APIs check `cs.credit`, `sales.quote`, `accounting.export`, `fleet.operate`, etc.
- Seed: admin/cs/sales/accounting/operator. Shared staff key is fallback only.
## 2026-09-11 — last three UI leftovers
- Swagger `/docs` stays stock; banner names it OpenAPI explorer.
- Shared template `verae-staff-ui` for CS + access-staff.
- `STAFF_COOKIE_DOMAIN` + nginx example for multi-host.
## 2026-09-11 — names, staff session, exclusive jobs.events
- Account-balance stores display names (`customer.put` + lookup by name). Edge writes names on customer create/edit; CS/sales/accounting/staff join from edge if the ledger has no name.
- New repo `verae-staff-session` (:3027). Department HTML redirects when `STAFF_AUTH=1`.
- `JOBS_EVENTS_EXCLUSIVE=1` makes `verae-jobs-events` the durable consumer; middleware webhook router backs off.
- Catalog index is cards. Disabled fleet machines are grey, not degraded yellow.
## 2026-09-11 — UI design-system pass
- Restyled CS/sales/accounting/access-staff to portal indigo; dollars + names.
- Fleet ⋯ menus, role chips, Docs cards, 0.0.0.0 copy, simulator indigo (orange = faults).
- Portal masked API key; admin customer drawer; empty invoice illustrations.
- Catalog system-ui. New independent repo **UI-Docs** (walkthrough, screenshots, UI-REVIEW.pdf).
## Second pass (25)
- Identity used from portal when `IDENTITY_URL` set.
- Leaf loads allow-list from nats-accounts `policy.json`.
- Staff plane is a first-class mailbox (`verae-access-staff`).
- jobs.events mailbox process `verae-jobs-events` (passthrough + count).