S00-S04: hub inventory, signed-config design, prompts saved

.gitignore

@@ -0,0 +1 @@
+.DS_Store

INVENTORY.md

@@ -0,0 +1,36 @@
+# Inventory — git.georgelambert.org
+
+Forgejo **11.0.16**. Scanned with the `marchon` API token (not stored here).
+
+## Orgs
+
+| Org | Repos | Messaging relevance |
+|---|---|---|
+| `marchon` | 61 | PFC, NATS, access, zapier, ops — **in scope** |
+| `infra` | `coredns`, `caddy` | Edge only; document Caddy docs hosts |
+| `verae` | 0 | Empty; do not invent content |
+| `websites` | 70+ | Marketing/site trees; **inventory only** this round |
+
+## marchon — NATS / DataCube (must stay consistent)
+
+- peergos-for-compliance, peergos-for-compliance-admin, peergos-for-compliance-ipfs, peergos-for-compliance-replication
+- peergos-compliance-{plan,architecture,docker,proxmox,nix,kubernetes,ipfs-pinning,aws-archive,iceberg,docs,config,go}
+- verae-nats-process, verae-nats-accounts, verae-nats-cluster
+- verae-access-{authz,web,api,leaf,zapier,staff}
+- verae-jobs-events, verae-ops, verae-deploy, verae-proxmox, verae-bootstrap
+- verae-archive-worm, verae-archive-aggregator, verae-keep, verae-uptime, verae-backup
+- verae-staff-{session,ui,iam}, verae-fleet, verae-tree-node, verae-middleware
+- verae-request-splitter, verae-activate, verae-zapier-*
+
+## marchon — other (link, do not rewrite)
+
+pulsepolls, datacube-pulsepolls, carpathia, zapier docs/pricing/identity, overview, UI-Docs, dev-overview
+
+## infra
+
+- caddy — future docs Caddy site files
+- coredns — DNS; no NATS subjects
+
+## websites
+
+Site HTML for datacubes.*, verae.*, veraetime.*, georgelambert.org, etc. No NATS endpoint catalog. Listed so the scan is complete; not forked this round.

LICENSE

@@ -0,0 +1,13 @@
+Copyright 2026 Verae / George Lambert
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND.

README.md

@@ -0,0 +1,19 @@
+# system-git-sync
+
+Hub for keeping `git.georgelambert.org` NATS names, signed configuration, and
+secure-messaging docs in sync.
+
+* [System-Git-Sync.MD](System-Git-Sync.MD) — prompts, design table, progress
+* [INVENTORY.md](INVENTORY.md) — every org scanned
+* [UserReview.MD](UserReview.MD) — decisions
+* [TODO-With-User.MD](TODO-With-User.MD) — guesses
+* [TODO.md](TODO.md) — dependency order
+
+Sibling repos:
+
+* https://git.georgelambert.org/marchon/nats-service-endpoints
+* https://git.georgelambert.org/marchon/secure-messaging
+* https://git.georgelambert.org/marchon/peergos-for-compliance
+* https://git.georgelambert.org/marchon/peergos-compliance-docs
+
+Not a HIPAA/SOC 2/ISO certificate. ns1: document-only until CI review.

System-Git-Sync.MD

@@ -0,0 +1,165 @@
+# System-Git-Sync
+
+Orchestrator for consistent NATS naming, signed configuration, DataCube admin
+history, and secure messaging across **every org** on
+`git.georgelambert.org`.
+
+This is **not** a HIPAA / SOC 2 / ISO certificate.
+
+Live **ns1** rule: **document-only first**, then code, then tests, then a
+**CI-CD review stack**, then deployment. Do not ship unsigned config or
+unreviewed binaries to `70.88.205.138`.
+
+---
+
+## Saved operator prompt (round 1)
+
+> There are mutiple parts to this request, part of it is architecture for
+> updating secure messaging service, part of it is requsting the
+> implementataion of additional git repos with code to implement this
+> functionality with the standard architecture, documentation, usage guide,
+> message tracing diagrams, message bundle diagrams and explainations, error
+> messages, error handling, and system logging. Then I need you to review
+> where this fits an needs to be embedded into the entire system, and then
+> make the code changes, committing and documenting every one, changing the
+> system documentation for that module if necessary and then please review,
+> modify as needed and both link to other repos, and once updates are all
+> changed and checked in, sync all of the documentation in all of the repos
+> to make them correct and consistant. make sure that all of the repos are
+> in sync, that the nats services have consistant naming, write a new git
+> repo that has the NATS-Service-Endpoints and lists what those endpoints
+> are for, what they do, who they expect messages from, what is the incoming
+> format, who they send messages too, and what is that format, what are the
+> rejection / security / filter paths / are incoming messages partially or
+> fully encrypted, if messages need to be and if so are fully encrypted and
+> the address is a passthrough make that clear, explain how encrypted
+> messages have routes and endpoints where the embedded data except for the
+> destination being encrypted, and the error handling blocks be encrypted
+> with the system public key for error handling and logging to process any
+> failed requests, and extract the return addresses with a public key
+> encryption token for returning secure message content in a message body
+> and any routing, or logging information in the message header as
+> specifically necessary inclusing error codes - and dead letter (service
+> request) failure, and an Network Error Bundle that is encoded with a
+> System Public Key that encodes the sender and ability to respond with
+> status messages, which will send an encrypted failure response to the
+> sender that only they can read - and an attached system error bounce
+> reportend bundle that is encrypted with that same system Public Key for
+> error recovery or logging while never revealing the contents of the
+> encrypted message by the intended recipient mailbox destination - not even
+> by the sender once sent (the sender will have to use their lookup ID to
+> identify the message source) and the same for the responder. in the event
+> of system failures make sure that the summary of those logs are sent to a
+> central logging system for review and correction. Please scan / close if
+> easier all repos on git.georgelambert.org to review and update. keep
+> itterating through the entire system logging every change and git checkin
+> for each project so that we have a valid system change and improvement
+> log. do not wait for any user input, if user decisions will need to be
+> made, write the into a UserReview.MD which we will review and make
+> decisions about and re-run if necessary. Save this prompt into the
+> "System-Git-Sync.MD" file with your log of progress and suggested changes
+> at the end and an internal todo list wich checks off todo items as
+> completed, but appends additional todo items in their order of
+> dependencies, and review the todo list from the top to make sure that all
+> system dependencies are resolved in order. The code needs to be in both
+> python for readability with full python line by line documentation, and go
+> profects using NATS communications - where if paractical using Internal
+> Nats Servers for performance and leaf nodes for connectivity. Note in the
+> ToDo list the choices made. All of the code and reviews documentation
+> should end up back on git.GeorgeLambert.org with rendered documentation in
+> both HTML and LaTex PDF (feel free to render those on the server and add
+> caddy endpoints as necessary) but make sure that all of the links in both
+> the HTML and LaTeX pdf's are resolvable into the entire git.georgelambert.org
+> repos - ideally with relative paths when practical. before you start,
+> please explain what you think is the task set so that I can refine and
+> review, and once you start, keep itterating over it until the project is
+> complete - if you are blocked and need user input, create a TODO-With-User.MD
+> file and do your best to work around it and keep going. If after a section,
+> you want user input - for this round make your best guesses and append the
+> Guesses and Assumptions that you make into the TODO-With-User.md file. are
+> we clar that I want you to understand, identify, itterate, and work on this
+> task until it is not only completed, but upon the first completion, you do
+> it again to make sure everything is in sync, and create a thesaurus with
+> links to ares in the documentation and code so that you can go back and
+> fix both vocabulary, variable names, paths, desscriptive NATS addresses to
+> make this entire set of repos totally in sync as much as possible. -
+> Please Confirm your understanding.
+
+## Saved operator prompt (round 2 — start now)
+
+> review and update every org on git.georgelambert.org, Start this loop now,
+> make the best system design choice, with information about how to select /
+> chose it in the configuration files, and make sure that to use a
+> configuration file it needs to be signed with a signed key wrapper to
+> prevent changes and hacking that can be regenerated from one of the admin
+> consoles after changes to the system configuration and the logging of both
+> the previous file, the new file, folowed by the diff of the files in an
+> append only history that uses a verae-datacube-history for admin changes.
+> Live ns1: document-only first, then code, test, build a CI-CD stack for
+> review and then deployment.
+
+---
+
+## Design choices (locked this round)
+
+Recorded so later work does not silently reverse them. How to **select**
+each choice is in signed config (`secure-messaging.json` inside the wrapper).
+
+| Choice | Value | Why | Config key |
+|---|---|---|---|
+| Crypto production | NPE HPKE (ChaCha20-Poly1305) | Already the Verae production E2E; do not invent a second suite | `crypto.mode = npe` |
+| Crypto lab | `pfc-lab-xor` or `plain-lab` | Lets tests run without NPE sidecar | `crypto.mode = lab-xor \| plain-lab` |
+| Config signature | Ed25519 over canonical JSON | Stdlib in Go; `cryptography` in Python; no extra CA | `config.sig.alg = ed25519` |
+| Config wrapper | `{payload, sig:{alg,key_id,signature}}` | Unsigned files are **rejected** | required |
+| Admin history | Verae DataCube chain (JSONL) `kind=admin-history` | Matches existing cube append-only model | `admin.history_cube` |
+| History payload | previous file, new file, unified diff, actor, hashes | Operator asked for prev + new + diff | chain `type=admin-config` |
+| Routing | **passthrough**: destination in the clear; body encrypted | Brokers are untrusted | `routing.mode = passthrough` |
+| After send | Sender cannot decrypt own ciphertext; **lookup_id** only | Operator rule | envelope `from_lookup_id` |
+| System errors | Network Error Bundle + bounce report, sealed to **system public key** | Logging without opening mail | `crypto.system_key_id` |
+| Sender failure notice | Sealed to **sender** public key only | Only they can read status | error header `ct_sender` |
+| Dead letter | `verae.sm.dead` + bundle | Service-request failure | subject catalog |
+| Logs | **summaries** only (codes, lookup_id, dest class) | No PHI / no body | `logging.mode = summary` |
+| Languages | Python (readable spec) + Go (in-process NATS + leaf) | Operator rule | n/a |
+| ns1 | docs first, then CI review, then deploy | Operator rule | n/a |
+| Website org | inventory only this round | Not NATS endpoints | n/a |
+
+---
+
+## Internal TODO (dependency order)
+
+Insert **new** dependencies **above** the blocked item.
+
+- [x] **S00** Confirm orgs on git.georgelambert.org (`marchon`, `infra`, `websites`; `verae` empty)
+- [x] **S01** Save prompts into this file
+- [x] **S02** Inventory repos (see `INVENTORY.md`)
+- [x] **S03** Lock design choices (table above)
+- [x] **S04** Hub repo `system-git-sync`
+- [x] **S05** `nats-service-endpoints` catalog schema + seed from PFC subjects
+- [x] **S06** `secure-messaging` Python (signed config, passthrough, error bundle, admin history)
+- [x] **S07** `secure-messaging` Go leaf (in-process NATS + hub leaf)
+- [x] **S08** Tests + Forgejo Actions CI
+- [x] **S09** Sphinx HTML + LaTeX PDF with git.georgelambert.org links
+- [ ] **S10** Embed: link from peergos-compliance-docs, peergos-for-compliance subjects.json, verae-nats-bus (docs only on ns1)
+- [ ] **S11** Thesaurus pass (vocabulary / subject names)
+- [ ] **S12** Second full sync of docs across NATS-related repos
+- [ ] **S13** CI-CD review pipeline docs; **no** unsigned deploy to ns1
+- [ ] **S14** Wire NPE sidecar (blocked on `PFC_REQUIRE_NPE` live cutover — see UserReview)
+
+---
+
+## Progress log
+
+Append-only. Newest at the bottom.
+
+- 2026-09-16 — Started loop. Forgejo 11.0.16. Orgs: marchon (61 repos), infra (coredns, caddy), websites (70+ site trees), verae (no repos). Token used via osxkeychain; not stored in git.
+- 2026-09-16 — Locked Ed25519 signed config + DataCube admin-history + passthrough routing + system-key error bundles.
+- 2026-09-16 — Created hub + nats-service-endpoints + secure-messaging (Python+Go) locally; CI workflows; Sphinx.
+- 2026-09-16 — Python 4/4 tests OK; Go leaf in-process test OK; endpoints PDF 17 pages. ns1: docs-only, no binary deploy.
+
+---
+
+## Suggested later changes
+
+- Merge `verae.ts.batch` owner to timestamp leaf JWT in verae-nats-bus.
+- Console UI: regenerate signed config + append admin-history cube (after S10).
+- Caddy `sync.pfc.georgelambert.org` for this hub’s HTML (after CI review).

TODO-With-User.MD

@@ -0,0 +1,22 @@
+# TODO-With-User
+
+Blocks plus **Guesses and Assumptions**. Work continues around these.
+
+## Guesses and assumptions
+
+- **Orgs in scope for code:** `marchon` NATS/PFC repos. `infra` docs only. `websites` inventory only. `verae` org empty.
+- **System key:** generate lab Ed25519 signing key + X25519 system box key; private files gitignored; public keys in signed config payload.
+- **Passthrough:** NATS subject and `to` mailbox id are in the clear; ciphertext is HPKE (prod) or lab-xor (lab).
+- **Lookup ID:** HMAC-SHA256 of sender identity with a per-message nonce; not reversible from the broker.
+- **Error bundle:** two ciphertexts: `ct_sender` (NaCl/X25519 box to sender) and `ct_system` (box to system key) containing bounce metadata **without** recipient payload.
+- **DataCube history:** JSONL chain compatible with `pfc.chain.Chain` so a real cube can ingest it; lab can run without Peergos FUSE.
+- **CI:** Forgejo Actions (`ubuntu-latest`). If runners are missing, the YAML still documents the review gate.
+- **ns1:** no binary deploy until a green CI run is recorded. Docs HTML may be copied to `/opt/pfc/docs/` as documentation-only.
+- **Python comments:** every executable line in `secure_messaging/` has a trailing or preceding comment.
+- **Go:** in-process `nats-server` leaf to `nats-a:7422` when `SM_LEAF_HUB` is set; otherwise in-memory only for tests.
+
+## Open blocks (workaround in place)
+
+- No HSM on the laptop — lab keys only.
+- Central Verae timestamp leaf may be down — `local-dual` receipts already used by PFC; same here for error timestamps.
+- Cannot rewrite 70 website repos meaningfully this round.

TODO.md

@@ -0,0 +1,21 @@
+# TODO (dependency order)
+
+Status: `[ ]` pending · `[x]` done (committed).
+
+Insert **new** dependencies **above** the blocked item.
+
+- [x] S00 Scan orgs
+- [x] S01 Save prompts
+- [x] S02 Inventory
+- [x] S03 Lock signed-config + datacube-history + passthrough + error-bundle
+- [x] S04 Hub repo
+- [x] S05 NATS-Service-Endpoints catalog
+- [x] S06 Python spec implementation
+- [x] S07 Go leaf
+- [x] S08 Tests + CI
+- [x] S09 Sphinx HTML/PDF
+- [ ] S10 Link PFC + compliance-docs (docs on ns1 only)
+- [ ] S11 Thesaurus
+- [ ] S12 Second sync pass
+- [ ] S13 CI-CD review then deploy (blocked on S08 green + UserReview #2)
+- [ ] S14 NPE sidecar live (blocked on UserReview #2)

UserReview.MD

@@ -0,0 +1,10 @@
+# UserReview
+
+Decisions to confirm later. Work continued with guesses in `TODO-With-User.MD`.
+
+1. **System public key custody** — lab Ed25519/X25519 generated into `keys/system/` (gitignored private). Production should be customer HSM. Rotate via signed config + admin-history cube.
+2. **Cut over `PFC_REQUIRE_NPE=1` on ns1** — not in this round (would fail-close the live console).
+3. **Caddy hostname** for this hub (`sync.pfc.georgelambert.org` vs a path under `docs.pfc`). Guess: path `/sync/` under existing docs host after CI.
+4. **verae org** is empty — leave empty unless you want it as the canonical NATS catalog owner instead of `marchon`.
+5. **Website org** — no NATS changes. Confirm if any site should deep-link the endpoint catalog.
+6. **Admin console that re-signs config** — PFC console vs peergos-compliance-config TUI. Guess: both may call the same Python `signed_config.sign`; PFC console first.

docs/thesaurus.md

@@ -0,0 +1,17 @@
+# Thesaurus
+
+Canonical term → where it lives. Use this to rename later.
+
+| Term | Use this | Not this | Where |
+|---|---|---|---|
+| Untrusted broker | NATS | "secure bus" | nats-service-endpoints |
+| Production E2E | NPE / HPKE | "TLS to NATS" | secure-messaging |
+| Lab envelope | pfc-lab-xor | "xor cipher" alone | peergos-for-compliance |
+| Dest in clear | passthrough | "plaintext message" | envelope.py |
+| Sender handle | lookup_id | "from username on wire" | envelope.py |
+| Config file | signed wrapper | raw JSON config | signed_config.py |
+| Admin audit | DataCube admin-history | SQL log | admin_history.py |
+| Failure to sender | ct_sender | "email bounce plaintext" | error_bundle.py |
+| Ops bounce | ct_system / Network Error Bundle | "log the ciphertext body" | error_bundle.py |
+| Inspect officers | share names | Google accounts | peergos-for-compliance |
+| Console SSO | PFC TOTP/Google | Peergos Drive login | pfc_admin |