# QA Retest — WP-M/N + BusinessStore v2 (MG3 probe) **Role:** QA (evaluate only; no product feature implementation) **Date:** 2026-10-03 (UTC probe ~06:34–06:37) **Host:** `NS1.GEORGELAMBERT.ORG` / `70.88.205.138` **Baseline:** [`2026-10-03-baseline.md`](./2026-10-03-baseline.md) **Prior:** [`2026-10-03-retest-wpl.md`](./2026-10-03-retest-wpl.md) **Scope:** Post WP-M (account-balance facade) + WP-N (CS/sales/acct `booksWrites:false`) retest against BusinessStore sqlite-file on live edge. Score userflows 1–7; judge MG3 prepaid-writer status. **Constraints honored:** No DB clears; no intentional ledger adjusts; Go `blockchain.db` **untouched** (mtime still 2026-09-25). Meter appends only from normal stamp probes. --- ## Summary judgment **WP-M/N lab progress is real:** `:3010` is a **billing.db facade** (`facade:true`, `source:"billing.db"`); `GET /statement/cust_1` usage **aligns** with edge `usage_entries` (n=8 after this probe); CS `:3011` / sales `:3012` / accounting-export `:3013` all advertise `edge: http://127.0.0.1:3040` and **`booksWrites: false`**; freeze doc is present at [`docs/DUAL-WRITER-FREEZE.md`](file:///opt/verae/zappier-edge/docs/DUAL-WRITER-FREEZE.md); digests-only create+idempotent on `:3040` still **202**; upstream Verae batch with admin JWT still **202**. **MG3 is not fully met.** There is **one live meter writer** (zappier-edge `SqliteFileBusinessStore` → `billing.db`). There is **not** yet one prepaid/adjust writer on **libSQL** (`LIBSQL_URL` absent; `BUSINESS_STORE=sqlite-file`). Freeze text still forbids production use of `verae.billing.balance.adjust` until re-pointed. `:3010` health remains `nats:false`. Flow 3 `completedAt` still epoch-zero. D004 leaf deny still **404**, not policy deny. Do **not** start WP-O host split. --- ## Checklist results | # | Check | Result | Evidence | |---|--------|--------|----------| | 1 | Edge BusinessStore sqlite-file still up; freeze doc at `docs/DUAL-WRITER-FREEZE.md` | **PASS** | Journal (last start `Oct 02 21:27:46`): `BusinessStore driver: sqlite-file`, `Billing customers/usage via SqliteFileBusinessStore (BILLING_DB)`. Secrets: `BUSINESS_STORE=sqlite-file`, `PORT=3040`, `ZAPPIER_UPSTREAM=http://127.0.0.1:3130`. File present: `/opt/verae/zappier-edge/docs/DUAL-WRITER-FREEZE.md` (350 B, authoritative meter+balance writes = edge BusinessStore; `:3010` legacy facade). | | 2 | `:3010` statement `cust_1` usage aligns with `billing.db` (facade) | **PASS** | Pre-probe: statement usage n=6, `source:"billing.db"`, prepaid 0; sqlite `usage_entries` cust_1 n=6 sum=60. Post-stamp: statement n=8, db n=8 sum=80, same source. Tip after probe `verifyChain` `{"ok":true,"length":8,"tipHash":"abf1c625…639792a"}`. Stale `books.json` still empty (mtime Sep 11) and **not** used for statement. | | 3 | CS/sales/acct health shows edge URL and `booksWrites: false` | **PASS** | `:3011` CS `{"edge":"http://127.0.0.1:3040","booksWrites":false,"nats":true}`; `:3012` sales same edge + `booksWrites:false`; `:3013` accounting-export same. | | 4 | Digest stamp flow on `:3040` (create + idempotent) | **PASS** | Digest `sha256("qa-retest-mg3-2026-10-03")`=`d638ddc8…d7b919`. POST1 → **202** `created:1` spend **10**; POST2 → **202** `created:0` `existing:1` spend **0**. Channel `api`. Period spend headers 70 then 80 after single follow-up. | | 5 | Optional Verae batch with admin token | **PASS** | `POST /auth/login` admin/admin123 → JWT. `POST /api/batch/timestamp` two digests-only items → **202** `jobIds` `1d027dcd-…`, `8db4f651-…`. | | 6 | Userflows 1–7 scored | See table below | — | | 7 | whattochange a–e for remaining gaps | See § whattochange | — | | 8 | MG3: one prepaid writer now? | **PARTIAL / not MG3** | See § MG3 status | ### Health matrix (this probe) | Port | Role | `/health` | Notes | |------|------|-----------|-------| | `:3040` | zappier-edge | **200** `role:zappier-edge` | BusinessStore sqlite-file | | `:3130` | middleware product | **200** `mockVerae:false`, `natsConnected:true` | Edge upstream | | `:3100` | middleware mock | **200** `mockVerae:true`, `natsConnected:false` | Do not smoke here | | `:3010` | account-balance | **200** `facade:true`, `d006:true`, `nats:false`, `billingDb:/opt/verae/zappier-edge/billing.db`, `edgeUrl:http://127.0.0.1:3040` | Read facade | | `:3011` | customer-service | **200** edge + `booksWrites:false` | WP-N | | `:3012` | sales-pricing | **200** edge + `booksWrites:false` | WP-N | | `:3013` | accounting-export | **200** edge + `booksWrites:false` | WP-N | | `:3021`–`:3025` | access planes | **200** | web/api/leaf/zapier/staff | ### Go DB (C9) - `/var/lib/timestamping-svc/data/blockchain.db` — mtime **2026-09-25**, size ~31 MB. **Not modified** (stat only). --- ## Dual-writer / statement vs meter (improved) | Store | Observation | |-------|-------------| | Edge `billing.db` | WAL; `usage_entries` length **8** after probe; `verifyChain` **ok**; Ada `balance_cents=0` | | `:3010/statement/cust_1` | **200** usage n=**8**, `source:"billing.db"`, prepaid 0 — **aligns** with ledger | | Legacy `books.json` | Exists under fleet runtime; empty counts; mtime Sep 11; **not** statement SoT | | C16 | **Mitigated for reads/meter:** facade reads BusinessStore file; CS/sales/acct claim no books writes. **Not closed for prepaid adjust:** freeze still bans production `balance.adjust`; NATS on `:3010` still `false`; store still sqlite-file not libSQL | --- ## Userflow readiness 1–7 | # | Userflow | Score | Evidence / remaining blocker | |---|----------|-------|------------------------------| | 1 | Signup → API key → Statement | **PARTIAL** | Seed `key-ada` works; statement now matches `billing.db` (was empty at baseline/WPL). No portal signup E2E harness; prepaid still 0; facade `nats:false`. | | 2 | Stamp digests single + batch + credit headers | **PASS** | Batch create+idempotent 202; credit headers; chain ok; upstream Verae batch 202. Still sqlite-file (not libSQL); zero-balance still meters. | | 3 | Async complete → Zapier REST Hook | **FAIL** | Single `POST /v1/timestamp` receipt `completedAt:"0001-01-01T00:00:00Z"` (jobId `a7c2842a-…`); no watch→events→webhook E2E. | | 4 | CS credit adjust → portal balance | **PARTIAL** | Health shows edge + `booksWrites:false` (WP-N intent). Freeze still says do not use `:3010` adjust for production credits; no adjust E2E in this QA (by design: no intentional ledger writes). | | 5 | Sales quote → meter match | **PARTIAL** | Sales `:3012` points at edge, `booksWrites:false`. Rate/quote↔meter harness not run; rates still edge SQLite. | | 6 | Accounting export consistency | **PARTIAL** | Acct `:3013` edge + `booksWrites:false`; statement↔usage now consistent so export *can* reconcile. Export E2E not executed. | | 7 | Leaf stamp OK / billing deny | **PARTIAL** | `:3023/health` 200; `POST :3023/v1/billing/adjust` → **404** `not found` (≠ D004 automated policy deny). | **Vs WPL retest:** flows 1, 4–6 moved from broken/unproven toward **PARTIAL** because facade statement aligns and staff planes advertise no books writes. Flow 2 remains green. Flows 3 and 7 unchanged in failure mode. --- ## MG3 status — is there one prepaid writer now? **Short answer: one meter writer, yes; full MG3 prepaid/libSQL writer, no.** | MG3 criterion (PM board) | This probe | |--------------------------|------------| | One prepaid writer; second paths removed | **Partial.** Meter appends only via edge BusinessStore → `billing.db`. `:3010` is read facade over that file. CS/sales/acct `booksWrites:false`. Stale `books.json` not serving statement. | | Runtime: one `LIBSQL_URL` serves customers/pricing/usage/invoices/balances | **FAIL.** No `LIBSQL_URL`; `BUSINESS_STORE=sqlite-file`. | | Grep: no prod SoT via `books.json` / per-host billing.db | **Partial.** Lab still uses host-local `billing.db` as SoT (accepted freeze stand-in until libSQL). `books.json` deprecated for statement but file remains. | | D004 leaf cannot adjust | **Weak.** 404 only; not automated deny proof. | | Freeze / adjust path | Freeze **explicitly** still blocks production `verae.billing.balance.adjust` until re-pointed at BusinessStore/libSQL → prepaid **adjust** writer not proven unified. | **Verdict:** Treat WP-M/N sqlite-file facade cutover as **accepted for lab read-path + meter SoT alignment**. **Do not mark MG3 PASS.** MG3 still requires libSQL SoT + proven single adjust path (or documented fold of adjust into edge BusinessStore) + D004 tests. --- ## a) What to change | # | Component | Problem | Evidence | Severity | Type | |---|-----------|---------|----------|----------|------| | 1 | Missing libSQL SoT (D006 / MG3) | Lab still sqlite-file `billing.db`; no `LIBSQL_URL` | secrets `BUSINESS_STORE=sqlite-file`; journal | **critical** | missing foundation | | 2 | Prepaid adjust path still frozen | Freeze forbids production `balance.adjust`; `:3010` `nats:false` | freeze doc; health JSON | **critical** | incomplete WP-M | | 3 | Async receipt / flow 3 | `completedAt` epoch-zero on single stamp | curl body `0001-01-01T00:00:00Z` | **high** | incomplete product path | | 4 | Userflow harness (WP-P) | Flows 1–7 not automated; 4–6 only health-level | this manual retest | **high** | missing gate | | 5 | D004 leaf deny | 404 ≠ policy deny | curl 404 | **medium** | missing test | | 6 | Zero-balance metering | Ada `balance_cents=0` still stamps | customers row + Balance header 0 | **medium** | missing prepaid gate | | 7 | Stale `books.json` artifact | Empty file remains in fleet runtime; confusion risk | path under `verae-fleet/.../books.json` | **low** | ops hygiene | ### What is working (preserve) - Live `BusinessStore driver: sqlite-file` + explicit `BUSINESS_STORE=sqlite-file` in secrets. - `docs/DUAL-WRITER-FREEZE.md` on live edge. - `:3010` facade reading `billing.db`; statement == `usage_entries` for `cust_1`. - CS/sales/acct `booksWrites:false` + edge URL. - Digests-only batch create+idempotent on `:3040` with credit headers; `verifyChain` ok. - Product upstream `:3130`; Verae Time `/auth/login` + `/api/batch/timestamp` digests-only. - Access planes healthy; Go `blockchain.db` untouched; no JetStream used for job-complete in this probe. --- ## b) What to change it to | # | Component | Target behavior | Acceptance criteria | Constraints | |---|-----------|-----------------|---------------------|-------------| | 1 | libSQL BusinessStore | `BUSINESS_STORE=libsql` + `LIBSQL_URL`; fail closed 503 if writer down | Migrator tip verify archived; live meter only to libSQL | Auth stays `zappier.db` (C13) | | 2 | Single adjust writer | CS adjust / `verae.billing.balance.adjust` → same BusinessStore; unfreeze after proof | Adjust visible on statement + edge balance headers; freeze doc updated | D001; do not dual-append chain; no books.json writes | | 3 | Async complete | Real `completedAt`; hook delivery path | E2E jobId → terminal event (+ hook attempt) | `verae.ts.*` **core NATS only** | | 4 | Harness 1–7 | Automated report under `plans/qa/` | Flows 1,2,4 MOCK then 3,5,6,7; exit 0 | — | | 5 | D004 | Automated leaf deny | Tests green; live deny ≠ bare 404 | Leaf may still stamp | | 6 | Prepaid gate | Documented 402 vs allow-with-debt | Seed funded + zero-balance fixture | Do not rewrite historical ledger | | 7 | Retire books.json | Import-only / archived; not in runtime SoT path | Ops note; facade never falls back to books for money | — | --- ## c) How to implement Aligned with PM board; **do not skip merge gates**. ```mermaid flowchart TD WPM_done[WP-M facade reads billing.db] --> WPN_done[WP-N booksWrites false] WPN_done --> AdjustProof[Prove adjust via BusinessStore / NATS] AdjustProof --> LibSQL[WP-K/O libSQL cutover + verifyChain] LibSQL --> MG3[MG3 PASS: one prepaid writer on libSQL] MG3 --> Harness[WP-P harness 1-7] MG3 --> Async[Fix completedAt / hook path] Harness --> MG4[MG4] ``` ### Phase A — Close adjust writer (hours–days) - Point `:3010` NATS adjust (or fold into edge HTTP) at BusinessStore; connect NATS or document HTTP-only facade. - Update freeze doc when adjust is safe; keep meter-only rule until then. - Prove: CS adjust → statement prepaid/credits change == edge headers (lab only). ### Phase B — MG3 libSQL - Migrator `billing.db` → libSQL; refuse on `verifyChain` fail; archive tip under `plans/qa/`. - Flip `BUSINESS_STORE=libsql`; facade/`BILLING_DB` read path follows libSQL or HTTP billing API. - Grep proof: no production money writes to `books.json` or private per-host SoT. ### Phase C — Product paths + harness - Fix single-stamp `completedAt`; prove flow 3 under harness (MOCK first). - Automate flows 1–7; D004 deny tests; prepaid policy fixture. **Validation for this retest exit:** checklist 1–5 green ⇒ WP-M/N sqlite-file facade **accepted**. MG3 / libSQL / adjust proof remain open — not deploy regressions. --- ## d) Expected results — and what remains incomplete ### After planned changes | Metric / behavior | Today (this retest) | Expected after Phases A–C / MG3 | |-------------------|---------------------|----------------------------------| | BusinessStore on edge | sqlite-file live | libSQL primary | | Statement vs meter | **Aligned** (n=8) | Same on libSQL | | CS/sales/acct booksWrites | false | false + proven adjust/export | | Freeze doc | Present; adjust banned | Present; adjust allowed via single writer | | Flow 3 `completedAt` | `0001-01-01…` | Real completion time | | Flow 7 D004 | 404 only | Automated deny | | `blockchain.db` | Untouched | Still untouched | ### Still missing even after those todos | Gap | Why | |-----|-----| | Zapier Platform live push / workspace Zaps | Operator Zapier login; out of cutover buffer | | NATS nkeys/mTLS | Lab isolation is network, not accounts | | libSQL HA / multi-primary | Launch = one primary + backup | | Peergos / Go timestamping redesign | Out of scope | ### Success tiers | Tier | Status after this retest | |------|--------------------------| | Sqlite-file BusinessStore + freeze doc | **Met** | | WP-M read facade (`statement` == ledger) | **Met** | | WP-N health (`booksWrites:false` + edge URL) | **Met** | | MG3 single prepaid writer on libSQL | **Not met** | | WP-Q / double userflow green | **Not met** | --- ## e) Prioritized todo list ``` Item 1 — [P0] — Prove prepaid adjust through BusinessStore (finish WP-M write path) Wire balance.adjust (NATS and/or edge HTTP) to same store meters use; update freeze when safe; keep books.json read-only/archived. PreReqs: (none for lab sqlite-file; libSQL preferred) Item 2 — [P0] — libSQL cutover with verifyChain gate (WP-K → MG3) Refuse cutover if chain fails; archive tip under plans/qa/; set BUSINESS_STORE=libsql + LIBSQL_URL. PreReqs: (Item 1 or explicit waiver that meter-only sqlite-file remains until cutover day) Item 3 — [P0] — MG3 proof artifact Grep + runtime proof: one writer; statement/adjust/meter same tip; no books.json SoT. PreReqs: (Item 2) Item 4 — [P1] — Fix single-stamp completedAt / async receipt (flow 3) Replace 0001-01-01 placeholders on product path used by Zapier wait/hook; core NATS only for verae.ts.*. PreReqs: (none) Item 5 — [P1] — Automate D004 leaf billing deny (WP-D) Replace 404-only smoke with authz allow-list tests. PreReqs: (none) Item 6 — [P1] — Userflow harness flows 1, 2, 4 then 3, 5, 6, 7 (WP-P) MOCK_VERAE report under plans/qa/; MG4. PreReqs: (Item 3 + Item 5) Item 7 — [P2] — Prepaid zero-balance policy gate Define 402 vs allow-with-debt; seed funded fixture. PreReqs: (Item 1) Item 8 — [P2] — Multi-host LIBSQL_URL topology (WP-O) Only after MG3 PASS. PreReqs: (Item 3) ``` **Dependency order (Do all):** Item 4 ∥ Item 5 → Item 1 → Item 2 → Item 3 → Item 6 → Item 7 → Item 8. --- ## Probe artifacts (raw) - Batch digest: `d638ddc81b601d6f025db6b982f42ce5abaee5aba0b6b2d496c57d764ed7b919` (`qa-retest-mg3-2026-10-03`) - Batch jobId: `231a1233-8a5c-49ee-8139-de8625039fe6` - Single digest: `f94d25942a900c8ca017aa5f4a1ddfbc071a0cc5993a575c1e058d707fdeb28b` (`qa-single-mg3`) - Single jobId: `a7c2842a-13b0-4196-a14b-4ec0c60f7013` (`completedAt` epoch-zero) - Chain tip after probe: `abf1c6253f4ef38fd42e06ca7a167883a64bac0450d0c94454318fd0c639792a` (length 8) - Upstream Verae jobIds: `1d027dcd-e976-4ca7-bb33-46f8275fe2b8`, `8db4f651-ad06-476c-bc8d-5356bc079710` - Statement post-probe: `usage_n=8`, `source=billing.db`, `prepaidCents=0` **QA note:** WP-M/N facade + `booksWrites:false` deploy is **accepted** for lab read/meter alignment. **MG3 remains open** until libSQL + proven single prepaid adjust writer. Formal WP-Q churn still waits on MG3 + harness skeleton.