16 KiB
QA Retest — WP-M/N + BusinessStore v2 (MG3 probe)
Role: QA (evaluate only; no product feature implementation)
Date: 2026-10-03 (UTC probe ~06:34–06:37)
Host: NS1.GEORGELAMBERT.ORG / 70.88.205.138
Baseline: 2026-10-03-baseline.md
Prior: 2026-10-03-retest-wpl.md
Scope: Post WP-M (account-balance facade) + WP-N (CS/sales/acct booksWrites:false) retest against BusinessStore sqlite-file on live edge. Score userflows 1–7; judge MG3 prepaid-writer status.
Constraints honored: No DB clears; no intentional ledger adjusts; Go blockchain.db untouched (mtime still 2026-09-25). Meter appends only from normal stamp probes.
Summary judgment
WP-M/N lab progress is real: :3010 is a billing.db facade (facade:true, source:"billing.db"); GET /statement/cust_1 usage aligns with edge usage_entries (n=8 after this probe); CS :3011 / sales :3012 / accounting-export :3013 all advertise edge: http://127.0.0.1:3040 and booksWrites: false; freeze doc is present at docs/DUAL-WRITER-FREEZE.md; digests-only create+idempotent on :3040 still 202; upstream Verae batch with admin JWT still 202.
MG3 is not fully met. There is one live meter writer (zappier-edge SqliteFileBusinessStore → billing.db). There is not yet one prepaid/adjust writer on libSQL (LIBSQL_URL absent; BUSINESS_STORE=sqlite-file). Freeze text still forbids production use of verae.billing.balance.adjust until re-pointed. :3010 health remains nats:false. Flow 3 completedAt still epoch-zero. D004 leaf deny still 404, not policy deny. Do not start WP-O host split.
Checklist results
| # | Check | Result | Evidence |
|---|---|---|---|
| 1 | Edge BusinessStore sqlite-file still up; freeze doc at docs/DUAL-WRITER-FREEZE.md |
PASS | Journal (last start Oct 02 21:27:46): BusinessStore driver: sqlite-file, Billing customers/usage via SqliteFileBusinessStore (BILLING_DB). Secrets: BUSINESS_STORE=sqlite-file, PORT=3040, ZAPPIER_UPSTREAM=http://127.0.0.1:3130. File present: /opt/verae/zappier-edge/docs/DUAL-WRITER-FREEZE.md (350 B, authoritative meter+balance writes = edge BusinessStore; :3010 legacy facade). |
| 2 | :3010 statement cust_1 usage aligns with billing.db (facade) |
PASS | Pre-probe: statement usage n=6, source:"billing.db", prepaid 0; sqlite usage_entries cust_1 n=6 sum=60. Post-stamp: statement n=8, db n=8 sum=80, same source. Tip after probe verifyChain {"ok":true,"length":8,"tipHash":"abf1c625…639792a"}. Stale books.json still empty (mtime Sep 11) and not used for statement. |
| 3 | CS/sales/acct health shows edge URL and booksWrites: false |
PASS | :3011 CS {"edge":"http://127.0.0.1:3040","booksWrites":false,"nats":true}; :3012 sales same edge + booksWrites:false; :3013 accounting-export same. |
| 4 | Digest stamp flow on :3040 (create + idempotent) |
PASS | Digest sha256("qa-retest-mg3-2026-10-03")=d638ddc8…d7b919. POST1 → 202 created:1 spend 10; POST2 → 202 created:0 existing:1 spend 0. Channel api. Period spend headers 70 then 80 after single follow-up. |
| 5 | Optional Verae batch with admin token | PASS | POST /auth/login admin/admin123 → JWT. POST /api/batch/timestamp two digests-only items → 202 jobIds 1d027dcd-…, 8db4f651-…. |
| 6 | Userflows 1–7 scored | See table below | — |
| 7 | whattochange a–e for remaining gaps | See § whattochange | — |
| 8 | MG3: one prepaid writer now? | PARTIAL / not MG3 | See § MG3 status |
Health matrix (this probe)
| Port | Role | /health |
Notes |
|---|---|---|---|
:3040 |
zappier-edge | 200 role:zappier-edge |
BusinessStore sqlite-file |
:3130 |
middleware product | 200 mockVerae:false, natsConnected:true |
Edge upstream |
:3100 |
middleware mock | 200 mockVerae:true, natsConnected:false |
Do not smoke here |
:3010 |
account-balance | 200 facade:true, d006:true, nats:false, billingDb:/opt/verae/zappier-edge/billing.db, edgeUrl:http://127.0.0.1:3040 |
Read facade |
:3011 |
customer-service | 200 edge + booksWrites:false |
WP-N |
:3012 |
sales-pricing | 200 edge + booksWrites:false |
WP-N |
:3013 |
accounting-export | 200 edge + booksWrites:false |
WP-N |
:3021–:3025 |
access planes | 200 | web/api/leaf/zapier/staff |
Go DB (C9)
/var/lib/timestamping-svc/data/blockchain.db— mtime 2026-09-25, size ~31 MB. Not modified (stat only).
Dual-writer / statement vs meter (improved)
| Store | Observation |
|---|---|
Edge billing.db |
WAL; usage_entries length 8 after probe; verifyChain ok; Ada balance_cents=0 |
:3010/statement/cust_1 |
200 usage n=8, source:"billing.db", prepaid 0 — aligns with ledger |
Legacy books.json |
Exists under fleet runtime; empty counts; mtime Sep 11; not statement SoT |
| C16 | Mitigated for reads/meter: facade reads BusinessStore file; CS/sales/acct claim no books writes. Not closed for prepaid adjust: freeze still bans production balance.adjust; NATS on :3010 still false; store still sqlite-file not libSQL |
Userflow readiness 1–7
| # | Userflow | Score | Evidence / remaining blocker |
|---|---|---|---|
| 1 | Signup → API key → Statement | PARTIAL | Seed key-ada works; statement now matches billing.db (was empty at baseline/WPL). No portal signup E2E harness; prepaid still 0; facade nats:false. |
| 2 | Stamp digests single + batch + credit headers | PASS | Batch create+idempotent 202; credit headers; chain ok; upstream Verae batch 202. Still sqlite-file (not libSQL); zero-balance still meters. |
| 3 | Async complete → Zapier REST Hook | FAIL | Single POST /v1/timestamp receipt completedAt:"0001-01-01T00:00:00Z" (jobId a7c2842a-…); no watch→events→webhook E2E. |
| 4 | CS credit adjust → portal balance | PARTIAL | Health shows edge + booksWrites:false (WP-N intent). Freeze still says do not use :3010 adjust for production credits; no adjust E2E in this QA (by design: no intentional ledger writes). |
| 5 | Sales quote → meter match | PARTIAL | Sales :3012 points at edge, booksWrites:false. Rate/quote↔meter harness not run; rates still edge SQLite. |
| 6 | Accounting export consistency | PARTIAL | Acct :3013 edge + booksWrites:false; statement↔usage now consistent so export can reconcile. Export E2E not executed. |
| 7 | Leaf stamp OK / billing deny | PARTIAL | :3023/health 200; POST :3023/v1/billing/adjust → 404 not found (≠ D004 automated policy deny). |
Vs WPL retest: flows 1, 4–6 moved from broken/unproven toward PARTIAL because facade statement aligns and staff planes advertise no books writes. Flow 2 remains green. Flows 3 and 7 unchanged in failure mode.
MG3 status — is there one prepaid writer now?
Short answer: one meter writer, yes; full MG3 prepaid/libSQL writer, no.
| MG3 criterion (PM board) | This probe |
|---|---|
| One prepaid writer; second paths removed | Partial. Meter appends only via edge BusinessStore → billing.db. :3010 is read facade over that file. CS/sales/acct booksWrites:false. Stale books.json not serving statement. |
Runtime: one LIBSQL_URL serves customers/pricing/usage/invoices/balances |
FAIL. No LIBSQL_URL; BUSINESS_STORE=sqlite-file. |
Grep: no prod SoT via books.json / per-host billing.db |
Partial. Lab still uses host-local billing.db as SoT (accepted freeze stand-in until libSQL). books.json deprecated for statement but file remains. |
| D004 leaf cannot adjust | Weak. 404 only; not automated deny proof. |
| Freeze / adjust path | Freeze explicitly still blocks production verae.billing.balance.adjust until re-pointed at BusinessStore/libSQL → prepaid adjust writer not proven unified. |
Verdict: Treat WP-M/N sqlite-file facade cutover as accepted for lab read-path + meter SoT alignment. Do not mark MG3 PASS. MG3 still requires libSQL SoT + proven single adjust path (or documented fold of adjust into edge BusinessStore) + D004 tests.
a) What to change
| # | Component | Problem | Evidence | Severity | Type |
|---|---|---|---|---|---|
| 1 | Missing libSQL SoT (D006 / MG3) | Lab still sqlite-file billing.db; no LIBSQL_URL |
secrets BUSINESS_STORE=sqlite-file; journal |
critical | missing foundation |
| 2 | Prepaid adjust path still frozen | Freeze forbids production balance.adjust; :3010 nats:false |
freeze doc; health JSON | critical | incomplete WP-M |
| 3 | Async receipt / flow 3 | completedAt epoch-zero on single stamp |
curl body 0001-01-01T00:00:00Z |
high | incomplete product path |
| 4 | Userflow harness (WP-P) | Flows 1–7 not automated; 4–6 only health-level | this manual retest | high | missing gate |
| 5 | D004 leaf deny | 404 ≠ policy deny | curl 404 | medium | missing test |
| 6 | Zero-balance metering | Ada balance_cents=0 still stamps |
customers row + Balance header 0 | medium | missing prepaid gate |
| 7 | Stale books.json artifact |
Empty file remains in fleet runtime; confusion risk | path under verae-fleet/.../books.json |
low | ops hygiene |
What is working (preserve)
- Live
BusinessStore driver: sqlite-file+ explicitBUSINESS_STORE=sqlite-filein secrets. docs/DUAL-WRITER-FREEZE.mdon live edge.:3010facade readingbilling.db; statement ==usage_entriesforcust_1.- CS/sales/acct
booksWrites:false+ edge URL. - Digests-only batch create+idempotent on
:3040with credit headers;verifyChainok. - Product upstream
:3130; Verae Time/auth/login+/api/batch/timestampdigests-only. - Access planes healthy; Go
blockchain.dbuntouched; no JetStream used for job-complete in this probe.
b) What to change it to
| # | Component | Target behavior | Acceptance criteria | Constraints |
|---|---|---|---|---|
| 1 | libSQL BusinessStore | BUSINESS_STORE=libsql + LIBSQL_URL; fail closed 503 if writer down |
Migrator tip verify archived; live meter only to libSQL | Auth stays zappier.db (C13) |
| 2 | Single adjust writer | CS adjust / verae.billing.balance.adjust → same BusinessStore; unfreeze after proof |
Adjust visible on statement + edge balance headers; freeze doc updated | D001; do not dual-append chain; no books.json writes |
| 3 | Async complete | Real completedAt; hook delivery path |
E2E jobId → terminal event (+ hook attempt) | verae.ts.* core NATS only |
| 4 | Harness 1–7 | Automated report under plans/qa/ |
Flows 1,2,4 MOCK then 3,5,6,7; exit 0 | — |
| 5 | D004 | Automated leaf deny | Tests green; live deny ≠ bare 404 | Leaf may still stamp |
| 6 | Prepaid gate | Documented 402 vs allow-with-debt | Seed funded + zero-balance fixture | Do not rewrite historical ledger |
| 7 | Retire books.json | Import-only / archived; not in runtime SoT path | Ops note; facade never falls back to books for money | — |
c) How to implement
Aligned with PM board; do not skip merge gates.
flowchart TD
WPM_done[WP-M facade reads billing.db] --> WPN_done[WP-N booksWrites false]
WPN_done --> AdjustProof[Prove adjust via BusinessStore / NATS]
AdjustProof --> LibSQL[WP-K/O libSQL cutover + verifyChain]
LibSQL --> MG3[MG3 PASS: one prepaid writer on libSQL]
MG3 --> Harness[WP-P harness 1-7]
MG3 --> Async[Fix completedAt / hook path]
Harness --> MG4[MG4]
Phase A — Close adjust writer (hours–days)
- Point
:3010NATS adjust (or fold into edge HTTP) at BusinessStore; connect NATS or document HTTP-only facade. - Update freeze doc when adjust is safe; keep meter-only rule until then.
- Prove: CS adjust → statement prepaid/credits change == edge headers (lab only).
Phase B — MG3 libSQL
- Migrator
billing.db→ libSQL; refuse onverifyChainfail; archive tip underplans/qa/. - Flip
BUSINESS_STORE=libsql; facade/BILLING_DBread path follows libSQL or HTTP billing API. - Grep proof: no production money writes to
books.jsonor private per-host SoT.
Phase C — Product paths + harness
- Fix single-stamp
completedAt; prove flow 3 under harness (MOCK first). - Automate flows 1–7; D004 deny tests; prepaid policy fixture.
Validation for this retest exit: checklist 1–5 green ⇒ WP-M/N sqlite-file facade accepted. MG3 / libSQL / adjust proof remain open — not deploy regressions.
d) Expected results — and what remains incomplete
After planned changes
| Metric / behavior | Today (this retest) | Expected after Phases A–C / MG3 |
|---|---|---|
| BusinessStore on edge | sqlite-file live | libSQL primary |
| Statement vs meter | Aligned (n=8) | Same on libSQL |
| CS/sales/acct booksWrites | false | false + proven adjust/export |
| Freeze doc | Present; adjust banned | Present; adjust allowed via single writer |
Flow 3 completedAt |
0001-01-01… |
Real completion time |
| Flow 7 D004 | 404 only | Automated deny |
blockchain.db |
Untouched | Still untouched |
Still missing even after those todos
| Gap | Why |
|---|---|
| Zapier Platform live push / workspace Zaps | Operator Zapier login; out of cutover buffer |
| NATS nkeys/mTLS | Lab isolation is network, not accounts |
| libSQL HA / multi-primary | Launch = one primary + backup |
| Peergos / Go timestamping redesign | Out of scope |
Success tiers
| Tier | Status after this retest |
|---|---|
| Sqlite-file BusinessStore + freeze doc | Met |
WP-M read facade (statement == ledger) |
Met |
WP-N health (booksWrites:false + edge URL) |
Met |
| MG3 single prepaid writer on libSQL | Not met |
| WP-Q / double userflow green | Not met |
e) Prioritized todo list
Item 1 — [P0] — Prove prepaid adjust through BusinessStore (finish WP-M write path)
Wire balance.adjust (NATS and/or edge HTTP) to same store meters use; update freeze when safe; keep books.json read-only/archived.
PreReqs: (none for lab sqlite-file; libSQL preferred)
Item 2 — [P0] — libSQL cutover with verifyChain gate (WP-K → MG3)
Refuse cutover if chain fails; archive tip under plans/qa/; set BUSINESS_STORE=libsql + LIBSQL_URL.
PreReqs: (Item 1 or explicit waiver that meter-only sqlite-file remains until cutover day)
Item 3 — [P0] — MG3 proof artifact
Grep + runtime proof: one writer; statement/adjust/meter same tip; no books.json SoT.
PreReqs: (Item 2)
Item 4 — [P1] — Fix single-stamp completedAt / async receipt (flow 3)
Replace 0001-01-01 placeholders on product path used by Zapier wait/hook; core NATS only for verae.ts.*.
PreReqs: (none)
Item 5 — [P1] — Automate D004 leaf billing deny (WP-D)
Replace 404-only smoke with authz allow-list tests.
PreReqs: (none)
Item 6 — [P1] — Userflow harness flows 1, 2, 4 then 3, 5, 6, 7 (WP-P)
MOCK_VERAE report under plans/qa/; MG4.
PreReqs: (Item 3 + Item 5)
Item 7 — [P2] — Prepaid zero-balance policy gate
Define 402 vs allow-with-debt; seed funded fixture.
PreReqs: (Item 1)
Item 8 — [P2] — Multi-host LIBSQL_URL topology (WP-O)
Only after MG3 PASS.
PreReqs: (Item 3)
Dependency order (Do all): Item 4 ∥ Item 5 → Item 1 → Item 2 → Item 3 → Item 6 → Item 7 → Item 8.
Probe artifacts (raw)
- Batch digest:
d638ddc81b601d6f025db6b982f42ce5abaee5aba0b6b2d496c57d764ed7b919(qa-retest-mg3-2026-10-03) - Batch jobId:
231a1233-8a5c-49ee-8139-de8625039fe6 - Single digest:
f94d25942a900c8ca017aa5f4a1ddfbc071a0cc5993a575c1e058d707fdeb28b(qa-single-mg3) - Single jobId:
a7c2842a-13b0-4196-a14b-4ec0c60f7013(completedAtepoch-zero) - Chain tip after probe:
abf1c6253f4ef38fd42e06ca7a167883a64bac0450d0c94454318fd0c639792a(length 8) - Upstream Verae jobIds:
1d027dcd-e976-4ca7-bb33-46f8275fe2b8,8db4f651-ad06-476c-bc8d-5356bc079710 - Statement post-probe:
usage_n=8,source=billing.db,prepaidCents=0
QA note: WP-M/N facade + booksWrites:false deploy is accepted for lab read/meter alignment. MG3 remains open until libSQL + proven single prepaid adjust writer. Formal WP-Q churn still waits on MG3 + harness skeleton.