docs: full Go MCP cutover test report (2026-09-29)

15 cases: 13 pass, 2 fail (laptop VeraeTime auth only). Includes unit,
public MCP, 20x global idempotent stamp, ApplicationClient, tsnats,
site pages, and sibling hostnames.
This commit is contained in:
George Lambert 2026-09-29 12:29:12 -04:00
commit 16609f5259
24 changed files with 319 additions and 0 deletions

View file

@ -0,0 +1,140 @@
# Go MCP cutover — full test report
**Run ID:** `2026-09-29T162528Z-go-mcp-cutover`
**Finished (UTC):** 2026-09-29T16:28:00Z (approx)
**Subject under test:** Go `verae-mcp-timestamping-server` on ns1 replacing Python
**Binary:** `/opt/verae/verae-mcp-timestamping-server-go/bin/verae-mcp-timestamping-server`
**Git (local):** `a249564` (+ Caddy fix for `mcp.georgelambert.org` during this run)
**Forgejo:** https://git.georgelambert.org/marchon/verae-mcp-timestamping-server-go
## Executive summary
| Metric | Value |
| --- | --- |
| Cases run | **15** |
| Passed | **13** |
| Failed | **2** |
| Go MCP live path | **Healthy** (`127.0.0.1:8000`, embedded NATS `127.0.0.1:4229`) |
Both failures are **laptop-side VeraeTime auth** (`~/.verae/auth.properties` token expired / missing), not Go MCP regressions. Equivalent global idempotency was proven through the live Go MCP path (L3, L12).
## Deploy snapshot
| Setting | Value |
| --- | --- |
| systemd unit | `verae-mcp-timestamping.service` |
| Drop-ins | `go-cutover.conf`, `verae-ai.conf` |
| `VERAE_MCP_PORT` | `8000` |
| `VERAE_NATS_LISTEN` | `127.0.0.1:4229` |
| `VERAE_MCP_REQUIRE_BEARER` | `0` (host-token fallback) |
| `VERAE_AUTH_PROPERTIES` | `/var/lib/verae-mcp/auth.properties` |
| Public hosts | `mcp.verae.ai`, `mcp.veraetime.net`, `mcp.georgelambert.org` → `127.0.0.1:8000` |
**During this run:** `mcp.georgelambert.org` Caddy upstream was corrected from `10.1.10.128:8000` (502) to `127.0.0.1:8000`. A `.bak` file that caused “ambiguous site definition” was moved out of `/etc/caddy/sites/`.
## Results matrix
| ID | Case | Status | Notes |
| --- | --- | --- | --- |
| U1 | Go unit tests (`go test ./...`) | **PASS** | authctx, embednats, ledger, server, tools, veraetime, digest, tsnats |
| U2 | Python client unit tests | **PASS** | 47 passed |
| U3 | Python server unit tests | **PASS** | 34 passed |
| L1 | Public MCP suite (`mcp.verae.ai`) | **PASS** | tools/list (7 tools), global hash_report, local stamp+verify, session valid admin |
| L2 | MCP global stamp (unique digest) | **PASS** | host-token fallback |
| L3 | MCP 20× same digest global stamp | **PASS** | unique=1, same=True, ~3.92s, CID `QmXgqQn8…` |
| L4 | Python `ApplicationClient` via MCP | **PASS** | local stamp, verify, hash_report |
| L5 | `ApplicationClient` direct VeraeTime | **FAIL** | HTTP 401 — laptop token missing/expired |
| L6 | CLI `verae-timestamp hash` | **PASS** | 64-hex digest |
| L7 | `transaction_log_test.py` (laptop → API) | **FAIL** | 20× HTTP 401 token expired; unique 0 / same False |
| L8 | `mcp.veraetime.net` tools/list | **PASS** | same Go process |
| L9 | `mcp.georgelambert.org` MCP | **PASS** | after Caddy upstream fix; session OK |
| L10 | `pkg/tsnats` → embedded NATS `:4229` | **PASS** | local stamp via SSH tunnel |
| L11 | Static site pages on `mcp.verae.ai` | **PASS** | `/`, developers, ways, openapi, app, explorer |
| L12 | ns1 loopback 5× global idempotent | **PASS** | unique=1, same CID `QmXgqQn8…` |
## Case details
### U1 — Go unit tests
All packages green. No FAIL lines in `logs/u1-go-unit.txt`.
### U2 / U3 — Python suites
Client 47 passed; server 34 passed. Confirms Application Client and Python GlobalClient libraries still build/test while Go owns production MCP.
### L1 — Public MCP suite (`https://mcp.verae.ai/mcp`)
1. `initialize` → `Mcp-Session-Id` present (stateful).
2. `tools/list` → `create_user`, `hash_report`, `login`, `session`, `signup`, `stamp`, `verify`.
3. `hash_report` global on Verae-Debug-Test digest → receipt `QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ`.
4. `stamp` scope=local → receipt issued; `verify` OK.
5. `session` → `valid: true`, `username: admin`, `role: admin` (host-token keeper).
### L2 / L3 — Global stamp paths
- L2: fresh digest stamped globally without client Bearer (`REQUIRE_BEARER=0`).
- L3: 20 sequential global stamps of digest `469e6159…` all returned the same CID; elapsed ~3.92s.
### L4 — ApplicationClient (MCP transport)
Local stamp → verify → hash_report through `https://mcp.verae.ai/mcp` succeeded.
### L5 / L7 — Laptop direct VeraeTime (expected fail)
These call `api.veraetime.net` with the **developer laptop** auth file, which returned **401 Unauthorized: token has expired** (L7) / auth required (L5). They do not exercise the ns1 Go process.
**Coverage substitute:** L3 + L12 prove the same digest idempotency and host-token path via Go MCP.
### L6 — CLI hash
`verae-timestamp hash` produced a valid SHA-256.
### L8 / L9 — Sibling public hostnames
- `mcp.veraetime.net`: PASS.
- `mcp.georgelambert.org`: initially 502 (wrong upstream); after Caddy fix → PASS.
### L10 — Embedded NATS client
`pkg/tsnats.Connect(nats://127.0.0.1:14229)` (tunnel to ns1 `:4229`) → `Stamp` scope=local → receipt ID returned.
### L11 — Static docs site
HTTP 200 for index, developers, ways, OpenAPI YAML, app, explorer.
### L12 — On-host MCP idempotency
From ns1 to `127.0.0.1:8000` with `Host: mcp.verae.ai`, 5× global stamp of Verae-Debug-Test → one CID.
## Failures — root cause and remediation
| ID | Root cause | Remediation |
| --- | --- | --- |
| L5 | No valid laptop `~/.verae/auth.properties` / `VERAE_GLOBAL_TOKEN` for direct API | Run `login` / refresh token on the laptop; not a Go cutover blocker |
| L7 | Same expired laptop token (20× 401 in debug-trace JSON) | Refresh token, or run GlobalClient test on ns1 as `verae-mcp`; L3/L12 already cover MCP-mediated idempotency |
## Artifacts
| Path | Contents |
| --- | --- |
| `results.csv` | Machine-readable PASS/FAIL |
| `logs/` | Per-case stdout/stderr |
| `logs/deploy-state.txt` | systemd drop-in snapshot |
| `logs/l3-20stamp.txt` | 20× CID list |
| `logs/l7-txlog.txt` | Laptop transaction_log output (401) |
Laptop `transaction_log_test` also wrote:
`verae-mcp-timestamping-server/debug-trace/2026-09-29T162648Z-verae-debug-test/`
## Conclusion
The Go MCP service on ns1 is **production-viable** for AI MCP clients and NATS bulk clients under the tested cases. Public tool parity, local ledger, global VeraeTime via host-token fallback, embedded NATS, and static docs all passed. Remaining fails are local developer credentials for direct API scripts.
## Rollback (reference)
```sh
sudo rm /etc/systemd/system/verae-mcp-timestamping.service.d/go-cutover.conf
sudo systemctl daemon-reload
sudo systemctl restart verae-mcp-timestamping.service
```

View file

@ -0,0 +1,15 @@
MainPID=799038
ActiveState=active
FragmentPath=/etc/systemd/system/verae-mcp-timestamping.service
DropInPaths=/etc/systemd/system/verae-mcp-timestamping.service.d/go-cutover.conf /etc/systemd/system/verae-mcp-timestamping.service.d/verae-ai.conf
[Service]
WorkingDirectory=/opt/verae/verae-mcp-timestamping-server-go
Environment=VERAE_MCP_PORT=8000
Environment=VERAE_NATS_LISTEN=127.0.0.1:4229
Environment=VERAE_MCP_REQUIRE_BEARER=0
Environment=VERAE_LEDGER_PATH=/var/lib/verae-mcp/mcp-timestamp-local.json
Environment=VERAE_AUTH_PROPERTIES=/var/lib/verae-mcp/auth.properties
Environment=VERAE_GLOBAL_URL=https://api.veraetime.net
# Clear Python ExecStart then set Go
ExecStart=
ExecStart=/opt/verae/verae-mcp-timestamping-server-go/bin/verae-mcp-timestamping-server

View file

@ -0,0 +1 @@
2026-09-29T16:28:26Z

View file

@ -0,0 +1,6 @@
TOOLS ['create_user', 'hash_report', 'login', 'session', 'signup', 'stamp', 'verify']
HASH_REPORT {"content": [{"type": "text", "text": "{\"receipts\":[{\"created_at\":\"2026-09-29T15:19:05.570331728Z\",\"hash\":\"469e615923b69c6b7291d8333699b5ad678b88d1afdc8d412c45a6d244fee0ec\",\"merkle\":\"113079\",\"receipt_id\":\"QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ\",\"scope\":\"global\"}]}"}], "structuredContent": {"receipts": [{"created_at": "2026-09-29T15:19:05.570331728Z", "hash": "469e615923b69c6b7291d8333699b5ad678b88d1afdc8d412c45a6d244fee0ec", "merkle": "113079", "receipt_id": "QmXgqQ
LOCAL_RECEIPT 14a344189150d80dd23880461751382d3228981484032f8e69f0ee5135645030
VERIFY None
SESSION {'expires_at': '2026-09-30T06:10:30-09:00', 'role': 'admin', 'username': 'admin', 'valid': True}
L1_OK

View file

@ -0,0 +1 @@
TSNATS_OK 1 56d0209cda360ab2210419699ca75acc2525ee0cf151fa79fe39737cef8f2bdf

View file

@ -0,0 +1,7 @@
200 https://mcp.verae.ai/
200 https://mcp.verae.ai/developers.html
200 https://mcp.verae.ai/ways.html
200 https://mcp.verae.ai/openapi/application-client.yaml
200 https://mcp.verae.ai/app.html
200 https://mcp.verae.ai/explorer.html
L11_OK

View file

@ -0,0 +1,7 @@
0 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
1 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
2 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
3 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
4 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
unique 1 same True
L12_OK

View file

@ -0,0 +1,2 @@
{"content": [{"type": "text", "text": "{\"receipts\":[{\"created_at\":\"2026-09-29T16:26:07.301696399Z\",\"hash\":\"14b7f33a56e1323382a60f6f00909b4ba9c3ef798cb8d7fb45c694eab874b426\",\"merkle\":\"\",\"receipt_id\":\"QmfQPVSAaYCTckDrv9jhf4BicQKiDvF1su2zcmKC2encve\",\"scope\":\"global\"}]}"}], "structuredContent": {"receipts": [{"created_at": "2026-09-29T16:26:07.301696399Z", "hash": "14b7f33a56e1323382a60f6f00909b4ba9c3ef798cb8d7fb45c694eab874b426", "merkle": "", "receipt_id": "QmfQPVSAaYCTckDrv9
L2_OK

View file

@ -0,0 +1,22 @@
0 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
1 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
2 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
3 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
4 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
5 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
6 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
7 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
8 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
9 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
10 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
11 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
12 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
13 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
14 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
15 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
16 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
17 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
18 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
19 QmXgqQn8eu6bpBmJ1Fm8xKUePfLREJdghsUkYijStGjVMQ
unique 1 same True elapsed_s 3.92
L3_OK

View file

@ -0,0 +1,2 @@
verify {'valid': True, 'error': None, 'receipt': {'receipt_id': '09c1697d2ab14b72f790fa9fd113cb875caa2f9105f2a12eeecc7b83cbc948d8', 'hash': '8b9ece235e6d6173b1bf192f32d3e5ef008e8303a3aab42b849cea50247d122b', 'scope': 'local', 'created_at': '2026-09-29T16:26:40Z', 'merkle': 'sha256:38d32058951e4c0288bce8ce1dfe99c2c515984da711a64ba198aeb398f77191', 'cube_id': '', 'org_id': '', 'restamp_of': ''}}
L4_OK 09c1697d2ab14b72f790fa9fd113cb875caa2f9105f2a12eeecc7b83cbc948d8

View file

@ -0,0 +1,23 @@
Traceback (most recent call last):
File "<stdin>", line 9, in <module>
File "/Users/marchon/.local/share/uv/python/cpython-3.13.7-macos-aarch64-none/lib/python3.13/asyncio/runners.py", line 195, in run
return runner.run(main)
~~~~~~~~~~^^^^^^
File "/Users/marchon/.local/share/uv/python/cpython-3.13.7-macos-aarch64-none/lib/python3.13/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/Users/marchon/.local/share/uv/python/cpython-3.13.7-macos-aarch64-none/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
return future.result()
~~~~~~~~~~~~~^^
File "<stdin>", line 6, in main
File "/Users/marchon/research/modelcontextprotocol/verae-mcp-timestamping-client/src/verae_mcp_timestamping_client/application_client.py", line 97, in hash_report
receipts = await asyncio.to_thread(self.veraetime.hash_report, digest)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/marchon/.local/share/uv/python/cpython-3.13.7-macos-aarch64-none/lib/python3.13/asyncio/threads.py", line 25, in to_thread
return await loop.run_in_executor(None, func_call)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/marchon/.local/share/uv/python/cpython-3.13.7-macos-aarch64-none/lib/python3.13/concurrent/futures/thread.py", line 59, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/marchon/research/modelcontextprotocol/verae-mcp-timestamping-client/src/verae_mcp_timestamping_client/veraetime_transport.py", line 359, in hash_report
raise _http_error(status, detail, what="hash report")
verae_mcp_timestamping_client.veraetime_transport.AuthError: hash report HTTP 401: auth required — call login or set VERAE_GLOBAL_TOKEN / ~/.verae/auth.properties

View file

@ -0,0 +1 @@
0f9e6d5df4b10c0e07ee45484c14008fb50afc7276d292a3bfa2a2fd442f95ca /tmp/verae-full-report-cli.txt

View file

@ -0,0 +1,4 @@
wrote /Users/marchon/research/modelcontextprotocol/verae-mcp-timestamping-server/debug-trace/2026-09-29T162648Z-verae-debug-test/transaction-log-test.html
source Verae-Debug-Test digest 469e615923b69c6b7291d8333699b5ad678b88d1afdc8d412c45a6d244fee0ec
unique 0 same False
txdir=debug-trace/2026-09-29T162648Z-verae-debug-test

View file

@ -0,0 +1,2 @@
['create_user', 'hash_report', 'login', 'session', 'signup', 'stamp', 'verify']
L8_OK

View file

@ -0,0 +1,5 @@
HTTP/2 502
alt-svc: h3=":443"; ma=2592000
server: Caddy
date: Tue, 29 Sep 2026 16:27:43 GMT

View file

@ -0,0 +1,2 @@
{"jsonrpc":"2.0","id":2,"result":{"content":[{"type":"text","text":"{\"expires_at\":\"2026-09-30T06:10:30-09:00\",\"role\":\"admin\",\"username\":\"admin\",\"valid\":true}"}],"structuredContent":{"expires_at":"2026-09-30T06:10:30-09:00","role":"admin","username":"admin","valid":true}}}
L9_OK

View file

@ -0,0 +1,19 @@
Traceback (most recent call last):
File "<stdin>", line 9, in <module>
File "<stdin>", line 7, in post
File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/urllib/request.py", line 187, in urlopen
return opener.open(url, data, timeout)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/urllib/request.py", line 493, in open
response = meth(req, response)
File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/urllib/request.py", line 602, in http_response
response = self.parent.error(
'http', request, response, code, msg, hdrs)
File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/urllib/request.py", line 531, in error
return self._call_chain(*args)
~~~~~~~~~~~~~~~~^^^^^^^
File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/urllib/request.py", line 464, in _call_chain
result = func(*args)
File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/urllib/request.py", line 611, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway

View file

@ -0,0 +1,9 @@
PASS L4 python-application-client-mcp
FAIL L5 appclient-direct-veraetime — raise _http_error(status, detail, what="hash report") verae_mcp_timestamping_client.veraetime_transport.AuthError: hash report HTTP 401: auth required — call login or set VERAE_GLOBAL_TOKEN / ~/
PASS L6 cli-hash
FAIL L7 transaction_log_test — wrote /Users/marchon/research/modelcontextprotocol/verae-mcp-timestamping-server/debug-trace/2026-09-29T162648Z-verae-debug-test/transaction-log-test.html source Verae-Debug-Test digest 469e615923b69c6b7291d8333699b5ad678b88d1afdc8d412c45a6d244fee0ec unique 0 same False
PASS L8 mcp-veraetime-net
FAIL L9 mcp-georgelambert-org — File "/opt/homebrew/Cellar/python@3.14/3.14.5/Frameworks/Python.framework/Versions/3.14/lib/python3.14/urllib/request.py", line 611, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 502: Bad Gateway
PASS L10 tsnats-embedded
PASS L11 static-site-pages
PASS L12 ns1-mcp-5x-global-idempotent

View file

@ -0,0 +1,15 @@
U1|go-unit|PASS
U2|python-client-unit|PASS
U3|python-server-unit|PASS
L1|public-mcp-suite|PASS
L2|mcp-global-stamp|PASS
L3|mcp-20x-global-stamp|PASS
L4|python-application-client-mcp|PASS
L5|appclient-direct-veraetime|FAIL| raise _http_error(status, detail, what="hash report") verae_mcp_timestamping_client.veraetime_transport.AuthError: hash report HTTP 401: auth required — call login or set VERAE_GLOBAL_TOKEN / ~/
L6|cli-hash|PASS
L7|transaction_log_test|FAIL|wrote /Users/marchon/research/modelcontextprotocol/verae-mcp-timestamping-server/debug-trace/2026-09-29T162648Z-verae-debug-test/transaction-log-test.html source Verae-Debug-Test digest 469e615923b69c6b7291d8333699b5ad678b88d1afdc8d412c45a6d244fee0ec unique 0 same False
L8|mcp-veraetime-net|PASS
L10|tsnats-embedded|PASS
L11|static-site-pages|PASS
L12|ns1-mcp-5x-global-idempotent|PASS
L9|mcp-georgelambert-org|PASS|fixed Caddy upstream 10.1.10.128→127.0.0.1 during report run
Can't render this file because it contains an unexpected character in line 8 and column 79.

View file

@ -0,0 +1,7 @@
id name status
U1 go-unit PASS
U2 python-client-unit PASS
U3 python-server-unit PASS
L1 public-mcp-suite PASS
L2 mcp-global-stamp PASS
L3 mcp-20x-global-stamp PASS
1 id name status
2 U1 go-unit PASS
3 U2 python-client-unit PASS
4 U3 python-server-unit PASS
5 L1 public-mcp-suite PASS
6 L2 mcp-global-stamp PASS
7 L3 mcp-20x-global-stamp PASS

Some files were not shown because too many files have changed in this diff Show more