marchon/master-zapier-plan-draft
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Add an NS1-host NATS study: run, charts, methodology, and tuning notes.
Some checks are pending
offline / test (push) Waiting to run
Details

Browse source 
Second ladder executed entirely on NS1.GEORGELAMBERT.ORG (70.88.205.138)
against LXC 511–513; HTML and PDF built on that host.
This commit is contained in:
George Lambert 2026-09-12 01:16:17 -04:00
parent de20a42109
commit c5d286dc6b
54 changed files with 4665 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

48
packages/verae-nats-cluster/results/20260912T051237Z/BENCH.md Normal file
View file

@ -0,0 +1,48 @@
# NATS cluster message speed
Run **`20260912T051237Z`** (UTC). Client: LXC **510** `verae-px-worker` (`10.10.10.20`), not a nats-* server. Servers: `nats-a/b/c` on `10.10.10.21–23` (`vmbr1` only).
Client URL:
```text
nats://10.10.10.21:4222,nats://10.10.10.22:4222,nats://10.10.10.23:4222
```
## Method
- **Core NATS** is fire-and-forget pub/sub (`nats bench`). No disk, no replica ack.
- **JetStream** uses **file** storage and **replicas=3** (same as product streams). The unique stream `benchstream` is deleted between JS loads.
- Throughput is **msgs/sec** from nats CLI **0.1.6** (`--no-progress --csv`). Its min/avg/max are publisher **rate spread**, not delay.
- **Ping** delay: one publisher, sequential publish-then-wait. This is one-message round-trip through the cluster.
- **Flood** delay: N publishers dump the whole batch, then the subscriber drains. This is **queueing under burst**, not wire RTT.
- Probe: `scripts/latency.mjs` (two connections, header timestamp).
## Throughput
| Run | Mode | Aggregate msgs/s | Pub msgs/s | Pub MB/s | Sub msgs/s | Sub MB/s |
|-----|------|------------------|------------|----------|------------|----------|
| `core-1p1s-50k-128` | core pub/sub | 776,331 | 502,502 | 61.34 | 420,976 | 51.39 |
| `core-4p4s-100k-128` | core pub/sub | 1,278,454 | 379,985 | 46.38 | 1,078,910 | 131.70 |
| `core-4p4s-50k-1k` | core pub/sub | 581,464 | 209,545 | 204.63 | 482,645 | 471.33 |
| `core-8p8s-200k-128` | core pub/sub | 2,065,217 | 333,731 | 40.74 | 1,847,242 | 225.49 |
| `js-2p2s-20k-128-r3` | jetstream r=3 file | 15,416 | — | — | — | — |
| `js-4p-20k-1k-r3` | jetstream r=3 file | — | 14,985 | 14.63 | — | — |
| `js-4p-50k-128-r3` | jetstream r=3 file | — | 17,986 | 2.20 | — | — |
## Round-trip delay
| Run | Kind | Count | Pubs | Size | min | avg | p50 | p90 | p99 | max |
|-----|------|-------|------|------|-----|-----|-----|-----|-----|-----|
| `lat-ping-1k-128` | ping (sequential RTT) | 1000 | 1 | 128 B | 0.341ms | 0.470ms | 0.395ms | 0.651ms | 1.377ms | 3.243ms |
| `lat-1p-5k-128` | flood (burst queueing) | 5000 | 1 | 128 B | 132.425ms | 203.717ms | 204.458ms | 241.750ms | 249.468ms | 249.640ms |
| `lat-4p-5k-1k` | flood (burst queueing) | 5000 | 4 | 1024 B | 196.422ms | 221.264ms | 224.039ms | 233.180ms | 236.798ms | 238.177ms |
| `lat-4p-10k-128` | flood (burst queueing) | 10000 | 4 | 128 B | 211.108ms | 294.925ms | 300.254ms | 314.223ms | 315.551ms | 316.091ms |
| `lat-8p-20k-128` | flood (burst queueing) | 20000 | 8 | 128 B | 280.608ms | 414.972ms | 409.802ms | 513.821ms | 535.394ms | 536.084ms |
## What the numbers mean
Product job/event/archive traffic is **JetStream r=3 file**. On this three-LXC stand that is about **16k durable 128 B pubs/s** (about **13k** at 1 KiB). Core NATS is an upper bound for non-durable fan-out: about **0.7–2.0M msgs/s** aggregate at 128 B, or **~630k msgs/s (~616 MB/s)** at 1 KiB with 4 publishers.
A quiet request-reply is **~0.3 ms** average, **p99 < 1 ms**. Flood rows in the **150–500 ms** band are the subscriber catching up after a burst, which is what a job-events mailbox sees if publishers outrun consumers.
Re-run on NS1: `bash scripts/bench.sh`. Raw logs/CSVs are under `results/<utc>/`.