0 releases 36 tags

RSS feed

• v1.7.2 27af672c5e

  Compare

  fix(list): use row-constructor cursor + restore per-phase publish timings + enable pg_stat_statements (#1215)

  Ghost released this 2026-04-27 15:35:59 -04:00 | 135 commits to main since this release

  Summary

  Three changes from the 2026-04-27 incident:

  1. Cursor SQL fix — addCursorCondition rewritten to use a
     row-constructor comparison (server_name, version) > ($1, $2) instead
     of the OR-decomposed form. Postgres can index-seek on the row
     constructor; it can't on the OR form, which scanned from the start of
     the index and filtered rows before the cursor. Cost grew linearly with
     cursor depth.
  2. Per-phase publish timings restored — createServerInTransaction
     was simplified during #1211 review to log only validate_ms. The
     incident showed publishes spending 50+ s in acquire_lock /
     version_checks / db_create while validate_ms reported a few
     hundred ms — the diagnostic signal was hidden. Restored timings for
     every phase, refactored into a small runPhase helper.
  3. pg_stat_statements enabled in the CNPG cluster spec. We had no
     aggregate query visibility during the incident — could only EXPLAIN
     queries we happened to suspect.

  Cursor fix evidence

  Local benchmark, 100K rows, cold cache:

  Form	Rows filtered	Buffer reads	Time
  OLD	80,001	7,679	31.6 ms
  NEW	2	1	0.05 ms

  Maps to prod's measured 8,911 buffer hits → 760 ms. End-to-end API:
  /v0/servers?limit=100&cursor=… returns in 4–7 ms regardless of depth.

  Risk check: confirmed (server_name, version) index exists in that
  column order (both servers_pkey and idx_servers_name_version), and
  both columns are NOT NULL so row-constructor comparison can't silently
  drop rows.

  Tests

  • New
    TestPostgreSQL_PerformanceScenarios/compound_cursor_across_versions_of_same_server
    pins multi-version pagination semantics. The existing cursor tests only
    exercised the fallback (single-component cursor) and degenerate (one
    version per server) cases.
  • make lint clean, go test -race ./internal/... ./cmd/... green.

  Deployment

  The cursor + slog changes are zero-downtime. The CNPG spec change
  triggers a brief PG restart (single-instance cluster). After the
  restart, run once on prod:

  kubectl exec -i registry-pg-1 -c postgres \
    --context gke_mcp-registry-prod_us-central1-b_mcp-registry-prod \
    -- psql -U postgres -c "CREATE EXTENSION IF NOT EXISTS pg_stat_statements"
  

  Time the merge for a low-traffic UTC window. v1.7.1's DB
  retry-with-backoff covers the brief PG restart.

  Out of scope

  MaxConns bump, per-IP rate limiting at nginx, response caching, the
  pre-existing superfluous WriteHeader warnings — separate follow-ups.

  🤖 Generated with Claude Code

  ---

  Co-authored-by: Claude Opus 4.7 (1M context) noreply@anthropic.com

  Downloads

  • Source code (ZIP)
    0 downloads
  • Source code (TAR.GZ)
    1 download