Mirror of https://github.com/modelcontextprotocol/registry — A community driven registry service for Model Context Protocol (MCP) servers.
  • Go 92.9%
  • PLpgSQL 3.1%
  • HTML 1.9%
  • Shell 1.6%
  • Makefile 0.5%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Radoslav Dimitrov 736d376858
fix(db): raise pgxpool MaxConns + PG max_connections + set PG resources (#1221)
## Summary

Follow-up to #1215 and #1220. Both of those addressed *individual* slow
queries; this PR addresses *concurrency* — under today's scraper load
the cursor query is fast on average (mean 42ms) but the connection pool
saturates and queue depth blows up at the Go HTTP layer.

## Diagnosis

`pg_stat_statements` (added in #1215) made this possible to see:

```
max_ms   mean_ms  calls    template
10,755     41.8   94,009   plain cursor pagination, no filter
 7,277    214.7   14,770   ILIKE substring filter
 7,638    330.8    7,656   ILIKE substring + is_latest filter
```

Mean times are healthy. But `max_exec_time` of 7–10s on the cursor
query, combined with sustained ~15 req/s from scrapers (ServiceNow's
148.139.x.x range, anonymous `node` user-agent, etc) saturates
`MaxConns=30 × 2 pods = 60`. New requests queue at the Go HTTP layer;
nginx-side p99 hits 35s; eventually scrapers time out at 60s, retry, and
amplify the queue.

Today's two ongoing alerts are both this pattern:
- 17:35–17:40 UTC `Availability dropped below 95%` — 4,526 GET requests
in 5 min, hundreds of 504s
- 18:17 UTC `Publish Endpoint Latency` re-fire — same scraper
concurrency dragging the publish path

Critically, the symptoms today were also visible during yesterday's
incident, but yesterday's broken cursor (#1215) was the dominant cause.
After #1215 the cursor is fast individually; concurrency now becomes the
next bottleneck.

## Changes

### pgxpool (`internal/database/postgres.go`)

| Setting | Before | After |
|---------|-------:|------:|
| `MaxConns` | 30 | **60** per pod |
| `MinConns` | 5 | **10** per pod |
| `MaxConnIdleTime` | 30 min | (unchanged) |
| `MaxConnLifetime` | 2 h | (unchanged) |

2 pods × 60 = 120 total app connections. Cuts queue depth roughly in
half at current scraper load.

### PG cluster (`deploy/pkg/k8s/postgres.go`)

- `max_connections: 100 → 200` — required to support the larger pool.
120 (app) + ~10 (PG internals: autovacuum, replication, admin) + 70
headroom.
- Added explicit `resources:` block — previously the pod had no resource
limits, making node-level OOM behaviour unpredictable.

| | Request | Limit |
|---|--------:|------:|
| memory | 512Mi | **4Gi** |
| cpu | 200m | 1500m |

## Resource budget

| Node | Now | After |
|------|----:|------:|
| dy89 (PG node) | 39% mem (~2.4 GiB / 6 GiB) | ~65% mem worst-case
(~3.9 GiB) |
| 2yxm | 36% mem | unchanged |

Both nodes well within capacity. CPU usage <50% on both, plenty of
headroom.

PG worst-case memory math:
- 200 conns × ~15 MB per backend = 3 GiB
- + `shared_buffers` 128 MiB
- + `maintenance_work_mem`, OS overhead, etc.
- ≈ 3–4 GiB total

## Deployment caveat

`max_connections` is a postmaster-level setting → CNPG triggers a PG
restart on the change. Same in-pod restart shape as the
pg_stat_statements deploy yesterday — registry pods see ~30s of DB
unavailability, covered by v1.7.1's retry-with-backoff (8 attempts, 1→8s
capped). One registry pod may bounce once before recovering, like
yesterday.

**Time the merge for a low-traffic UTC window.**

Order of operations matters within the deploy itself:
1. Pulumi applies the new CNPG spec → PG restarts with
`max_connections=200`
2. Rolling deploy of registry pods picks up `MaxConns=60` config
3. New conns are accepted up to 200 limit

Pulumi's standard ordering does step 1 before step 2 in this scenario
(Pulumi resource graph: CNPG cluster precedes Deployment). If for any
reason it doesn't, the worst case is `too many connections` errors
during a small window — Self-correcting once the rollout completes.

## Test plan

- [x] `go build ./...` clean for app + deploy
- [x] `make lint` clean
- [x] `go test -race ./internal/database/...` green
- [ ] On merge: staging deploy applies the spec change cleanly; PG
restarts; pool reaches 60 max
- [ ] On prod deploy: same; verify no `too many connections` errors
during the rollout window
- [ ] After deploy: query `SHOW max_connections` returns 200; query
pg_stat_statements after a scraper burst, confirm `max_exec_time` for
cursor query no longer hits 10s

## Out of scope

- **ILIKE substring search** (`server_name ILIKE '%foo%'`) is
unindexable. Three pg_stat_statements variants run with means 141–333ms
and max 4–7s. Worth replacing with a `pg_trgm` GIN index or full-text
search in a separate PR.
- **Per-IP nginx rate limiting** — defends against scraper retry storms
regardless of pool size.
- **`Cache-Control` on `/v0/servers`** — let nginx absorb
scraper-repeated cursors.
- **Pre-existing `superfluous WriteHeader` warnings** — separate Huma
framework issue.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 15:21:24 -04:00
.githooks Add linting documentation and pre-commit hooks (#177) 2025-07-15 09:31:17 -07:00
.github build(deps): bump goreleaser/goreleaser-action from 7.1.0 to 7.2.1 in the actions group (#1218) 2026-04-28 03:23:31 -04:00
.vscode Move publisher CLI from tools/ to cmd/ (#326) 2025-08-29 14:45:36 +01:00
cmd fix: harden registry availability and add publish phase timings (#1211) 2026-04-27 09:09:50 -04:00
data chore: Release server.json schema version 2025-12-11 (#841) 2025-12-12 09:33:57 -08:00
deploy fix(db): raise pgxpool MaxConns + PG max_connections + set PG resources (#1221) 2026-04-28 15:21:24 -04:00
docs fix(auth): better DNS verification errors and wrong-selector hint (#1202) 2026-04-25 11:22:06 -04:00
internal fix(db): raise pgxpool MaxConns + PG max_connections + set PG resources (#1221) 2026-04-28 15:21:24 -04:00
pkg feat(oci): add support for Quay.io registry (#1150) 2026-04-24 06:11:46 -04:00
scripts docs: update API endpoint documentation from /v0/ to /v0.1/ (#802) 2025-12-04 13:02:45 -08:00
tests/integration fix: store tokens in ~/.config/mcp-publisher/ instead of cwd (#1166) 2026-04-15 03:23:07 -04:00
tools feat(oci): add support for Quay.io registry (#1150) 2026-04-24 06:11:46 -04:00
.env.example fix: seed from production API by default to ensure validation passes (#611) 2025-10-03 23:58:37 +03:00
.gitignore Enhanced server.json validation (phase 1) (#636) 2026-01-16 00:49:39 +02:00
.golangci.yml Stop pinning Go to patch versions, use stable in CI (#1137) 2026-04-09 11:57:21 -04:00
.goreleaser.yaml Update Goreleaser and Cosign (#839) 2025-12-11 08:43:57 -08:00
.ko.dockerignore Use ko in favour of docker build (#708) 2025-10-29 15:16:11 +00:00
.ko.yaml Use ko in favour of docker build (#708) 2025-10-29 15:16:11 +00:00
CHANGES.md Added /validate endpoint and modified mcp-publish so that all validation happens there (#896) 2026-02-04 15:52:07 +02:00
CLAUDE.md Add contributing guidelines to prevent seed.json PRs (#1079) 2026-03-22 07:42:18 -07:00
complete.md fix: store tokens in ~/.config/mcp-publisher/ instead of cwd (#1166) 2026-04-15 03:23:07 -04:00
CONTRIBUTING.md Add contributing guidelines to prevent seed.json PRs (#1079) 2026-03-22 07:42:18 -07:00
docker-compose.yml Use ko in favour of docker build (#708) 2025-10-29 15:16:11 +00:00
go.mod build(deps): bump github.com/jackc/pgx/v5 from 5.9.1 to 5.9.2 (#1188) 2026-04-21 05:02:43 -04:00
go.sum build(deps): bump github.com/jackc/pgx/v5 from 5.9.1 to 5.9.2 (#1188) 2026-04-21 05:02:43 -04:00
LICENSE chore: update licensing to Apache 2.0 for new contributions (#887) 2026-01-13 11:11:20 +02:00
Makefile Ensure tests are failing CI (#840) 2025-12-11 09:56:57 +02:00
README.md docs: note PostgreSQL requirement for prebuilt Docker image (#1196) 2026-04-25 07:15:00 -07:00
SECURITY.md Update SECURITY.md to use GitHub Security Advisories (#977) 2026-02-18 23:36:23 -08:00

MCP Registry

The MCP registry provides MCP clients with a list of MCP servers, like an app store for MCP servers.

📤 Publish my MCP server | ⚡️ Live API docs | 👀 Ecosystem vision | 📖 Full documentation

Development Status

2025-10-24 update: The Registry API has entered an API freeze (v0.1) 🎉. For the next month or more, the API will remain stable with no breaking changes, allowing integrators to confidently implement support. This freeze applies to v0.1 while development continues on v0. We'll use this period to validate the API in real-world integrations and gather feedback to shape v1 for general availability. Thank you to everyone for your contributions and patience—your involvement has been key to getting us here!

2025-09-08 update: The registry has launched in preview 🎉 (announcement blog post). While the system is now more stable, this is still a preview release and breaking changes or data resets may occur. A general availability (GA) release will follow later. We'd love your feedback in GitHub discussions or in the #registry-dev Discord (joining details here).

Current key maintainers:

Contributing

We use multiple channels for collaboration - see modelcontextprotocol.io/community/communication.

Often (but not always) ideas flow through this pipeline:

  • Discord - Real-time community discussions
  • Discussions - Propose and discuss product/technical requirements
  • Issues - Track well-scoped technical work
  • Pull Requests - Contribute work towards issues

Quick start:

Pre-requisites

Running the server

# Start full development environment
make dev-compose

This starts the registry at localhost:8080 with PostgreSQL. The database uses ephemeral storage and is reset each time you restart the containers, ensuring a clean state for development and testing.

Note: The registry uses ko to build container images. The make dev-compose command automatically builds the registry image with ko and loads it into your local Docker daemon before starting the services.

By default, the registry seeds from the production API with a filtered subset of servers (to keep startup fast). This ensures your local environment mirrors production behavior and all seed data passes validation. For offline development you can seed from a file without validation with MCP_REGISTRY_SEED_FROM=data/seed.json MCP_REGISTRY_ENABLE_REGISTRY_VALIDATION=false make dev-compose.

The setup can be configured with environment variables in docker-compose.yml - see .env.example for a reference.

Alternative: Running a pre-built Docker image

Pre-built Docker images are automatically published to GitHub Container Registry. Note that the image does not bundle PostgreSQL, so you need to run your own and point the registry at it via MCP_REGISTRY_DATABASE_URL (see docker-compose.yml for a working example):

# Run latest stable release
docker run -p 8080:8080 ghcr.io/modelcontextprotocol/registry:latest

# Run latest from main branch (continuous deployment)
docker run -p 8080:8080 ghcr.io/modelcontextprotocol/registry:main

# Run specific release version
docker run -p 8080:8080 ghcr.io/modelcontextprotocol/registry:v1.0.0

# Run development build from main branch
docker run -p 8080:8080 ghcr.io/modelcontextprotocol/registry:main-20250906-abc123d

Available tags:

  • Releases: latest, v1.0.0, v1.1.0, etc.
  • Continuous: main (latest main branch build)
  • Development: main-<date>-<sha> (specific commit builds)

Publishing a server

To publish a server, we've built a simple CLI. You can use it with:

# Build the latest CLI
make publisher

# Use it!
./bin/mcp-publisher --help

See the publisher guide for more details.

Other commands

# Run lint, unit tests and integration tests
make check

There are also a few more helpful commands for development. Run make help to learn more, or look in Makefile.

Architecture

Project Structure

├── cmd/                     # Application entry points
│   └── publisher/           # Server publishing tool
├── data/                    # Seed data
├── deploy/                  # Deployment configuration (Pulumi)
├── docs/                    # Documentation
├── internal/                # Private application code
│   ├── api/                 # HTTP handlers and routing
│   ├── auth/                # Authentication (GitHub OAuth, JWT, namespace blocking)
│   ├── config/              # Configuration management
│   ├── database/            # Data persistence (PostgreSQL)
│   ├── service/             # Business logic
│   ├── telemetry/           # Metrics and monitoring
│   └── validators/          # Input validation
├── pkg/                     # Public packages
│   ├── api/                 # API types and structures
│   │   └── v0/              # Version 0 API types
│   └── model/               # Data models for server.json
├── scripts/                 # Development and testing scripts
├── tests/                   # Integration tests
└── tools/                   # CLI tools and utilities
    └── validate-*.sh        # Schema validation tools

Authentication

Publishing supports multiple authentication methods:

  • GitHub OAuth - For publishing by logging into GitHub
  • GitHub OIDC - For publishing from GitHub Actions
  • DNS verification - For proving ownership of a domain and its subdomains
  • HTTP verification - For proving ownership of a domain

The registry validates namespace ownership when publishing. E.g. to publish...:

  • io.github.domdomegg/my-cool-mcp you must login to GitHub as domdomegg, or be in a GitHub Action on domdomegg's repos
  • me.adamjones/my-cool-mcp you must prove ownership of adamjones.me via DNS or HTTP challenge

Community Projects

Check out community projects to explore notable registry-related work created by the community.

More documentation

See the documentation for more details if your question has not been answered here!