- Go 92.9%
- PLpgSQL 3.1%
- HTML 1.9%
- Shell 1.6%
- Makefile 0.5%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
## Summary Follow-up to #1215 and #1220. Both of those addressed *individual* slow queries; this PR addresses *concurrency* — under today's scraper load the cursor query is fast on average (mean 42ms) but the connection pool saturates and queue depth blows up at the Go HTTP layer. ## Diagnosis `pg_stat_statements` (added in #1215) made this possible to see: ``` max_ms mean_ms calls template 10,755 41.8 94,009 plain cursor pagination, no filter 7,277 214.7 14,770 ILIKE substring filter 7,638 330.8 7,656 ILIKE substring + is_latest filter ``` Mean times are healthy. But `max_exec_time` of 7–10s on the cursor query, combined with sustained ~15 req/s from scrapers (ServiceNow's 148.139.x.x range, anonymous `node` user-agent, etc) saturates `MaxConns=30 × 2 pods = 60`. New requests queue at the Go HTTP layer; nginx-side p99 hits 35s; eventually scrapers time out at 60s, retry, and amplify the queue. Today's two ongoing alerts are both this pattern: - 17:35–17:40 UTC `Availability dropped below 95%` — 4,526 GET requests in 5 min, hundreds of 504s - 18:17 UTC `Publish Endpoint Latency` re-fire — same scraper concurrency dragging the publish path Critically, the symptoms today were also visible during yesterday's incident, but yesterday's broken cursor (#1215) was the dominant cause. After #1215 the cursor is fast individually; concurrency now becomes the next bottleneck. ## Changes ### pgxpool (`internal/database/postgres.go`) | Setting | Before | After | |---------|-------:|------:| | `MaxConns` | 30 | **60** per pod | | `MinConns` | 5 | **10** per pod | | `MaxConnIdleTime` | 30 min | (unchanged) | | `MaxConnLifetime` | 2 h | (unchanged) | 2 pods × 60 = 120 total app connections. Cuts queue depth roughly in half at current scraper load. ### PG cluster (`deploy/pkg/k8s/postgres.go`) - `max_connections: 100 → 200` — required to support the larger pool. 120 (app) + ~10 (PG internals: autovacuum, replication, admin) + 70 headroom. - Added explicit `resources:` block — previously the pod had no resource limits, making node-level OOM behaviour unpredictable. | | Request | Limit | |---|--------:|------:| | memory | 512Mi | **4Gi** | | cpu | 200m | 1500m | ## Resource budget | Node | Now | After | |------|----:|------:| | dy89 (PG node) | 39% mem (~2.4 GiB / 6 GiB) | ~65% mem worst-case (~3.9 GiB) | | 2yxm | 36% mem | unchanged | Both nodes well within capacity. CPU usage <50% on both, plenty of headroom. PG worst-case memory math: - 200 conns × ~15 MB per backend = 3 GiB - + `shared_buffers` 128 MiB - + `maintenance_work_mem`, OS overhead, etc. - ≈ 3–4 GiB total ## Deployment caveat `max_connections` is a postmaster-level setting → CNPG triggers a PG restart on the change. Same in-pod restart shape as the pg_stat_statements deploy yesterday — registry pods see ~30s of DB unavailability, covered by v1.7.1's retry-with-backoff (8 attempts, 1→8s capped). One registry pod may bounce once before recovering, like yesterday. **Time the merge for a low-traffic UTC window.** Order of operations matters within the deploy itself: 1. Pulumi applies the new CNPG spec → PG restarts with `max_connections=200` 2. Rolling deploy of registry pods picks up `MaxConns=60` config 3. New conns are accepted up to 200 limit Pulumi's standard ordering does step 1 before step 2 in this scenario (Pulumi resource graph: CNPG cluster precedes Deployment). If for any reason it doesn't, the worst case is `too many connections` errors during a small window — Self-correcting once the rollout completes. ## Test plan - [x] `go build ./...` clean for app + deploy - [x] `make lint` clean - [x] `go test -race ./internal/database/...` green - [ ] On merge: staging deploy applies the spec change cleanly; PG restarts; pool reaches 60 max - [ ] On prod deploy: same; verify no `too many connections` errors during the rollout window - [ ] After deploy: query `SHOW max_connections` returns 200; query pg_stat_statements after a scraper burst, confirm `max_exec_time` for cursor query no longer hits 10s ## Out of scope - **ILIKE substring search** (`server_name ILIKE '%foo%'`) is unindexable. Three pg_stat_statements variants run with means 141–333ms and max 4–7s. Worth replacing with a `pg_trgm` GIN index or full-text search in a separate PR. - **Per-IP nginx rate limiting** — defends against scraper retry storms regardless of pool size. - **`Cache-Control` on `/v0/servers`** — let nginx absorb scraper-repeated cursors. - **Pre-existing `superfluous WriteHeader` warnings** — separate Huma framework issue. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
| .githooks | ||
| .github | ||
| .vscode | ||
| cmd | ||
| data | ||
| deploy | ||
| docs | ||
| internal | ||
| pkg | ||
| scripts | ||
| tests/integration | ||
| tools | ||
| .env.example | ||
| .gitignore | ||
| .golangci.yml | ||
| .goreleaser.yaml | ||
| .ko.dockerignore | ||
| .ko.yaml | ||
| CHANGES.md | ||
| CLAUDE.md | ||
| complete.md | ||
| CONTRIBUTING.md | ||
| docker-compose.yml | ||
| go.mod | ||
| go.sum | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| SECURITY.md | ||
MCP Registry
The MCP registry provides MCP clients with a list of MCP servers, like an app store for MCP servers.
📤 Publish my MCP server | ⚡️ Live API docs | 👀 Ecosystem vision | 📖 Full documentation
Development Status
2025-10-24 update: The Registry API has entered an API freeze (v0.1) 🎉. For the next month or more, the API will remain stable with no breaking changes, allowing integrators to confidently implement support. This freeze applies to v0.1 while development continues on v0. We'll use this period to validate the API in real-world integrations and gather feedback to shape v1 for general availability. Thank you to everyone for your contributions and patience—your involvement has been key to getting us here!
2025-09-08 update: The registry has launched in preview 🎉 (announcement blog post). While the system is now more stable, this is still a preview release and breaking changes or data resets may occur. A general availability (GA) release will follow later. We'd love your feedback in GitHub discussions or in the #registry-dev Discord (joining details here).
Current key maintainers:
- Adam Jones (Anthropic) @domdomegg
- Tadas Antanavicius (PulseMCP) @tadasant
- Toby Padilla (GitHub) @toby
- Radoslav (Rado) Dimitrov (Stacklok) @rdimitrov
Contributing
We use multiple channels for collaboration - see modelcontextprotocol.io/community/communication.
Often (but not always) ideas flow through this pipeline:
- Discord - Real-time community discussions
- Discussions - Propose and discuss product/technical requirements
- Issues - Track well-scoped technical work
- Pull Requests - Contribute work towards issues
Quick start:
Pre-requisites
- Docker
- Go 1.24.x
- ko - Container image builder for Go (installation instructions)
- golangci-lint v2.4.0
Running the server
# Start full development environment
make dev-compose
This starts the registry at localhost:8080 with PostgreSQL. The database uses ephemeral storage and is reset each time you restart the containers, ensuring a clean state for development and testing.
Note: The registry uses ko to build container images. The make dev-compose command automatically builds the registry image with ko and loads it into your local Docker daemon before starting the services.
By default, the registry seeds from the production API with a filtered subset of servers (to keep startup fast). This ensures your local environment mirrors production behavior and all seed data passes validation. For offline development you can seed from a file without validation with MCP_REGISTRY_SEED_FROM=data/seed.json MCP_REGISTRY_ENABLE_REGISTRY_VALIDATION=false make dev-compose.
The setup can be configured with environment variables in docker-compose.yml - see .env.example for a reference.
Alternative: Running a pre-built Docker image
Pre-built Docker images are automatically published to GitHub Container Registry. Note that the image does not bundle PostgreSQL, so you need to run your own and point the registry at it via MCP_REGISTRY_DATABASE_URL (see docker-compose.yml for a working example):
# Run latest stable release
docker run -p 8080:8080 ghcr.io/modelcontextprotocol/registry:latest
# Run latest from main branch (continuous deployment)
docker run -p 8080:8080 ghcr.io/modelcontextprotocol/registry:main
# Run specific release version
docker run -p 8080:8080 ghcr.io/modelcontextprotocol/registry:v1.0.0
# Run development build from main branch
docker run -p 8080:8080 ghcr.io/modelcontextprotocol/registry:main-20250906-abc123d
Available tags:
- Releases:
latest,v1.0.0,v1.1.0, etc. - Continuous:
main(latest main branch build) - Development:
main-<date>-<sha>(specific commit builds)
Publishing a server
To publish a server, we've built a simple CLI. You can use it with:
# Build the latest CLI
make publisher
# Use it!
./bin/mcp-publisher --help
See the publisher guide for more details.
Other commands
# Run lint, unit tests and integration tests
make check
There are also a few more helpful commands for development. Run make help to learn more, or look in Makefile.
Architecture
Project Structure
├── cmd/ # Application entry points
│ └── publisher/ # Server publishing tool
├── data/ # Seed data
├── deploy/ # Deployment configuration (Pulumi)
├── docs/ # Documentation
├── internal/ # Private application code
│ ├── api/ # HTTP handlers and routing
│ ├── auth/ # Authentication (GitHub OAuth, JWT, namespace blocking)
│ ├── config/ # Configuration management
│ ├── database/ # Data persistence (PostgreSQL)
│ ├── service/ # Business logic
│ ├── telemetry/ # Metrics and monitoring
│ └── validators/ # Input validation
├── pkg/ # Public packages
│ ├── api/ # API types and structures
│ │ └── v0/ # Version 0 API types
│ └── model/ # Data models for server.json
├── scripts/ # Development and testing scripts
├── tests/ # Integration tests
└── tools/ # CLI tools and utilities
└── validate-*.sh # Schema validation tools
Authentication
Publishing supports multiple authentication methods:
- GitHub OAuth - For publishing by logging into GitHub
- GitHub OIDC - For publishing from GitHub Actions
- DNS verification - For proving ownership of a domain and its subdomains
- HTTP verification - For proving ownership of a domain
The registry validates namespace ownership when publishing. E.g. to publish...:
io.github.domdomegg/my-cool-mcpyou must login to GitHub asdomdomegg, or be in a GitHub Action on domdomegg's reposme.adamjones/my-cool-mcpyou must prove ownership ofadamjones.mevia DNS or HTTP challenge
Community Projects
Check out community projects to explore notable registry-related work created by the community.
More documentation
See the documentation for more details if your question has not been answered here!