Commit graph registry/deploy/Pulumi.local.yaml
Author SHA1 Message Date
pree-dew
a9ff705d80
Add monitoring infrastructure: victoriametrics and grafana (#328)
This includes infra setup for metrics collection and visualisation

## Motivation and Context
It is important to have basic observability setup, so that common issues
can be identified quickly

## How Has This Been Tested?
- Tested on local
- Tested on one of the production setup

## Breaking Changes
No

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [X] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [X] My code follows the repository's style guidelines
- [X] New and existing tests pass locally
- [X] I have added appropriate error handling
- [ ] I have added or updated documentation as needed

## Additional context
- This includes set up of following components:
- Victoriametrics single node cluster for storing metrics data with
persistent storage configuration for local and production env.
- Vmagent with target discovery for mcp-registry pods with scrape
interval of 30s.
  - Grafana for visualising metrics and setting alerts.
     - Datasource of victoriametrics is pre-configured as config map
     - Persistent volume for basic configuration
- Sqlite for local and Postgres for production setup for storing
dashboard, alerts etc details
 
 ## Metrics in action
<img width="1422" height="666" alt="Screenshot 2025-08-31 at 4 35 28 AM"
src="https://github.com/user-attachments/assets/e54ce846-653c-4fa3-aa7a-fe277f0810e6"
/>

---------

Co-authored-by: Adam Jones <adamj@anthropic.com>
Co-authored-by: adam jones <adamj+git@anthropic.com>
2025-09-02 12:40:43 +00:00
adam jones
ebaf4c9c99
Refactor database implementation to use PostgreSQL instead of MongoDB (#289)
Fixes #228

## Motivation and Context

See #19

## How Has This Been Tested?

- Run locally
- Run in docker compose
- Run infra in minikube
- Unit and integration tests all passing :)

## Breaking Changes

Yes: users will not be able to access any data they used to have in
custom mongodb. We are accepting this breaking change, given the
registry is in experimental development status.

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [x] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [x] My code follows the repository's style guidelines
- [x] New and existing tests pass locally
- [x] I have added appropriate error handling
- [x] I have added or updated documentation as needed

## Additional context

I have not implemented connection pooling support for simplicity, and
because cloudnative-pg already provides built-in connection pooling with
pgBouncer so if we want it later we can switch to this endpoint easily.
2025-08-22 01:40:28 +01:00
adam jones
e9fe6f9ab8
feat: Add creds for GitHub auth, add "none" auth to publisher (#276)
## Motivation and Context

- Adds credentials for the new MCP Registry Login GitHub Apps (local,
staging, prod)
- Adds support for "none" auth to publisher

## How Has This Been Tested?

- Tested publisher workflows locally
- Unit and integration tests passing

## Breaking Changes

None

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [x] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [x] My code follows the repository's style guidelines
- [x] New and existing tests pass locally
- [x] I have added appropriate error handling
- [x] I have added or updated documentation as needed

---

Merging based on Tadas's approval review, and only making minor docs changes on top of that.
2025-08-18 15:49:04 +01:00
adam jones
667e2840b8
feat(infra): Add Pulumi-based Kubernetes deployment infrastructure (#237)
Original PR: #227

- Add Pulumi-based infrastructure as code for deploying MCP Registry to
Kubernetes
- Support for both local development (minikube) and Azure Kubernetes
Service (AKS)
- Complete deployment orchestration including:
- cluster setup: e.g. you point this at an Azure account, and it can set
up and manage the cluster for you. e.g. K8s version, number of nodes,
type of nodes, ...
  - cloud agnostic K8s services: cert-manager, nginx-ingress
- app services: MongoDB, and registry application (currently using nginx
as a placeholder, blocked on #225 (as is #190). but should be a 1 line
change)

## How is this different to #190

- Supports cluster setup and management. This enables:
- Non-hosting maintainers managing many devops workflows (e.g. scaling
up the cluster, or bumping K8s versions). Without this, we'd need to
bug/page the organisation hosting the registry when we need these things
changed.
- Makes it easy to spin up things like staging/temporary clusters, as
well as enables contributors to replicate the stack exactly on their own
Azure accounts.
- Sets up cloud-agnostic services. For example, rather than using the
Azure-managed ingresses and CA, we install nginx-ingress and
cert-manager. This enables:
- Running the entire infra stack can also run locally (e.g. in minikube,
k3s, orbstack, colima) - making it much easier for contributors to test
changes to infra stuff.
- Moving between cloud providers much more easily, e.g. we could shift
from Azure to GCP/AWS/other with minimal hassle.
- Everything stays written in Go, rather than Helm templates. This means
we get things like type-checking etc. for free (which from my experience
makes AI tools wayyy better at editing K8s stuff), and contributors
don't need to learn a new language if they're already using Go.

## Testing

I've got this running well:
- locally in minikube
- on cloud in Azure (my personal Azure account)

<details><summary>Claude written architecture and security
review</summary>
<p>

## Deployment Review & Assessment

### Current Architecture Strengths
**Pulumi IaC Approach**
- Well-structured infrastructure as code using Pulumi
- Multi-provider support (AKS, local) with clean abstraction
- Good separation of concerns in `pkg/` directory

**Security Fundamentals**
- Non-root container execution (`appuser` with UID 10001)
- Secrets properly managed via Kubernetes secrets
- TLS/SSL certificate management with cert-manager and Let's Encrypt

### Critical Issues & High-Priority Improvements

**1. Production Deployment Not Ready** 🚨
The registry deployment uses `nginx:alpine` placeholder image instead of
the actual MCP registry:
- `deploy/pkg/k8s/registry.go:67` - TODO comments indicate incomplete
setup
- Health probes are commented out
- Port mapping doesn't match actual application (80 vs 8080)

**Fix:** Build and publish actual registry container image to GHCR,
update deployment

**2. Database Security Considerations** 🔒
- MongoDB deployed without authentication
- No backup/disaster recovery strategy
- Database credentials hardcoded

*Note: MongoDB is not exposed externally (ClusterIP service), so this is
not a critical security risk but should be addressed for production.*

**3. Monitoring & Observability Gaps** 📊
- No Prometheus/Grafana monitoring stack
- No log aggregation (ELK/Loki)
- No application metrics/health dashboards
- No alerting configured

**4. High Availability & Reliability** ⚠️
- Single MongoDB instance (no replication)
- No persistent volume backup strategy
- Fixed 10Gi storage without growth planning
- Only 2 replicas for registry service
- No pod disruption budgets
- No horizontal pod autoscaling

### Recommended Improvements

**Immediate (High Priority)**
1. Complete Registry Deployment - Build proper container image pipeline,
enable health checks
2. Secure MongoDB - Add authentication credentials, implement backup
strategy

**Medium Priority**
3. Add Monitoring Stack - Prometheus, Grafana deployment
4. Security Hardening (Nice to Have) - RBAC policies, Network Policies,
Pod Security Standards
5. CI/CD Pipeline Enhancement - Container image building/publishing,
automated deployment

**Lower Priority**
6. High Availability - MongoDB replica set, HPA for registry pods
7. Operational Excellence - Kubernetes dashboard, cost optimization

### Configuration Issues
- Production config has test credentials: `deploy/Pulumi.prod.yaml:4-5`
- Missing environment-specific resource sizing
- Hardcoded domain names (`example.com`)

The deployment setup shows good architectural foundations but needs
significant work before production readiness. The most critical issue is
the placeholder nginx container - priority should be completing the
actual registry application deployment before addressing the other
improvements. Security measures like RBAC and Network Policies are nice
to have but not strictly necessary given that MongoDB is not exposed
externally.

🤖 Generated with [Claude Code](https://claude.ai/code)

</p>
</details> 


## Metadata

Working towards #91

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-11 19:43:49 +01:00