deploy: Deploy the registry to GCP (#255)
Adds the Pulumi code to: - Deploy the registry (and associated services e.g. mongodb) to Google Cloud Platform (GCP), on top of Google Kubernetes Engine (GKE) - Sets up proper environments and secrets management - Uses the real container image, now that it's published in #225. At the moment attached to latest, we might want to pin the version later (or perhaps always use `latest` in staging, and pin prod) - Uses real domains (`staging.registry.modelcontextprotocol.io`) rather than examples (``) ## Motivation and Context Setting up infrastructure to deploy it. I set something up in Azure in #227, although not super robust (e.g. no service accounts etc.). Think we will use GCP as: - the maintainers have experience with GCP, but none with Azure - costs are quite low, and Anthropic is happy to cover them in the short term - means we only have to maintain one login system (just Google Cloud Identity), not two (Google Workspace + Azure) ## How Has This Been Tested? Deployed this to a staging and production cluster. Try it yourself at: ```bash curl -H "Host: staging.registry.modelcontextprotocol.io" -k https://35.222.36.75/v0/ping ``` (will be sorting out domains very soon) ## Breaking Changes NA - just adds support for GCP deployment ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. --> - [x] I have read the [MCP Documentation](https://modelcontextprotocol.io) - [x] My code follows the repository's style guidelines - [ ] New and existing tests pass locally - [x] I have added appropriate error handling - [x] I have added or updated documentation as needed ## Additional context <!-- Add any other context, implementation notes, or design decisions --> Expected follow-ups: - GitHub Action setup to deploy things to the cluster from GitHub, to avoid gatekeeping to just the people with the secrets.
This commit is contained in:
parent
667e2840b8
commit
fe52766676
1 changed files with 395 additions and 144 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# MCP Registry Kubernetes Deployment
|
||||
|
||||
This directory contains Pulumi infrastructure code to deploy the MCP Registry service to a Kubernetes cluster. It supports multiple Kubernetes providers: Azure Kubernetes Service (AKS) and local (using existing kubeconfig).
|
||||
This directory contains Pulumi infrastructure code to deploy the MCP Registry service to a Kubernetes cluster. It supports deploying the infrastructure locally (using an existing kubeconfig, e.g. with minikube) or to Google Cloud Platform (GCP).
|
||||
|
||||
## Quick Start
|
||||
|
||||
|
|
@ -19,17 +19,53 @@ Pre-requisites:
|
|||
|
||||
# To use your local kubeconfig (default)
|
||||
pulumi config set mcp-registry:provider local
|
||||
# Alternative: To use AKS
|
||||
# pulumi config set mcp-registry:provider aks
|
||||
|
||||
# GitHub OAuth
|
||||
pulumi config set mcp-registry:githubClientId <your-github-client-id>
|
||||
pulumi config set --secret mcp-registry:githubClientSecret <your-github-client-secret>
|
||||
```
|
||||
4. Deploy: `go build && PULUMI_CONFIG_PASSPHRASE="" pulumi up --yes`
|
||||
5. Access the repository via the ingress load balancer. You can find its external IP with `kubectl get svc nginx-ingress-ingress-nginx-controller -n ingress-nginx` (with minikube, if it's 'pending' you might need `minikube tunnel`). Then run `curl -H "Host: mcp-registry-local.example.com" -k https://<EXTERNAL-IP>/v0/ping` to check that the service is up.
|
||||
4. Deploy: `make local-up`
|
||||
5. Access the repository via the ingress load balancer. You can find its external IP with `kubectl get svc ingress-nginx-controller -n ingress-nginx` (with minikube, if it's 'pending' you might need `minikube tunnel`). Then run `curl -H "Host: local.registry.modelcontextprotocol.io" -k https://<EXTERNAL-IP>/v0/ping` to check that the service is up.
|
||||
|
||||
### Production Deployment (AKS)
|
||||
### Production Deployment (GCP)
|
||||
|
||||
**Note:** This is how the production deployment will be set up once. But then the plan will be future updates are effectively a login + `pulumi up` from GitHub Actions.
|
||||
|
||||
Pre-requisites:
|
||||
- [Pulumi CLI installed](https://www.pulumi.com/docs/iac/download-install/)
|
||||
- A Google Cloud Platform (GCP) account
|
||||
- A GCP Service Account with appropriate permissions
|
||||
|
||||
1. Create a project: `gcloud projects create mcp-registry-prod`
|
||||
2. Set the project: `gcloud config set project mcp-registry-prod`
|
||||
3. Enable required APIs: `gcloud services enable storage.googleapis.com && gcloud services enable container.googleapis.com`
|
||||
4. Create a service account with necessary permissions, and get the key:
|
||||
```bash
|
||||
gcloud iam service-accounts create pulumi-svc
|
||||
sleep 10
|
||||
gcloud projects add-iam-policy-binding mcp-registry-prod --member="serviceAccount:pulumi-svc@mcp-registry-prod.iam.gserviceaccount.com" --role="roles/container.admin"
|
||||
gcloud projects add-iam-policy-binding mcp-registry-prod --member="serviceAccount:pulumi-svc@mcp-registry-prod.iam.gserviceaccount.com" --role="roles/compute.admin"
|
||||
gcloud projects add-iam-policy-binding mcp-registry-prod --member="serviceAccount:pulumi-svc@mcp-registry-prod.iam.gserviceaccount.com" --role="roles/storage.admin"
|
||||
gcloud iam service-accounts add-iam-policy-binding $(gcloud projects describe mcp-registry-prod --format="value(projectNumber)")-compute@developer.gserviceaccount.com --member="serviceAccount:pulumi-svc@mcp-registry-prod.iam.gserviceaccount.com" --role="roles/iam.serviceAccountUser"
|
||||
gcloud iam service-accounts keys create sa-key.json --iam-account=pulumi-svc@mcp-registry-prod.iam.gserviceaccount.com
|
||||
```
|
||||
5. Create a GCS bucket for Pulumi state: `gsutil mb gs://mcp-registry-prod-pulumi-state`
|
||||
6. Set Pulumi's backend to GCS: `pulumi login gs://mcp-registry-prod-pulumi-state`
|
||||
7. Get the passphrase file `passphrase.prod.txt` from @domdomegg
|
||||
- TODO: avoid dependence on one person! Probably will shift all of this into CI.
|
||||
8. Init the GCP stack: `PULUMI_CONFIG_PASSPHRASE_FILE=passphrase.prod.txt pulumi stack init gcpProd`
|
||||
9. Set the GCP credentials in Pulumi config:
|
||||
```bash
|
||||
# Base64 encode the service account key and set it
|
||||
pulumi config set --secret gcp:credentials "$(base64 < sa-key.json)"
|
||||
```
|
||||
10. Deploy: `make prod-up`
|
||||
11. Access the repository via the ingress load balancer. You can find its external IP with: `kubectl get svc ingress-nginx-controller -n ingress-nginx`.
|
||||
Then run `curl -H "Host: prod.registry.modelcontextprotocol.io" -k https://<EXTERNAL-IP>/v0/ping` to check that the service is up.
|
||||
|
||||
<!--
|
||||
|
||||
### Production Deployment (Azure)
|
||||
|
||||
**Note:** This is how the production deployment will be set up once. But then the plan will be future updates are effectively a login + `pulumi up` from GitHub Actions.
|
||||
|
||||
|
|
@ -45,10 +81,12 @@ Pre-requisites:
|
|||
5. Add the 'Storage Blob Data Contributor' role assignment for yourself on the storage account: `az role assignment create --assignee $(az ad signed-in-user show --query id -o tsv) --role "Storage Blob Data Contributor" --scope "/subscriptions/$(az account show --query id -o tsv)/resourceGroups/official-mcp-registry-prod"`
|
||||
6. Create a container: `az storage container create --name pulumi-state --account-name officialmcpregistryprod`
|
||||
7. Set Pulumi's backend to Azure: `pulumi login 'azblob://pulumi-state?storage_account=officialmcpregistryprod'`
|
||||
8. Init the production stack: `pulumi stack init prod`
|
||||
8. Init the production stack: `pulumi stack init aksProd`
|
||||
- TODO: This has a password that maybe needs to be shared with select contributors?
|
||||
9. Deploy: `go build && PULUMI_CONFIG_PASSPHRASE="" pulumi up --yes`
|
||||
10. Access the repository via the ingress load balancer. You can find its external IP with `kubectl get svc nginx-ingress-ingress-nginx-controller -n ingress-nginx` or view it in the Pulumi outputs. Then run `curl -H "Host: mcp-registry-prod.example.com" -k https://<EXTERNAL-IP>/v0/ping` to check that the service is up.
|
||||
10. Access the repository via the ingress load balancer. You can find its external IP with `kubectl get svc ingress-nginx-controller -n ingress-nginx` or view it in the Pulumi outputs. Then run `curl -H "Host: prod.registry.modelcontextprotocol.io" -k https://<EXTERNAL-IP>/v0/ping` to check that the service is up.
|
||||
|
||||
-->
|
||||
|
||||
## Structure
|
||||
|
||||
|
|
@ -56,7 +94,8 @@ Pre-requisites:
|
|||
├── main.go # Pulumi program entry point
|
||||
├── Pulumi.yaml # Project configuration
|
||||
├── Pulumi.local.yaml # Local stack configuration
|
||||
├── Pulumi.prod.yaml # Production stack configuration
|
||||
├── Pulumi.prod.yaml # Production stack configuration (Azure)
|
||||
├── Pulumi.gcp.yaml # Production stack configuration (GCP)
|
||||
├── Makefile # Build and deployment targets
|
||||
├── go.mod # Go module dependencies
|
||||
├── go.sum # Go module checksums
|
||||
|
|
@ -70,6 +109,7 @@ Pre-requisites:
|
|||
└── providers/ # Kubernetes cluster providers
|
||||
├── types.go # Provider interface definitions
|
||||
├── aks/ # Azure Kubernetes Service provider
|
||||
├── gcp/ # Google Kubernetes Engine provider
|
||||
└── local/ # Local kubeconfig provider
|
||||
```
|
||||
|
||||
|
|
@ -91,9 +131,11 @@ Pre-requisites:
|
|||
| Parameter | Description | Required |
|
||||
|-----------|-------------|----------|
|
||||
| `environment` | Deployment environment (dev/prod) | Yes |
|
||||
| `provider` | Kubernetes provider (local/aks) | No (default: local) |
|
||||
| `provider` | Kubernetes provider (local/aks/gcp) | No (default: local) |
|
||||
| `githubClientId` | GitHub OAuth Client ID | Yes |
|
||||
| `githubClientSecret` | GitHub OAuth Client Secret | Yes |
|
||||
| `gcpProjectId` | GCP Project ID (required when provider=gcp) | No |
|
||||
| `gcpRegion` | GCP Region (default: us-central1) | No |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue