registry/internal/validators/registries/mcpb.go
Tadas Antanavicius 56e8b4879b
Introduce serverId, rename id to versionId (#465)
Wielded Claude Code to do this; across many iterations and have reviewed
all the code + largely rewritten the PR commentary here.

This PR implements the server ID consistency changes from
[#396](https://github.com/modelcontextprotocol/registry/issues/396). Our
goal is to:
- Rename `id` to `versionId` to make it clear that it is the ID for a
specific version, not a server
- Introduce `serverId` as a net new concept. This will be 1:1 with
`name`. While not strictly necessary (we could just use name as a PK),
the convention of having a UUID as an ID on our objects is just a
helpful subjective architecture decision. Plus also leaves open the door
to reconsidering the "should we allow renames" question.

## Key Changes

### 🗄️ Database Schema Changes
- **Renamed**: `id` column → `version_id` (unique identifier for each
server version)

### 📋 Metadata Structure Updates
- **JSON Schema**: Updated
`_meta.io.modelcontextprotocol.registry/official` to include:
  - `serverId`: Consistent ID across all versions of a server
  - `versionId`: Unique ID for this specific version (renamed from `id`)

### 🔧 API Endpoint Updates
- **New endpoint structure**: Changed from version-ID-based to
server-ID-based URLs:
  - `GET /v0/servers/{server_id}` - Get latest version of server
  - `GET /v0/servers/{server_id}?version=X.X.X` - Get specific version  
- `GET /v0/servers/{server_id}/versions` - List all versions of a server
  - `PUT /v0/servers/{server_id}?version=X.X.X` - Edit specific version
- **Consistent URL patterns**: All server endpoints now use `server_id`
as the primary identifier
 
### Database Migration Strategy
The migration (`005_add_server_id_rename_version_id.sql`):
1. Adds new `version_id` column  
2. Generates deterministic UUIDs for servers with the same name
3. Updates JSON metadata to add `serverId` and rename `id` to
`versionId`
4. Copies existing `id` values to `version_id` column
5. Drops old `id` column and makes `version_id` the new primary key

## Testing

Besides of course automated tests passing, I:
1. Set up my local db with the pre-005-migration state, imported all the
production data
2. Ran the migration
3. Spot checked some diffs of the data that resulted --

```json
// before
{
  "name": "io.github.ruvnet/ruv-swarm",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "id": "0a9cda7f-2b73-4839-844b-6582f335de42",
      "is_latest": false,
      "updated_at": "2025-09-10T17:21:34.92137012Z",
      "published_at": "2025-09-10T17:15:32.69690433Z"
    }
  },
  "version": "1.0.18",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "Neural network swarm orchestration with WebAssembly acceleration and MCP integration"
}

// after
{
  "name": "io.github.ruvnet/ruv-swarm",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "e8538952-96af-4a33-9e31-7fbed012521e",
      "is_latest": false,
      "versionId": "0a9cda7f-2b73-4839-844b-6582f335de42",
      "updated_at": "2025-09-10T17:21:34.92137012Z",
      "published_at": "2025-09-10T17:15:32.69690433Z"
    }
  },
  "version": "1.0.18",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "Neural network swarm orchestration with WebAssembly acceleration and MCP integration"
}

```

Servers that have multiple versions stayed consistent in their
`serverId`:

```json
// First version
{
  "name": "app.getdialer/dialer",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "80f08bdb-f3e4-4f19-b072-1676ae49b663",
      "is_latest": false,
      "versionId": "04f25b17-331d-48d7-b08b-37baece208a7",
      "updated_at": "2025-09-09T00:16:49.162620072Z",
      "published_at": "2025-09-08T23:47:09.452137971Z"
    }
  },
  "status": "active",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
  "remotes": [
    {
      "url": "https://getdialer.app/mcp",
      "type": "streamable-http"
    },
    {
      "url": "https://getdialer.app/sse",
      "type": "sse"
    }
  ],
  "version": "1.0.0",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "An MCP server that provides your you make outbound phone calls using your own phone number"
}

// Second version
{
  "name": "app.getdialer/dialer",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "80f08bdb-f3e4-4f19-b072-1676ae49b663",
      "is_latest": true,
      "versionId": "faa26cb9-3aad-415e-aece-f715fff5fe5e",
      "updated_at": "2025-09-09T00:16:49.107842808Z",
      "published_at": "2025-09-09T00:16:49.107842808Z"
    }
  },
  "status": "active",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
  "remotes": [
    {
      "url": "https://getdialer.app/mcp",
      "type": "streamable-http"
    },
    {
      "url": "https://getdialer.app/sse",
      "type": "sse"
    }
  ],
  "version": "1.0.1",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "An MCP server that provides lets you make outbound phone calls using your own phone number"
}
```

And I also tried publishing a test server anew; looked good:

```bash
⏺ Bash(curl -s "http://localhost:8080/v0/servers/102dc4e4-6ca8-40e6-aa11-cd45763d3352" | jq '.')
  ⎿  {                                                                                             
       "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
       "name": "io.modelcontextprotocol.anonymous/test-claude-migration-server",
       "description": "A test server to verify the new migration and API endpoints work correctly",
       "status": "active",
       "repository": {
         "url": "https://github.com/example/test-claude-migration-server",
         "source": "github",
         "id": "example/test-claude-migration-server"
       },
       "version": "1.0.0",
       "_meta": {
         "io.modelcontextprotocol.registry/official": {
           "serverId": "102dc4e4-6ca8-40e6-aa11-cd45763d3352",
           "versionId": "48c9cd55-7bc4-4ac2-896e-59e81eb653e4",
           "published_at": "2025-09-12T18:33:23.264203-07:00",
           "updated_at": "2025-09-12T18:33:23.264203-07:00",
           "is_latest": true
         }
       }
     }

⏺ Bash(curl -s "http://localhost:8080/v0/servers/102dc4e4-6ca8-40e6-aa11-cd45763d3352/versions" | jq '.')
  ⎿  {                                                                                                 
       "servers": [
         {
           "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
           "name": "io.modelcontextprotocol.anonymous/test-claude-migration-server",
           "description": "A test server to verify the new migration and API endpoints work correctly",
           "status": "active",
           "repository": {
             "url": "https://github.com/example/test-claude-migration-server",
             "source": "github",
             "id": "example/test-claude-migration-server"
           },
           "version": "1.0.0",
           "_meta": {
             "io.modelcontextprotocol.registry/official": {
               "serverId": "102dc4e4-6ca8-40e6-aa11-cd45763d3352",
               "versionId": "48c9cd55-7bc4-4ac2-896e-59e81eb653e4",
               "published_at": "2025-09-12T18:33:23.264203-07:00",
               "updated_at": "2025-09-12T18:33:23.264203-07:00",
               "is_latest": true
             }
           }
         }
       ],
       "metadata": {
         "count": 1
       }
     }
```

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Preeti Dewani <preetidewani1990@gmail.com>
Co-authored-by: adam jones <adamj+git@anthropic.com>
Co-authored-by: Adam Jones <adamj@anthropic.com>
2025-09-21 15:00:09 +03:00

172 lines
5.5 KiB
Go

package registries
import (
"context"
"fmt"
"net/http"
"net/url"
"regexp"
"strings"
"time"
"github.com/modelcontextprotocol/registry/pkg/model"
)
func ValidateMCPB(ctx context.Context, pkg model.Package, _ string) error {
// MCPB packages must include a file hash for integrity verification
if pkg.FileSHA256 == "" {
return fmt.Errorf("MCPB package must include a fileSha256 hash for integrity verification")
}
err := validateMCPBUrl(pkg.Identifier)
if err != nil {
return err
}
inferredBaseURL, err := inferMCPBRegistryBaseURL(pkg.Identifier)
if err != nil {
return err
}
if pkg.RegistryBaseURL == "" {
pkg.RegistryBaseURL = inferredBaseURL
} else if pkg.RegistryBaseURL != inferredBaseURL {
return fmt.Errorf("MCPB package '%s' has inconsistent registry base URL: %s (expected: %s)",
pkg.Identifier, pkg.RegistryBaseURL, inferredBaseURL)
}
// Parse the URL to validate format
url, err := url.Parse(pkg.Identifier)
if err != nil {
return fmt.Errorf("invalid MCPB package URL: %w", err)
}
if url.Scheme != "https" {
return fmt.Errorf("invalid MCPB package URL, must use HTTPS: %s", pkg.Identifier)
}
// Check that the URL contains 'mcp' somewhere (case-insensitive)
if !strings.Contains(strings.ToLower(pkg.Identifier), "mcp") {
return fmt.Errorf("MCPB package URL must contain 'mcp': %s", pkg.Identifier)
}
// Verify the file exists and is publicly accessible
client := &http.Client{Timeout: 10 * time.Second}
req, err := http.NewRequestWithContext(ctx, http.MethodHead, pkg.Identifier, nil)
if err != nil {
return fmt.Errorf("failed to create request: %w", err)
}
req.Header.Set("User-Agent", "MCP-Registry-Validator/1.0")
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("failed to verify MCPB package accessibility: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("MCPB package '%s' is not publicly accessible (status: %d)", pkg.Identifier, resp.StatusCode)
}
return nil
}
func validateMCPBUrl(fullURL string) error {
parsedURL, err := url.Parse(fullURL)
if err != nil {
return fmt.Errorf("invalid MCPB package URL: %w", err)
}
host := strings.ToLower(parsedURL.Host)
allowedHosts := []string{
"github.com",
"www.github.com",
"gitlab.com",
"www.gitlab.com",
}
isAllowed := false
for _, allowed := range allowedHosts {
if host == allowed {
isAllowed = true
break
}
}
if !isAllowed {
return fmt.Errorf("MCPB packages must be hosted on allowlisted providers (GitHub or GitLab). Host '%s' is not allowed", host)
}
// Validate URL path is a proper release URL with strict structure validation
path := parsedURL.Path
switch host {
case "github.com", "www.github.com":
// GitHub release URLs must match: /owner/repo/releases/download/tag/filename
if !isValidGitHubReleaseURL(path) {
return fmt.Errorf("GitHub MCPB packages must be release assets following the pattern '/owner/repo/releases/download/tag/filename'")
}
case "gitlab.com", "www.gitlab.com":
// GitLab release URLs must match specific patterns
if !isValidGitLabReleaseURL(path) {
return fmt.Errorf("GitLab MCPB packages must be release assets following patterns '/owner/repo/-/releases/tag/downloads/filename' or '/owner/repo/-/package_files/id/download'")
}
}
return nil
}
// isValidGitHubReleaseURL validates that a path follows the GitHub release asset pattern
// Pattern: /owner/repo/releases/download/tag/filename
func isValidGitHubReleaseURL(path string) bool {
// GitHub release URL pattern: /owner/repo/releases/download/tag/filename
// - owner: username or organization (1-39 chars, alphanumeric + hyphens, no consecutive hyphens)
// - repo: repository name (similar rules to owner)
// - tag: release tag (can contain various characters but not empty)
// - filename: asset filename (not empty)
pattern := `^/([a-zA-Z0-9]([a-zA-Z0-9\-]{0,37}[a-zA-Z0-9])?)/([a-zA-Z0-9._\-]+)/releases/download/([^/]+)/([^/]+)$`
matched, _ := regexp.MatchString(pattern, path)
return matched
}
// isValidGitLabReleaseURL validates that a path follows GitLab release asset patterns
func isValidGitLabReleaseURL(path string) bool {
// GitLab release URL patterns:
// 1. /owner/repo/-/releases/tag/downloads/filename
// 2. /owner/repo/-/package_files/id/download
// 3. /group/subgroup/repo/-/releases/tag/downloads/filename (nested groups)
// The key insight is that GitLab URLs have "/-/" as a delimiter that separates the
// project path from the GitLab-specific routes. Everything before "/-/" is the project path.
// Pattern 1: Release downloads with /-/releases/tag/downloads/filename
releasePattern := `^/([a-zA-Z0-9._\-]+(?:/[a-zA-Z0-9._\-]+)*)/-/releases/([^/]+)/downloads/([^/]+)$`
if matched, _ := regexp.MatchString(releasePattern, path); matched {
return true
}
// Pattern 2: Package files with /-/package_files/id/download
packagePattern := `^/([a-zA-Z0-9._\-]+(?:/[a-zA-Z0-9._\-]+)*)/-/package_files/([0-9]+)/download$`
if matched, _ := regexp.MatchString(packagePattern, path); matched {
return true
}
return false
}
// inferMCPBRegistryBaseURL infers the registry base URL from an MCPB identifier
func inferMCPBRegistryBaseURL(identifier string) (string, error) {
parsedURL, err := url.Parse(identifier)
if err != nil {
return "", err
}
host := strings.ToLower(parsedURL.Host)
switch host {
case "github.com", "www.github.com":
return model.RegistryURLGitHub, nil
case "gitlab.com", "www.gitlab.com":
return model.RegistryURLGitLab, nil
default:
return "", fmt.Errorf("invalid host for MCPB package: %s, expected github or gitlab", host)
}
}