registry/internal/validators/validators.go
Tadas Antanavicius 56e8b4879b
Introduce serverId, rename id to versionId (#465)
Wielded Claude Code to do this; across many iterations and have reviewed
all the code + largely rewritten the PR commentary here.

This PR implements the server ID consistency changes from
[#396](https://github.com/modelcontextprotocol/registry/issues/396). Our
goal is to:
- Rename `id` to `versionId` to make it clear that it is the ID for a
specific version, not a server
- Introduce `serverId` as a net new concept. This will be 1:1 with
`name`. While not strictly necessary (we could just use name as a PK),
the convention of having a UUID as an ID on our objects is just a
helpful subjective architecture decision. Plus also leaves open the door
to reconsidering the "should we allow renames" question.

## Key Changes

### 🗄️ Database Schema Changes
- **Renamed**: `id` column → `version_id` (unique identifier for each
server version)

### 📋 Metadata Structure Updates
- **JSON Schema**: Updated
`_meta.io.modelcontextprotocol.registry/official` to include:
  - `serverId`: Consistent ID across all versions of a server
  - `versionId`: Unique ID for this specific version (renamed from `id`)

### 🔧 API Endpoint Updates
- **New endpoint structure**: Changed from version-ID-based to
server-ID-based URLs:
  - `GET /v0/servers/{server_id}` - Get latest version of server
  - `GET /v0/servers/{server_id}?version=X.X.X` - Get specific version  
- `GET /v0/servers/{server_id}/versions` - List all versions of a server
  - `PUT /v0/servers/{server_id}?version=X.X.X` - Edit specific version
- **Consistent URL patterns**: All server endpoints now use `server_id`
as the primary identifier
 
### Database Migration Strategy
The migration (`005_add_server_id_rename_version_id.sql`):
1. Adds new `version_id` column  
2. Generates deterministic UUIDs for servers with the same name
3. Updates JSON metadata to add `serverId` and rename `id` to
`versionId`
4. Copies existing `id` values to `version_id` column
5. Drops old `id` column and makes `version_id` the new primary key

## Testing

Besides of course automated tests passing, I:
1. Set up my local db with the pre-005-migration state, imported all the
production data
2. Ran the migration
3. Spot checked some diffs of the data that resulted --

```json
// before
{
  "name": "io.github.ruvnet/ruv-swarm",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "id": "0a9cda7f-2b73-4839-844b-6582f335de42",
      "is_latest": false,
      "updated_at": "2025-09-10T17:21:34.92137012Z",
      "published_at": "2025-09-10T17:15:32.69690433Z"
    }
  },
  "version": "1.0.18",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "Neural network swarm orchestration with WebAssembly acceleration and MCP integration"
}

// after
{
  "name": "io.github.ruvnet/ruv-swarm",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "e8538952-96af-4a33-9e31-7fbed012521e",
      "is_latest": false,
      "versionId": "0a9cda7f-2b73-4839-844b-6582f335de42",
      "updated_at": "2025-09-10T17:21:34.92137012Z",
      "published_at": "2025-09-10T17:15:32.69690433Z"
    }
  },
  "version": "1.0.18",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "Neural network swarm orchestration with WebAssembly acceleration and MCP integration"
}

```

Servers that have multiple versions stayed consistent in their
`serverId`:

```json
// First version
{
  "name": "app.getdialer/dialer",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "80f08bdb-f3e4-4f19-b072-1676ae49b663",
      "is_latest": false,
      "versionId": "04f25b17-331d-48d7-b08b-37baece208a7",
      "updated_at": "2025-09-09T00:16:49.162620072Z",
      "published_at": "2025-09-08T23:47:09.452137971Z"
    }
  },
  "status": "active",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
  "remotes": [
    {
      "url": "https://getdialer.app/mcp",
      "type": "streamable-http"
    },
    {
      "url": "https://getdialer.app/sse",
      "type": "sse"
    }
  ],
  "version": "1.0.0",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "An MCP server that provides your you make outbound phone calls using your own phone number"
}

// Second version
{
  "name": "app.getdialer/dialer",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "80f08bdb-f3e4-4f19-b072-1676ae49b663",
      "is_latest": true,
      "versionId": "faa26cb9-3aad-415e-aece-f715fff5fe5e",
      "updated_at": "2025-09-09T00:16:49.107842808Z",
      "published_at": "2025-09-09T00:16:49.107842808Z"
    }
  },
  "status": "active",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
  "remotes": [
    {
      "url": "https://getdialer.app/mcp",
      "type": "streamable-http"
    },
    {
      "url": "https://getdialer.app/sse",
      "type": "sse"
    }
  ],
  "version": "1.0.1",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "An MCP server that provides lets you make outbound phone calls using your own phone number"
}
```

And I also tried publishing a test server anew; looked good:

```bash
⏺ Bash(curl -s "http://localhost:8080/v0/servers/102dc4e4-6ca8-40e6-aa11-cd45763d3352" | jq '.')
  ⎿  {                                                                                             
       "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
       "name": "io.modelcontextprotocol.anonymous/test-claude-migration-server",
       "description": "A test server to verify the new migration and API endpoints work correctly",
       "status": "active",
       "repository": {
         "url": "https://github.com/example/test-claude-migration-server",
         "source": "github",
         "id": "example/test-claude-migration-server"
       },
       "version": "1.0.0",
       "_meta": {
         "io.modelcontextprotocol.registry/official": {
           "serverId": "102dc4e4-6ca8-40e6-aa11-cd45763d3352",
           "versionId": "48c9cd55-7bc4-4ac2-896e-59e81eb653e4",
           "published_at": "2025-09-12T18:33:23.264203-07:00",
           "updated_at": "2025-09-12T18:33:23.264203-07:00",
           "is_latest": true
         }
       }
     }

⏺ Bash(curl -s "http://localhost:8080/v0/servers/102dc4e4-6ca8-40e6-aa11-cd45763d3352/versions" | jq '.')
  ⎿  {                                                                                                 
       "servers": [
         {
           "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
           "name": "io.modelcontextprotocol.anonymous/test-claude-migration-server",
           "description": "A test server to verify the new migration and API endpoints work correctly",
           "status": "active",
           "repository": {
             "url": "https://github.com/example/test-claude-migration-server",
             "source": "github",
             "id": "example/test-claude-migration-server"
           },
           "version": "1.0.0",
           "_meta": {
             "io.modelcontextprotocol.registry/official": {
               "serverId": "102dc4e4-6ca8-40e6-aa11-cd45763d3352",
               "versionId": "48c9cd55-7bc4-4ac2-896e-59e81eb653e4",
               "published_at": "2025-09-12T18:33:23.264203-07:00",
               "updated_at": "2025-09-12T18:33:23.264203-07:00",
               "is_latest": true
             }
           }
         }
       ],
       "metadata": {
         "count": 1
       }
     }
```

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Preeti Dewani <preetidewani1990@gmail.com>
Co-authored-by: adam jones <adamj+git@anthropic.com>
Co-authored-by: Adam Jones <adamj@anthropic.com>
2025-09-21 15:00:09 +03:00

513 lines
No EOL
16 KiB
Go

package validators
import (
"context"
"encoding/json"
"fmt"
"net/url"
"regexp"
"slices"
"strings"
"github.com/modelcontextprotocol/registry/internal/config"
apiv0 "github.com/modelcontextprotocol/registry/pkg/api/v0"
"github.com/modelcontextprotocol/registry/pkg/model"
)
// Regexes to detect semver range syntaxes
var (
// Case 1: comparator ranges
// - "^1.2.3",
// - "~1.2.3",
// - ">=1.0.0",
// - "<=1.0.0",
// - ">1.0.0",
// - "<1.0.0",
// - "=1.0.0",
comparatorRangeRe = regexp.MustCompile(`^\s*(?:\^|~|>=|<=|>|<|=)\s*v?\d+(?:\.\d+){0,3}(?:-[0-9A-Za-z.-]+)?\s*$`)
// Case 2: hyphen ranges
// - "1.2.3 - 2.0.0",
hyphenRangeRe = regexp.MustCompile(`^\s*v?\d+(?:\.\d+){0,3}(?:-[0-9A-Za-z.-]+)?\s-\s*v?\d+(?:\.\d+){0,3}(?:-[0-9A-Za-z.-]+)?\s*$`)
// Case 3: OR ranges
// - "1.2 || 1.3",
orRangeRe = regexp.MustCompile(`^\s*(?:v?\d+(?:\.\d+){0,3}(?:-[0-9A-Za-z.-]+)?\s*)(?:\|\|\s*v?\d+(?:\.\d+){0,3}(?:-[0-9A-Za-z.-]+)?\s*)+$`)
// Case 4: dotted version wildcards
// - "1.2.*",
// - "1.2.x",
// - "1.2.X",
// - "1.x",
// etc.
dottedVersionLikeRe = regexp.MustCompile(`^\s*(?:v?\d+|x|X|\*)(?:\.(?:\d+|x|X|\*)){1,2}(?:-[0-9A-Za-z.-]+)?\s*$`)
)
func ValidateServerJSON(serverJSON *apiv0.ServerJSON) error {
// Validate server name exists and format
if _, err := parseServerName(*serverJSON); err != nil {
return err
}
// Validate top-level server version is a specific version (not a range) & not "latest"
if err := validateVersion(serverJSON.Version); err != nil {
return err
}
// Validate repository
if err := validateRepository(&serverJSON.Repository); err != nil {
return err
}
// Validate website URL if provided
if err := validateWebsiteURL(serverJSON.WebsiteURL); err != nil {
return err
}
// Validate all packages (basic field validation)
// Detailed package validation (including registry checks) is done during publish
for _, pkg := range serverJSON.Packages {
if err := validatePackageField(&pkg); err != nil {
return err
}
}
// Validate all remotes
for _, remote := range serverJSON.Remotes {
if err := validateRemoteTransport(&remote); err != nil {
return err
}
}
// Validate reverse-DNS namespace matching for remote URLs
if err := validateRemoteNamespaceMatch(*serverJSON); err != nil {
return err
}
// Validate reverse-DNS namespace matching for website URL
if err := validateWebsiteURLNamespaceMatch(*serverJSON); err != nil {
return err
}
return nil
}
func validateRepository(obj *model.Repository) error {
// Skip validation for empty repository (optional field)
if obj.URL == "" && obj.Source == "" {
return nil
}
// validate the repository source
repoSource := RepositorySource(obj.Source)
if !IsValidRepositoryURL(repoSource, obj.URL) {
return fmt.Errorf("%w: %s", ErrInvalidRepositoryURL, obj.URL)
}
// validate subfolder if present
if obj.Subfolder != "" && !IsValidSubfolderPath(obj.Subfolder) {
return fmt.Errorf("%w: %s", ErrInvalidSubfolderPath, obj.Subfolder)
}
return nil
}
func validateWebsiteURL(websiteURL string) error {
// Skip validation if website URL is not provided (optional field)
if websiteURL == "" {
return nil
}
// Parse the URL to ensure it's valid
parsedURL, err := url.Parse(websiteURL)
if err != nil {
return fmt.Errorf("invalid websiteUrl: %w", err)
}
// Ensure it's an absolute URL with valid scheme
if !parsedURL.IsAbs() {
return fmt.Errorf("websiteUrl must be absolute (include scheme): %s", websiteURL)
}
// Only allow HTTP/HTTPS schemes for security
if parsedURL.Scheme != "http" && parsedURL.Scheme != "https" {
return fmt.Errorf("websiteUrl must use http or https scheme: %s", websiteURL)
}
return nil
}
func validatePackageField(obj *model.Package) error {
if !HasNoSpaces(obj.Identifier) {
return ErrPackageNameHasSpaces
}
// Validate version string
if err := validateVersion(obj.Version); err != nil {
return err
}
// Validate runtime arguments
for _, arg := range obj.RuntimeArguments {
if err := validateArgument(&arg); err != nil {
return fmt.Errorf("invalid runtime argument: %w", err)
}
}
// Validate package arguments
for _, arg := range obj.PackageArguments {
if err := validateArgument(&arg); err != nil {
return fmt.Errorf("invalid package argument: %w", err)
}
}
// Validate transport with template variable support
availableVariables := collectAvailableVariables(obj)
if err := validatePackageTransport(&obj.Transport, availableVariables); err != nil {
return fmt.Errorf("invalid transport: %w", err)
}
return nil
}
// validateVersion validates the version string.
// NB: we decided that we would not enforce strict semver for version strings
func validateVersion(version string) error {
if version == "latest" {
return ErrReservedVersionString
}
// Reject semver range-like inputs
if looksLikeVersionRange(version) {
return fmt.Errorf("%w: %q", ErrVersionLooksLikeRange, version)
}
return nil
}
// looksLikeVersionRange detects common semver range syntaxes and wildcard patterns.
// that indicate the value is not a single, specific version.
// Examples that should return true:
// - "^1.2.3",
// - "~1.2.3",
// - ">=1.0.0",
// - "1.x",
// - "1.2.*",
// - "1 - 2",
// - "1.2 || 1.3"
func looksLikeVersionRange(version string) bool {
trimmed := strings.TrimSpace(version)
if trimmed == "" {
return false
}
if comparatorRangeRe.MatchString(trimmed) {
return true
}
if hyphenRangeRe.MatchString(trimmed) {
return true
}
if orRangeRe.MatchString(trimmed) {
return true
}
if dottedVersionLikeRe.MatchString(trimmed) {
// wildcard in a dotted version (x/X/*) implies range-like intent
return strings.Contains(trimmed, "x") || strings.Contains(trimmed, "X") || strings.Contains(trimmed, "*")
}
return false
}
// validateArgument validates argument details
func validateArgument(obj *model.Argument) error {
if obj.Type == model.ArgumentTypeNamed {
// Validate named argument name format
if err := validateNamedArgumentName(obj.Name); err != nil {
return err
}
// Validate value and default don't start with the name
if err := validateArgumentValueFields(obj.Name, obj.Value, obj.Default); err != nil {
return err
}
}
return nil
}
func validateNamedArgumentName(name string) error {
// Check if name is required for named arguments
if name == "" {
return ErrNamedArgumentNameRequired
}
// Check for invalid characters that suggest embedded values or descriptions
// Valid: "--directory", "--port", "-v", "config", "verbose"
// Invalid: "--directory <absolute_path_to_adfin_mcp_folder>", "--port 8080"
if strings.Contains(name, "<") || strings.Contains(name, ">") ||
strings.Contains(name, " ") || strings.Contains(name, "$") {
return fmt.Errorf("%w: %s", ErrInvalidNamedArgumentName, name)
}
return nil
}
func validateArgumentValueFields(name, value, defaultValue string) error {
// Check if value starts with the argument name (using startsWith, not contains)
if value != "" && strings.HasPrefix(value, name) {
return fmt.Errorf("%w: value starts with argument name '%s': %s", ErrArgumentValueStartsWithName, name, value)
}
if defaultValue != "" && strings.HasPrefix(defaultValue, name) {
return fmt.Errorf("%w: default starts with argument name '%s': %s", ErrArgumentDefaultStartsWithName, name, defaultValue)
}
return nil
}
// collectAvailableVariables collects all available template variables from a package
func collectAvailableVariables(pkg *model.Package) []string {
var variables []string
// Add environment variable names
for _, env := range pkg.EnvironmentVariables {
variables = append(variables, env.Name)
}
// Add runtime argument names and value hints
for _, arg := range pkg.RuntimeArguments {
if arg.Name != "" {
variables = append(variables, arg.Name)
}
if arg.ValueHint != "" {
variables = append(variables, arg.ValueHint)
}
}
// Add package argument names and value hints
for _, arg := range pkg.PackageArguments {
if arg.Name != "" {
variables = append(variables, arg.Name)
}
if arg.ValueHint != "" {
variables = append(variables, arg.ValueHint)
}
}
return variables
}
// validatePackageTransport validates a package's transport with templating support
func validatePackageTransport(transport *model.Transport, availableVariables []string) error {
// Validate transport type is supported
switch transport.Type {
case model.TransportTypeStdio:
// Validate that URL is empty for stdio transport
if transport.URL != "" {
return fmt.Errorf("url must be empty for %s transport type, got: %s", transport.Type, transport.URL)
}
return nil
case model.TransportTypeStreamableHTTP, model.TransportTypeSSE:
// URL is required for streamable-http and sse
if transport.URL == "" {
return fmt.Errorf("url is required for %s transport type", transport.Type)
}
// Validate URL format with template variable support
if !IsValidTemplatedURL(transport.URL, availableVariables, true) {
// Check if it's a template variable issue or basic URL issue
templateVars := extractTemplateVariables(transport.URL)
if len(templateVars) > 0 {
return fmt.Errorf("%w: template variables in URL %s reference undefined variables. Available variables: %v",
ErrInvalidRemoteURL, transport.URL, availableVariables)
}
return fmt.Errorf("%w: %s", ErrInvalidRemoteURL, transport.URL)
}
return nil
default:
return fmt.Errorf("unsupported transport type: %s", transport.Type)
}
}
// validateRemoteTransport validates a remote transport (no templating allowed)
func validateRemoteTransport(obj *model.Transport) error {
// Validate transport type is supported - remotes only support streamable-http and sse
switch obj.Type {
case model.TransportTypeStreamableHTTP, model.TransportTypeSSE:
// URL is required for streamable-http and sse
if obj.URL == "" {
return fmt.Errorf("url is required for %s transport type", obj.Type)
}
// Validate URL format (no templates allowed for remotes, no localhost)
if !IsValidRemoteURL(obj.URL) {
return fmt.Errorf("%w: %s", ErrInvalidRemoteURL, obj.URL)
}
return nil
default:
return fmt.Errorf("unsupported transport type for remotes: %s (only streamable-http and sse are supported)", obj.Type)
}
}
// ValidatePublishRequest validates a complete publish request including extensions
func ValidatePublishRequest(req apiv0.ServerJSON, cfg *config.Config) error {
// Validate publisher extensions in _meta
if err := validatePublisherExtensions(req); err != nil {
return err
}
// Validate the server detail (includes all nested validation)
if err := ValidateServerJSON(&req); err != nil {
return err
}
// Validate registry ownership for all packages if validation is enabled and server is not deleted
if cfg.EnableRegistryValidation && req.Status != model.StatusDeleted {
ctx := context.Background()
for i, pkg := range req.Packages {
if err := ValidatePackage(ctx, pkg, req.Name); err != nil {
return fmt.Errorf("registry validation failed for package %d (%s): %w", i, pkg.Identifier, err)
}
}
}
return nil
}
func validatePublisherExtensions(req apiv0.ServerJSON) error {
const maxExtensionSize = 4 * 1024 // 4KB limit
// Check size limit for _meta publisher-provided extension
if req.Meta != nil && req.Meta.PublisherProvided != nil {
extensionsJSON, err := json.Marshal(req.Meta.PublisherProvided)
if err != nil {
return fmt.Errorf("failed to marshal _meta.io.modelcontextprotocol.registry/publisher-provided extension: %w", err)
}
if len(extensionsJSON) > maxExtensionSize {
return fmt.Errorf("_meta.io.modelcontextprotocol.registry/publisher-provided extension exceeds 4KB limit (%d bytes)", len(extensionsJSON))
}
}
if req.Meta != nil {
// Validate that only publisher-provided data is allowed in _meta during publish (no official registry metadata should be present)
if req.Meta.Official != nil {
return fmt.Errorf("official registry metadata '_meta.io.modelcontextprotocol.registry/official' is not allowed during publish")
}
}
return nil
}
func parseServerName(serverJSON apiv0.ServerJSON) (string, error) {
name := serverJSON.Name
if name == "" {
return "", fmt.Errorf("server name is required and must be a string")
}
// Validate format: dns-namespace/name
if !strings.Contains(name, "/") {
return "", fmt.Errorf("server name must be in format 'dns-namespace/name' (e.g., 'com.example.api/server')")
}
// Check for multiple slashes - reject if found
slashCount := strings.Count(name, "/")
if slashCount > 1 {
return "", ErrMultipleSlashesInServerName
}
parts := strings.SplitN(name, "/", 2)
if len(parts) != 2 || parts[0] == "" || parts[1] == "" {
return "", fmt.Errorf("server name must be in format 'dns-namespace/name' with non-empty namespace and name parts")
}
return name, nil
}
// validateRemoteNamespaceMatch validates that remote URLs match the reverse-DNS namespace
func validateRemoteNamespaceMatch(serverJSON apiv0.ServerJSON) error {
namespace := serverJSON.Name
for _, remote := range serverJSON.Remotes {
if err := validateRemoteURLMatchesNamespace(remote.URL, namespace); err != nil {
return fmt.Errorf("remote URL %s does not match namespace %s: %w", remote.URL, namespace, err)
}
}
return nil
}
// validateWebsiteURLNamespaceMatch validates that website URL matches the reverse-DNS namespace
func validateWebsiteURLNamespaceMatch(serverJSON apiv0.ServerJSON) error {
// Skip validation if website URL is not provided
if serverJSON.WebsiteURL == "" {
return nil
}
namespace := serverJSON.Name
if err := validateRemoteURLMatchesNamespace(serverJSON.WebsiteURL, namespace); err != nil {
return fmt.Errorf("websiteUrl %s does not match namespace %s: %w", serverJSON.WebsiteURL, namespace, err)
}
return nil
}
// validateRemoteURLMatchesNamespace checks if a remote URL's hostname matches the publisher domain from the namespace
func validateRemoteURLMatchesNamespace(remoteURL, namespace string) error {
// Parse the URL to extract the hostname
parsedURL, err := url.Parse(remoteURL)
if err != nil {
return fmt.Errorf("invalid URL format: %w", err)
}
hostname := parsedURL.Hostname()
if hostname == "" {
return fmt.Errorf("URL must have a valid hostname")
}
// Skip validation for localhost and local development URLs
if hostname == "localhost" || strings.HasSuffix(hostname, ".localhost") || hostname == "127.0.0.1" {
return nil
}
// Extract publisher domain from reverse-DNS namespace
publisherDomain := extractPublisherDomainFromNamespace(namespace)
if publisherDomain == "" {
return fmt.Errorf("invalid namespace format: cannot extract domain from %s", namespace)
}
// Check if the remote URL hostname matches the publisher domain or is a subdomain
if !isValidHostForDomain(hostname, publisherDomain) {
return fmt.Errorf("remote URL host %s does not match publisher domain %s", hostname, publisherDomain)
}
return nil
}
// extractPublisherDomainFromNamespace converts reverse-DNS namespace to normal domain format
// e.g., "com.example" -> "example.com"
func extractPublisherDomainFromNamespace(namespace string) string {
// Extract the namespace part before the first slash
namespacePart := namespace
if slashIdx := strings.Index(namespace, "/"); slashIdx != -1 {
namespacePart = namespace[:slashIdx]
}
// Split into parts and reverse them to get normal domain format
parts := strings.Split(namespacePart, ".")
if len(parts) < 2 {
return ""
}
// Reverse the parts to convert from reverse-DNS to normal domain
slices.Reverse(parts)
return strings.Join(parts, ".")
}
// isValidHostForDomain checks if a hostname is the domain or a subdomain of the publisher domain
func isValidHostForDomain(hostname, publisherDomain string) bool {
// Exact match
if hostname == publisherDomain {
return true
}
// Subdomain match - hostname should end with "." + publisherDomain
if strings.HasSuffix(hostname, "."+publisherDomain) {
return true
}
return false
}