Commit graph registry/internal/database/migrations
Author SHA1 Message Date
Radoslav Dimitrov
0cb1f6bf52
fix: recalculate isLatest after status changes (#1081) (#1201)
## Summary

Fixes #1081. When the version flagged `is_latest` was soft-deleted, no
other version was promoted, and because `GetCurrentLatestVersion` didn't
filter deleted rows, subsequent publishes with a lower semver never
became latest — leaving `/v0/servers/{name}/versions/latest` permanently
404.

This PR has two parts:

### 1. Code fix — recalculate `is_latest` on every status change

- Recalculate `is_latest` after every status mutation
(`UpdateServerStatus`, `UpdateAllVersionsStatus`, and the inline status
change in `UpdateServer`), in the same transaction that already holds
the per-server advisory lock.
- Winner selection reuses the existing `CompareVersions` logic: highest
non-deleted version gets the flag; if every version is deleted, the
highest deleted version keeps it so admin lookups via
`includeDeleted=true` still resolve the server (important for restore
flows).
- New `SetLatestVersion` DB primitive issues two statements (clear old,
set new) — `idx_unique_latest_per_server` is a non-deferrable unique
partial index, so a single `UPDATE` that flips two rows trips the
constraint.

### 2. One-shot migration — heal servers already affected (no manual SQL
needed)

`014_heal_is_latest.sql` promotes the highest non-deleted version on
every server whose `is_latest` flag is on a deleted row (or absent
entirely). Picker parses `major.minor.patch` numerically and falls back
to `published_at` for non-semver or as a tiebreaker. Heals
`io.github.containers/kubernetes-mcp-server` to `0.0.61` automatically
and catches any other servers silently affected by the original bug,
without hardcoded version numbers.

Picker behavior vs. `CompareVersions`:
- Standard `M.m.p` semver: matches exactly, including the
backport-after-delete case (e.g. published 1.0.1 then 1.0.0 hotfix after
deleting 2.0.0 → picks 1.0.1, not 1.0.0).
- Non-semver: falls through to `published_at DESC`. Reasonable.
- Prereleases (`1.0.0-alpha` vs `1.0.0`): both parse to `(1,0,0)`,
tiebreak by `published_at`. Diverges from `CompareVersions` (which
always says `1.0.0 > 1.0.0-alpha`). Edge case requires (a) server hit by
this bug AND (b) prereleases at the top of the version list. Acceptable;
the publisher can flip it with one `mcp-publisher status` call (which
now triggers proper recalc thanks to part 1).

## Verification

End-to-end curl repro of the exact scenario from the issue, run against
both `main` and this branch:

**Before (main):**
```
publish 1.0.0 → 200
publish 0.0.59 → 200
PATCH .../1.0.0/status {deleted} → 200
GET /versions/latest → 404            ← bug
publish 0.0.60 → 200
GET /versions/latest → 404            ← bug persists, 0.0.60 has isLatest=False
```

**After (this branch):**
```
publish 1.0.0 → 200
publish 0.0.59 → 200
PATCH .../1.0.0/status {deleted} → 200
GET /versions/latest → 200 (0.0.59, isLatest=true)
publish 0.0.60 → 200
GET /versions/latest → 200 (0.0.60, isLatest=true)
```

## Test plan

- [x] Service tests covering: soft-delete latest → next highest active
promoted; the exact #1081 repro end-to-end through the service;
all-versions-deleted still keeps highest addressable for admin;
restore-to-higher-version reclaims latest.
- [x] Migration tests covering: kubernetes-mcp-server scenario (heals to
highest semver); backport scenario (semver beats most-recent-published);
no-is_latest-row defensive case; all-deleted server left untouched;
healthy server left untouched; non-semver versions fall back to
`published_at`.
- [x] Full unit test suite (`./internal/... ./cmd/...`) passes against a
real Postgres.
- [x] `make lint` (0 issues) and `make validate` pass.
- [x] Manual curl repro against a running registry confirms the fix
resolves the failure mode and `main` still reproduces it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 11:59:28 -04:00
Preeti Dewani
bb84ea69e8
Publisher-managed "deleted" support (#893)
(Updated auto-generated PR description from @tadasant / Claude on Feb
22, 2026)

## Motivation and Context

Fixes #637

Server lifecycle management (deprecation and yanking) is a standard
requirement for package registries. Use cases like rebranding, security
vulnerabilities, and end-of-life servers demand the ability to mark
servers as deprecated or deleted, with optional messaging to guide
consumers toward alternatives.

## Changes

### New API endpoints

- **`PATCH /v0/servers/{serverName}/versions/{version}/status`** —
Update the lifecycle status of a single server version
- **`PATCH /v0/servers/{serverName}/status`** — Update the lifecycle
status of all versions of a server in a single transaction

Both endpoints accept:
```json
{
  "status": "active|deprecated|deleted",
  "statusMessage": "Optional message (max 500 chars) explaining the change"
}
```

### New CLI command

- **`mcp-publisher status <serverName> [version]`** — Update server
lifecycle status from the command line
  - `--status` (required): `active`, `deprecated`, or `deleted`
  - `--message`: Optional status message
- `--all-versions`: Apply to all versions of the server (replaces the
`<version>` argument)

### API response changes

`RegistryExtensions` now includes two new fields:
- `statusChangedAt` — Timestamp of the last status change
- `statusMessage` — Optional free-form message (e.g., deprecation
reason, migration guidance)

### List endpoint changes

- New `include_deleted` query parameter on `GET /v0/servers` — defaults
to `false`, hiding deleted servers from listings
- When `updated_since` is provided, `include_deleted` is automatically
set to `true` for incremental sync

### Database

- Migration `013_add_status_fields.sql`: Adds `status_changed_at` (NOT
NULL, initialized from `published_at`) and `status_message` columns to
the `servers` table

### Design decisions (from review discussion)

- **Dedicated status endpoints** rather than overloading the existing
edit endpoint
- **Bidirectional status transitions** — all transitions between
`active`, `deprecated`, and `deleted` are allowed
- **`statusMessage` as free-form text** — no structured `nextName` or
`alternativeUrl` fields (per maintainer feedback, to avoid complexity
around chaining, trust transfer, etc.)
- **Publish permission** grants status change access (not limited to
admin/edit)
- **Status message auto-cleared** when transitioning back to `active`
Good, now I have the full picture. Here's the addendum:

### Consumer-facing API changes

The new status fields live inside
`_meta["io.modelcontextprotocol.registry/official"]` on every
`ServerResponse`. This is the existing `RegistryExtensions` object — two
new fields are added alongside the existing ones:

**Before:**
```json
{
  "server": { "name": "com.example/my-server", "...": "..." },
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "status": "active",
      "publishedAt": "2025-06-01T00:00:00Z",
      "updatedAt": "2025-06-15T00:00:00Z",
      "isLatest": true
    }
  }
}
```

**After:**
```json
{
  "server": { "name": "com.example/my-server", "...": "..." },
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "status": "deprecated",
      "statusChangedAt": "2026-02-01T12:00:00Z",
      "statusMessage": "Deprecated in favor of com.example/my-server-v2",
      "publishedAt": "2025-06-01T00:00:00Z",
      "updatedAt": "2026-02-01T12:00:00Z",
      "isLatest": true
    }
  }
}
```

| New field | Type | Present when |
|-----------|------|-------------|
| `statusChangedAt` | `string` (RFC3339) | Always — initialized to
`publishedAt` for existing records |
| `statusMessage` | `string` (max 500 chars) | Only when set —
`null`/omitted for active servers (auto-cleared on transition to active)
|

These fields appear on **every endpoint that returns a
`ServerResponse`**: `GET /v0/servers`, `GET /v0/servers/{name}`, `GET
/v0/servers/{name}/versions/{version}`, and the new PATCH status
endpoints.

### List endpoint filtering

`GET /v0/servers` gains an `include_deleted` query parameter:

| Scenario | Behavior |
|----------|----------|
| No `include_deleted` param | Deleted servers hidden (default `false`)
|
| `include_deleted=true` | Deleted servers included in results |
| `updated_since` provided | `include_deleted` forced to `true` for
incremental sync |

## How Has This Been Tested?

- Unit tests for status handler (`status_test.go`) covering status
transitions, permission checks, validation errors, and the all-versions
endpoint
- Unit tests for the CLI command (`status_test.go`)
- Integration with existing server/edit handler tests
- Database layer tests for the new status update and query methods
- Tested locally with `make dev-compose`

## Types of changes

- [x] New feature (non-breaking change which adds functionality)

## Checklist

- [x] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [x] My code follows the repository's style guidelines
- [x] New and existing tests pass locally
- [x] I have added appropriate error handling
- [x] I have added or updated documentation as needed

## Open items from review

- [ ] `include_deleted` silent override: When `updated_since` is
provided and `include_deleted=false` is explicitly passed, should return
a 400 error rather than silently overriding
([thread](https://github.com/modelcontextprotocol/registry/pull/893#discussion_r2100879360))
— agreed upon but may not yet be implemented
2026-02-27 09:00:07 -08:00
Joel Verhagen
80cb03ff7a
Fix NuGet.org base URL default, improve API integration (#813)
<!-- Provide a brief summary of your changes -->

## Motivation and Context
<!-- Why is this change needed? What problem does it solve? -->

This resolves
https://github.com/modelcontextprotocol/registry/issues/738. There are
two issues with the NuGet.org integration today:

1. Change the `registryBaseUrl` for NuGet to
`https://api.nuget.org/v3/index.json`. `https://api.nuget.org` is not
the entry point for our service and is not specific enough.
`https://api.nuget.org/v3/index.json` is the value that users put into
their tooling to connect to our public registry (it is installed by
default in .NET client tooling).
2. Use the service index API to get the URL pattern for package READMEs.
- This URL is not guaranteed to be stable. The service index URL
(`registryBaseUrl`) is, however.

Additionally, I polished up the error messages so various error modes
are clearer. When packages are published to NuGet.org, they take 2-5
minutes to become available due to validations we run.

Some possible states that are now handled and described more helpfully
are:
- Version exists, but the package has no README
- The ID exists, but the version does not
- The ID does not exist at all

There are only 3 authors (me, another MSFT person, and a community
member) explicitly setting the old value. The DB migration will fix the
issue. A future publish will fail with the validation error but the
error message is clear.

`registry type and base URL do not match: 'https://api.nuget.org' is not
valid for registry type 'nuget'. Expected:
https://api.nuget.org/v3/index.json`

## How Has This Been Tested?
<!-- Have you tested this in a real application? Which scenarios were
tested? -->

- Unit tests, new and existing
- Manual test locally
- Set old data in the DB, `docker compose up`, confirm the migration
updates the specific package

## Breaking Changes

Yes, but minimal impact expected, unless there is a dependency on the
specific `https://api.nuget.org` value downstream. I am not aware of any
such dependency, and the new value is more proper and specific.

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [x] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [x] My code follows the repository's style guidelines
- [x] New and existing tests pass locally
- [x] I have added appropriate error handling
- [x] I have added or updated documentation as needed

## Additional context
<!-- Add any other context, implementation notes, or design decisions
-->

For more information about the NuGet API see:
https://learn.microsoft.com/en-us/nuget/api/overview#service-index (I
wrote most of the docs).
2025-12-09 07:19:38 -08:00
Tadas Antanavicius
8f8fc89801
Fix missing $schema entries - data migration (#806)
Drafted by Claude Code, reviewed by me.

## Summary

This PR adds a data migration to fix 17 server entries that were
published with empty `$schema` fields.

**This is a pure data migration** - no code changes are needed because:

- Validation for the `$schema` field was already added in commit
[6d2001b](6d2001b)
on **2025-10-07**
- That validation was deployed to production on **2025-10-14** with
v1.3.0 ([PR
#668](https://github.com/modelcontextprotocol/registry/pull/668))
- All 17 affected entries were published before deployment (see dates
below)

## Affected Entries (17 total)

| Server | Version | Published |
|--------|---------|-----------|
| io.github.ruvnet/claude-flow | 2.0.0-alpha.104 | 2025-09-10 |
| io.github.ruvnet/claude-flow | 2.0.0-alpha.105 | 2025-09-10 |
| io.github.ruvnet/ruv-swarm | 1.0.18 | 2025-09-10 |
| io.github.ruvnet/ruv-swarm | 1.0.19 | 2025-09-10 |
| travel.kismet/mcp-server | 0.0.0 | 2025-09-10 |
| io.github.jkakar/cookwith-mcp | 1.0.0 | 2025-09-11 |
| io.github.jkakar/cookwith-mcp | 1.0.1 | 2025-09-12 |
| io.github.jkakar/cookwith-mcp | 1.0.2 | 2025-09-12 |
| io.github.OtherVibes/mcp-as-a-judge | 0.3.12 | 2025-09-18 |
| io.github.OtherVibes/mcp-as-a-judge | 0.3.13 | 2025-09-18 |
| io.github.OtherVibes/mcp-as-a-judge | 0.3.14 | 2025-09-18 |
| io.github.OtherVibes/mcp-as-a-judge | 0.3.20 | 2025-09-20 |
| io.github.Skills03/scrimba-teaching | 1.0.1 | 2025-09-21 |
| io.github.Skills03/scrimba-teaching | 1.1.0 | 2025-09-21 |
| io.github.Skills03/scrimba-teaching | 1.2.0 | 2025-09-21 |
| io.github.toby/mirror-mcp | 0.0.4 | 2025-10-01 |
| io.github.antuelle78/weather-mcp | 1.0.0 | 2025-10-09 ⚠️ |

**Note on outlier:** `io.github.antuelle78/weather-mcp` was published on
2025-10-09, which is after the validation code was committed
(2025-10-07) but before it was deployed (2025-10-14). This confirms the
entries slipped through due to the gap between merge and deployment.

## Dry Run SQL

To verify affected rows before running the migration:

```sql
SELECT server_name, version, value->>'$schema' as schema_value
FROM servers
WHERE (server_name, version) IN (
    ('io.github.OtherVibes/mcp-as-a-judge', '0.3.12'),
    ('io.github.OtherVibes/mcp-as-a-judge', '0.3.13'),
    ('io.github.OtherVibes/mcp-as-a-judge', '0.3.14'),
    ('io.github.OtherVibes/mcp-as-a-judge', '0.3.20'),
    ('io.github.Skills03/scrimba-teaching', '1.0.1'),
    ('io.github.Skills03/scrimba-teaching', '1.1.0'),
    ('io.github.Skills03/scrimba-teaching', '1.2.0'),
    ('io.github.antuelle78/weather-mcp', '1.0.0'),
    ('io.github.jkakar/cookwith-mcp', '1.0.0'),
    ('io.github.jkakar/cookwith-mcp', '1.0.1'),
    ('io.github.jkakar/cookwith-mcp', '1.0.2'),
    ('io.github.ruvnet/claude-flow', '2.0.0-alpha.104'),
    ('io.github.ruvnet/claude-flow', '2.0.0-alpha.105'),
    ('io.github.ruvnet/ruv-swarm', '1.0.18'),
    ('io.github.ruvnet/ruv-swarm', '1.0.19'),
    ('io.github.toby/mirror-mcp', '0.0.4'),
    ('travel.kismet/mcp-server', '0.0.0')
)
AND (value->>'$schema' = '' OR value->>'$schema' IS NULL);
```

Result against prod:

```sql
app=> SELECT server_name, version, value->>'$schema' as schema_value
FROM servers
WHERE (server_name, version) IN (
    ('io.github.OtherVibes/mcp-as-a-judge', '0.3.12'),
    ('io.github.OtherVibes/mcp-as-a-judge', '0.3.13'),
    ('io.github.OtherVibes/mcp-as-a-judge', '0.3.14'),
    ('io.github.OtherVibes/mcp-as-a-judge', '0.3.20'),
    ('io.github.Skills03/scrimba-teaching', '1.0.1'),
    ('io.github.Skills03/scrimba-teaching', '1.1.0'),
    ('io.github.Skills03/scrimba-teaching', '1.2.0'),
    ('io.github.antuelle78/weather-mcp', '1.0.0'),
    ('io.github.jkakar/cookwith-mcp', '1.0.0'),
    ('io.github.jkakar/cookwith-mcp', '1.0.1'),
    ('io.github.jkakar/cookwith-mcp', '1.0.2'),
    ('io.github.ruvnet/claude-flow', '2.0.0-alpha.104'),
    ('io.github.ruvnet/claude-flow', '2.0.0-alpha.105'),
    ('io.github.ruvnet/ruv-swarm', '1.0.18'),
    ('io.github.ruvnet/ruv-swarm', '1.0.19'),
    ('io.github.toby/mirror-mcp', '0.0.4'),
    ('travel.kismet/mcp-server', '0.0.0')
)
AND (value->>'$schema' = '' OR value->>'$schema' IS NULL);
             server_name             |     version     | schema_value 
-------------------------------------+-----------------+--------------
 io.github.toby/mirror-mcp           | 0.0.4           | 
 io.github.jkakar/cookwith-mcp       | 1.0.0           | 
 io.github.ruvnet/ruv-swarm          | 1.0.19          | 
 io.github.jkakar/cookwith-mcp       | 1.0.1           | 
 io.github.ruvnet/ruv-swarm          | 1.0.18          | 
 io.github.antuelle78/weather-mcp    | 1.0.0           | 
 io.github.jkakar/cookwith-mcp       | 1.0.2           | 
 io.github.OtherVibes/mcp-as-a-judge | 0.3.12          | 
 travel.kismet/mcp-server            | 0.0.0           | 
 io.github.Skills03/scrimba-teaching | 1.0.1           | 
 io.github.OtherVibes/mcp-as-a-judge | 0.3.20          | 
 io.github.Skills03/scrimba-teaching | 1.2.0           | 
 io.github.ruvnet/claude-flow        | 2.0.0-alpha.105 | 
 io.github.ruvnet/claude-flow        | 2.0.0-alpha.104 | 
 io.github.Skills03/scrimba-teaching | 1.1.0           | 
 io.github.OtherVibes/mcp-as-a-judge | 0.3.14          | 
 io.github.OtherVibes/mcp-as-a-judge | 0.3.13          | 
```

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-30 08:41:02 -08:00
Radoslav Dimitrov
d9099e9117
Fix migration numbering and add migration 010 for package format (#675)
<!-- Provide a brief summary of your changes -->

## Motivation and Context
<!-- Why is this change needed? What problem does it solve? -->
The following PR fixes an issue we introduced on September, 30th that
caused some of the migrations to not apply. This PR fixes it. After
merging we'll have to reset staging as this would not work there (this
is expected).

## How Has This Been Tested?
<!-- Have you tested this in a real application? Which scenarios were
tested? -->

## Breaking Changes
<!-- Will users need to update their code or configurations? -->

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [ ] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [ ] My code follows the repository's style guidelines
- [ ] New and existing tests pass locally
- [ ] I have added appropriate error handling
- [ ] I have added or updated documentation as needed

## Additional context
<!-- Add any other context, implementation notes, or design decisions
-->

Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
2025-10-15 17:00:39 +03:00
Radoslav Dimitrov
632cbce815
Refactor package types to use dedicated schema types per package type (NPM, PyPI, NuGet, OCI, MCPB) (#634) 2025-10-11 10:53:45 +03:00
Tadas Antanavicius
e968cfdbca
fix: renumber migrations to fix staging deployment (#592)
## Summary

Fixes staging deployment by renumbering migrations to match what staging
has already applied.

## Problem

Staging already applied migration 008 (metadata separation) before we
inserted the new cleanup migration. When we added the cleanup as 008 and
renumbered metadata to 009, staging tried to apply 009 again and failed
with "column already exists".

## Current State

- **Staging**: Has run original 008 (metadata separation), marked as
complete. Failed trying to run 009 (which is the same migration
renumbered).
- **Production**: Has not run either 008 or 009 yet.

## Solution

- Rename `008_clean_invalid_data.sql` → `007a_clean_invalid_data.sql`
- Rename `009_separate_official_metadata.sql` →
`008_separate_official_metadata.sql` (back to original)

## How This Fixes It

With this renumbering:

1. **Staging behavior**:
- Sees 008 is already complete (the metadata migration it already ran)
- Sees 007a as new and runs it, but it will be a no-op (skips because
staging doesn't have the problematic production data)
   - Continues normally

2. **Production behavior**:
   - Runs 007a (cleanup migration) - removes invalid data
- Runs 008 (metadata separation) - applies successfully because data is
now clean
   - Both migrations run in the correct order as intended

## Testing

- Verified migration order works correctly
- Tested on empty database (test environment) - 007a skips as expected
- Tested with production data snapshot - 007a cleans data, 008 applies
successfully

This is a critical fix to unblock staging deployment.
2025-09-30 20:39:33 +03:00
Tadas Antanavicius
272ee9322d
fix: add server name validation and clean invalid data before migration 009 (#591)
## Summary

This PR adds two important improvements to ensure data integrity:

1. **Adds server name format validation at the API layer** to match the
database constraint from migration 008
2. **Includes migration 008 to clean up invalid data** before applying
strict constraints in migration 009

## Changes

### Server Name Validation
- Enforces validation at the API layer matching the database constraint
- **Namespace pattern**: `[a-zA-Z0-9][a-zA-Z0-9.-]*[a-zA-Z0-9]` 
  - Must start and end with alphanumeric characters
  - Can contain dots and hyphens in the middle
- **Name pattern**: `[a-zA-Z0-9][a-zA-Z0-9._-]*[a-zA-Z0-9]`
  - Must start and end with alphanumeric characters  
  - Can contain dots, underscores, and hyphens in the middle
- Provides clear, user-friendly error messages indicating which part is
invalid
- Uses DRY approach with composed regex patterns

### Migration 008: Data Cleanup
- Removes servers with invalid names or empty versions (5 servers
affected in production)
- Fixes invalid status values by setting them to 'active' (1 server
affected)
- Removes duplicate name+version combinations
- Includes comprehensive safety checks to prevent accidental data loss
- Must run before migration 009 which adds strict constraints

## Testing
- All validator tests pass
- Migration includes safety checks that match production data
- CI verification included

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-30 20:07:42 +03:00
Tadas Antanavicius
2d49e5b68c
fix: prevent NULL status values during migration 008 (#587)
## Summary
- Fixes migration 008 failing with "column contains null values" error
- Adds COALESCE protection to handle 126 servers without status fields
- Includes test utilities to reproduce and verify fixes with production
data

## Problem

Migration 008 was failing in production because:
- 126 out of 781 servers have no `status` field in their JSON
- When extracting `rec.value->>'status'`, these became NULL values
- The `ALTER TABLE servers ALTER COLUMN status SET NOT NULL` constraint
then failed

## Solution

1. **Prevent NULLs during migration**: Use COALESCE to provide 'active'
default when status is NULL/empty
2. **Enhanced debugging**: Added RAISE NOTICE statements to log
problematic values
3. **Safety check**: Double-check that throws exception if any NULLs
remain
4. **Test utilities**: Created mirror_data tools to test migrations with
real production data

## Testing

Created and used `scripts/mirror_data/` tools to:
- Fetch all 781 servers from production API
- Load them into a test database
- Verify the migration fix works correctly

### Results:
- **Before fix**: 126 NULL status values created during migration
- **After fix**: 0 NULL status values, all servers have valid status
- Migration completes successfully with NOT NULL constraint

## Test plan
- [x] Tested locally with production data using mirror_data tools
- [x] Verified 0 NULL values after migration (vs 126 before)
- [x] Confirmed NOT NULL constraint adds successfully
- [ ] Deploy to staging and verify migration succeeds
- [ ] Monitor production deployment

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-30 06:43:26 -07:00
Radoslav Dimitrov
e39b008b41
Ensure we populate sane defaults for NULL values (#583)
<!-- Provide a brief summary of your changes -->

## Motivation and Context
<!-- Why is this change needed? What problem does it solve? -->
The following PR tries to ensure sane defaults for entries that for one
reason or another have NULL values for some of their properties.

## How Has This Been Tested?
<!-- Have you tested this in a real application? Which scenarios were
tested? -->

## Breaking Changes
<!-- Will users need to update their code or configurations? -->

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [ ] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [ ] My code follows the repository's style guidelines
- [ ] New and existing tests pass locally
- [ ] I have added appropriate error handling
- [ ] I have added or updated documentation as needed

## Additional context
<!-- Add any other context, implementation notes, or design decisions
-->

---------

Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
2025-09-30 15:47:18 +03:00
Tadas Antanavicius
3dc4f93370
fix: handle NULL status values in migration 008 (#578)
## Problem

Production deployment is failing with CrashLoopBackOff. Migration 008
fails with:
```
ERROR: column "status" of relation "servers" contains null values (SQLSTATE 23502)
```

## Root Cause

The migration extracts the `status` field from an unexpected location in
the data structure.

### Actual Data Structure (Before Migration)

In the current format, `status` is at the **top level** of the JSON, not
nested in official metadata:

```json
{
  "name": "com.example/server",
  "version": "1.0.0",
  "status": "active",           ← STATUS IS HERE (top level)
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "uuid",
      "publishedAt": "2025-09-01T00:00:00Z",
      "isLatest": true
      // NO STATUS FIELD HERE
    }
  }
}
```

For 120 out of 765 servers, the status field is **missing entirely**:
```json
{
  "name": "com.biodnd/agent-ip",
  "version": "0.1.2",
  // ← NO STATUS FIELD AT ALL
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "uuid",
      "publishedAt": "2025-09-23T09:47:07Z",
      "isLatest": true
      // STILL NO STATUS FIELD HERE
    }
  }
}
```

### Original Migration Logic

```sql
IF official_meta IS NOT NULL THEN
    UPDATE servers
    SET
        status = (official_meta->>'status'),  -- Extracts NULL for all servers
```

**Result:** 
- All 765 servers would have NULL status extracted
- Because `official_meta->>'status'` returns NULL (field doesn't exist
there)
- The migration tries to add `NOT NULL` constraint → fails

### Fixed Migration Logic

```sql
IF official_meta IS NOT NULL THEN
    UPDATE servers
    SET
        status = COALESCE(NULLIF(rec.value->>'status', 'null'), 'active'),
```

**Result:**
- Extracts from top-level JSON where status actually exists
- `NULLIF(rec.value->>'status', 'null')` handles edge case of literal
string "null"
- `COALESCE(..., 'active')` provides fallback when status is missing
- All servers get valid status value

### Impact Breakdown

**645 servers with `"status": "active"`:**
- Before fix: Would extract NULL → migration fails
- After fix: Extracts "active" correctly

**120 servers with missing status field:**
- Before fix: Would extract NULL → migration fails  
- After fix: Defaults to 'active' via COALESCE

## Example Servers with Missing Status

(First 9 discovered during investigation)
- com.biodnd/agent-ip
- io.github.jkakar/cookwith-mcp
- com.joelverhagen.mcp/Knapcode.SampleMcpServer
- io.github.Lyellr88/marm-mcp-server
- com.biodnd/agent-press
- io.github.timheuer/sampledotnetmcpserver
- io.github.CodeCraftersLLC/local-voice-mcp
- com.biodnd/agent-fin
- io.github.ruvnet/ruv-swarm

## Testing

Verified against production data (765 total servers):
- ✅ All servers have official metadata
- ✅ None have status in official metadata (expected)
- ✅ 645 have "status": "active" at top level
- ✅ 120 have missing status field (no field present)
- ✅ Fix extracts from correct location for all cases

## Impact

This unblocks production deployment. Once merged and released as v1.2.1,
the migration will:
1. Extract status from the correct location (top-level JSON)
2. Handle missing status fields with sensible defaults
3. Successfully add NOT NULL constraint
4. Allow pods to start normally

---
🤖 Generated with [Claude Code](https://claude.com/claude-code)
2025-09-30 10:21:48 +03:00
Radoslav Dimitrov
b21c6f564d
Changes on the server.json format and the API (#562) 2025-09-29 23:30:35 +03:00
adam jones
60c801c0c2
Fix race conditions in concurrent server publishing (#528)
## Summary

Fixes race conditions in concurrent server publishing that could result
in data integrity violations:
- Multiple versions marked as `isLatest=true` for the same server
- Duplicate (serverId, version) combinations
- Same server name assigned different serverIDs

## Solution

Uses two complementary approaches for correctness:

### 1. PostgreSQL Advisory Locks
- Serializes concurrent publishes per server name using
`pg_advisory_xact_lock`
- Uses FNV-1a hash to convert server name to lock ID
- Transaction-scoped locks auto-release on commit/rollback
- **Required after #465**: The serverId introduction means concurrent
publishes must atomically coordinate whether to reuse an existing
serverId (for additional versions) or generate a new one (for first
version). Without locks, race conditions would assign different
serverIDs to versions of the same server.

### 2. Unique Constraints (Migration 007)
Two new indexes provide data integrity guarantees:
- **idx_unique_server_version**: Ensures (serverId, version) is unique
across all versions
- **idx_unique_latest_version**: Partial index ensuring only one version
per serverId has `isLatest=true`

These constraints serve as defense-in-depth safety nets. Even if lock
logic had a bug, the database would reject integrity violations.

Also drops redundant `idx_servers_version_id` since `version_id` is
already the primary key.

## Why WithPublishLock validation is needed after #465

The `ctx.Err()` check at the start of `WithPublishLock` is critical
because:
1. After #465 introduced serverId, the publish flow must atomically
check existing versions and assign/reuse serverId
2. If the context is cancelled during lock acquisition, we could
deadlock
3. Checking ctx validity before acquiring locks prevents these deadlocks

## Testing

Added `TestPublishConcurrentVersionsNoRace` with 100 concurrent
goroutines publishing different versions of the same server. All succeed
with:
- Exactly one version marked as latest
- All versions sharing the same serverId
- No duplicate (serverId, version) combinations

NB: this only covers the in memory db, not the postgres db. It'd be nice
to write tests for this one as well, but our current setup is a bit
annoying for this. Separately looking into ways to easily run all our
tests with real postgres and ideally scrap memory.go completely.
2025-09-22 12:16:05 +01:00
Preeti Dewani
53f808a2c1
add migraiton file to convert snake case to camel case (#522)
## Motivation and Context
- Make all entries in current database matching the new schema

## How Has This Been Tested?
- locally

## Breaking Changes
- Yes

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [X] Breaking change (fix or feature that would cause existing
functionality to change)
- [X] Documentation update

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [ ] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [X] My code follows the repository's style guidelines
- [X] New and existing tests pass locally
- [ ] I have added appropriate error handling
- [X] I have added or updated documentation as needed

Before migration:

```
{
      "name": "io.github.kenjihikmatullah/productboard-mcp",
      "description": "Integrate the Productboard API into agentic workflows via MCP",
      "status": "active",
      "repository": {
        "url": "https://github.com/kenjihikmatullah/productboard-mcp",
        "source": "github",
        "id": "939104710"
      },
      "version": "0.0.1-seed",
      "packages": [
        {
          "registry_type": "npm",
          "identifier": "productboard-mcp",
          "version": "1.0.1",
          "transport": {
            "type": "stdio"
          }
        }
      ],
      "_meta": {
        "io.modelcontextprotocol.registry/official": {
          "id": "00613acb-73e2-4f93-8b96-296df17316c8",
          "published_at": "0001-01-01T00:00:00Z",
          "updated_at": "0001-01-01T00:00:00Z",
          "is_latest": true
        }
      }
    }
    
```

After migration:

```
 {
      "name": "io.github.kenjihikmatullah/productboard-mcp",
      "description": "Integrate the Productboard API into agentic workflows via MCP",
      "status": "active",
      "repository": {
        "url": "https://github.com/kenjihikmatullah/productboard-mcp",
        "source": "github",
        "id": "939104710"
      },
      "version": "0.0.1-seed",
      "packages": [
        {
          "registryType": "npm",
          "identifier": "productboard-mcp",
          "version": "1.0.1",
          "transport": {
            "type": "stdio"
          },
        }
      ],
      "_meta": {
        "io.modelcontextprotocol.registry/official": {
          "serverId": "83cfe55c-55c5-4e7e-b9c0-6d7d369f52ed",
          "versionId": "00613acb-73e2-4f93-8b96-296df17316c8",
          "publishedAt": "0001-01-01T00:00:00Z",
          "updatedAt": "0001-01-01T00:00:00Z",
          "isLatest": true
        }
      }
    }
2025-09-21 15:26:15 +03:00
Tadas Antanavicius
56e8b4879b
Introduce serverId, rename id to versionId (#465)
Wielded Claude Code to do this; across many iterations and have reviewed
all the code + largely rewritten the PR commentary here.

This PR implements the server ID consistency changes from
[#396](https://github.com/modelcontextprotocol/registry/issues/396). Our
goal is to:
- Rename `id` to `versionId` to make it clear that it is the ID for a
specific version, not a server
- Introduce `serverId` as a net new concept. This will be 1:1 with
`name`. While not strictly necessary (we could just use name as a PK),
the convention of having a UUID as an ID on our objects is just a
helpful subjective architecture decision. Plus also leaves open the door
to reconsidering the "should we allow renames" question.

## Key Changes

### 🗄️ Database Schema Changes
- **Renamed**: `id` column → `version_id` (unique identifier for each
server version)

### 📋 Metadata Structure Updates
- **JSON Schema**: Updated
`_meta.io.modelcontextprotocol.registry/official` to include:
  - `serverId`: Consistent ID across all versions of a server
  - `versionId`: Unique ID for this specific version (renamed from `id`)

### 🔧 API Endpoint Updates
- **New endpoint structure**: Changed from version-ID-based to
server-ID-based URLs:
  - `GET /v0/servers/{server_id}` - Get latest version of server
  - `GET /v0/servers/{server_id}?version=X.X.X` - Get specific version  
- `GET /v0/servers/{server_id}/versions` - List all versions of a server
  - `PUT /v0/servers/{server_id}?version=X.X.X` - Edit specific version
- **Consistent URL patterns**: All server endpoints now use `server_id`
as the primary identifier
 
### Database Migration Strategy
The migration (`005_add_server_id_rename_version_id.sql`):
1. Adds new `version_id` column  
2. Generates deterministic UUIDs for servers with the same name
3. Updates JSON metadata to add `serverId` and rename `id` to
`versionId`
4. Copies existing `id` values to `version_id` column
5. Drops old `id` column and makes `version_id` the new primary key

## Testing

Besides of course automated tests passing, I:
1. Set up my local db with the pre-005-migration state, imported all the
production data
2. Ran the migration
3. Spot checked some diffs of the data that resulted --

```json
// before
{
  "name": "io.github.ruvnet/ruv-swarm",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "id": "0a9cda7f-2b73-4839-844b-6582f335de42",
      "is_latest": false,
      "updated_at": "2025-09-10T17:21:34.92137012Z",
      "published_at": "2025-09-10T17:15:32.69690433Z"
    }
  },
  "version": "1.0.18",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "Neural network swarm orchestration with WebAssembly acceleration and MCP integration"
}

// after
{
  "name": "io.github.ruvnet/ruv-swarm",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "e8538952-96af-4a33-9e31-7fbed012521e",
      "is_latest": false,
      "versionId": "0a9cda7f-2b73-4839-844b-6582f335de42",
      "updated_at": "2025-09-10T17:21:34.92137012Z",
      "published_at": "2025-09-10T17:15:32.69690433Z"
    }
  },
  "version": "1.0.18",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "Neural network swarm orchestration with WebAssembly acceleration and MCP integration"
}

```

Servers that have multiple versions stayed consistent in their
`serverId`:

```json
// First version
{
  "name": "app.getdialer/dialer",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "80f08bdb-f3e4-4f19-b072-1676ae49b663",
      "is_latest": false,
      "versionId": "04f25b17-331d-48d7-b08b-37baece208a7",
      "updated_at": "2025-09-09T00:16:49.162620072Z",
      "published_at": "2025-09-08T23:47:09.452137971Z"
    }
  },
  "status": "active",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
  "remotes": [
    {
      "url": "https://getdialer.app/mcp",
      "type": "streamable-http"
    },
    {
      "url": "https://getdialer.app/sse",
      "type": "sse"
    }
  ],
  "version": "1.0.0",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "An MCP server that provides your you make outbound phone calls using your own phone number"
}

// Second version
{
  "name": "app.getdialer/dialer",
  "_meta": {
    "io.modelcontextprotocol.registry/official": {
      "serverId": "80f08bdb-f3e4-4f19-b072-1676ae49b663",
      "is_latest": true,
      "versionId": "faa26cb9-3aad-415e-aece-f715fff5fe5e",
      "updated_at": "2025-09-09T00:16:49.107842808Z",
      "published_at": "2025-09-09T00:16:49.107842808Z"
    }
  },
  "status": "active",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
  "remotes": [
    {
      "url": "https://getdialer.app/mcp",
      "type": "streamable-http"
    },
    {
      "url": "https://getdialer.app/sse",
      "type": "sse"
    }
  ],
  "version": "1.0.1",
  "repository": {
    "url": "",
    "source": ""
  },
  "description": "An MCP server that provides lets you make outbound phone calls using your own phone number"
}
```

And I also tried publishing a test server anew; looked good:

```bash
⏺ Bash(curl -s "http://localhost:8080/v0/servers/102dc4e4-6ca8-40e6-aa11-cd45763d3352" | jq '.')
  ⎿  {                                                                                             
       "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
       "name": "io.modelcontextprotocol.anonymous/test-claude-migration-server",
       "description": "A test server to verify the new migration and API endpoints work correctly",
       "status": "active",
       "repository": {
         "url": "https://github.com/example/test-claude-migration-server",
         "source": "github",
         "id": "example/test-claude-migration-server"
       },
       "version": "1.0.0",
       "_meta": {
         "io.modelcontextprotocol.registry/official": {
           "serverId": "102dc4e4-6ca8-40e6-aa11-cd45763d3352",
           "versionId": "48c9cd55-7bc4-4ac2-896e-59e81eb653e4",
           "published_at": "2025-09-12T18:33:23.264203-07:00",
           "updated_at": "2025-09-12T18:33:23.264203-07:00",
           "is_latest": true
         }
       }
     }

⏺ Bash(curl -s "http://localhost:8080/v0/servers/102dc4e4-6ca8-40e6-aa11-cd45763d3352/versions" | jq '.')
  ⎿  {                                                                                                 
       "servers": [
         {
           "$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
           "name": "io.modelcontextprotocol.anonymous/test-claude-migration-server",
           "description": "A test server to verify the new migration and API endpoints work correctly",
           "status": "active",
           "repository": {
             "url": "https://github.com/example/test-claude-migration-server",
             "source": "github",
             "id": "example/test-claude-migration-server"
           },
           "version": "1.0.0",
           "_meta": {
             "io.modelcontextprotocol.registry/official": {
               "serverId": "102dc4e4-6ca8-40e6-aa11-cd45763d3352",
               "versionId": "48c9cd55-7bc4-4ac2-896e-59e81eb653e4",
               "published_at": "2025-09-12T18:33:23.264203-07:00",
               "updated_at": "2025-09-12T18:33:23.264203-07:00",
               "is_latest": true
             }
           }
         }
       ],
       "metadata": {
         "count": 1
       }
     }
```

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Preeti Dewani <preetidewani1990@gmail.com>
Co-authored-by: adam jones <adamj+git@anthropic.com>
Co-authored-by: Adam Jones <adamj@anthropic.com>
2025-09-21 15:00:09 +03:00
adam jones
ef4607720d
Fix _meta field format for MCP spec compliance (#367)
## Summary

Updates the `_meta` field structure to be compliant with the MCP
specification by using proper `prefix/name` format for vendor
extensions.

### Changes

- **Registry metadata**: `io.modelcontextprotocol.registry` →
`io.modelcontextprotocol.registry/official`
- **Publisher metadata**: `publisher` →
`io.modelcontextprotocol.registry/publisher-provided`
- **Database migration**: Updates indexes to use new field paths
- **Updated all code**: Service layer, validators, tests, and seed data

### Format Before
```json
{
  "_meta": {
    "publisher": { "license": "MIT" },
    "io.modelcontextprotocol.registry": { "id": "...", "is_latest": true }
  }
}
```

### Format After
```json
{
  "_meta": {
    "io.modelcontextprotocol.registry/publisher-provided": { "license": "MIT" },
    "io.modelcontextprotocol.registry/official": { "id": "...", "is_latest": true }
  }
}
```

This resolves the naming collision with the planned top-level
`publisher` field and ensures compliance with MCP specification
requirements.

Fixes #356
2025-09-08 11:40:04 -07:00
adam jones
218b0b422d
Implement flattened _meta format for server extensions (#331)
## Summary

This PR implements the flattened `_meta` format for server extensions as
discussed in #284, replacing the confusing wrapper format with a clean
single-file approach.

**Before (wrapper format):**
```json
{
  "server": { "name": "my-server", "description": "..." },
  "x-publisher": { "build_info": "..." },
  "x-io.modelcontextprotocol.registry": { "id": "...", "published_at": "..." }
}
```

**After (flattened format):**
```json
{
  "name": "my-server",
  "description": "...",
  "_meta": {
    "publisher": { "build_info": "..." },
    "io.modelcontextprotocol.registry": { "id": "...", "published_at": "..." }
  }
}
```

## Key Changes

- **Flattened Format**: Single `ServerJSON` file with structured `_meta`
field
- **Better DX**: Publishers maintain one file instead of separate
server.json + extensions.json
- **MCP Compliance**: Uses standard `_meta` mechanism instead of `x-*`
prefixes
- **Simplified Database**: PostgreSQL now uses simple `id, value` table
structure
- **Type Safety**: Structured `ServerMeta` with known extension fields

## Database Migration

Added migration `003_simplify_to_key_value.sql` that drops the complex
relational structure and creates a simple key-value table. This
significantly simplifies database management.

## Testing

- All core tests passing (validators, service layer)
- Schema validation working for all 10 examples
- Seed data successfully migrated (396 entries)
- Publishing functionality verified

Fixes #284
2025-09-01 19:27:23 +01:00
Radoslav Dimitrov
8d7e471fd3
Allow for vendor extensions on servers (#298)
<!-- Provide a brief summary of your changes -->
This PR serves as a placeholder and brings together all of the changes
related to supporting vendor extensions passed through the API. The idea
is to have a single place to track and review the work as it comes
together (even if it’s still a bit of a work in progress).

**The changes in this PR cover:**
* Updates on the data model
* Updates on both databases (in-memory and postgresql), added a new
table to PSQL for tracking the extensions leaving the server table only
for the clean server.json properties
* Updates the seed.json file (thus the big line change)
* Adds a seed-migrate tool to help migrating an existing seed file to
the new format
* Updates the seed import mechanism to support this
* Updates the existing tests - unit, integration and validation tests
* Adds the publish handler to the OpenAPI spec (it was missing)
* Fixes linting errors
* Updates on the API side on (publish, list, get) to support the new
format of `ServerResponse` -
```
{
  "server": {},
  "x-io.modelcontextprotocol.registry": {}, /*not available for publish since this gets populated by the registry*/
  "x-publisher": {}
}
```

## Motivation and Context
<!-- Why is this change needed? What problem does it solve? -->
Motivated by the discussions in:
https://github.com/modelcontextprotocol/registry/discussions/284

## How Has This Been Tested?
<!-- Have you tested this in a real application? Which scenarios were
tested? -->
Locally and by running all of the tests

## Breaking Changes
<!-- Will users need to update their code or configurations? -->
Yes, it changes both the API and the format of the seed file
 
## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

---

Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
Co-authored-by: adam jones <domdomegg+git@gmail.com>
2025-08-23 09:59:03 +01:00
adam jones
ebaf4c9c99
Refactor database implementation to use PostgreSQL instead of MongoDB (#289)
Fixes #228

## Motivation and Context

See #19

## How Has This Been Tested?

- Run locally
- Run in docker compose
- Run infra in minikube
- Unit and integration tests all passing :)

## Breaking Changes

Yes: users will not be able to access any data they used to have in
custom mongodb. We are accepting this breaking change, given the
registry is in experimental development status.

## Types of changes
<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [x] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [x] My code follows the repository's style guidelines
- [x] New and existing tests pass locally
- [x] I have added appropriate error handling
- [x] I have added or updated documentation as needed

## Additional context

I have not implemented connection pooling support for simplicity, and
because cloudnative-pg already provides built-in connection pooling with
pgBouncer so if we want it later we can switch to this endpoint easily.
2025-08-22 01:40:28 +01:00