Commit graph ruby-sdk/conformance/server.rb
Author SHA1 Message Date
Koichi ITO
bbe637612b Support progress notifications per MCP specification
## Motivation and Context

The MCP specification defines `notifications/progress` for reporting incremental progress
during long-running tool operations. The Ruby SDK did not yet support this, so tools had no way to
send progress updates to clients. This caused the `tools-call-with-progress` conformance test to fail.

This adds an `MCP::ServerContext` wrapper that provides a `report_progress` method on
`server_context`. Tools call `server_context.report_progress(value, total:, message:)` to send
notifications back to the client. The method is a no-op when no `progressToken` was provided,
so tools can call it unconditionally without checking for a token.

This design is consistent with the TypeScript and Python SDKs, which provide progress
reporting through a context object rather than a separate parameter.

On the client side, `Client#call_tool` now accepts a `progress_token:` parameter that sets
`_meta.progressToken` in the request.

Ref: https://modelcontextprotocol.io/specification/latest/server/utilities/progress

## How Has This Been Tested?

Added comprehensive tests in `test/mcp/server_progress_test.rb` covering `notify_progress`,
tool integration with progress, block-defined tools, `**kwargs` tools, and no-op behavior
without a token.

Added client tests for `progress_token:` parameter inclusion and omission.

Added `test/mcp/server_context_test.rb` test for non-Hash `server_context` preservation
when `_meta` is present.

Conformance: `tools-call-with-progress` removed from expected failures.

## Breaking Change

`server_context` is now always an `MCP::ServerContext` instance when passed to tools,
even if the server has no context configured. Code that checks `server_context ?` to
detect a nil context will need to be updated, as `ServerContext` is always truthy.
`ServerContext` delegates method calls to the underlying context object via
`method_missing`, so `server_context[:key]` and other accessor patterns continue to work.

In practice, this is unlikely to affect users. A tool that declares `server_context:`
is opting in to use the context; if the server has no context configured, that is a
server configuration issue rather than something the tool should handle with a nil check.
2026-03-10 01:49:09 +09:00
Koichi ITO
13bbd2937d Support running MCP conformance tests against the Ruby SDK
## Motivation

To assess and track the Ruby SDK's compliance with the MCP
specification, integrate `@modelcontextprotocol/conformance` as the
standard evaluation harness. This makes it possible to measure
conformance objectively, catch regressions early, and produce the kind
of evaluation data needed for SDK tier assessments.

## Scope

Only server conformance is included at this point. Client conformance
requires protocol-level features that MCP::Client does not yet
support (initialize handshake, notifications, ping, etc.).

## Usage

Run conformance tests:

```console
bundle exec rake conformance
```

Start the conformance server in the foreground for iterating on a single
scenario without the per-run startup overhead:

```console
bundle exec rake conformance_server
```

## Default Rake Task (`bundle exec rake`)

It is configured to run in order of lower execution cost: `rubocop`, `test`, and then `conformance`.

## CI

`.github/workflows/conformance.yml` file has been added to run the conformance tests in CI.

## SDK Tier Report

The MCP SDK Tier system (https://modelcontextprotocol.io/community/sdk-tiers)
requires SDK maintainers to self-assess and report results to the SDK Working
Group via https://github.com/modelcontextprotocol/modelcontextprotocol/issues.

To generate a full tier assessment report, use the `/mcp-sdk-tier-audit`
slash command from the https://github.com/modelcontextprotocol/conformance repository with
the conformance server running:

```console
# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp
```

The skill evaluates conformance pass rate, issue label taxonomy, triage
metrics, documentation coverage, and policy compliance, then produces a
markdown report suitable for tier advancement submissions.
2026-03-02 14:50:03 +09:00