Commit graph ruby-sdk/conformance
Author SHA1 Message Date
Koichi ITO
da5f9225ac Support sampling/createMessage per MCP specification
## Motivation and Context

The MCP specification defines `sampling/createMessage` to request LLM completions from the client.
The Ruby SDK did not yet support server-to-client requests,
so the server had no way to ask the client to sample an LLM.
This caused the `tools-call-sampling` conformance test to fail.

This adds `Server#create_sampling_message` to send a `sampling/createMessage` request
to the client via the transport layer. The method validates that the client declared
the `sampling` capability during initialization, and optionally checks the
`sampling.tools` capability when tools or tool_choice parameters are provided.

On the transport side, `Transport#send_request` is introduced as an abstract method,
with implementations in both `StdioTransport` and `StreamableHTTPTransport`.
Each transport assigns a unique request ID, sends the JSON-RPC request to the client,
and blocks until the client responds.

Client capabilities are stored per session via `ServerSession`, so validation is scoped
to the originating client. `ServerSession#create_sampling_message` uses
`build_sampling_params` for capability validation and sends the request directly through
the transport with the session ID, following the same pattern as `notify_progress`.

`Server#create_sampling_message` is the public API for single-client transports
(e.g., StdioTransport).

For multi-client transports (e.g., StreamableHTTPTransport),
`ServerSession#create_sampling_message` must be used to route requests to the correct client.
Tools using `server_context.create_sampling_message` automatically route to the correct session
regardless of transport.

`StreamableHTTPTransport#send_request` requires `session_id` to prevent broadcasting sampling requests
to all connected clients. Missing or invalid session IDs raise explicit errors instead of silently failing.

The `include_context` parameter is soft-deprecated in the MCP specification but is included
for compatibility with existing clients that declare the `sampling.context` capability.
Python and TypeScript SDKs also retain this parameter.

Ref: https://modelcontextprotocol.io/specification/latest/server/utilities/sampling

## How Has This Been Tested?

Added comprehensive tests in `test/mcp/server_sampling_test.rb` covering required and optional parameters,
capability validation, error handling, nil-param omission, per-session capability isolation,
and HTTP init capability scoping.

Added transport tests in `test/mcp/server/transports/stdio_transport_test.rb` and
`test/mcp/server/transports/streamable_http_transport_test.rb` for `send_request` round-trip behavior,
error responses, stateless mode rejection, missing session_id, invalid session_id, and no-active-stream error.

Conformance: `tools-call-sampling` is removed from expected failures. The conformance server's `TestSampling`
tool now calls `server_context.create_sampling_message` instead of returning a stub error.

This aligns the Ruby SDK with the MCP specification and other SDK implementations.

## Breaking Changes

None for end users.
`Transport#send_request` is a new abstract method. `ServerSession#create_sampling_message` is a new method.
2026-04-04 05:55:27 +09:00
Koichi ITO
88e1f9a840 Support completion/complete per MCP specification
## Motivation and Context

The MCP specification defines `completion/complete` for providing autocompletion
suggestions for prompt arguments and resource template URIs.
The Ruby SDK previously had only a no-op handler that returned empty results,
with no way for users to define custom completion logic.

This aligns the Ruby SDK with the Python and TypeScript SDKs,
both of which support user-defined completion handlers.

## How Has This Been Tested?

Server tests cover: default handler, custom handler for `ref/prompt` and `ref/resource`,
context argument passing, 100-item truncation, and error responses for nonexistent prompts,
nonexistent resource templates, and invalid ref types.

Client tests cover: request structure, context parameter inclusion,
and fallback when result is missing.

## Breaking Changes

None. The existing default no-op handler behavior is preserved.
The `completion_handler` method is purely additive. The only behavioral change
is that `completion/complete` requests now validate that the referenced prompt
or resource template exists before calling the handler,
returning an `invalid_params` error for unknown references.
2026-04-03 01:35:07 +09:00
Koichi ITO
bbe637612b Support progress notifications per MCP specification
## Motivation and Context

The MCP specification defines `notifications/progress` for reporting incremental progress
during long-running tool operations. The Ruby SDK did not yet support this, so tools had no way to
send progress updates to clients. This caused the `tools-call-with-progress` conformance test to fail.

This adds an `MCP::ServerContext` wrapper that provides a `report_progress` method on
`server_context`. Tools call `server_context.report_progress(value, total:, message:)` to send
notifications back to the client. The method is a no-op when no `progressToken` was provided,
so tools can call it unconditionally without checking for a token.

This design is consistent with the TypeScript and Python SDKs, which provide progress
reporting through a context object rather than a separate parameter.

On the client side, `Client#call_tool` now accepts a `progress_token:` parameter that sets
`_meta.progressToken` in the request.

Ref: https://modelcontextprotocol.io/specification/latest/server/utilities/progress

## How Has This Been Tested?

Added comprehensive tests in `test/mcp/server_progress_test.rb` covering `notify_progress`,
tool integration with progress, block-defined tools, `**kwargs` tools, and no-op behavior
without a token.

Added client tests for `progress_token:` parameter inclusion and omission.

Added `test/mcp/server_context_test.rb` test for non-Hash `server_context` preservation
when `_meta` is present.

Conformance: `tools-call-with-progress` removed from expected failures.

## Breaking Change

`server_context` is now always an `MCP::ServerContext` instance when passed to tools,
even if the server has no context configured. Code that checks `server_context ?` to
detect a nil context will need to be updated, as `ServerContext` is always truthy.
`ServerContext` delegates method calls to the underlying context object via
`method_missing`, so `server_context[:key]` and other accessor patterns continue to work.

In practice, this is unlikely to affect users. A tool that declares `server_context:`
is opting in to use the context; if the server has no context configured, that is a
server configuration issue rather than something the tool should handle with a nil check.
2026-03-10 01:49:09 +09:00
Koichi ITO
13bbd2937d Support running MCP conformance tests against the Ruby SDK
## Motivation

To assess and track the Ruby SDK's compliance with the MCP
specification, integrate `@modelcontextprotocol/conformance` as the
standard evaluation harness. This makes it possible to measure
conformance objectively, catch regressions early, and produce the kind
of evaluation data needed for SDK tier assessments.

## Scope

Only server conformance is included at this point. Client conformance
requires protocol-level features that MCP::Client does not yet
support (initialize handshake, notifications, ping, etc.).

## Usage

Run conformance tests:

```console
bundle exec rake conformance
```

Start the conformance server in the foreground for iterating on a single
scenario without the per-run startup overhead:

```console
bundle exec rake conformance_server
```

## Default Rake Task (`bundle exec rake`)

It is configured to run in order of lower execution cost: `rubocop`, `test`, and then `conformance`.

## CI

`.github/workflows/conformance.yml` file has been added to run the conformance tests in CI.

## SDK Tier Report

The MCP SDK Tier system (https://modelcontextprotocol.io/community/sdk-tiers)
requires SDK maintainers to self-assess and report results to the SDK Working
Group via https://github.com/modelcontextprotocol/modelcontextprotocol/issues.

To generate a full tier assessment report, use the `/mcp-sdk-tier-audit`
slash command from the https://github.com/modelcontextprotocol/conformance repository with
the conformance server running:

```console
# Terminal 1 (this repository): start the conformance server
bundle exec rake conformance_server

# Terminal 2 (conformance repository): run the tier audit skill as a slash command in Claude Code
/mcp-sdk-tier-audit /path/to/modelcontextprotocol/ruby-sdk http://localhost:9292/mcp
```

The skill evaluates conformance pass rate, issue label taxonomy, triage
metrics, documentation coverage, and policy compliance, then produces a
markdown report suitable for tier advancement submissions.
2026-03-02 14:50:03 +09:00