Support pagination per MCP specification
## Motivation and Context The MCP specification defines cursor-based pagination for list operations that may return large result sets: https://modelcontextprotocol.io/specification/2025-11-25/server/utilities/pagination Pagination allows servers to yield results in smaller chunks rather than all at once, which is especially important when connecting to external services over the internet. The Ruby SDK previously returned complete arrays for all list endpoints (`tools/list`, `resources/list`, `prompts/list`, `resources/templates/list`) without pagination support. This adds cursor-based pagination. ### Server-side A new `MCP::Server::Pagination` module is introduced (mixed into `MCP::Server`) that provides the `paginate` helper for slicing a collection by cursor and a `cursor_from` helper that extracts the cursor from the request params while rejecting non-Hash inputs with `-32602 Invalid params` per the specification. `MCP::Server.new` accepts a new `page_size:` keyword argument. When `nil` (the default), all items are returned in a single response, preserving the existing behavior. When set to a positive integer, list responses include a `nextCursor` field if more pages are available. `Server#page_size=` is a validating setter that rejects anything other than `nil` or a positive `Integer`, raising `ArgumentError` on invalid inputs. The constructor routes through the setter, so both `Server.new(page_size: 0)` and `server.page_size = -1` raise. Cursors are string tokens carrying a zero-based offset, treated as opaque by clients. Cursor inputs are validated to be strings per the MCP spec; invalid cursors (non-string, non-numeric, negative, or out-of-range) raise `RequestHandlerError` with error code `-32602 (Invalid params)` per the spec. ### Client-side: two complementary APIs The SDK exposes two API shapes for listing, each serving a different use case: 1. **Single-page with cursor** (`client.list_tools(cursor:)`, `list_resources`, `list_resource_templates`, `list_prompts`): each call issues one JSON-RPC request and returns a result object (`MCP::Client::ListToolsResult` etc.) carrying the page items and an optional `next_cursor`. These methods match the single-page-with-cursor convention used by the Python SDK (`session.list_tools(params=PaginatedRequestParams(cursor=...))`) and the TypeScript SDK (`client.listTools({ cursor })`). The method name and the `cursor` parameter name are identical across the three SDKs, so pagination code translates directly between languages. 2. **Whole-collection** (`client.tools`, `client.resources`, `client.resource_templates`, `client.prompts`): auto-iterates through all pages and returns a plain array of items, guaranteeing the full collection regardless of the server's `page_size` setting. The auto-pagination loop tracks previously seen cursors in a set and exits if the server revisits any of them, guarding against both immediate repeats and multi-cursor cycles (e.g. `A -> B -> A`). This mirrors Rust SDK's `list_all_*` convenience methods, though the Ruby names are preserved from the pre-pagination 0.13.0 API for backward compatibility. Future rename to `list_all_*` is left as a TODO in the source. Result classes are `Struct`s named to mirror Python SDK's `ListToolsResult` / `ListPromptsResult` / etc. to align naming with other SDKs. Each struct exposes its page items (e.g. `result.tools`), an optional `next_cursor` for continuation, and an optional `meta` field mirroring the MCP `Result` type's `_meta` response field so servers that decorate list responses do not lose metadata through the client. ### Ruby-specific ergonomics `Server.new(page_size:)` and the `Pagination` module are Ruby-specific. The Python and TypeScript SDKs do not expose a built-in `page_size` helper; developers there re-implement cursor slicing inside each handler. In Ruby, setting `page_size:` on the server is enough for the SDK to handle the slicing across the four built-in list endpoints. ### README A Pagination section documents both the server-side `page_size:` option and the client-side iteration patterns, including an explicit note that each `list_*` call is a single JSON-RPC round trip whose response size depends on the server's `page_size`, and that `client.tools` / `.resources` / `.resource_templates` / `.prompts` return the complete collection when needed. ## How Has This Been Tested? - Added tests for the `Pagination` module (`test/mcp/server/pagination_test.rb`), server-side pagination (`test/mcp/server_test.rb`), and client-side pagination (`test/mcp/client_test.rb`). - Added and existing tests pass. `rake rubocop` is clean and `rake conformance` passes. ## Breaking Changes None. This is a new feature addition. Existing code continues to work unchanged: `page_size` defaults to `nil` on the server (returns all items, no `nextCursor`), and the client's existing `tools` / `resources` / `resource_templates` / `prompts` methods continue to return complete arrays by transparently iterating through pages.
This commit is contained in:
Koichi ITO
parent
7b5533c809
commit
b97b922a36
4 changed files with 970 additions and 35 deletions
|
|
@ -2,6 +2,7 @@
|
|||
|
||||
require_relative "client/stdio"
|
||||
require_relative "client/http"
|
||||
require_relative "client/paginated_result"
|
||||
require_relative "client/tool"
|
||||
|
||||
module MCP
|
||||
|
|
@ -43,8 +44,41 @@ module MCP
|
|||
# So keeping it public
|
||||
attr_reader :transport
|
||||
|
||||
# Returns the list of tools available from the server.
|
||||
# Each call will make a new request – the result is not cached.
|
||||
# Returns a single page of tools from the server.
|
||||
#
|
||||
# @param cursor [String, nil] Cursor from a previous page response.
|
||||
# @return [MCP::Client::ListToolsResult] Result with `tools` (Array<MCP::Client::Tool>)
|
||||
# and `next_cursor` (String or nil).
|
||||
#
|
||||
# @example Iterate all pages
|
||||
# cursor = nil
|
||||
# loop do
|
||||
# page = client.list_tools(cursor: cursor)
|
||||
# page.tools.each { |tool| puts tool.name }
|
||||
# cursor = page.next_cursor
|
||||
# break unless cursor
|
||||
# end
|
||||
def list_tools(cursor: nil)
|
||||
params = cursor ? { cursor: cursor } : nil
|
||||
response = request(method: "tools/list", params: params)
|
||||
result = response["result"] || {}
|
||||
|
||||
tools = (result["tools"] || []).map do |tool|
|
||||
Tool.new(
|
||||
name: tool["name"],
|
||||
description: tool["description"],
|
||||
input_schema: tool["inputSchema"],
|
||||
)
|
||||
end
|
||||
|
||||
ListToolsResult.new(tools: tools, next_cursor: result["nextCursor"], meta: result["_meta"])
|
||||
end
|
||||
|
||||
# Returns every tool available on the server. Iterates through all pages automatically
|
||||
# when the server paginates, so the full collection is returned regardless of the server's `page_size` setting.
|
||||
# Use {#list_tools} when you need fine-grained cursor control.
|
||||
#
|
||||
# Each call will make a new request - the result is not cached.
|
||||
#
|
||||
# @return [Array<MCP::Client::Tool>] An array of available tools.
|
||||
#
|
||||
|
|
@ -54,45 +88,151 @@ module MCP
|
|||
# puts tool.name
|
||||
# end
|
||||
def tools
|
||||
response = request(method: "tools/list")
|
||||
# TODO: consider renaming to `list_all_tools`.
|
||||
all_tools = []
|
||||
seen = Set.new
|
||||
cursor = nil
|
||||
|
||||
response.dig("result", "tools")&.map do |tool|
|
||||
Tool.new(
|
||||
name: tool["name"],
|
||||
description: tool["description"],
|
||||
input_schema: tool["inputSchema"],
|
||||
)
|
||||
end || []
|
||||
loop do
|
||||
page = list_tools(cursor: cursor)
|
||||
all_tools.concat(page.tools)
|
||||
next_cursor = page.next_cursor
|
||||
break if next_cursor.nil? || seen.include?(next_cursor)
|
||||
|
||||
seen << next_cursor
|
||||
cursor = next_cursor
|
||||
end
|
||||
|
||||
all_tools
|
||||
end
|
||||
|
||||
# Returns the list of resources available from the server.
|
||||
# Each call will make a new request – the result is not cached.
|
||||
# Returns a single page of resources from the server.
|
||||
#
|
||||
# @param cursor [String, nil] Cursor from a previous page response.
|
||||
# @return [MCP::Client::ListResourcesResult] Result with `resources` (Array<Hash>)
|
||||
# and `next_cursor` (String or nil).
|
||||
def list_resources(cursor: nil)
|
||||
params = cursor ? { cursor: cursor } : nil
|
||||
response = request(method: "resources/list", params: params)
|
||||
result = response["result"] || {}
|
||||
|
||||
ListResourcesResult.new(
|
||||
resources: result["resources"] || [],
|
||||
next_cursor: result["nextCursor"],
|
||||
meta: result["_meta"],
|
||||
)
|
||||
end
|
||||
|
||||
# Returns every resource available on the server. Iterates through all pages automatically
|
||||
# when the server paginates, so the full collection is returned regardless of the server's `page_size` setting.
|
||||
# Use {#list_resources} when you need fine-grained cursor control.
|
||||
#
|
||||
# Each call will make a new request - the result is not cached.
|
||||
#
|
||||
# @return [Array<Hash>] An array of available resources.
|
||||
def resources
|
||||
response = request(method: "resources/list")
|
||||
# TODO: consider renaming to `list_all_resources`.
|
||||
all_resources = []
|
||||
seen = Set.new
|
||||
cursor = nil
|
||||
|
||||
response.dig("result", "resources") || []
|
||||
loop do
|
||||
page = list_resources(cursor: cursor)
|
||||
all_resources.concat(page.resources)
|
||||
next_cursor = page.next_cursor
|
||||
break if next_cursor.nil? || seen.include?(next_cursor)
|
||||
|
||||
seen << next_cursor
|
||||
cursor = next_cursor
|
||||
end
|
||||
|
||||
all_resources
|
||||
end
|
||||
|
||||
# Returns the list of resource templates available from the server.
|
||||
# Each call will make a new request – the result is not cached.
|
||||
# Returns a single page of resource templates from the server.
|
||||
#
|
||||
# @param cursor [String, nil] Cursor from a previous page response.
|
||||
# @return [MCP::Client::ListResourceTemplatesResult] Result with `resource_templates`
|
||||
# (Array<Hash>) and `next_cursor` (String or nil).
|
||||
def list_resource_templates(cursor: nil)
|
||||
params = cursor ? { cursor: cursor } : nil
|
||||
response = request(method: "resources/templates/list", params: params)
|
||||
result = response["result"] || {}
|
||||
|
||||
ListResourceTemplatesResult.new(
|
||||
resource_templates: result["resourceTemplates"] || [],
|
||||
next_cursor: result["nextCursor"],
|
||||
meta: result["_meta"],
|
||||
)
|
||||
end
|
||||
|
||||
# Returns every resource template available on the server. Iterates through all pages automatically
|
||||
# when the server paginates, so the full collection is returned regardless of the server's `page_size` setting.
|
||||
# Use {#list_resource_templates} when you need fine-grained cursor control.
|
||||
#
|
||||
# Each call will make a new request - the result is not cached.
|
||||
#
|
||||
# @return [Array<Hash>] An array of available resource templates.
|
||||
def resource_templates
|
||||
response = request(method: "resources/templates/list")
|
||||
# TODO: consider renaming to `list_all_resource_templates`.
|
||||
all_templates = []
|
||||
seen = Set.new
|
||||
cursor = nil
|
||||
|
||||
response.dig("result", "resourceTemplates") || []
|
||||
loop do
|
||||
page = list_resource_templates(cursor: cursor)
|
||||
all_templates.concat(page.resource_templates)
|
||||
next_cursor = page.next_cursor
|
||||
break if next_cursor.nil? || seen.include?(next_cursor)
|
||||
|
||||
seen << next_cursor
|
||||
cursor = next_cursor
|
||||
end
|
||||
|
||||
all_templates
|
||||
end
|
||||
|
||||
# Returns the list of prompts available from the server.
|
||||
# Each call will make a new request – the result is not cached.
|
||||
# Returns a single page of prompts from the server.
|
||||
#
|
||||
# @param cursor [String, nil] Cursor from a previous page response.
|
||||
# @return [MCP::Client::ListPromptsResult] Result with `prompts` (Array<Hash>)
|
||||
# and `next_cursor` (String or nil).
|
||||
def list_prompts(cursor: nil)
|
||||
params = cursor ? { cursor: cursor } : nil
|
||||
response = request(method: "prompts/list", params: params)
|
||||
result = response["result"] || {}
|
||||
|
||||
ListPromptsResult.new(
|
||||
prompts: result["prompts"] || [],
|
||||
next_cursor: result["nextCursor"],
|
||||
meta: result["_meta"],
|
||||
)
|
||||
end
|
||||
|
||||
# Returns every prompt available on the server. Iterates through all pages automatically
|
||||
# when the server paginates, so the full collection is returned regardless of the server's `page_size` setting.
|
||||
# Use {#list_prompts} when you need fine-grained cursor control.
|
||||
#
|
||||
# Each call will make a new request - the result is not cached.
|
||||
#
|
||||
# @return [Array<Hash>] An array of available prompts.
|
||||
def prompts
|
||||
response = request(method: "prompts/list")
|
||||
# TODO: consider renaming to `list_all_prompts`.
|
||||
all_prompts = []
|
||||
seen = Set.new
|
||||
cursor = nil
|
||||
|
||||
response.dig("result", "prompts") || []
|
||||
loop do
|
||||
page = list_prompts(cursor: cursor)
|
||||
all_prompts.concat(page.prompts)
|
||||
next_cursor = page.next_cursor
|
||||
break if next_cursor.nil? || seen.include?(next_cursor)
|
||||
|
||||
seen << next_cursor
|
||||
cursor = next_cursor
|
||||
end
|
||||
|
||||
all_prompts
|
||||
end
|
||||
|
||||
# Calls a tool via the transport layer and returns the full response from the server.
|
||||
|
|
|
|||
13
lib/mcp/client/paginated_result.rb
Normal file
13
lib/mcp/client/paginated_result.rb
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
# frozen_string_literal: true
|
||||
|
||||
module MCP
|
||||
class Client
|
||||
# Result objects returned by `list_tools`, `list_prompts`, `list_resources`, and `list_resource_templates`.
|
||||
# Each carries the page items, an optional opaque `next_cursor` string for continuing pagination,
|
||||
# and an optional `meta` hash mirroring the MCP `_meta` response field.
|
||||
ListToolsResult = Struct.new(:tools, :next_cursor, :meta, keyword_init: true)
|
||||
ListPromptsResult = Struct.new(:prompts, :next_cursor, :meta, keyword_init: true)
|
||||
ListResourcesResult = Struct.new(:resources, :next_cursor, :meta, keyword_init: true)
|
||||
ListResourceTemplatesResult = Struct.new(:resource_templates, :next_cursor, :meta, keyword_init: true)
|
||||
end
|
||||
end
|
||||
|
|
@ -6,6 +6,7 @@ require_relative "methods"
|
|||
require_relative "logging_message_notification"
|
||||
require_relative "progress"
|
||||
require_relative "server_context"
|
||||
require_relative "server/pagination"
|
||||
require_relative "server/transports"
|
||||
|
||||
module MCP
|
||||
|
|
@ -65,9 +66,10 @@ module MCP
|
|||
end
|
||||
|
||||
include Instrumentation
|
||||
include Pagination
|
||||
|
||||
attr_accessor :description, :icons, :name, :title, :version, :website_url, :instructions, :tools, :prompts, :resources, :server_context, :configuration, :capabilities, :transport, :logging_message_notification
|
||||
attr_reader :client_capabilities
|
||||
attr_reader :page_size, :client_capabilities
|
||||
|
||||
def initialize(
|
||||
description: nil,
|
||||
|
|
@ -84,6 +86,7 @@ module MCP
|
|||
server_context: nil,
|
||||
configuration: nil,
|
||||
capabilities: nil,
|
||||
page_size: nil,
|
||||
transport: nil
|
||||
)
|
||||
@description = description
|
||||
|
|
@ -100,6 +103,7 @@ module MCP
|
|||
@resource_templates = resource_templates
|
||||
@resource_index = index_resources_by_uri(resources)
|
||||
@server_context = server_context
|
||||
self.page_size = page_size
|
||||
@configuration = MCP.configuration.merge(configuration)
|
||||
@client = nil
|
||||
|
||||
|
|
@ -183,6 +187,14 @@ module MCP
|
|||
@handlers[method_name] = block
|
||||
end
|
||||
|
||||
def page_size=(page_size)
|
||||
unless page_size.nil? || (page_size.is_a?(Integer) && page_size > 0)
|
||||
raise ArgumentError, "page_size must be nil or a positive integer"
|
||||
end
|
||||
|
||||
@page_size = page_size
|
||||
end
|
||||
|
||||
def notify_tools_list_changed
|
||||
return unless @transport
|
||||
|
||||
|
|
@ -377,16 +389,8 @@ module MCP
|
|||
result = case method
|
||||
when Methods::INITIALIZE
|
||||
init(params, session: session)
|
||||
when Methods::TOOLS_LIST
|
||||
{ tools: @handlers[Methods::TOOLS_LIST].call(params) }
|
||||
when Methods::PROMPTS_LIST
|
||||
{ prompts: @handlers[Methods::PROMPTS_LIST].call(params) }
|
||||
when Methods::RESOURCES_LIST
|
||||
{ resources: @handlers[Methods::RESOURCES_LIST].call(params) }
|
||||
when Methods::RESOURCES_READ
|
||||
{ contents: @handlers[Methods::RESOURCES_READ].call(params) }
|
||||
when Methods::RESOURCES_TEMPLATES_LIST
|
||||
{ resourceTemplates: @handlers[Methods::RESOURCES_TEMPLATES_LIST].call(params) }
|
||||
when Methods::TOOLS_CALL
|
||||
call_tool(params, session: session, related_request_id: related_request_id)
|
||||
when Methods::COMPLETION_COMPLETE
|
||||
|
|
@ -488,7 +492,9 @@ module MCP
|
|||
end
|
||||
|
||||
def list_tools(request)
|
||||
@tools.values.map(&:to_h)
|
||||
page = paginate(@tools.values, cursor: cursor_from(request), page_size: @page_size, request: request, &:to_h)
|
||||
|
||||
{ tools: page[:items], nextCursor: page[:next_cursor] }.compact
|
||||
end
|
||||
|
||||
def call_tool(request, session: nil, related_request_id: nil)
|
||||
|
|
@ -536,7 +542,9 @@ module MCP
|
|||
end
|
||||
|
||||
def list_prompts(request)
|
||||
@prompts.values.map(&:to_h)
|
||||
page = paginate(@prompts.values, cursor: cursor_from(request), page_size: @page_size, request: request, &:to_h)
|
||||
|
||||
{ prompts: page[:items], nextCursor: page[:next_cursor] }.compact
|
||||
end
|
||||
|
||||
def get_prompt(request)
|
||||
|
|
@ -556,7 +564,9 @@ module MCP
|
|||
end
|
||||
|
||||
def list_resources(request)
|
||||
@resources.map(&:to_h)
|
||||
page = paginate(@resources, cursor: cursor_from(request), page_size: @page_size, request: request, &:to_h)
|
||||
|
||||
{ resources: page[:items], nextCursor: page[:next_cursor] }.compact
|
||||
end
|
||||
|
||||
# Server implementation should set `resources_read_handler` to override no-op default
|
||||
|
|
@ -566,7 +576,9 @@ module MCP
|
|||
end
|
||||
|
||||
def list_resource_templates(request)
|
||||
@resource_templates.map(&:to_h)
|
||||
page = paginate(@resource_templates, cursor: cursor_from(request), page_size: @page_size, request: request, &:to_h)
|
||||
|
||||
{ resourceTemplates: page[:items], nextCursor: page[:next_cursor] }.compact
|
||||
end
|
||||
|
||||
def complete(params)
|
||||
|
|
|
|||
42
lib/mcp/server/pagination.rb
Normal file
42
lib/mcp/server/pagination.rb
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
# frozen_string_literal: true
|
||||
|
||||
module MCP
|
||||
class Server
|
||||
module Pagination
|
||||
private
|
||||
|
||||
def cursor_from(request)
|
||||
return if request.nil?
|
||||
|
||||
unless request.is_a?(Hash)
|
||||
raise RequestHandlerError.new("Invalid params", request, error_type: :invalid_params)
|
||||
end
|
||||
|
||||
request[:cursor]
|
||||
end
|
||||
|
||||
def paginate(items, cursor:, page_size:, request:, &block)
|
||||
start_index = 0
|
||||
|
||||
if cursor
|
||||
unless cursor.is_a?(String)
|
||||
raise RequestHandlerError.new("Invalid cursor", request, error_type: :invalid_params)
|
||||
end
|
||||
|
||||
start_index = Integer(cursor, exception: false)
|
||||
if start_index.nil? || start_index < 0 || start_index >= items.size
|
||||
raise RequestHandlerError.new("Invalid cursor", request, error_type: :invalid_params)
|
||||
end
|
||||
end
|
||||
|
||||
end_index = page_size ? start_index + page_size : items.size
|
||||
page = items[start_index...end_index]
|
||||
page = page.map(&block) if block
|
||||
|
||||
result = { items: page }
|
||||
result[:next_cursor] = end_index.to_s if end_index < items.size
|
||||
result
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
Loading…
Reference in a new issue