Add the "Multi Round-Trip Requests" track briefing doc to the repo. Note that there was some previous discussion in the [original location](https://docs.google.com/document/d/1i12c1NXhDBaCQYlL-GOpC2fh6Yr_2itlc07WP7-JqAE/edit?usp=sharing).
85 lines
4.6 KiB
Markdown
85 lines
4.6 KiB
Markdown
# Multi Round-Trip Requests: Brief
|
|
|
|
**Background:** There are two main types of tool implementations that may
|
|
need multi round-trip requests (e.g., elicitation within the context of a
|
|
tool call):
|
|
|
|
* **Ephemeral:** Tools that do not need to store any state on the server
|
|
side in the course of handling a tool call. Examples: getting the
|
|
weather, or reading email messages from an inbox. The majority of
|
|
tool calls will be this category.
|
|
* **Persistent:** Tools that *do* need to store state on the server side
|
|
in the course of handling a tool call. Example: Modifying the state of a VM.
|
|
|
|
Note: This doc uses tools as the primary example, but the mechanism is
|
|
intended to work for any client-initiated request.
|
|
|
|
**Goal** To make it cheaper, easier, and more fault tolerant to implement
|
|
mutli round-trip requests, especially for ephemeral tool implementations.
|
|
|
|
Context:
|
|
|
|
* [MCP Streamless Round-Trip Proposal Sketch](https://docs.google.com/document/d/1EQSttkKGOUNXzJM998VDhBu3288f4IqRM0jd1tWbdos/edit?usp=sharing)
|
|
* [MCP Multi Round-Trip Request Problem](https://docs.google.com/presentation/d/1Hi1pUNQWEb59TnvSUkMBTy59vHQ87_Gobe5NF9lz7UA/edit?usp=sharing)
|
|
|
|
**Core Principle: Ability to Pass State Through Client instead of Storing
|
|
it on Server**
|
|
|
|
* **Approach:** Provide mechanisms to allow passing state through the client
|
|
between requests rather than storing it on the server side, either in a
|
|
shared storage tier or in memory (which requires stateful LB routing).
|
|
* **Requirement:** We need to make this case easier and cheaper for ephemeral
|
|
tools, but we need to not make things worse for persistent tools. (As a
|
|
bonus, we might even improve things for some cases for persistent tools,
|
|
but that's not a strict requirement.)
|
|
|
|
**Requirements** To facilitate storing state on the client, we will support
|
|
the following:
|
|
|
|
* **Support Incomplete Responses and Resuming Requests:** When a tool
|
|
implementation needs to make a request to the client for more
|
|
information (e.g., elicitation or sampling), instead of keeping the
|
|
original SSE stream from the tool call response open, the server will
|
|
send an incomplete response indicating the requests that it needs
|
|
the client to respond to, and then it will terminate the stream.
|
|
The client will then generate the responses to the server's requests
|
|
and issue a new tool call that includes those responses.
|
|
* **Ability to Pass Arbitrary State Through the Client:** We need a
|
|
cookie-like mechanism, perhaps similar to SEP-1685, to allow servers
|
|
to pass state through the client, even if that state is generated on
|
|
the server instead of being generated by the client in response to a
|
|
server-initiated request.
|
|
* Note: This mechanism may alleviate the need for a shared storage tier
|
|
on the server side in both ephemeral and persistent tool cases.
|
|
* **Support for Persistent Tasks:** For tools that may continue processing
|
|
the request while waiting for clients to respond to the server-initiated
|
|
requests, the client needs to know that that is happening, so that it
|
|
can inform the server when the tool call needs to be cancelled.
|
|
|
|
**Key Design Decisions (Open Questions)**
|
|
|
|
* **Mechanism for Passing State Through the Client:** Should we use something
|
|
like HTTP cookies, or should we use an MCP-specific solution like SEP-1685?
|
|
* *Consideration:* This overlaps somewhat with discussions from the session
|
|
track (see [Sessions Track:
|
|
Brief](https://docs.google.com/document/d/1W0pJMbZUCVwo7SLpxgujrr-3yHvOzfLgaZqRsh7Z-CA/edit?usp=sharing)),
|
|
where we are deciding whether to use HTTP cookies for sessions or
|
|
introducing an MCP-specific mechanism.
|
|
* *Consideration:* For the multi round-trip request track, the scope of
|
|
the state that we need to pass through the client is fairly short;
|
|
we will need it only until the client issues the next RPC associated
|
|
with the same request. If the client is simply providing a cookie
|
|
jar for each session, that may make it hard to know when a cookie
|
|
is no longer needed without relying on TTLs.
|
|
* **Overlap with Tasks:** The changes that we're making here seem to overlap
|
|
a lot with tasks. We may want to simplify the approach to tasks as part
|
|
of this work.
|
|
|
|
**Impact Assessment** A critical part of this track is weighing the impact
|
|
on **current SDK implementations**.
|
|
|
|
* **Backwards Compatibility:** For tool implementations that are currently
|
|
written to do an async await of the elicitation or sampling request and
|
|
then resume the tool call on the same stream, we need to figure out if
|
|
there's a way we can keep that code working with the new approach for the
|
|
wire protocol.
|