Multi Round-Trip Requests Track: Brief (#5)
Add the "Multi Round-Trip Requests" track briefing doc to the repo. Note that there was some previous discussion in the [original location](https://docs.google.com/document/d/1i12c1NXhDBaCQYlL-GOpC2fh6Yr_2itlc07WP7-JqAE/edit?usp=sharing).
This commit is contained in:
parent
1de2540eb3
commit
1edadf9718
1 changed files with 85 additions and 0 deletions
85
proposals/multi-round-trip-requests-track-brief.md
Normal file
85
proposals/multi-round-trip-requests-track-brief.md
Normal file
|
|
@ -0,0 +1,85 @@
|
||||||
|
# Multi Round-Trip Requests: Brief
|
||||||
|
|
||||||
|
**Background:** There are two main types of tool implementations that may
|
||||||
|
need multi round-trip requests (e.g., elicitation within the context of a
|
||||||
|
tool call):
|
||||||
|
|
||||||
|
* **Ephemeral:** Tools that do not need to store any state on the server
|
||||||
|
side in the course of handling a tool call. Examples: getting the
|
||||||
|
weather, or reading email messages from an inbox. The majority of
|
||||||
|
tool calls will be this category.
|
||||||
|
* **Persistent:** Tools that *do* need to store state on the server side
|
||||||
|
in the course of handling a tool call. Example: Modifying the state of a VM.
|
||||||
|
|
||||||
|
Note: This doc uses tools as the primary example, but the mechanism is
|
||||||
|
intended to work for any client-initiated request.
|
||||||
|
|
||||||
|
**Goal** To make it cheaper, easier, and more fault tolerant to implement
|
||||||
|
mutli round-trip requests, especially for ephemeral tool implementations.
|
||||||
|
|
||||||
|
Context:
|
||||||
|
|
||||||
|
* [MCP Streamless Round-Trip Proposal Sketch](https://docs.google.com/document/d/1EQSttkKGOUNXzJM998VDhBu3288f4IqRM0jd1tWbdos/edit?usp=sharing)
|
||||||
|
* [MCP Multi Round-Trip Request Problem](https://docs.google.com/presentation/d/1Hi1pUNQWEb59TnvSUkMBTy59vHQ87_Gobe5NF9lz7UA/edit?usp=sharing)
|
||||||
|
|
||||||
|
**Core Principle: Ability to Pass State Through Client instead of Storing
|
||||||
|
it on Server**
|
||||||
|
|
||||||
|
* **Approach:** Provide mechanisms to allow passing state through the client
|
||||||
|
between requests rather than storing it on the server side, either in a
|
||||||
|
shared storage tier or in memory (which requires stateful LB routing).
|
||||||
|
* **Requirement:** We need to make this case easier and cheaper for ephemeral
|
||||||
|
tools, but we need to not make things worse for persistent tools. (As a
|
||||||
|
bonus, we might even improve things for some cases for persistent tools,
|
||||||
|
but that's not a strict requirement.)
|
||||||
|
|
||||||
|
**Requirements** To facilitate storing state on the client, we will support
|
||||||
|
the following:
|
||||||
|
|
||||||
|
* **Support Incomplete Responses and Resuming Requests:** When a tool
|
||||||
|
implementation needs to make a request to the client for more
|
||||||
|
information (e.g., elicitation or sampling), instead of keeping the
|
||||||
|
original SSE stream from the tool call response open, the server will
|
||||||
|
send an incomplete response indicating the requests that it needs
|
||||||
|
the client to respond to, and then it will terminate the stream.
|
||||||
|
The client will then generate the responses to the server's requests
|
||||||
|
and issue a new tool call that includes those responses.
|
||||||
|
* **Ability to Pass Arbitrary State Through the Client:** We need a
|
||||||
|
cookie-like mechanism, perhaps similar to SEP-1685, to allow servers
|
||||||
|
to pass state through the client, even if that state is generated on
|
||||||
|
the server instead of being generated by the client in response to a
|
||||||
|
server-initiated request.
|
||||||
|
* Note: This mechanism may alleviate the need for a shared storage tier
|
||||||
|
on the server side in both ephemeral and persistent tool cases.
|
||||||
|
* **Support for Persistent Tasks:** For tools that may continue processing
|
||||||
|
the request while waiting for clients to respond to the server-initiated
|
||||||
|
requests, the client needs to know that that is happening, so that it
|
||||||
|
can inform the server when the tool call needs to be cancelled.
|
||||||
|
|
||||||
|
**Key Design Decisions (Open Questions)**
|
||||||
|
|
||||||
|
* **Mechanism for Passing State Through the Client:** Should we use something
|
||||||
|
like HTTP cookies, or should we use an MCP-specific solution like SEP-1685?
|
||||||
|
* *Consideration:* This overlaps somewhat with discussions from the session
|
||||||
|
track (see [Sessions Track:
|
||||||
|
Brief](https://docs.google.com/document/d/1W0pJMbZUCVwo7SLpxgujrr-3yHvOzfLgaZqRsh7Z-CA/edit?usp=sharing)),
|
||||||
|
where we are deciding whether to use HTTP cookies for sessions or
|
||||||
|
introducing an MCP-specific mechanism.
|
||||||
|
* *Consideration:* For the multi round-trip request track, the scope of
|
||||||
|
the state that we need to pass through the client is fairly short;
|
||||||
|
we will need it only until the client issues the next RPC associated
|
||||||
|
with the same request. If the client is simply providing a cookie
|
||||||
|
jar for each session, that may make it hard to know when a cookie
|
||||||
|
is no longer needed without relying on TTLs.
|
||||||
|
* **Overlap with Tasks:** The changes that we're making here seem to overlap
|
||||||
|
a lot with tasks. We may want to simplify the approach to tasks as part
|
||||||
|
of this work.
|
||||||
|
|
||||||
|
**Impact Assessment** A critical part of this track is weighing the impact
|
||||||
|
on **current SDK implementations**.
|
||||||
|
|
||||||
|
* **Backwards Compatibility:** For tool implementations that are currently
|
||||||
|
written to do an async await of the elicitation or sampling request and
|
||||||
|
then resume the tool call on the same stream, we need to figure out if
|
||||||
|
there's a way we can keep that code working with the new approach for the
|
||||||
|
wire protocol.
|
||||||
Loading…
Reference in a new issue