diff --git a/proposals/multi-round-trip-requests-track-brief.md b/proposals/multi-round-trip-requests-track-brief.md new file mode 100644 index 0000000..50ae92a --- /dev/null +++ b/proposals/multi-round-trip-requests-track-brief.md @@ -0,0 +1,85 @@ +# Multi Round-Trip Requests: Brief + +**Background:** There are two main types of tool implementations that may +need multi round-trip requests (e.g., elicitation within the context of a +tool call): + +* **Ephemeral:** Tools that do not need to store any state on the server + side in the course of handling a tool call. Examples: getting the + weather, or reading email messages from an inbox. The majority of + tool calls will be this category. +* **Persistent:** Tools that *do* need to store state on the server side + in the course of handling a tool call. Example: Modifying the state of a VM. + +Note: This doc uses tools as the primary example, but the mechanism is +intended to work for any client-initiated request. + +**Goal** To make it cheaper, easier, and more fault tolerant to implement +mutli round-trip requests, especially for ephemeral tool implementations. + +Context: + +* [MCP Streamless Round-Trip Proposal Sketch](https://docs.google.com/document/d/1EQSttkKGOUNXzJM998VDhBu3288f4IqRM0jd1tWbdos/edit?usp=sharing) +* [MCP Multi Round-Trip Request Problem](https://docs.google.com/presentation/d/1Hi1pUNQWEb59TnvSUkMBTy59vHQ87_Gobe5NF9lz7UA/edit?usp=sharing) + +**Core Principle: Ability to Pass State Through Client instead of Storing +it on Server** + +* **Approach:** Provide mechanisms to allow passing state through the client + between requests rather than storing it on the server side, either in a + shared storage tier or in memory (which requires stateful LB routing). +* **Requirement:** We need to make this case easier and cheaper for ephemeral + tools, but we need to not make things worse for persistent tools. (As a + bonus, we might even improve things for some cases for persistent tools, + but that's not a strict requirement.) + +**Requirements** To facilitate storing state on the client, we will support +the following: + +* **Support Incomplete Responses and Resuming Requests:** When a tool + implementation needs to make a request to the client for more + information (e.g., elicitation or sampling), instead of keeping the + original SSE stream from the tool call response open, the server will + send an incomplete response indicating the requests that it needs + the client to respond to, and then it will terminate the stream. + The client will then generate the responses to the server's requests + and issue a new tool call that includes those responses. +* **Ability to Pass Arbitrary State Through the Client:** We need a + cookie-like mechanism, perhaps similar to SEP-1685, to allow servers + to pass state through the client, even if that state is generated on + the server instead of being generated by the client in response to a + server-initiated request. + * Note: This mechanism may alleviate the need for a shared storage tier + on the server side in both ephemeral and persistent tool cases. +* **Support for Persistent Tasks:** For tools that may continue processing + the request while waiting for clients to respond to the server-initiated + requests, the client needs to know that that is happening, so that it + can inform the server when the tool call needs to be cancelled. + +**Key Design Decisions (Open Questions)** + +* **Mechanism for Passing State Through the Client:** Should we use something + like HTTP cookies, or should we use an MCP-specific solution like SEP-1685? + * *Consideration:* This overlaps somewhat with discussions from the session + track (see [Sessions Track: + Brief](https://docs.google.com/document/d/1W0pJMbZUCVwo7SLpxgujrr-3yHvOzfLgaZqRsh7Z-CA/edit?usp=sharing)), + where we are deciding whether to use HTTP cookies for sessions or + introducing an MCP-specific mechanism. + * *Consideration:* For the multi round-trip request track, the scope of + the state that we need to pass through the client is fairly short; + we will need it only until the client issues the next RPC associated + with the same request. If the client is simply providing a cookie + jar for each session, that may make it hard to know when a cookie + is no longer needed without relying on TTLs. +* **Overlap with Tasks:** The changes that we're making here seem to overlap + a lot with tasks. We may want to simplify the approach to tasks as part + of this work. + +**Impact Assessment** A critical part of this track is weighing the impact +on **current SDK implementations**. + +* **Backwards Compatibility:** For tool implementations that are currently + written to do an async await of the elicitation or sampling request and + then resume the tool call on the same stream, we need to figure out if + there's a way we can keep that code working with the new approach for the + wire protocol.