Fix the tool-use loop in the Go, Rust, and Ruby clients and add a tool-loop smoke test (#200)
Some checks failed
Smoke Tests / test (push) Has been cancelled

* Fix the tool-use loop in the Go, Rust, and Ruby clients and add a tool-loop smoke test

## Motivation and Context

Only the Python and TypeScript clients implemented a correct tool-use loop. The other three
had bugs that break ordinary multi-step requests:

- **Go, Rust, Ruby** did a single tool round and then called Claude once more *without*
  `tools`, so any request that needs a second tool call stalled. Go silently dropped
  `tool_use` blocks from that final response.
- **Ruby** called Claude once per `tool_use` block, each time with a single `tool_result`.
  Parallel tool calls produced a malformed message sequence and an API 400.
- **max_tokens** was 1000 in TypeScript and Ruby, 1024 in Go, and unset in Rust. Sonnet 5
  thinks adaptively, so 1000 tokens is regularly exhausted mid-thought. Python already used
  10000 with a comment explaining why.
- **Ruby** never forwarded MCP `isError` as `is_error`, so the model could not tell a failed
  tool call from a successful one. Rust cannot set the flag either (see below).
- **TypeScript** forwarded raw MCP content blocks to Anthropic. The two block shapes differ,
  so a non-text result would have failed.

The existing smoke test never noticed because it only checks that each client calls
`tools/list`. None of these bugs was reachable without an API key.

### What changed

All five clients now run the same loop: call tools until Claude answers without one, send
every `tool_result` of a turn in one user message, pass `tools` on every call, and stop
after `MAX_TOOL_TURNS = 10`. `max_tokens` is 10000 everywhere, with the same comment.
Python and TypeScript also keep the last response's text when the turn cap is hit; it used
to be dropped.

Rust keeps the `genai` crate. Its tool-response type has no `is_error` field, so an error
result is marked with an `Error:` prefix in the text and the limitation is commented.

### New smoke test

`tests/helpers/tool-loop-test.ts` starts a fake Anthropic Messages API on a loopback port,
points a client at it through `ANTHROPIC_BASE_URL`, pipes in one query, and checks every
request the client sends. It scripts two parallel tool calls, then a call to a tool the mock
server lacks (an `isError` result), then a final answer, and asserts that tools are passed
each time, `max_tokens` is 10000, all `tool_result`s arrive in one message with matching
ids, and the error is forwarded. `smoke-test.sh` runs it for Python, TypeScript, Go, and
Ruby.

The Rust client is excluded for now: `genai` reads no endpoint variable, and the client
negotiates protocol `2025-11-25`, under which the mock's array-rooted tool is refused at call
time. Both will be addressed in a follow-up that brings all examples to current SDK versions.

## How Has This Been Tested?

- `./tests/smoke-test.sh` passes for every example.
- The new tool-loop test passes all 19 checks for Python, TypeScript, Go, and Ruby.
- The Rust client was driven through the same scripted conversation with a temporary
  endpoint resolver (not committed) and passes.
- As a negative check, the Ruby client from `main` was run through the new test. It fails 10
  of 19 checks, on exactly the bugs listed above.
- Go and Ruby were built and run in `golang:1.25` and `ruby:3.4` containers.

## Breaking Changes

None. The clients' command lines, output format, and no-key behavior are unchanged.

## Types of changes

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] Documentation update

## Checklist

- [x] I have read the [MCP Documentation](https://modelcontextprotocol.io)
- [x] My code follows the repository's style guidelines
- [x] New and existing tests pass locally
- [x] I have added appropriate error handling
- [x] I have added or updated documentation as needed

## Additional context

Two pre-existing issues surfaced while testing and are left for a follow-up:

- The Rust client (rmcp 3.1.2) negotiates `2025-11-25`. The TypeScript mock server and the
  Python weather server both reject its calls to array-rooted tools under that revision.
  Even current rmcp defaults to `2025-11-25`, so the client must request `2026-07-28`
  explicitly.
- The Go SDK's tool-result helper sends content as a list of text blocks rather than a
  string. The API accepts both, so it is unchanged.

AI assistance: the analysis, code changes, test helper, and this description were prepared
partially manually, partially with Claude Code and reviewed by me.

* Improvements

* Address comments
This commit is contained in:
Alex Akimov 2026-09-20 03:25:02 +02:00 • committed by GitHub
commit 470d1ba328
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -8,6 +8,7 @@ The smoke tests verify:
- **Servers**: Each weather server (Python, TypeScript, Rust, Go, Ruby) can start, respond to MCP protocol requests, and honour the output schemas it advertises
- **Clients**: Each MCP client (Python, TypeScript, Ruby, Go, Rust) can connect to a mock server and list tools
- **Tool loop**: Each client (Python, TypeScript, Ruby, Go) runs a query through a scripted tool-use loop against a fake Anthropic API and forwards tool results correctly
## Structured content
@ -20,6 +21,20 @@ The array case is the one worth guarding. A server that advertises `{"type": "ar
Tool calls reach the live NWS API. When it is unreachable the tools return an error result, which the test reports as a skip rather than a failure — someone else's outage should not fail the build.
## Tool loop
Connecting and listing tools does not exercise the chat loop, so each client is also driven through three scripted queries with `tool-loop-test.ts`. No real API is involved: the helper starts a fake Anthropic Messages API on a loopback port and hands it to the client through `ANTHROPIC_BASE_URL`, which every quickstart SDK reads. The client talks to the mock MCP server as in the no-key test. The helper types each query only after the client shows its `Query:` prompt, as a person would.
The fake API picks a script from the query text and checks every request the client sends:
- **parallel tools** — a response with two `tool_use` blocks, then one for a tool the mock server lacks, then an answer. `tools` must be passed on every call, `max_tokens` must be 10000 (room for the model's adaptive thinking), every `tool_use` must get a `tool_result` in a single following user message with matching `tool_use_id`s, and the MCP `isError` result must be forwarded as `is_error`.
- **ten tool turns** — exactly `MAX_TOOL_TURNS` tool calls, then an answer. The client must print the answer and no stop notice: nothing was cut short.
- **endless tool turns** — a tool call on every response. The client must stop after `MAX_TOOL_TURNS` rounds, print `[Stopped after 10 tool-use turns]` once, and make no further call.
Finally the client must exit 0 on `quit`. A client that does one tool round and stops, drops `tools` on the follow-up call, sends one `tool_result` per message, or gets the turn cap wrong passes the no-key test and fails this one.
The Rust client is not covered yet. Its `genai` crate reads no environment variable for the endpoint, and it negotiates protocol `2025-11-25`, under which the mock's array-rooted tool is refused at call time.
## Running Tests
```bash
@ -94,6 +109,22 @@ It advertises two tools whose output schemas cover both shapes a structured resu
node tests/helpers/build/mock-mcp-server.js
```
### tool-loop-test.ts
Runs a client through the scripted tool loop described above. It starts the fake Anthropic API, spawns the client command with `ANTHROPIC_BASE_URL` and a dummy `ANTHROPIC_API_KEY` set, types the three queries and then `quit` at the client's prompts, and reports which checks failed along with the client's output.
**Usage**:
```bash
node tests/helpers/build/tool-loop-test.js <client command> [args...]
```
**Example**:
```bash
node tests/helpers/build/tool-loop-test.js node mcp-client-typescript/build/index.js tests/helpers/build/mock-mcp-server.js
```
## CI/CD Integration
Tests run automatically on pull requests via GitHub Actions. See `.github/workflows/ci.yml` for the CI configuration.