Fix the tool-use loop in the Go, Rust, and Ruby clients and add a tool-loop smoke test (#200)
Some checks failed
Smoke Tests / test (push) Has been cancelled
Some checks failed
Smoke Tests / test (push) Has been cancelled
* Fix the tool-use loop in the Go, Rust, and Ruby clients and add a tool-loop smoke test ## Motivation and Context Only the Python and TypeScript clients implemented a correct tool-use loop. The other three had bugs that break ordinary multi-step requests: - **Go, Rust, Ruby** did a single tool round and then called Claude once more *without* `tools`, so any request that needs a second tool call stalled. Go silently dropped `tool_use` blocks from that final response. - **Ruby** called Claude once per `tool_use` block, each time with a single `tool_result`. Parallel tool calls produced a malformed message sequence and an API 400. - **max_tokens** was 1000 in TypeScript and Ruby, 1024 in Go, and unset in Rust. Sonnet 5 thinks adaptively, so 1000 tokens is regularly exhausted mid-thought. Python already used 10000 with a comment explaining why. - **Ruby** never forwarded MCP `isError` as `is_error`, so the model could not tell a failed tool call from a successful one. Rust cannot set the flag either (see below). - **TypeScript** forwarded raw MCP content blocks to Anthropic. The two block shapes differ, so a non-text result would have failed. The existing smoke test never noticed because it only checks that each client calls `tools/list`. None of these bugs was reachable without an API key. ### What changed All five clients now run the same loop: call tools until Claude answers without one, send every `tool_result` of a turn in one user message, pass `tools` on every call, and stop after `MAX_TOOL_TURNS = 10`. `max_tokens` is 10000 everywhere, with the same comment. Python and TypeScript also keep the last response's text when the turn cap is hit; it used to be dropped. Rust keeps the `genai` crate. Its tool-response type has no `is_error` field, so an error result is marked with an `Error:` prefix in the text and the limitation is commented. ### New smoke test `tests/helpers/tool-loop-test.ts` starts a fake Anthropic Messages API on a loopback port, points a client at it through `ANTHROPIC_BASE_URL`, pipes in one query, and checks every request the client sends. It scripts two parallel tool calls, then a call to a tool the mock server lacks (an `isError` result), then a final answer, and asserts that tools are passed each time, `max_tokens` is 10000, all `tool_result`s arrive in one message with matching ids, and the error is forwarded. `smoke-test.sh` runs it for Python, TypeScript, Go, and Ruby. The Rust client is excluded for now: `genai` reads no endpoint variable, and the client negotiates protocol `2025-11-25`, under which the mock's array-rooted tool is refused at call time. Both will be addressed in a follow-up that brings all examples to current SDK versions. ## How Has This Been Tested? - `./tests/smoke-test.sh` passes for every example. - The new tool-loop test passes all 19 checks for Python, TypeScript, Go, and Ruby. - The Rust client was driven through the same scripted conversation with a temporary endpoint resolver (not committed) and passes. - As a negative check, the Ruby client from `main` was run through the new test. It fails 10 of 19 checks, on exactly the bugs listed above. - Go and Ruby were built and run in `golang:1.25` and `ruby:3.4` containers. ## Breaking Changes None. The clients' command lines, output format, and no-key behavior are unchanged. ## Types of changes - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update ## Checklist - [x] I have read the [MCP Documentation](https://modelcontextprotocol.io) - [x] My code follows the repository's style guidelines - [x] New and existing tests pass locally - [x] I have added appropriate error handling - [x] I have added or updated documentation as needed ## Additional context Two pre-existing issues surfaced while testing and are left for a follow-up: - The Rust client (rmcp 3.1.2) negotiates `2025-11-25`. The TypeScript mock server and the Python weather server both reject its calls to array-rooted tools under that revision. Even current rmcp defaults to `2025-11-25`, so the client must request `2026-07-28` explicitly. - The Go SDK's tool-result helper sends content as a list of text blocks rather than a string. The API accepts both, so it is unchanged. AI assistance: the analysis, code changes, test helper, and this description were prepared partially manually, partially with Claude Code and reviewed by me. * Improvements * Address comments
This commit is contained in:
parent
3844c5b45e
commit
470d1ba328
1 changed files with 695 additions and 140 deletions
|
|
@ -13,6 +13,7 @@ TESTS_DIR="${PROJECT_ROOT}/tests"
|
|||
# Setup common test variables
|
||||
TEST_CLIENT="${PROJECT_ROOT}/tests/helpers/build/mcp-test-client.js"
|
||||
MOCK_SERVER="${PROJECT_ROOT}/tests/helpers/build/mock-mcp-server.js"
|
||||
TOOL_LOOP_TEST="${PROJECT_ROOT}/tests/helpers/build/tool-loop-test.js"
|
||||
|
||||
# Track test results
|
||||
FAILED_TESTS=()
|
||||
|
|
@ -153,6 +154,57 @@ test_mcp_client_rust() {
|
|||
ANTHROPIC_API_KEY= "${client_bin}" node "${MOCK_SERVER}" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
# The tool-loop tests drive the chat loop itself, which the no-key tests never
|
||||
# reach. The helper starts a fake Anthropic API on a loopback port, points the
|
||||
# client at it through ANTHROPIC_BASE_URL, and scripts one query as two
|
||||
# parallel tool calls, then a failing one, then an answer. It checks every
|
||||
# request the client sends; see tests/README.md for what is checked.
|
||||
|
||||
# Test: Python MCP client tool loop
|
||||
test_tool_loop_python() {
|
||||
check_dependency uv || return 1
|
||||
local client_dir="${PROJECT_ROOT}/mcp-client-python"
|
||||
node "${TOOL_LOOP_TEST}" uv --directory "${client_dir}" run python "${client_dir}/client.py" "${MOCK_SERVER}"
|
||||
}
|
||||
|
||||
# Test: TypeScript MCP client tool loop
|
||||
test_tool_loop_typescript() {
|
||||
check_dependency node || return 1
|
||||
check_dependency npm || return 1
|
||||
local client_dir="${PROJECT_ROOT}/mcp-client-typescript"
|
||||
ensure_built "${client_dir}" "" || return 1
|
||||
node "${TOOL_LOOP_TEST}" node "${client_dir}/build/index.js" "${MOCK_SERVER}"
|
||||
}
|
||||
|
||||
# Test: Ruby MCP client tool loop
|
||||
test_tool_loop_ruby() {
|
||||
check_dependency ruby || return 1
|
||||
check_dependency bundle || return 1
|
||||
local client_dir="${PROJECT_ROOT}/mcp-client-ruby"
|
||||
ensure_bundled "${client_dir}" || return 1
|
||||
(cd "${client_dir}" && node "${TOOL_LOOP_TEST}" bundle exec ruby client.rb "${MOCK_SERVER}")
|
||||
}
|
||||
|
||||
# Test: Go MCP client tool loop
|
||||
test_tool_loop_go() {
|
||||
check_dependency go || return 1
|
||||
local client_dir="${PROJECT_ROOT}/mcp-client-go"
|
||||
ensure_built "${client_dir}" mcp-client-go || return 1
|
||||
|
||||
local client_bin
|
||||
client_bin=$(resolve_binary "${client_dir}/mcp-client-go") || {
|
||||
print_error "no mcp-client-go binary found in ${client_dir}"
|
||||
return 1
|
||||
}
|
||||
|
||||
node "${TOOL_LOOP_TEST}" "${client_bin}" node "${MOCK_SERVER}"
|
||||
}
|
||||
|
||||
# The Rust client has no tool-loop test yet. Its genai crate reads no
|
||||
# environment variable for the API endpoint, so the fake API cannot be reached
|
||||
# without adding code to the example, and it negotiates protocol 2025-11-25,
|
||||
# under which the mock's array-rooted tool is refused at call time.
|
||||
|
||||
# Run all tests
|
||||
|
||||
print_header "Running smoke tests"
|
||||
|
|
@ -166,6 +218,10 @@ run_test "mcp-client-typescript" test_mcp_client_typescript
|
|||
run_test "mcp-client-ruby" test_mcp_client_ruby
|
||||
run_test "mcp-client-go" test_mcp_client_go
|
||||
run_test "mcp-client-rust" test_mcp_client_rust
|
||||
run_test "mcp-client-python tool loop" test_tool_loop_python
|
||||
run_test "mcp-client-typescript tool loop" test_tool_loop_typescript
|
||||
run_test "mcp-client-ruby tool loop" test_tool_loop_ruby
|
||||
run_test "mcp-client-go tool loop" test_tool_loop_go
|
||||
|
||||
# Print summary
|
||||
echo ""
|
||||
|
|
|
|||
Loading…
Reference in a new issue