Some checks failed
Smoke Tests / test (push) Has been cancelled
* Fix the tool-use loop in the Go, Rust, and Ruby clients and add a tool-loop smoke test ## Motivation and Context Only the Python and TypeScript clients implemented a correct tool-use loop. The other three had bugs that break ordinary multi-step requests: - **Go, Rust, Ruby** did a single tool round and then called Claude once more *without* `tools`, so any request that needs a second tool call stalled. Go silently dropped `tool_use` blocks from that final response. - **Ruby** called Claude once per `tool_use` block, each time with a single `tool_result`. Parallel tool calls produced a malformed message sequence and an API 400. - **max_tokens** was 1000 in TypeScript and Ruby, 1024 in Go, and unset in Rust. Sonnet 5 thinks adaptively, so 1000 tokens is regularly exhausted mid-thought. Python already used 10000 with a comment explaining why. - **Ruby** never forwarded MCP `isError` as `is_error`, so the model could not tell a failed tool call from a successful one. Rust cannot set the flag either (see below). - **TypeScript** forwarded raw MCP content blocks to Anthropic. The two block shapes differ, so a non-text result would have failed. The existing smoke test never noticed because it only checks that each client calls `tools/list`. None of these bugs was reachable without an API key. ### What changed All five clients now run the same loop: call tools until Claude answers without one, send every `tool_result` of a turn in one user message, pass `tools` on every call, and stop after `MAX_TOOL_TURNS = 10`. `max_tokens` is 10000 everywhere, with the same comment. Python and TypeScript also keep the last response's text when the turn cap is hit; it used to be dropped. Rust keeps the `genai` crate. Its tool-response type has no `is_error` field, so an error result is marked with an `Error:` prefix in the text and the limitation is commented. ### New smoke test `tests/helpers/tool-loop-test.ts` starts a fake Anthropic Messages API on a loopback port, points a client at it through `ANTHROPIC_BASE_URL`, pipes in one query, and checks every request the client sends. It scripts two parallel tool calls, then a call to a tool the mock server lacks (an `isError` result), then a final answer, and asserts that tools are passed each time, `max_tokens` is 10000, all `tool_result`s arrive in one message with matching ids, and the error is forwarded. `smoke-test.sh` runs it for Python, TypeScript, Go, and Ruby. The Rust client is excluded for now: `genai` reads no endpoint variable, and the client negotiates protocol `2025-11-25`, under which the mock's array-rooted tool is refused at call time. Both will be addressed in a follow-up that brings all examples to current SDK versions. ## How Has This Been Tested? - `./tests/smoke-test.sh` passes for every example. - The new tool-loop test passes all 19 checks for Python, TypeScript, Go, and Ruby. - The Rust client was driven through the same scripted conversation with a temporary endpoint resolver (not committed) and passes. - As a negative check, the Ruby client from `main` was run through the new test. It fails 10 of 19 checks, on exactly the bugs listed above. - Go and Ruby were built and run in `golang:1.25` and `ruby:3.4` containers. ## Breaking Changes None. The clients' command lines, output format, and no-key behavior are unchanged. ## Types of changes - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update ## Checklist - [x] I have read the [MCP Documentation](https://modelcontextprotocol.io) - [x] My code follows the repository's style guidelines - [x] New and existing tests pass locally - [x] I have added appropriate error handling - [x] I have added or updated documentation as needed ## Additional context Two pre-existing issues surfaced while testing and are left for a follow-up: - The Rust client (rmcp 3.1.2) negotiates `2025-11-25`. The TypeScript mock server and the Python weather server both reject its calls to array-rooted tools under that revision. Even current rmcp defaults to `2025-11-25`, so the client must request `2026-07-28` explicitly. - The Go SDK's tool-result helper sends content as a list of text blocks rather than a string. The API accepts both, so it is unchanged. AI assistance: the analysis, code changes, test helper, and this description were prepared partially manually, partially with Claude Code and reviewed by me. * Improvements * Address comments
159 lines
4.6 KiB
Shell
159 lines
4.6 KiB
Shell
#!/bin/bash
|
|
# Shared utilities for test scripts
|
|
|
|
# Colors for output
|
|
RED='\033[0;31m'
|
|
GREEN='\033[0;32m'
|
|
YELLOW='\033[1;33m'
|
|
NC='\033[0m' # No Color
|
|
|
|
# Print functions
|
|
print_header() {
|
|
echo -e "${YELLOW}==== $1 ====${NC}"
|
|
}
|
|
|
|
print_success() {
|
|
echo -e "${GREEN}✓ $1${NC}"
|
|
}
|
|
|
|
print_error() {
|
|
echo -e "${RED}✗ $1${NC}"
|
|
}
|
|
|
|
# Get project root directory
|
|
get_project_root() {
|
|
cd "$(dirname "$0")/.." && pwd
|
|
}
|
|
|
|
# Check if command exists
|
|
command_exists() {
|
|
command -v "$1" >/dev/null 2>&1
|
|
}
|
|
|
|
# Check dependency and fail the calling test if not found.
|
|
#
|
|
# Returns rather than exits: a missing toolchain should fail its own test and
|
|
# let the rest of the suite run, not abort the whole script before the summary
|
|
# is printed.
|
|
check_dependency() {
|
|
local cmd=$1
|
|
if ! command_exists "${cmd}"; then
|
|
print_error "${cmd} is not installed"
|
|
return 1
|
|
fi
|
|
}
|
|
|
|
# The executable suffix for this platform: ".exe" under MSYS/Git Bash, empty
|
|
# elsewhere. Windows will not run a binary without it, whatever the file is
|
|
# actually named.
|
|
exe_suffix() {
|
|
case "$(uname -s)" in
|
|
MINGW* | MSYS* | CYGWIN*) echo ".exe" ;;
|
|
*) echo "" ;;
|
|
esac
|
|
}
|
|
|
|
# Echo the path to a built binary, accounting for the .exe suffix on Windows.
|
|
# Prints nothing and returns 1 when neither exists.
|
|
resolve_binary() {
|
|
local path=$1
|
|
if [ -f "${path}$(exe_suffix)" ]; then
|
|
echo "${path}$(exe_suffix)"
|
|
elif [ -f "${path}" ]; then
|
|
echo "${path}"
|
|
else
|
|
return 1
|
|
fi
|
|
}
|
|
|
|
# Setup test environment
|
|
setup_test() {
|
|
local test_name=$1
|
|
print_header "Testing ${test_name}"
|
|
|
|
PROJECT_ROOT=$(get_project_root)
|
|
SERVER_DIR="${PROJECT_ROOT}/${test_name}"
|
|
CLIENT_DIR="${PROJECT_ROOT}/${test_name}"
|
|
TEST_CLIENT="${PROJECT_ROOT}/tests/helpers/build/mcp-test-client.js"
|
|
MOCK_SERVER="${PROJECT_ROOT}/tests/helpers/build/mock-mcp-server.js"
|
|
TOOL_LOOP_TEST="${PROJECT_ROOT}/tests/helpers/build/tool-loop-test.js"
|
|
}
|
|
|
|
# Ensure test helpers are built
|
|
ensure_helpers_built() {
|
|
if [ ! -f "${TEST_CLIENT}" ] || [ ! -f "${MOCK_SERVER}" ] || [ ! -f "${TOOL_LOOP_TEST}" ]; then
|
|
print_error "Test helpers not built"
|
|
print_header "Building test helpers..."
|
|
cd "${PROJECT_ROOT}/tests/helpers" || return 1
|
|
npm install >/dev/null 2>&1
|
|
npm run build >/dev/null 2>&1
|
|
cd - >/dev/null || return 1
|
|
print_success "Test helpers built"
|
|
fi
|
|
}
|
|
|
|
# Run a build step, showing its output only if it fails.
|
|
#
|
|
# Builds used to be run with `>/dev/null 2>&1` unconditionally, which hid
|
|
# compile errors: a failed build left no binary behind and the smoke test
|
|
# reported the downstream `spawn ... ENOENT` instead of the actual error.
|
|
run_build() {
|
|
local what=$1
|
|
shift
|
|
local output
|
|
if ! output=$("$@" 2>&1); then
|
|
print_error "${what} failed"
|
|
echo "${output}"
|
|
return 1
|
|
fi
|
|
}
|
|
|
|
# Ensure a project directory is built (TypeScript/Rust/Go)
|
|
#
|
|
# `bin` is the name of the binary a Rust or Go project produces (unused for
|
|
# TypeScript). Rust names it after the package, so it is looked for under
|
|
# target/; Go is told to write it to `bin` in the project directory.
|
|
#
|
|
# Returns rather than exits, for the same reason check_dependency does: one
|
|
# project failing to build should fail its own test, not abort the suite
|
|
# before the summary is printed.
|
|
ensure_built() {
|
|
local dir=$1
|
|
local bin=$2
|
|
cd "${dir}" || return 1
|
|
|
|
# Install npm dependencies if needed
|
|
if [ -f "package.json" ] && [ ! -d "node_modules" ]; then
|
|
run_build "npm install in ${dir}" npm install || return 1
|
|
fi
|
|
|
|
# Build TypeScript if needed
|
|
if [ -f "tsconfig.json" ] && [ ! -f "build/index.js" ]; then
|
|
run_build "npm run build in ${dir}" npm run build || return 1
|
|
fi
|
|
|
|
# Build Rust if needed
|
|
if [ -f "Cargo.toml" ] \
|
|
&& ! resolve_binary "target/release/${bin}" >/dev/null \
|
|
&& ! resolve_binary "target/debug/${bin}" >/dev/null; then
|
|
run_build "cargo build in ${dir}" cargo build --release || return 1
|
|
fi
|
|
|
|
# Build Go if needed
|
|
if [ -f "go.mod" ] && ! resolve_binary "${bin}" >/dev/null; then
|
|
run_build "go build in ${dir}" go build -o "${bin}$(exe_suffix)" . || return 1
|
|
fi
|
|
}
|
|
|
|
# Ensure a Ruby project directory has its gems installed.
|
|
#
|
|
# Unlike ensure_built there is no artefact to test for -- a Ruby example has no
|
|
# build step and Gemfile.lock is gitignored -- so ask Bundler directly.
|
|
ensure_bundled() {
|
|
local dir=$1
|
|
cd "${dir}" || return 1
|
|
|
|
if ! bundle check >/dev/null 2>&1; then
|
|
run_build "bundle install in ${dir}" bundle install || return 1
|
|
fi
|
|
}
|