* Assert on structured output in the smoke tests The smoke tests connected to each server, listed its tools and stopped there, so they could not catch a broken structured result. mcp-test-client now calls every tool that declares an outputSchema and checks the answer against it: the result must carry structuredContent, and a tool with an array-rooted schema must return a top-level JSON array rather than an object wrapping one. Tools with no output schema are left alone, so this is not a new requirement on any example. The client negotiates with mode "auto", so it exercises servers of either protocol era. Tool calls reach the live NWS API; an error result is reported as a skip rather than a failure. mock-mcp-server moves to the low-level Server and advertises one array-rooted and one object-rooted tool, with the schemas written out literally. A client that compiles every declared outputSchema at connect time fails against the array-rooted one if it assumes output schemas are always objects. utils.sh no longer swallows build output. A failed build left no binary behind and the test reported a downstream spawn ENOENT instead of the compiler error, which is what makes a broken build hard to diagnose from CI logs; #143 is the current example. check_dependency returns instead of exiting, so a missing toolchain fails its own test and the suite still prints its summary. Binaries resolve through a helper that accounts for the .exe suffix. The test helpers move to the 2.0 MCP SDK packages, where the 2026-07-28 support lives; the examples themselves are untouched. Those require Node 20 or newer, so CI moves off Node 18 to Node 24, the Active LTS - Node 20 reached end of life in April 2026. The Go examples and the Rust client stay uncovered here. Both clients abort when no .env file is present, so that coverage lands with those examples. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Update to the stable 2.0 SDK packages and address review feedback @modelcontextprotocol/{client,server} 2.0.0 shipped on 2026-07-27, so the helpers move off 2.0.0-beta.5. Review fixes: - strip the UTF-8 BOM that preceded smoke-test.sh's shebang - ensure_built and ensure_helpers_built return rather than exit, so one project failing to build no longer aborts the suite before the summary - engines.node and the documented Node requirement go to 20, matching what the 2.0 SDK packages actually require - close the client on the failure path, so a failing test cannot leave the spawned server running - the README claimed Go and Rust client coverage that did not exist Adds the Go weather server to the smoke test, with the Go toolchain CI was missing. The Go and Rust clients stay uncovered: on main both abort without a .env, so credential-free startup has to land in their own directories. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
138 lines
4 KiB
Shell
Executable file
138 lines
4 KiB
Shell
Executable file
#!/bin/bash
|
|
set -e
|
|
|
|
# Source utilities
|
|
source "$(dirname "$0")/utils.sh"
|
|
|
|
print_header "MCP Quickstart Smoke Tests"
|
|
|
|
# Get project root
|
|
PROJECT_ROOT=$(get_project_root)
|
|
TESTS_DIR="${PROJECT_ROOT}/tests"
|
|
|
|
# Setup common test variables
|
|
TEST_CLIENT="${PROJECT_ROOT}/tests/helpers/build/mcp-test-client.js"
|
|
MOCK_SERVER="${PROJECT_ROOT}/tests/helpers/build/mock-mcp-server.js"
|
|
|
|
# Track test results
|
|
FAILED_TESTS=()
|
|
PASSED_TESTS=()
|
|
|
|
# Build test helpers
|
|
ensure_helpers_built
|
|
|
|
# Helper function to run a test and track results
|
|
run_test() {
|
|
local test_name=$1
|
|
echo ""
|
|
print_header "Testing ${test_name}"
|
|
|
|
if "${@:2}"; then
|
|
PASSED_TESTS+=("${test_name}")
|
|
print_success "${test_name} test passed"
|
|
else
|
|
FAILED_TESTS+=("${test_name}")
|
|
print_error "${test_name} test failed"
|
|
fi
|
|
}
|
|
|
|
# Test: Python weather server
|
|
test_weather_server_python() {
|
|
check_dependency uv || return 1
|
|
local server_dir="${PROJECT_ROOT}/weather-server-python"
|
|
node "${TEST_CLIENT}" uv --directory "${server_dir}" run weather.py
|
|
}
|
|
|
|
# Test: TypeScript weather server
|
|
test_weather_server_typescript() {
|
|
check_dependency node || return 1
|
|
check_dependency npm || return 1
|
|
local server_dir="${PROJECT_ROOT}/weather-server-typescript"
|
|
ensure_built "${server_dir}" || return 1
|
|
node "${TEST_CLIENT}" node "${server_dir}/build/index.js"
|
|
}
|
|
|
|
# Test: Rust weather server
|
|
test_weather_server_rust() {
|
|
check_dependency cargo || return 1
|
|
local server_dir="${PROJECT_ROOT}/weather-server-rust"
|
|
ensure_built "${server_dir}" || return 1
|
|
|
|
# Determine which binary to use
|
|
local server_bin
|
|
server_bin=$(resolve_binary "${server_dir}/target/release/weather" \
|
|
|| resolve_binary "${server_dir}/target/debug/weather") || {
|
|
print_error "no weather binary found in ${server_dir}/target"
|
|
return 1
|
|
}
|
|
|
|
node "${TEST_CLIENT}" "${server_bin}"
|
|
}
|
|
|
|
# Test: Go weather server
|
|
test_weather_server_go() {
|
|
check_dependency go || return 1
|
|
local server_dir="${PROJECT_ROOT}/weather-server-go"
|
|
ensure_built "${server_dir}" || return 1
|
|
|
|
local server_bin
|
|
server_bin=$(resolve_binary "${server_dir}/server") || {
|
|
print_error "no server binary found in ${server_dir}"
|
|
return 1
|
|
}
|
|
|
|
node "${TEST_CLIENT}" "${server_bin}"
|
|
}
|
|
|
|
# Test: Python MCP client
|
|
test_mcp_client_python() {
|
|
check_dependency uv || return 1
|
|
local client_dir="${PROJECT_ROOT}/mcp-client-python"
|
|
uv --directory "${client_dir}" run python "${client_dir}/client.py" "${MOCK_SERVER}" >/dev/null 2>&1
|
|
}
|
|
|
|
# Test: TypeScript MCP client
|
|
test_mcp_client_typescript() {
|
|
check_dependency node || return 1
|
|
check_dependency npm || return 1
|
|
local client_dir="${PROJECT_ROOT}/mcp-client-typescript"
|
|
ensure_built "${client_dir}" || return 1
|
|
node "${client_dir}/build/index.js" "${MOCK_SERVER}" >/dev/null 2>&1
|
|
}
|
|
|
|
# Run all tests
|
|
#
|
|
# All four servers are covered. The Go and Rust clients are not: on main both
|
|
# abort when no .env file is present, so they cannot be driven without
|
|
# credentials. Making them start credential-free is a change in their own
|
|
# directories, so their coverage lands with those changes rather than here.
|
|
print_header "Running smoke tests"
|
|
run_test "weather-server-python" test_weather_server_python
|
|
run_test "weather-server-typescript" test_weather_server_typescript
|
|
run_test "weather-server-rust" test_weather_server_rust
|
|
run_test "weather-server-go" test_weather_server_go
|
|
run_test "mcp-client-python" test_mcp_client_python
|
|
run_test "mcp-client-typescript" test_mcp_client_typescript
|
|
|
|
# Print summary
|
|
echo ""
|
|
print_header "Test Summary"
|
|
echo "Passed: ${#PASSED_TESTS[@]}"
|
|
for test in "${PASSED_TESTS[@]}"; do
|
|
print_success "${test}"
|
|
done
|
|
|
|
if [ ${#FAILED_TESTS[@]} -gt 0 ]; then
|
|
echo ""
|
|
echo "Failed: ${#FAILED_TESTS[@]}"
|
|
for test in "${FAILED_TESTS[@]}"; do
|
|
print_error "${test}"
|
|
done
|
|
echo ""
|
|
print_error "Some tests failed"
|
|
exit 1
|
|
else
|
|
echo ""
|
|
print_success "All tests passed!"
|
|
exit 0
|
|
fi
|