quickstart-resources/tests/utils.sh
Ola Hungerford f587167452
Assert on structured output in the smoke tests and update to v2 (#163)
* Assert on structured output in the smoke tests

The smoke tests connected to each server, listed its tools and stopped there,
so they could not catch a broken structured result.

mcp-test-client now calls every tool that declares an outputSchema and checks
the answer against it: the result must carry structuredContent, and a tool
with an array-rooted schema must return a top-level JSON array rather than an
object wrapping one. Tools with no output schema are left alone, so this is
not a new requirement on any example. The client negotiates with mode "auto",
so it exercises servers of either protocol era. Tool calls reach the live NWS
API; an error result is reported as a skip rather than a failure.

mock-mcp-server moves to the low-level Server and advertises one array-rooted
and one object-rooted tool, with the schemas written out literally. A client
that compiles every declared outputSchema at connect time fails against the
array-rooted one if it assumes output schemas are always objects.

utils.sh no longer swallows build output. A failed build left no binary behind
and the test reported a downstream spawn ENOENT instead of the compiler error,
which is what makes a broken build hard to diagnose from CI logs; #143 is the
current example. check_dependency returns instead of exiting, so a missing
toolchain fails its own test and the suite still prints its summary. Binaries
resolve through a helper that accounts for the .exe suffix.

The test helpers move to the 2.0 MCP SDK packages, where the 2026-07-28
support lives; the examples themselves are untouched. Those require Node 20 or
newer, so CI moves off Node 18 to Node 24, the Active LTS - Node 20 reached
end of life in April 2026.

The Go examples and the Rust client stay uncovered here. Both clients abort
when no .env file is present, so that coverage lands with those examples.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Update to the stable 2.0 SDK packages and address review feedback

@modelcontextprotocol/{client,server} 2.0.0 shipped on 2026-07-27, so the
helpers move off 2.0.0-beta.5.

Review fixes:
- strip the UTF-8 BOM that preceded smoke-test.sh's shebang
- ensure_built and ensure_helpers_built return rather than exit, so one
  project failing to build no longer aborts the suite before the summary
- engines.node and the documented Node requirement go to 20, matching what
  the 2.0 SDK packages actually require
- close the client on the failure path, so a failing test cannot leave the
  spawned server running
- the README claimed Go and Rust client coverage that did not exist

Adds the Go weather server to the smoke test, with the Go toolchain CI was
missing. The Go and Rust clients stay uncovered: on main both abort without
a .env, so credential-free startup has to land in their own directories.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 19:42:57 +02:00

140 lines
3.8 KiB
Shell

#!/bin/bash
# Shared utilities for test scripts
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Print functions
print_header() {
echo -e "${YELLOW}==== $1 ====${NC}"
}
print_success() {
echo -e "${GREEN}✓ $1${NC}"
}
print_error() {
echo -e "${RED}✗ $1${NC}"
}
# Get project root directory
get_project_root() {
cd "$(dirname "$0")/.." && pwd
}
# Check if command exists
command_exists() {
command -v "$1" >/dev/null 2>&1
}
# Check dependency and fail the calling test if not found.
#
# Returns rather than exits: a missing toolchain should fail its own test and
# let the rest of the suite run, not abort the whole script before the summary
# is printed.
check_dependency() {
local cmd=$1
if ! command_exists "${cmd}"; then
print_error "${cmd} is not installed"
return 1
fi
}
# The executable suffix for this platform: ".exe" under MSYS/Git Bash, empty
# elsewhere. Windows will not run a binary without it, whatever the file is
# actually named.
exe_suffix() {
case "$(uname -s)" in
MINGW* | MSYS* | CYGWIN*) echo ".exe" ;;
*) echo "" ;;
esac
}
# Echo the path to a built binary, accounting for the .exe suffix on Windows.
# Prints nothing and returns 1 when neither exists.
resolve_binary() {
local path=$1
if [ -f "${path}$(exe_suffix)" ]; then
echo "${path}$(exe_suffix)"
elif [ -f "${path}" ]; then
echo "${path}"
else
return 1
fi
}
# Setup test environment
setup_test() {
local test_name=$1
print_header "Testing ${test_name}"
PROJECT_ROOT=$(get_project_root)
SERVER_DIR="${PROJECT_ROOT}/${test_name}"
CLIENT_DIR="${PROJECT_ROOT}/${test_name}"
TEST_CLIENT="${PROJECT_ROOT}/tests/helpers/build/mcp-test-client.js"
MOCK_SERVER="${PROJECT_ROOT}/tests/helpers/build/mock-mcp-server.js"
}
# Ensure test helpers are built
ensure_helpers_built() {
if [ ! -f "${TEST_CLIENT}" ] || [ ! -f "${MOCK_SERVER}" ]; then
print_error "Test helpers not built"
print_header "Building test helpers..."
cd "${PROJECT_ROOT}/tests/helpers" || return 1
npm install >/dev/null 2>&1
npm run build >/dev/null 2>&1
cd - >/dev/null || return 1
print_success "Test helpers built"
fi
}
# Run a build step, showing its output only if it fails.
#
# Builds used to be run with `>/dev/null 2>&1` unconditionally, which hid
# compile errors: a failed build left no binary behind and the smoke test
# reported the downstream `spawn ... ENOENT` instead of the actual error.
run_build() {
local what=$1
shift
local output
if ! output=$("$@" 2>&1); then
print_error "${what} failed"
echo "${output}"
return 1
fi
}
# Ensure a project directory is built (TypeScript/Rust/Go)
#
# Returns rather than exits, for the same reason check_dependency does: one
# project failing to build should fail its own test, not abort the suite
# before the summary is printed.
ensure_built() {
local dir=$1
cd "${dir}" || return 1
# Install npm dependencies if needed
if [ -f "package.json" ] && [ ! -d "node_modules" ]; then
run_build "npm install in ${dir}" npm install || return 1
fi
# Build TypeScript if needed
if [ -f "tsconfig.json" ] && [ ! -f "build/index.js" ]; then
run_build "npm run build in ${dir}" npm run build || return 1
fi
# Build Rust if needed
if [ -f "Cargo.toml" ] \
&& ! resolve_binary "target/release/weather" >/dev/null \
&& ! resolve_binary "target/debug/weather" >/dev/null; then
run_build "cargo build in ${dir}" cargo build --release || return 1
fi
# Build Go if needed
if [ -f "go.mod" ] && ! resolve_binary "server" >/dev/null; then
run_build "go build in ${dir}" go build -o "server$(exe_suffix)" . || return 1
fi
}