Most MCP servers get a handful of assert result == ... tests bolted on and nothing more. That misses the failures that actually bite: a tool quietly renamed, a parameter that changed type, a result that grew a field, an error that stopped being an error. A server is a contract a model depends on, and the contract needs its own tests.

This tutorial builds a small, deterministic FastMCP server and then a layered test suite around it with four kinds of tests, each catching a different class of regression:

  1. Contract tests pin the public interface — the exact tool set and each tool’s input schema.
  2. Conformance tests assert invariants that must hold for every tool, and validate real results against each tool’s advertised outputSchema.
  3. Snapshot tests freeze rich structured results in golden files so an unnoticed change fails loudly.
  4. Fixtures (an in-memory client, a snapshot helper) keep all of the above short and fast.

The stack is Mac-native: uv, make, and pytest.

The idea: test the contract, not just the math

Two questions have different answers and deserve different tests:

  • Does this tool compute the right value? Assert it directly, or snapshot it.
  • Is this tool still the same tool the client discovered yesterday? That is the contract: names, schemas, annotations, error behavior. It breaks silently during refactors and is what a conformance test defends.

The trick that keeps conformance tests useful as a server grows is to write them against the tool list at runtime, not a hardcoded name. A rule like “every tool has a description” then protects tools you have not written yet.

What you will build

  • server.py: the server under test — two tools with structured outputs, a resource, and a prompt, all pure and deterministic.
  • tests/conftest.py: a client fixture and a snapshot fixture.
  • tests/test_contract.py: contract and conformance tests.
  • tests/test_snapshots.py: golden-file snapshots plus explicit edge cases.
  • A make wrapper whose snapshots target regenerates the golden files.

Prerequisites

  • macOS 13+ with Homebrew (brew.sh).
  • uv 0.5+ — brew install uv; verify uv --version.
  • Xcode Command Line Tools (xcode-select --install) for make.
  • Familiarity with FastMCP tools and the in-memory Client (see Build an MCP Server with FastMCP). Structured output is covered in Return Structured Output from a FastMCP Server.

Step 1: Add project hygiene

Create the file

mkdir -p testing-strategy-mcp-macos
cd testing-strategy-mcp-macos
touch .gitignore

Add the code: .gitignore

# Python
__pycache__/
*.py[cod]
.venv/
.uv/
.pytest_cache/
.ruff_cache/
.mypy_cache/
*.log
# OS / editor noise
.DS_Store

Detailed breakdown

  • Standard Python ignores plus .DS_Store, created first so the virtualenv and caches never get committed. Note that tests/snapshots/ is not ignored — the golden files are committed artifacts the suite compares against.

Step 2: Initialize the project with uv

Create the files

uv init --name macmcp --no-workspace
rm -f main.py hello.py
uv add fastmcp
uv add --dev pytest pytest-asyncio jsonschema

Add the code: pyproject.toml

[project]
name = "macmcp"
version = "0.1.0"
description = "A testing strategy for MCP servers with FastMCP and pytest"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "fastmcp>=3.4.4",
]

[dependency-groups]
dev = [
    "jsonschema>=4.26.0",
    "pytest>=9.1.1",
    "pytest-asyncio>=1.4.0",
]

Detailed breakdown

  • jsonschema is a dev dependency: it validates tool results against the outputSchema the server advertises, and only the tests need it. FastMCP already validates outputs internally, but checking again from the outside proves the advertised contract is what a real client would enforce.

Step 3: The server under test

Keep the server pure so results are a function of inputs alone. No clock, no randomness, no I/O means no flaky tests and no need to mock anything.

Create the file

touch server.py

Add the code: server.py

"""The server under test.

A small, deterministic FastMCP server used to demonstrate a testing method: two
tools with structured (Pydantic) outputs, one resource, and one prompt. It is
intentionally pure — no clock, no randomness, no I/O — so every result is a
function of its inputs and the tests can assert exact values.
"""

from fastmcp import FastMCP
from fastmcp.exceptions import ToolError
from mcp.types import ToolAnnotations
from pydantic import BaseModel, Field

mcp = FastMCP(name="text-tools")

READ_ONLY = ToolAnnotations(readOnlyHint=True, idempotentHint=True, openWorldHint=False)


class WordStats(BaseModel):
    """Structured result for analyze_text."""

    words: int = Field(description="Total whitespace-separated tokens")
    characters: int = Field(description="Character count including spaces")
    unique_words: int = Field(description="Distinct tokens, case-insensitive")


@mcp.tool(annotations=READ_ONLY)
def analyze_text(text: str) -> WordStats:
    """Count words, characters, and distinct words in a string."""
    tokens = text.split()
    return WordStats(
        words=len(tokens),
        characters=len(text),
        unique_words=len({t.lower() for t in tokens}),
    )


@mcp.tool(annotations=READ_ONLY)
def to_celsius(fahrenheit: float) -> float:
    """Convert a Fahrenheit temperature to Celsius, rounded to 2 decimals."""
    if fahrenheit < -459.67:
        raise ToolError("temperature below absolute zero")
    return round((fahrenheit - 32) * 5 / 9, 2)


@mcp.resource("catalog://info")
def catalog_info() -> str:
    """A short description of what this server offers."""
    return "text-tools: analyze_text, to_celsius"


@mcp.prompt
def summarize(text: str) -> str:
    """Build a prompt that asks a model to summarize the given text."""
    return f"Summarize the following in one sentence:\n\n{text}"


if __name__ == "__main__":
    mcp.run()

Detailed breakdown

  • analyze_text returns a WordStats model, so FastMCP advertises an outputSchema and returns structuredContent. That structured result is exactly what the conformance and snapshot tests inspect.
  • to_celsius returns a bare float, which FastMCP wraps under a result key (with x-fastmcp-wrap-result: true in the schema). Its guard raises a ToolError below absolute zero — an error path the suite tests explicitly.
  • The ToolAnnotations are part of the contract too; a conformance test checks they are present. The resource and prompt round out a realistic surface.

Step 4: Fixtures — a client and a snapshot helper

Every test needs a connected client, and the snapshot tests need a golden-file comparator. Both live in conftest.py so pytest injects them by name.

Create the file

mkdir -p tests/snapshots
touch tests/__init__.py
touch tests/conftest.py

Add the code: tests/conftest.py

"""Shared fixtures for the MCP test suite.

Two fixtures do most of the work:

- `client` opens the server over the in-memory transport once per test, so no test
  spins up its own connection or reaches the network.
- `snapshot` compares a value against a committed golden JSON file, and rewrites it
  when `SNAPSHOT_UPDATE=1` — the pattern behind tool-result snapshot tests.
"""

import json
import os
from pathlib import Path

import pytest
import pytest_asyncio
from fastmcp import Client

from server import mcp

SNAPSHOT_DIR = Path(__file__).parent / "snapshots"


@pytest_asyncio.fixture
async def client():
    """An in-memory client connected to the server under test."""
    async with Client(mcp) as c:
        yield c


@pytest.fixture
def snapshot():
    """Assert a JSON-serializable value equals its committed snapshot.

    Set `SNAPSHOT_UPDATE=1` to (re)write snapshots after an intentional change,
    then review the diff before committing.
    """
    update = os.environ.get("SNAPSHOT_UPDATE") == "1"

    def assert_match(name: str, value) -> None:
        path = SNAPSHOT_DIR / f"{name}.json"
        serialized = json.dumps(value, indent=2, sort_keys=True)
        if update or not path.exists():
            path.write_text(serialized + "\n")
            return
        expected = path.read_text().rstrip("\n")
        assert serialized == expected, (
            f"snapshot '{name}' changed.\n"
            f"--- expected ---\n{expected}\n--- actual ---\n{serialized}\n"
            f"If intended, re-run with SNAPSHOT_UPDATE=1."
        )

    return assert_match

Detailed breakdown

  • client is an async fixture (pytest_asyncio.fixture) that opens one in-memory Client per test and tears it down after. Tests just declare a client parameter; there is no connection boilerplate anywhere else.
  • snapshot returns a comparator. json.dumps(..., sort_keys=True) makes the serialization stable regardless of key order, so a snapshot only changes when the data changes. Missing file or SNAPSHOT_UPDATE=1 writes the golden file; otherwise it compares and fails with a readable diff.
  • The snapshots/ directory is committed, so a fresh clone has the golden files to compare against. The first run generates them; every run after that guards them.

Step 5: Contract and conformance tests

Create the file

touch tests/test_contract.py

Add the code: tests/test_contract.py

"""Contract and spec-conformance tests.

These do not check *what a tool computes* — the snapshot tests do that. They check
that the server's advertised interface is stable and well-formed:

- **Contract**: the exact set of tools, and each tool's input schema, so an
  accidental rename or a changed parameter type fails loudly.
- **Conformance**: generic invariants that must hold for *every* tool (a
  description, an input schema, annotations) plus validation of real results
  against each tool's own advertised `outputSchema`.

The conformance tests are written against the tool list, not hardcoded names, so
they keep protecting new tools as the server grows.
"""

import jsonschema
import pytest

# --- Contract: pin the public interface ---------------------------------------


async def test_tool_set_is_stable(client):
    names = {t.name for t in await client.list_tools()}
    assert names == {"analyze_text", "to_celsius"}


async def test_analyze_text_input_contract(client):
    tools = {t.name: t for t in await client.list_tools()}
    schema = tools["analyze_text"].inputSchema
    assert schema["required"] == ["text"]
    assert schema["properties"]["text"]["type"] == "string"


async def test_to_celsius_input_contract(client):
    tools = {t.name: t for t in await client.list_tools()}
    schema = tools["to_celsius"].inputSchema
    assert schema["required"] == ["fahrenheit"]
    assert schema["properties"]["fahrenheit"]["type"] == "number"


# --- Conformance: invariants that hold for every tool -------------------------


async def test_every_tool_has_a_description_and_schema(client):
    for tool in await client.list_tools():
        assert tool.description, f"{tool.name} has no description"
        assert tool.inputSchema.get("type") == "object", f"{tool.name} input not object"


async def test_every_tool_names_snake_case(client):
    for tool in await client.list_tools():
        assert tool.name.islower() and " " not in tool.name
        assert tool.name.replace("_", "").isalnum()


@pytest.mark.parametrize(
    "name,args",
    [
        ("analyze_text", {"text": "hello world"}),
        ("to_celsius", {"fahrenheit": 98.6}),
    ],
)
async def test_results_conform_to_output_schema(client, name, args):
    """Every tool result must validate against the schema the tool advertises."""
    tools = {t.name: t for t in await client.list_tools()}
    out_schema = tools[name].outputSchema
    result = await client.call_tool(name, args)
    jsonschema.validate(instance=result.structured_content, schema=out_schema)

Detailed breakdown

  • test_tool_set_is_stable is the single most valuable test here: a rename, an accidental deletion, or an extra tool leaking in all fail it immediately.
  • The two input-contract tests pin the required parameters and their types, so a change from float to str, or a newly required argument, is caught.
  • test_every_tool_has_a_description_and_schema and test_every_tool_names_snake_case iterate the live tool list. Add a tool with no docstring or a CamelCase name and these fail without being edited — that is the point of writing them against the runtime list.
  • test_results_conform_to_output_schema calls each tool and validates the real structured_content against the outputSchema the server published, with jsonschema. It is parametrized, so one function covers every tool.

Step 6: Snapshot tests and explicit edge cases

Create the file

touch tests/test_snapshots.py

Add the code: tests/test_snapshots.py

"""Tool-result snapshot tests.

Snapshots capture the exact shape and values a tool returns and compare them to a
committed golden file. They catch regressions you did not think to assert by hand:
a reordered field, a changed rounding rule, a newly added key. When a change is
intentional, regenerate with `SNAPSHOT_UPDATE=1` and review the diff.

Prefer snapshots for rich, structured results; prefer explicit assertions (below)
for a single scalar or a specific edge case you want to document.
"""

import pytest

# --- Snapshots for structured results -----------------------------------------


async def test_analyze_text_snapshot(client, snapshot):
    result = await client.call_tool(
        "analyze_text", {"text": "the cat sat on the mat"}
    )
    snapshot("analyze_text_basic", result.structured_content)


async def test_analyze_text_unicode_snapshot(client, snapshot):
    result = await client.call_tool("analyze_text", {"text": "café Café CAFÉ"})
    snapshot("analyze_text_unicode", result.structured_content)


# --- Explicit assertions for scalars and edge cases ---------------------------


@pytest.mark.parametrize(
    "fahrenheit,expected",
    [
        (32.0, 0.0),
        (212.0, 100.0),
        (98.6, 37.0),
        (-40.0, -40.0),
    ],
)
async def test_to_celsius_values(client, fahrenheit, expected):
    result = await client.call_tool("to_celsius", {"fahrenheit": fahrenheit})
    assert result.data == expected


async def test_to_celsius_rejects_below_absolute_zero(client):
    from fastmcp.exceptions import ToolError

    with pytest.raises(ToolError, match="absolute zero"):
        await client.call_tool("to_celsius", {"fahrenheit": -500})


async def test_empty_text_is_all_zeros(client):
    result = await client.call_tool("analyze_text", {"text": ""})
    assert result.structured_content == {
        "words": 0,
        "characters": 0,
        "unique_words": 0,
    }

Detailed breakdown

  • The two snapshot tests capture the full structured_content for a normal and a Unicode input. The Unicode case (café Café CAFÉ) documents real behavior: three words, one unique word (case-insensitive), and 14 characters because the precomposed é counts as one code point.
  • test_to_celsius_values uses result.data, the typed return value FastMCP parses from the wrapped result, so the assertion is against 100.0, not {"result": 100.0}. Parametrizing documents four reference conversions in one function.
  • test_to_celsius_rejects_below_absolute_zero pins the error path: a ToolError whose message matches absolute zero. Error contracts regress as easily as success ones.
  • test_empty_text_is_all_zeros is the boundary case an LLM will eventually send; asserting it explicitly keeps the edge documented next to the snapshots.

Step 7: Wrap it in a Makefile

Create the file

touch Makefile

Add the code: Makefile

.DEFAULT_GOAL := help

.PHONY: help install demo serve test test-v snapshots clean

help:  ## Show this help screen
	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) \
		| awk 'BEGIN {FS = ":.*?## "}; {printf "  \033[36m%-12s\033[0m %s\n", $$1, $$2}'

install:  ## Sync runtime and dev dependencies
	uv sync

demo:  ## Smoke-run the server under test
	uv run python client.py

serve:  ## Run the server over stdio (for a real MCP client)
	uv run python server.py

test:  ## Run the full test suite
	uv run pytest -q

test-v:  ## Run the suite verbosely
	uv run pytest -v

snapshots:  ## Regenerate committed snapshots (review the diff before committing)
	SNAPSHOT_UPDATE=1 uv run pytest -q tests/test_snapshots.py

clean:  ## Remove caches
	rm -rf .pytest_cache __pycache__ tests/__pycache__

Detailed breakdown

  • .DEFAULT_GOAL := help makes a bare make print the help screen. Recipe bodies must be tab-indented or make errors.
  • make snapshots is the deliberate escape hatch: after an intentional change to a tool’s output, regenerate the golden files, then read the git diff before committing so a real regression can never be rubber-stamped.

Step 8: The smoke demo (optional but useful)

A tiny client.py gives a human-readable look at the server the tests exercise.

Create the file

touch client.py

Add the code: client.py

"""Drive the server under test in-memory.

A quick smoke run of the same server the test suite exercises: list the tools,
call each one, and read the resource. The real coverage lives in `tests/`; this is
just a human-readable look at what the tools return.
"""

import asyncio

from fastmcp import Client

from server import mcp


async def main() -> None:
    async with Client(mcp) as client:
        print("tools:    ", [t.name for t in await client.list_tools()])
        print("resources:", [str(r.uri) for r in await client.list_resources()])
        print("prompts:  ", [p.name for p in await client.list_prompts()])

        r = await client.call_tool("analyze_text", {"text": "the cat sat on the mat"})
        print("\nanalyze_text('the cat sat on the mat'):")
        print("  ", r.structured_content)

        r = await client.call_tool("to_celsius", {"fahrenheit": 98.6})
        print("to_celsius(98.6):")
        print("  ", r.data)

        print("read catalog://info:")
        print("  ", (await client.read_resource("catalog://info"))[0].text)


if __name__ == "__main__":
    asyncio.run(main())

Detailed breakdown

  • This is not a test; it is a quick way to eyeball the surface. The suite is the source of truth, but a smoke script is handy when adding a tool.

Run it:

uv run python client.py

Expected output:

tools:     ['analyze_text', 'to_celsius']
resources: ['catalog://info']
prompts:   ['summarize']

analyze_text('the cat sat on the mat'):
   {'words': 6, 'characters': 22, 'unique_words': 5}
to_celsius(98.6):
   37.0
read catalog://info:
   text-tools: analyze_text, to_celsius

Step 9: Run the suite

make          # prints the help screen
make demo     # smoke-runs the server (output in Step 8)
make test     # runs the full suite

The suite passes:

...............                                                          [100%]
15 passed in 0.04s

Run it verbosely to see the layers:

make test-v

tests/test_contract.py::test_tool_set_is_stable PASSED
tests/test_contract.py::test_analyze_text_input_contract PASSED
tests/test_contract.py::test_to_celsius_input_contract PASSED
tests/test_contract.py::test_every_tool_has_a_description_and_schema PASSED
tests/test_contract.py::test_every_tool_names_snake_case PASSED
tests/test_contract.py::test_results_conform_to_output_schema[analyze_text-args0] PASSED
tests/test_contract.py::test_results_conform_to_output_schema[to_celsius-args1] PASSED
tests/test_snapshots.py::test_analyze_text_snapshot PASSED
tests/test_snapshots.py::test_analyze_text_unicode_snapshot PASSED
tests/test_snapshots.py::test_to_celsius_values[32.0-0.0] PASSED
tests/test_snapshots.py::test_to_celsius_values[212.0-100.0] PASSED
tests/test_snapshots.py::test_to_celsius_values[98.6-37.0] PASSED
tests/test_snapshots.py::test_to_celsius_values[-40.0--40.0] PASSED
tests/test_snapshots.py::test_to_celsius_rejects_below_absolute_zero PASSED
tests/test_snapshots.py::test_empty_text_is_all_zeros PASSED

Applying this to your own server

  • Start with test_tool_set_is_stable. One test that pins the tool names pays for itself the first time a refactor drops or renames a tool.
  • Write conformance tests against the live list. “Every tool has a description”, “every result validates against its outputSchema” — these cost nothing per new tool and never go stale.
  • Snapshot structured results, assert scalars. A rich dict is a snapshot; a single number or a specific edge case is a plain assert that documents intent.
  • Keep the server pure, or make time and randomness injectable. If a tool reads the clock or a database, pass those in so tests can pin them; otherwise snapshots flake.
  • Test error paths as contracts. pytest.raises(ToolError, match=...) guards the message and behavior a client depends on when input is bad.
  • Regenerate snapshots deliberately. SNAPSHOT_UPDATE=1 then a diff review is the workflow; never regenerate blind.

Troubleshooting

  • ModuleNotFoundError: No module named 'server' under pytest. tests/ needs an __init__.py so pytest puts the project root on sys.path.
  • Snapshots always “pass” and never compare. The golden file was missing, so the fixture wrote it. Commit tests/snapshots/*.json and rerun; the second run compares.
  • A snapshot fails after an intentional change. Run make snapshots, review the diff, and commit the updated golden file.
  • outputSchema is None for a tool. The tool returns an unannotated value. Give it a return type (a Pydantic model or a typed scalar) so FastMCP publishes a schema for the conformance test to check.
  • Async tests are skipped or error. asyncio_mode = auto must be set in pytest.ini and pytest-asyncio installed.

Recap

The suite has four layers, each guarding a different failure: contract tests pin the tool set and input schemas, conformance tests enforce per-tool invariants and validate results against the advertised outputSchema, snapshot tests freeze structured results in reviewed golden files, and shared fixtures keep every test to a few lines. Together they treat the server as the contract a model depends on, not just a bag of functions.

Next improvements:

  • Add a JSON-Schema-conformance check that also validates each tool’s input schema is a valid draft, catching malformed hand-written schemas.
  • Snapshot the full tools/list payload to detect any interface drift in one test.
  • Wire make test into CI so the contract is checked on every push.