docs: load media examples from disk instead of inline base64 (#3108)
This commit is contained in:
Max
GitHub
parent
ebcc4dc3fb
commit
3a6f2996cd
6 changed files with 108 additions and 50 deletions
|
|
@ -6,17 +6,17 @@ The SDK ships two helpers for binary results (**`Image`** and **`Audio`**) and a
|
|||
|
||||
## Returning an image
|
||||
|
||||
Annotate the return type as `Image` and return one:
|
||||
Annotate the return type as `Image`, point it at a file, and return it:
|
||||
|
||||
```python title="server.py" hl_lines="14 16"
|
||||
```python title="server.py" hl_lines="8 12 14"
|
||||
--8<-- "docs_src/media/tutorial001.py"
|
||||
```
|
||||
|
||||
* `Image` takes exactly one of `data` (raw bytes) or `path` (a file to read).
|
||||
* `format="png"` becomes the MIME type the client sees: `image/png`.
|
||||
* The bytes here are a one-pixel placeholder so the file runs on its own. In a real server they come from Pillow, matplotlib, a headless browser, or anything else that hands you `bytes`.
|
||||
* `Image` takes exactly one of `path` (a file to read) or `data` (raw bytes).
|
||||
* The MIME type the client sees is guessed from the suffix: `logo.png` is announced as `image/png`.
|
||||
* Nothing here is special about logos. Any PNG next to `server.py` works: a chart your code rendered, a diagram, a photo.
|
||||
|
||||
`Image` is an SDK convenience, not a protocol type. On the wire your return value becomes an **`ImageContent`** block (your bytes base64-encoded, plus the MIME type):
|
||||
`Image` is an SDK convenience, not a protocol type. On the wire your return value becomes an **`ImageContent`** block (the file's bytes base64-encoded, plus the MIME type):
|
||||
|
||||
```python
|
||||
result.content # [ImageContent(type="image", data="iVBORw0KGgoAAAANSUhEUg...", mime_type="image/png")]
|
||||
|
|
@ -25,7 +25,7 @@ result.structured_content # None
|
|||
|
||||
Two things to notice:
|
||||
|
||||
* `data` is base64. You returned raw `bytes`; the SDK did the encoding.
|
||||
* `data` is base64. You never touched the bytes; the SDK read the file and did the encoding.
|
||||
* `structured_content` is `None`. An `Image` is content for the model to look at, not data for the application to parse: there is no output schema. (Contrast **[Structured Output](structured-output.md)**, where the return annotation *is* the schema.)
|
||||
|
||||
!!! info
|
||||
|
|
@ -35,32 +35,40 @@ Two things to notice:
|
|||
|
||||
### Try it
|
||||
|
||||
Drop any PNG next to `server.py`, name it `logo.png`, and run:
|
||||
|
||||
```console
|
||||
uv run mcp dev server.py
|
||||
```
|
||||
|
||||
Open the **Tools** tab and call `logo`. The result is not a string: it is an `image` content block, and the Inspector renders it as a picture. You returned `bytes`; everything between that and the pixels on screen was the SDK.
|
||||
Open the **Tools** tab and call `logo`. The result is not a string: it is an `image` content block, and the Inspector renders your picture. Everything between the file on disk and the pixels on screen was the SDK.
|
||||
|
||||
## Returning audio
|
||||
|
||||
`Audio` is the same shape:
|
||||
`Audio` is the same shape. Keep `logo.png` where it was, and put any WAV beside it as `chime.wav`:
|
||||
|
||||
```python title="server.py" hl_lines="21-24"
|
||||
```python title="server.py" hl_lines="18-21"
|
||||
--8<-- "docs_src/media/tutorial002.py"
|
||||
```
|
||||
|
||||
The result is an **`AudioContent`** block:
|
||||
|
||||
```python
|
||||
result.content # [AudioContent(type="audio", data="UklGRjQAAABXQVZFZm1...", mime_type="audio/wav")]
|
||||
result.content # [AudioContent(type="audio", data="UklGR...", mime_type="audio/wav")]
|
||||
result.structured_content # None
|
||||
```
|
||||
|
||||
Same deal: raw bytes in, base64 and a MIME type out, no output schema.
|
||||
Same deal: a file on disk in, base64 and a MIME type out, no output schema.
|
||||
|
||||
## Bytes or a file
|
||||
|
||||
Both helpers also accept `path=` instead of `data=`. The file is read when the result is built, and the MIME type is guessed from the suffix:
|
||||
Both helpers also accept `data=` (raw bytes) instead of `path=`. That is the mode for bytes that never came from a file of their own — a database column, an HTTP response, something Pillow just drew:
|
||||
|
||||
```python title="server.py" hl_lines="14 15"
|
||||
--8<-- "docs_src/media/tutorial003.py"
|
||||
```
|
||||
|
||||
With `path=` there is nothing to declare: the file is read when the result is built, and the MIME type is guessed from the suffix:
|
||||
|
||||
* `Image`: `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`.
|
||||
* `Audio`: `.wav`, `.mp3`, `.ogg`, `.flac`, `.aac`, `.m4a`.
|
||||
|
|
@ -78,7 +86,7 @@ A suffix it doesn't recognise falls back to `application/octet-stream`.
|
|||
An `Icon` is metadata, not content. It doesn't carry the image; it points at one with a URI, and a client may fetch it and show it next to your server's name, a tool, a resource, or a prompt.
|
||||
|
||||
```python title="server.py" hl_lines="5-6 8 11 17"
|
||||
--8<-- "docs_src/media/tutorial003.py"
|
||||
--8<-- "docs_src/media/tutorial004.py"
|
||||
```
|
||||
|
||||
* `src` is a URI the client can resolve: `https:`, or a `data:` URI if you want the icon embedded with no extra fetch.
|
||||
|
|
@ -100,7 +108,7 @@ A tool's icons are on the `Tool` object from `tools/list`, a resource's on the `
|
|||
## Recap
|
||||
|
||||
* Return an `Image` or `Audio` from a tool and the client receives an `ImageContent` / `AudioContent` block: your bytes base64-encoded, with a MIME type.
|
||||
* Build one from in-memory `data=` plus an explicit `format=`, or from a `path=` and let the suffix decide.
|
||||
* Build one from a `path=` and let the suffix decide the MIME type, or from in-memory `data=` plus an explicit `format=`.
|
||||
* Media results carry no `structured_content` and no output schema.
|
||||
* An `Icon` is a pointer: a `src` URI plus optional `mime_type`, `sizes`, and `theme`.
|
||||
* `icons=[...]` works on the server, on tools, on resources, and on prompts, and clients find them on the matching objects.
|
||||
|
|
|
|||
|
|
@ -1,16 +1,14 @@
|
|||
import base64
|
||||
from pathlib import Path
|
||||
|
||||
from mcp.server import MCPServer
|
||||
from mcp.server.mcpserver import Image
|
||||
|
||||
mcp = MCPServer("Brand kit")
|
||||
|
||||
LOGO_PNG = base64.b64decode(
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQ9bsBAAHPAURf8l/aAAAAAElFTkSuQmCC"
|
||||
)
|
||||
LOGO_FILE = Path(__file__).parent / "logo.png" # or the path to your file on disk
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
def logo() -> Image:
|
||||
"""The brand logo as a PNG."""
|
||||
return Image(data=LOGO_PNG, format="png")
|
||||
return Image(path=LOGO_FILE)
|
||||
|
|
|
|||
|
|
@ -1,24 +1,21 @@
|
|||
import base64
|
||||
from pathlib import Path
|
||||
|
||||
from mcp.server import MCPServer
|
||||
from mcp.server.mcpserver import Audio, Image
|
||||
|
||||
mcp = MCPServer("Brand kit")
|
||||
|
||||
LOGO_PNG = base64.b64decode(
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQ9bsBAAHPAURf8l/aAAAAAElFTkSuQmCC"
|
||||
)
|
||||
|
||||
CHIME_WAV = base64.b64decode("UklGRjQAAABXQVZFZm10IBAAAAABAAEAQB8AAIA+AAACABAAZGF0YRAAAAAAAAAAAAAAAAAAAAAAAAAA")
|
||||
LOGO_FILE = Path(__file__).parent / "logo.png"
|
||||
CHIME_FILE = Path(__file__).parent / "chime.wav"
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
def logo() -> Image:
|
||||
"""The brand logo as a PNG."""
|
||||
return Image(data=LOGO_PNG, format="png")
|
||||
return Image(path=LOGO_FILE)
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
def chime() -> Audio:
|
||||
"""The notification chime as a WAV."""
|
||||
return Audio(data=CHIME_WAV, format="wav")
|
||||
return Audio(path=CHIME_FILE)
|
||||
|
|
|
|||
|
|
@ -1,20 +1,15 @@
|
|||
from mcp_types import Icon
|
||||
from pathlib import Path
|
||||
|
||||
from mcp.server import MCPServer
|
||||
from mcp.server.mcpserver import Image
|
||||
|
||||
LOGO = Icon(src="https://example.com/brand-kit.png", mime_type="image/png", sizes=["48x48"])
|
||||
PALETTE = Icon(src="https://example.com/palette.svg", mime_type="image/svg+xml", sizes=["any"])
|
||||
mcp = MCPServer("Brand kit")
|
||||
|
||||
mcp = MCPServer("Brand kit", icons=[LOGO])
|
||||
LOGO_FILE = Path(__file__).parent / "logo.png"
|
||||
|
||||
|
||||
@mcp.tool(icons=[PALETTE])
|
||||
def palette() -> list[str]:
|
||||
"""The brand colour palette as hex codes."""
|
||||
return ["#1d4ed8", "#f59e0b", "#10b981"]
|
||||
|
||||
|
||||
@mcp.resource("brand://guidelines", icons=[LOGO])
|
||||
def guidelines() -> str:
|
||||
"""How to use the brand assets."""
|
||||
return "Use the primary colour for calls to action."
|
||||
@mcp.tool()
|
||||
def logo_from_bytes() -> Image:
|
||||
"""The brand logo as a PNG."""
|
||||
png = LOGO_FILE.read_bytes() # a database read, an HTTP response, Pillow output...
|
||||
return Image(data=png, format="png")
|
||||
|
|
|
|||
20
docs_src/media/tutorial004.py
Normal file
20
docs_src/media/tutorial004.py
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
from mcp_types import Icon
|
||||
|
||||
from mcp.server import MCPServer
|
||||
|
||||
LOGO = Icon(src="https://example.com/brand-kit.png", mime_type="image/png", sizes=["48x48"])
|
||||
PALETTE = Icon(src="https://example.com/palette.svg", mime_type="image/svg+xml", sizes=["any"])
|
||||
|
||||
mcp = MCPServer("Brand kit", icons=[LOGO])
|
||||
|
||||
|
||||
@mcp.tool(icons=[PALETTE])
|
||||
def palette() -> list[str]:
|
||||
"""The brand colour palette as hex codes."""
|
||||
return ["#1d4ed8", "#f59e0b", "#10b981"]
|
||||
|
||||
|
||||
@mcp.resource("brand://guidelines", icons=[LOGO])
|
||||
def guidelines() -> str:
|
||||
"""How to use the brand assets."""
|
||||
return "Use the primary colour for calls to action."
|
||||
|
|
@ -1,11 +1,12 @@
|
|||
"""`docs/servers/media.md`: every claim the page makes, proved against the real SDK."""
|
||||
|
||||
import base64
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from mcp_types import AudioContent, Icon, ImageContent
|
||||
|
||||
from docs_src.media import tutorial001, tutorial002, tutorial003
|
||||
from docs_src.media import tutorial001, tutorial002, tutorial003, tutorial004
|
||||
from mcp import Client
|
||||
from mcp.server.mcpserver import Audio, Image
|
||||
|
||||
|
|
@ -13,16 +14,32 @@ from mcp.server.mcpserver import Audio, Image
|
|||
pytestmark = [pytest.mark.anyio, pytest.mark.filterwarnings("error::mcp.MCPDeprecationWarning")]
|
||||
|
||||
|
||||
async def test_image_return_becomes_an_image_content_block() -> None:
|
||||
"""tutorial001: `-> Image` reaches the client as a base64 `ImageContent` block, not text."""
|
||||
@pytest.fixture
|
||||
def logo_file(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
|
||||
"""The PNG the tutorials expect next to `server.py`, fabricated on disk.
|
||||
|
||||
The SDK never parses the contents, so opaque bytes suffice.
|
||||
"""
|
||||
path = tmp_path / "logo.png"
|
||||
path.write_bytes(b"fake png data")
|
||||
monkeypatch.setattr(tutorial001, "LOGO_FILE", path)
|
||||
monkeypatch.setattr(tutorial002, "LOGO_FILE", path)
|
||||
monkeypatch.setattr(tutorial003, "LOGO_FILE", path)
|
||||
return path
|
||||
|
||||
|
||||
async def test_image_return_becomes_an_image_content_block(logo_file: Path) -> None:
|
||||
"""tutorial001: `-> Image` reaches the client as a base64 `ImageContent` block, not text,
|
||||
with the MIME type guessed from the `.png` suffix."""
|
||||
async with Client(tutorial001.mcp) as client:
|
||||
result = await client.call_tool("logo", {})
|
||||
assert not result.is_error
|
||||
assert result.content == [
|
||||
ImageContent(type="image", data=base64.b64encode(tutorial001.LOGO_PNG).decode(), mime_type="image/png")
|
||||
ImageContent(type="image", data=base64.b64encode(logo_file.read_bytes()).decode(), mime_type="image/png")
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.usefixtures("logo_file")
|
||||
async def test_image_result_has_no_structured_content_and_no_output_schema() -> None:
|
||||
"""tutorial001: media is content for the model, not data for the application."""
|
||||
async with Client(tutorial001.mcp) as client:
|
||||
|
|
@ -32,27 +49,50 @@ async def test_image_result_has_no_structured_content_and_no_output_schema() ->
|
|||
assert result.structured_content is None
|
||||
|
||||
|
||||
async def test_audio_return_becomes_an_audio_content_block() -> None:
|
||||
async def test_audio_return_becomes_an_audio_content_block(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""tutorial002: `Audio` is the same shape as `Image`."""
|
||||
chime_file = tmp_path / "chime.wav"
|
||||
chime_file.write_bytes(b"fake wav data")
|
||||
monkeypatch.setattr(tutorial002, "CHIME_FILE", chime_file)
|
||||
|
||||
async with Client(tutorial002.mcp) as client:
|
||||
result = await client.call_tool("chime", {})
|
||||
assert not result.is_error
|
||||
assert result.content == [
|
||||
AudioContent(type="audio", data=base64.b64encode(tutorial002.CHIME_WAV).decode(), mime_type="audio/wav")
|
||||
AudioContent(type="audio", data=base64.b64encode(chime_file.read_bytes()).decode(), mime_type="audio/wav")
|
||||
]
|
||||
assert result.structured_content is None
|
||||
|
||||
|
||||
async def test_in_memory_bytes_with_a_format_become_the_same_image_content_block(logo_file: Path) -> None:
|
||||
"""tutorial003: `data=` plus `format=` produces the same wire block as `path=`."""
|
||||
async with Client(tutorial003.mcp) as client:
|
||||
result = await client.call_tool("logo_from_bytes", {})
|
||||
assert not result.is_error
|
||||
assert result.content == [
|
||||
ImageContent(type="image", data=base64.b64encode(logo_file.read_bytes()).decode(), mime_type="image/png")
|
||||
]
|
||||
|
||||
|
||||
def test_path_file_is_read_when_the_result_is_built() -> None:
|
||||
"""The page's `path=` claim (SDK-defined): the file is opened when the result is built,
|
||||
not when the helper is constructed — `server.py` can name a file that appears later."""
|
||||
image = Image(path="does-not-exist.png")
|
||||
with pytest.raises(FileNotFoundError):
|
||||
image.to_image_content()
|
||||
|
||||
|
||||
def test_raw_data_without_a_format_falls_back_to_a_default_mime_type() -> None:
|
||||
"""The `!!! check`: with `data=` there is no suffix to guess from, so `format=` decides."""
|
||||
assert Image(data=b"\x89PNG\r\n\x1a\n", format="png").to_image_content().mime_type == "image/png"
|
||||
assert Image(data=b"\x89PNG\r\n\x1a\n").to_image_content().mime_type == "image/png"
|
||||
assert Audio(data=b"\xff\xfb", format="wav").to_audio_content().mime_type == "audio/wav"
|
||||
assert Audio(data=b"\xff\xfb").to_audio_content().mime_type == "audio/wav"
|
||||
|
||||
|
||||
async def test_icons_are_visible_where_they_were_declared() -> None:
|
||||
"""tutorial003: server icons land on `server_info`, tool icons on the `Tool`, resource icons on the `Resource`."""
|
||||
async with Client(tutorial003.mcp) as client:
|
||||
"""tutorial004: server icons land on `server_info`, tool icons on the `Tool`, resource icons on the `Resource`."""
|
||||
async with Client(tutorial004.mcp) as client:
|
||||
assert client.server_info.icons == [
|
||||
Icon(src="https://example.com/brand-kit.png", mime_type="image/png", sizes=["48x48"])
|
||||
]
|
||||
|
|
|
|||
Loading…
Reference in a new issue