experimental-ext-tool-annot.../docs/trust-model.md
Sam Morrow 2aea5dc6be
docs: scaffold trust/privacy extension repo from SEP-1913 carve
Carve the schema-bearing parts of SEP-1913 into three independent
experimental extensions, per @localden's "narrower first cut" ask and
the Tool Annotations IG's 2026-05-28 extension-first decision:

- io.modelcontextprotocol/trust-annotations (headline): narrow
  data-classification taxonomy (sensitive, untrusted) + open-ended
  evidenceRef pointer.
- io.modelcontextprotocol/action-metadata: carries forward SEP-2061
  (inputMetadata/returnMetadata/outcomes + requiresReview).
- io.modelcontextprotocol/ifc-fides: FIDES information-flow control as
  a profile of evidenceRef, not a wire root.

Adds repo meta (README, MAINTAINERS, CONTRIBUTING, LICENSE) and docs
(sep-disposition, intent-comment, decisions, open-questions,
trust-model, related-work). FIDES demoted to a profile; DataClass and
requiresReview rehomed; maliciousActivityHint and propagation parked on
the SEP-1913 umbrella. Citations index on public sources only.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-10 23:50:22 +02:00

53 lines
2.8 KiB
Markdown

# Trust model
A single statement of the enforcement model shared by all extensions in this
repository, so individual specs don't re-litigate it.
## Annotations are claims, not guarantees
An annotation on a tool result or definition is a **claim** made by whoever
produced it. Nothing in these extensions assumes the producer is honest or
competent. The value of an annotation comes from two things:
1. **Verifiability.** Where a claim needs to be trusted, the `evidenceRef`
pointer (see [`trust-annotations`](../specification/draft/trust-annotations.mdx))
lets a consumer resolve and check the evidence behind it — re-hash the
referenced record, verify a signature, check an inclusion proof — rather
than taking the claim on faith.
2. **Accountability.** Enforcement lives with the parties that can impose
consequences: **registries and marketplaces** that admit servers,
**hosts/clients** that gate actions, and **operators** that set policy. The
annotation gives those parties a machine-readable surface to act on.
This mirrors the framing repeated throughout the SEP-1913 discussion: trust
comes from the ecosystem verifying annotations, not from developer good faith.
A server that lies in its annotations is a server a registry can refuse to list
and a host can refuse to trust — the same accountability model as any other
declared capability.
## Defense in depth, not a single gate
As framed in the IG's inaugural meeting, these annotations are **defense in
depth**: they reduce the likelihood of unintended actions (unnecessary
destructive operations, data crossing a boundary it shouldn't), they don't
claim to be a complete security boundary. A host SHOULD combine them with its
own checks (its own injection detection, its own policy engine) rather than
treating any single annotation as authoritative.
## Human-in-the-loop on the risky edges
A recurring pattern from the IG research (Joanna's "phases" work, and the
SEP-1913 thread): rather than blanket-blocking flows a policy engine is unsure
about, **flag the specific call for user confirmation**. This preserves utility
while keeping a human on the genuinely risky edges, and is the recommended
default for `requiresReview` ([`action-metadata`](../specification/draft/action-metadata.mdx))
and for IFC policy violations ([`ifc-fides`](../specification/draft/ifc-fides.mdx)).
## Cross-domain is the hard case
Policy engines work well inside a single user/organization universe and
struggle across universes (cross-org, cross-domain flows). These extensions
provide the *signal* (where data came from, how sensitive it is, what a tool
does with it); they do not solve cross-domain enforcement on their own. That
remains an [open question](./open-questions.md) and a likely area for future
work (e.g. asymmetric crypto for domain integrity verification).