diff --git a/docs/decisions.md b/docs/decisions.md index a451955..e3a574b 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -117,3 +117,31 @@ that extension's `evidenceRef` slot. **Rationale.** Separate PRs let each piece be reviewed and graduate on its own clock. FIDES stacks on `trust-annotations` because a scheme has no meaning without the slot it fills. + +## 2026-06-17 — Schemes carry data labels; host architectures do not + +**Decision.** `schemes/` holds **data-labelling** approaches a server attaches to +a result (FIDES, Permissive IFC, AirGapAgent, `data-class`, attestation +envelopes). **Host architectures** — control-flow designs the client/host runs +(CaMeL, the "Design Patterns for Securing LLM Agents" catalogue, Dual-LLM) — are +prior art in [`related-work.md`](./related-work.md), not candidate schemes. + +**Rationale.** A scheme produces a label; an architecture decides what to do with +one. Conflating them would invite a `schemes/camel.md` that has no per-result +payload to define. A capability token such an architecture issues can still be +*referenced* through `evidenceRef`, but the architecture itself is not a scheme. + +## 2026-06-17 — Early SEP-1913 feedback recorded as cited open questions + +**Decision.** The substantive concerns from the original issue (#711) and SEP +(#1913) review — the set-theoretic critique of linear sensitivity, org-defined +vocabularies, the class+regulatory pairing, taint persistence across storage, +per-block byte ranges, sequence-shape, and the false-security risk — are +captured with reviewer attributions in [`open-questions.md`](./open-questions.md) +rather than silently dropped by the narrower cut. + +**Rationale.** The narrow first cut (`sensitive: boolean`) deliberately omits a +lot of debated design. Recording *why*, with links to the people who raised each +point, keeps the history visible and gives each parked item a home to graduate +from (a scheme, an `action-metadata` field, or a future extension) instead of +being re-litigated from scratch. diff --git a/docs/open-questions.md b/docs/open-questions.md index 7adb9d9..8432a37 100644 --- a/docs/open-questions.md +++ b/docs/open-questions.md @@ -16,13 +16,50 @@ Tracked here rather than in the spec drafts, so the drafts stay non-temporal. ## trust-annotations -- Is `sensitive` the right single coarse signal, or do we need the - `data-class.v1` profile from day one? +- **Shape of the sensitivity signal.** Is `sensitive: boolean` the right single + coarse signal, or does the narrow cut overcorrect? The SEP-1913 thread rejected + a linear `sensitiveHint: low|medium|high` because sensitivity is set-theoretic, + not a scale ([@JustinCappos](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2967516811): + medical results to a mail MCP *and* a card number to a payment MCP, but neither + crossed). Reviewers then pushed for org-defined vocabularies + ([@olaservo](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2968743154), + [@Mossaka](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2971788308)) + and a class+regulatory pairing such as `confidential:hipaa` + ([@krubenok](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#discussion_r3103485194)). + Against that, [@localden](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4037623595) + warned a baked-in taxonomy is hard to remove. Open: keep the boolean and let + the [`data-class.v1` scheme](../schemes/data-class.md) carry the taxonomy, or + also expose a structured escape hatch on the wire so regulated flows are + expressible without a scheme? +- **Enforcement vs. advisory.** A self-declared `sensitive: true` from a + poorly-configured or malicious server could create a false sense of security + ([@localden](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4037623595)). + [docs/trust-model.md](./trust-model.md) puts verification in registries; is + that enough without a normative client-side check path? - Content-block-level vs. result-level attachment — does the draft need a - worked multi-result example before it's implementable? + worked multi-result example before it's implementable? Per-block annotation + with **byte/codepoint ranges** was requested so clients can highlight the + flagged span, not just the whole result + ([@connor4312](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-3849207989)). - `list_changed`: confirmed response-level annotations don't participate; revisit only if trust vocabulary ever attaches to tool definitions. +## Naming (under review) + +These are explicitly unsettled and are being reviewed against the SEP-1913 +record before any rename lands: + +- **Umbrella name.** The original SEP was "Trust *& Sensitivity* Annotations" — + two axes (integrity *and* confidentiality). `trust-annotations` reads as the + integrity half; does the name hide the `sensitive` (confidentiality) half? +- **`untrusted` vs `openWorldHint`.** Same concept, different layer (result vs. + tool definition). Share a name/vocabulary, or keep them deliberately distinct? +- **`evidenceRef.type` vs `evidenceRef.scheme`.** The repo calls these values + "schemes" (`schemes/`); the selector field is `type`. Align the field name? +- **`evidenceRef` itself.** Its ancestor is [@pshkv](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4196867926)'s + `decision_ref` / `attestation_ref` / `policy_profile`. Is `evidenceRef` the + clearest umbrella, or should the pointer name the kind of thing it references? + ## action-metadata - Coexistence vs. replacement of legacy `destructiveHint` / `readOnlyHint` / @@ -42,8 +79,20 @@ Tracked here rather than in the spec drafts, so the drafts stay non-temporal. ## Parked (SEP-1913 umbrella) -- **`maliciousActivityHint`** — if it returns, it is per-`ContentBlock` with - spans, driven by the host's own detection, not a server-attested boolean. +- **`maliciousActivityHint`** — killed in review on three grounds + ([@connor4312](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-3849207989)): + it can't be resolved before execution for dynamic tools; a boolean gives no + UX or ranges; and clients won't trust a server's self-report of its own + maliciousness. If it returns, it is per-`ContentBlock` with spans, driven by + the **host's** own detection, not a server-attested boolean. +- **Taint persistence across the store boundary** — a label must survive + round-tripping through storage: write a card number to a file, read it back, + and the sensitivity label must not silently disappear + ([@JustinCappos](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2967516811)). + This is a propagation property no single response annotation can guarantee on + its own; it needs the taxonomy and `evidenceRef` stable first. - **Session-level propagation rules** — escalation semantics and the - sequence-shape gap ("this was call N in a flagged sequence" has no response - annotation surface today). + **sequence-shape** gap: "this was call N in a flagged sequence" has no response + annotation surface today ([marras0914](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913)). + A candidate `evidenceRef` scheme could carry a sequence assertion; tracked in + [`schemes/README.md`](../schemes/README.md). diff --git a/docs/related-work.md b/docs/related-work.md index 5cdb88f..858391f 100644 --- a/docs/related-work.md +++ b/docs/related-work.md @@ -15,10 +15,22 @@ annotation work. Several were surfaced in IG meetings (notably 2026-05-28). ## Research - **FIDES** — *Information-flow control for LLM agents.* [arXiv:2505.23643](https://arxiv.org/abs/2505.23643). Basis for the `ifc.fides.v1` scheme in [`schemes/`](../schemes/). -- **Design Patterns for Securing LLM Agents** — IBM/Google/Microsoft. [arXiv:2506.08837](https://arxiv.org/abs/2506.08837). Plan-Then-Execute, Dual LLM, Map-Reduce, etc. +- **Permissive Information-Flow Analysis for LLMs** — relaxes IFC join so a label propagates only when an input actually influences an output. [arXiv:2410.03055](https://arxiv.org/abs/2410.03055). Candidate `evidenceRef` scheme (per-result label), like FIDES. +- **AirGapAgent** — contextual-integrity minimisation: restrict per-task data to what the context warrants. [arXiv:2405.05175](https://arxiv.org/abs/2405.05175). Candidate scheme: emits a contextual-integrity classification per result. +- **CaMeL — Defeating Prompt Injections by Design** — capability-based control/data-flow extraction. [arXiv:2503.18813](https://arxiv.org/abs/2503.18813). A **host architecture**, not a data-label scheme (see note below); a capability token it issues could be referenced via `evidenceRef`, but the architecture itself is not a scheme. +- **Design Patterns for Securing LLM Agents** — IBM/Google/Microsoft. [arXiv:2506.08837](https://arxiv.org/abs/2506.08837). Plan-Then-Execute, Dual LLM, Map-Reduce, etc. Also **host architectures**, not schemes. - **Trail of Bits** — prompt-injection via hidden content in GitHub issues. [blog](https://blog.trailofbits.com/2025/08/06/prompt-injection-engineering-for-attackers-exploiting-github-copilot/). - **OpenAI Auto Review** — https://alignment.openai.com/auto-review/ (shared in IG chat). +### Schemes vs. host architectures + +The `evidenceRef` slot carries **data labels** — a per-result record a server +can attach (FIDES, Permissive IFC, AirGapAgent, data-class, attestation +envelopes). It does **not** carry **host architectures** — control-flow designs +the *client/host* runs (CaMeL, the Design-Patterns catalogue, Dual-LLM). These +are complementary: an architecture decides what to do with a label, the label is +what a scheme produces. Only the former belong in [`schemes/`](../schemes/). + ## Implementations & tooling - [`kapil8811/mcp-trust-annotations`](https://github.com/kapil8811/mcp-trust-annotations) — reference Python SDK PoC for `trust-annotations`. diff --git a/specification/draft/trust-annotations.mdx b/specification/draft/trust-annotations.mdx index 395e3c8..4ed0689 100644 --- a/specification/draft/trust-annotations.mdx +++ b/specification/draft/trust-annotations.mdx @@ -137,6 +137,26 @@ are present, the content-block annotation refines the result-level one for that block; it MUST NOT *weaken* a result-level claim (union semantics — once `true`, stays `true`). +### Relationship to existing `*Hint` annotations + +MCP tool definitions already carry an `openWorldHint` (alongside `readOnlyHint` / +`destructiveHint` / `idempotentHint`). `untrusted` is deliberately **not** a +synonym for `openWorldHint`: + +- `openWorldHint` is a property of the **tool definition** — "this tool reaches + an open, attacker-influenceable world" (e.g. a web fetch). It is known at + registration time and does not vary per call. +- `untrusted` is a property of a **specific result** — "*this* returned data + originated from an open-world / untrusted source." A tool whose `openWorldHint` + is `true` may still return trusted data on a given call, and a tool whose + `openWorldHint` is `false` can surface untrusted data it read from storage. + +The original SEP-1913 proposal reused `openWorldHint` for the result-level +"untrusted source" signal and then drew exactly this distinction +([issue #711](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711)). +Whether the two should share a name or vocabulary is an +[open question](../../docs/open-questions.md). + ### Lifecycle and `list_changed` Trust annotations defined here are **response-level**: they describe a specific