Record SEP-1913 feedback as cited open questions; clarify untrusted vs openWorldHint

Enrich the base extension docs so the early review history is preserved rather
than erased by the narrow first cut:

- trust-annotations.mdx: add a normative subsection distinguishing result-level
  untrusted from tool-definition openWorldHint (the SEP-1913 #711 distinction).
- open-questions.md: capture the sensitivity-field shape debate, enforcement-vs-
  advisory risk, per-block byte ranges, taint persistence, and sequence-shape,
  each attributed to the reviewer who raised it; add a Naming (under review)
  section for the contested field/extension names.
- related-work.md: add verified FIDES alternatives (Permissive IFC 2410.03055,
  AirGapAgent 2405.05175, CaMeL 2503.18813) and a schemes-vs-host-architectures
  distinction.
- decisions.md: log the scheme/architecture boundary and the feedback-capture
  decision.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Sam Morrow 2026-06-16 08:56:46 +02:00
commit 0be45d9b95
No known key found for this signature in database
4 changed files with 117 additions and 8 deletions

View file

@ -117,3 +117,31 @@ that extension's `evidenceRef` slot.
**Rationale.** Separate PRs let each piece be reviewed and graduate on its own
clock. FIDES stacks on `trust-annotations` because a scheme has no meaning
without the slot it fills.
## 2026-06-17 — Schemes carry data labels; host architectures do not
**Decision.** `schemes/` holds **data-labelling** approaches a server attaches to
a result (FIDES, Permissive IFC, AirGapAgent, `data-class`, attestation
envelopes). **Host architectures** — control-flow designs the client/host runs
(CaMeL, the "Design Patterns for Securing LLM Agents" catalogue, Dual-LLM) — are
prior art in [`related-work.md`](./related-work.md), not candidate schemes.
**Rationale.** A scheme produces a label; an architecture decides what to do with
one. Conflating them would invite a `schemes/camel.md` that has no per-result
payload to define. A capability token such an architecture issues can still be
*referenced* through `evidenceRef`, but the architecture itself is not a scheme.
## 2026-06-17 — Early SEP-1913 feedback recorded as cited open questions
**Decision.** The substantive concerns from the original issue (#711) and SEP
(#1913) review — the set-theoretic critique of linear sensitivity, org-defined
vocabularies, the class+regulatory pairing, taint persistence across storage,
per-block byte ranges, sequence-shape, and the false-security risk — are
captured with reviewer attributions in [`open-questions.md`](./open-questions.md)
rather than silently dropped by the narrower cut.
**Rationale.** The narrow first cut (`sensitive: boolean`) deliberately omits a
lot of debated design. Recording *why*, with links to the people who raised each
point, keeps the history visible and gives each parked item a home to graduate
from (a scheme, an `action-metadata` field, or a future extension) instead of
being re-litigated from scratch.

View file

@ -16,13 +16,50 @@ Tracked here rather than in the spec drafts, so the drafts stay non-temporal.
## trust-annotations
- Is `sensitive` the right single coarse signal, or do we need the
`data-class.v1` profile from day one?
- **Shape of the sensitivity signal.** Is `sensitive: boolean` the right single
coarse signal, or does the narrow cut overcorrect? The SEP-1913 thread rejected
a linear `sensitiveHint: low|medium|high` because sensitivity is set-theoretic,
not a scale ([@JustinCappos](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2967516811):
medical results to a mail MCP *and* a card number to a payment MCP, but neither
crossed). Reviewers then pushed for org-defined vocabularies
([@olaservo](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2968743154),
[@Mossaka](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2971788308))
and a class+regulatory pairing such as `confidential:hipaa`
([@krubenok](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#discussion_r3103485194)).
Against that, [@localden](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4037623595)
warned a baked-in taxonomy is hard to remove. Open: keep the boolean and let
the [`data-class.v1` scheme](../schemes/data-class.md) carry the taxonomy, or
also expose a structured escape hatch on the wire so regulated flows are
expressible without a scheme?
- **Enforcement vs. advisory.** A self-declared `sensitive: true` from a
poorly-configured or malicious server could create a false sense of security
([@localden](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4037623595)).
[docs/trust-model.md](./trust-model.md) puts verification in registries; is
that enough without a normative client-side check path?
- Content-block-level vs. result-level attachment — does the draft need a
worked multi-result example before it's implementable?
worked multi-result example before it's implementable? Per-block annotation
with **byte/codepoint ranges** was requested so clients can highlight the
flagged span, not just the whole result
([@connor4312](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-3849207989)).
- `list_changed`: confirmed response-level annotations don't participate; revisit
only if trust vocabulary ever attaches to tool definitions.
## Naming (under review)
These are explicitly unsettled and are being reviewed against the SEP-1913
record before any rename lands:
- **Umbrella name.** The original SEP was "Trust *& Sensitivity* Annotations" —
two axes (integrity *and* confidentiality). `trust-annotations` reads as the
integrity half; does the name hide the `sensitive` (confidentiality) half?
- **`untrusted` vs `openWorldHint`.** Same concept, different layer (result vs.
tool definition). Share a name/vocabulary, or keep them deliberately distinct?
- **`evidenceRef.type` vs `evidenceRef.scheme`.** The repo calls these values
"schemes" (`schemes/`); the selector field is `type`. Align the field name?
- **`evidenceRef` itself.** Its ancestor is [@pshkv](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4196867926)'s
`decision_ref` / `attestation_ref` / `policy_profile`. Is `evidenceRef` the
clearest umbrella, or should the pointer name the kind of thing it references?
## action-metadata
- Coexistence vs. replacement of legacy `destructiveHint` / `readOnlyHint` /
@ -42,8 +79,20 @@ Tracked here rather than in the spec drafts, so the drafts stay non-temporal.
## Parked (SEP-1913 umbrella)
- **`maliciousActivityHint`** — if it returns, it is per-`ContentBlock` with
spans, driven by the host's own detection, not a server-attested boolean.
- **`maliciousActivityHint`** — killed in review on three grounds
([@connor4312](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-3849207989)):
it can't be resolved before execution for dynamic tools; a boolean gives no
UX or ranges; and clients won't trust a server's self-report of its own
maliciousness. If it returns, it is per-`ContentBlock` with spans, driven by
the **host's** own detection, not a server-attested boolean.
- **Taint persistence across the store boundary** — a label must survive
round-tripping through storage: write a card number to a file, read it back,
and the sensitivity label must not silently disappear
([@JustinCappos](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2967516811)).
This is a propagation property no single response annotation can guarantee on
its own; it needs the taxonomy and `evidenceRef` stable first.
- **Session-level propagation rules** — escalation semantics and the
sequence-shape gap ("this was call N in a flagged sequence" has no response
annotation surface today).
**sequence-shape** gap: "this was call N in a flagged sequence" has no response
annotation surface today ([marras0914](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913)).
A candidate `evidenceRef` scheme could carry a sequence assertion; tracked in
[`schemes/README.md`](../schemes/README.md).

View file

@ -15,10 +15,22 @@ annotation work. Several were surfaced in IG meetings (notably 2026-05-28).
## Research
- **FIDES** — *Information-flow control for LLM agents.* [arXiv:2505.23643](https://arxiv.org/abs/2505.23643). Basis for the `ifc.fides.v1` scheme in [`schemes/`](../schemes/).
- **Design Patterns for Securing LLM Agents** — IBM/Google/Microsoft. [arXiv:2506.08837](https://arxiv.org/abs/2506.08837). Plan-Then-Execute, Dual LLM, Map-Reduce, etc.
- **Permissive Information-Flow Analysis for LLMs** — relaxes IFC join so a label propagates only when an input actually influences an output. [arXiv:2410.03055](https://arxiv.org/abs/2410.03055). Candidate `evidenceRef` scheme (per-result label), like FIDES.
- **AirGapAgent** — contextual-integrity minimisation: restrict per-task data to what the context warrants. [arXiv:2405.05175](https://arxiv.org/abs/2405.05175). Candidate scheme: emits a contextual-integrity classification per result.
- **CaMeL — Defeating Prompt Injections by Design** — capability-based control/data-flow extraction. [arXiv:2503.18813](https://arxiv.org/abs/2503.18813). A **host architecture**, not a data-label scheme (see note below); a capability token it issues could be referenced via `evidenceRef`, but the architecture itself is not a scheme.
- **Design Patterns for Securing LLM Agents** — IBM/Google/Microsoft. [arXiv:2506.08837](https://arxiv.org/abs/2506.08837). Plan-Then-Execute, Dual LLM, Map-Reduce, etc. Also **host architectures**, not schemes.
- **Trail of Bits** — prompt-injection via hidden content in GitHub issues. [blog](https://blog.trailofbits.com/2025/08/06/prompt-injection-engineering-for-attackers-exploiting-github-copilot/).
- **OpenAI Auto Review** — https://alignment.openai.com/auto-review/ (shared in IG chat).
### Schemes vs. host architectures
The `evidenceRef` slot carries **data labels** — a per-result record a server
can attach (FIDES, Permissive IFC, AirGapAgent, data-class, attestation
envelopes). It does **not** carry **host architectures** — control-flow designs
the *client/host* runs (CaMeL, the Design-Patterns catalogue, Dual-LLM). These
are complementary: an architecture decides what to do with a label, the label is
what a scheme produces. Only the former belong in [`schemes/`](../schemes/).
## Implementations & tooling
- [`kapil8811/mcp-trust-annotations`](https://github.com/kapil8811/mcp-trust-annotations) — reference Python SDK PoC for `trust-annotations`.

View file

@ -137,6 +137,26 @@ are present, the content-block annotation refines the result-level one for that
block; it MUST NOT *weaken* a result-level claim (union semantics — once
`true`, stays `true`).
### Relationship to existing `*Hint` annotations
MCP tool definitions already carry an `openWorldHint` (alongside `readOnlyHint` /
`destructiveHint` / `idempotentHint`). `untrusted` is deliberately **not** a
synonym for `openWorldHint`:
- `openWorldHint` is a property of the **tool definition** — "this tool reaches
an open, attacker-influenceable world" (e.g. a web fetch). It is known at
registration time and does not vary per call.
- `untrusted` is a property of a **specific result** — "*this* returned data
originated from an open-world / untrusted source." A tool whose `openWorldHint`
is `true` may still return trusted data on a given call, and a tool whose
`openWorldHint` is `false` can surface untrusted data it read from storage.
The original SEP-1913 proposal reused `openWorldHint` for the result-level
"untrusted source" signal and then drew exactly this distinction
([issue #711](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711)).
Whether the two should share a name or vocabulary is an
[open question](../../docs/open-questions.md).
### Lifecycle and `list_changed`
Trust annotations defined here are **response-level**: they describe a specific