Record SEP-1913 feedback as cited open questions; clarify untrusted vs openWorldHint
Enrich the base extension docs so the early review history is preserved rather than erased by the narrow first cut: - trust-annotations.mdx: add a normative subsection distinguishing result-level untrusted from tool-definition openWorldHint (the SEP-1913 #711 distinction). - open-questions.md: capture the sensitivity-field shape debate, enforcement-vs- advisory risk, per-block byte ranges, taint persistence, and sequence-shape, each attributed to the reviewer who raised it; add a Naming (under review) section for the contested field/extension names. - related-work.md: add verified FIDES alternatives (Permissive IFC 2410.03055, AirGapAgent 2405.05175, CaMeL 2503.18813) and a schemes-vs-host-architectures distinction. - decisions.md: log the scheme/architecture boundary and the feedback-capture decision. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
parent
67f84f72e8
commit
0be45d9b95
4 changed files with 117 additions and 8 deletions
|
|
@ -117,3 +117,31 @@ that extension's `evidenceRef` slot.
|
|||
**Rationale.** Separate PRs let each piece be reviewed and graduate on its own
|
||||
clock. FIDES stacks on `trust-annotations` because a scheme has no meaning
|
||||
without the slot it fills.
|
||||
|
||||
## 2026-06-17 — Schemes carry data labels; host architectures do not
|
||||
|
||||
**Decision.** `schemes/` holds **data-labelling** approaches a server attaches to
|
||||
a result (FIDES, Permissive IFC, AirGapAgent, `data-class`, attestation
|
||||
envelopes). **Host architectures** — control-flow designs the client/host runs
|
||||
(CaMeL, the "Design Patterns for Securing LLM Agents" catalogue, Dual-LLM) — are
|
||||
prior art in [`related-work.md`](./related-work.md), not candidate schemes.
|
||||
|
||||
**Rationale.** A scheme produces a label; an architecture decides what to do with
|
||||
one. Conflating them would invite a `schemes/camel.md` that has no per-result
|
||||
payload to define. A capability token such an architecture issues can still be
|
||||
*referenced* through `evidenceRef`, but the architecture itself is not a scheme.
|
||||
|
||||
## 2026-06-17 — Early SEP-1913 feedback recorded as cited open questions
|
||||
|
||||
**Decision.** The substantive concerns from the original issue (#711) and SEP
|
||||
(#1913) review — the set-theoretic critique of linear sensitivity, org-defined
|
||||
vocabularies, the class+regulatory pairing, taint persistence across storage,
|
||||
per-block byte ranges, sequence-shape, and the false-security risk — are
|
||||
captured with reviewer attributions in [`open-questions.md`](./open-questions.md)
|
||||
rather than silently dropped by the narrower cut.
|
||||
|
||||
**Rationale.** The narrow first cut (`sensitive: boolean`) deliberately omits a
|
||||
lot of debated design. Recording *why*, with links to the people who raised each
|
||||
point, keeps the history visible and gives each parked item a home to graduate
|
||||
from (a scheme, an `action-metadata` field, or a future extension) instead of
|
||||
being re-litigated from scratch.
|
||||
|
|
|
|||
|
|
@ -16,13 +16,50 @@ Tracked here rather than in the spec drafts, so the drafts stay non-temporal.
|
|||
|
||||
## trust-annotations
|
||||
|
||||
- Is `sensitive` the right single coarse signal, or do we need the
|
||||
`data-class.v1` profile from day one?
|
||||
- **Shape of the sensitivity signal.** Is `sensitive: boolean` the right single
|
||||
coarse signal, or does the narrow cut overcorrect? The SEP-1913 thread rejected
|
||||
a linear `sensitiveHint: low|medium|high` because sensitivity is set-theoretic,
|
||||
not a scale ([@JustinCappos](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2967516811):
|
||||
medical results to a mail MCP *and* a card number to a payment MCP, but neither
|
||||
crossed). Reviewers then pushed for org-defined vocabularies
|
||||
([@olaservo](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2968743154),
|
||||
[@Mossaka](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2971788308))
|
||||
and a class+regulatory pairing such as `confidential:hipaa`
|
||||
([@krubenok](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#discussion_r3103485194)).
|
||||
Against that, [@localden](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4037623595)
|
||||
warned a baked-in taxonomy is hard to remove. Open: keep the boolean and let
|
||||
the [`data-class.v1` scheme](../schemes/data-class.md) carry the taxonomy, or
|
||||
also expose a structured escape hatch on the wire so regulated flows are
|
||||
expressible without a scheme?
|
||||
- **Enforcement vs. advisory.** A self-declared `sensitive: true` from a
|
||||
poorly-configured or malicious server could create a false sense of security
|
||||
([@localden](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4037623595)).
|
||||
[docs/trust-model.md](./trust-model.md) puts verification in registries; is
|
||||
that enough without a normative client-side check path?
|
||||
- Content-block-level vs. result-level attachment — does the draft need a
|
||||
worked multi-result example before it's implementable?
|
||||
worked multi-result example before it's implementable? Per-block annotation
|
||||
with **byte/codepoint ranges** was requested so clients can highlight the
|
||||
flagged span, not just the whole result
|
||||
([@connor4312](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-3849207989)).
|
||||
- `list_changed`: confirmed response-level annotations don't participate; revisit
|
||||
only if trust vocabulary ever attaches to tool definitions.
|
||||
|
||||
## Naming (under review)
|
||||
|
||||
These are explicitly unsettled and are being reviewed against the SEP-1913
|
||||
record before any rename lands:
|
||||
|
||||
- **Umbrella name.** The original SEP was "Trust *& Sensitivity* Annotations" —
|
||||
two axes (integrity *and* confidentiality). `trust-annotations` reads as the
|
||||
integrity half; does the name hide the `sensitive` (confidentiality) half?
|
||||
- **`untrusted` vs `openWorldHint`.** Same concept, different layer (result vs.
|
||||
tool definition). Share a name/vocabulary, or keep them deliberately distinct?
|
||||
- **`evidenceRef.type` vs `evidenceRef.scheme`.** The repo calls these values
|
||||
"schemes" (`schemes/`); the selector field is `type`. Align the field name?
|
||||
- **`evidenceRef` itself.** Its ancestor is [@pshkv](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-4196867926)'s
|
||||
`decision_ref` / `attestation_ref` / `policy_profile`. Is `evidenceRef` the
|
||||
clearest umbrella, or should the pointer name the kind of thing it references?
|
||||
|
||||
## action-metadata
|
||||
|
||||
- Coexistence vs. replacement of legacy `destructiveHint` / `readOnlyHint` /
|
||||
|
|
@ -42,8 +79,20 @@ Tracked here rather than in the spec drafts, so the drafts stay non-temporal.
|
|||
|
||||
## Parked (SEP-1913 umbrella)
|
||||
|
||||
- **`maliciousActivityHint`** — if it returns, it is per-`ContentBlock` with
|
||||
spans, driven by the host's own detection, not a server-attested boolean.
|
||||
- **`maliciousActivityHint`** — killed in review on three grounds
|
||||
([@connor4312](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913#issuecomment-3849207989)):
|
||||
it can't be resolved before execution for dynamic tools; a boolean gives no
|
||||
UX or ranges; and clients won't trust a server's self-report of its own
|
||||
maliciousness. If it returns, it is per-`ContentBlock` with spans, driven by
|
||||
the **host's** own detection, not a server-attested boolean.
|
||||
- **Taint persistence across the store boundary** — a label must survive
|
||||
round-tripping through storage: write a card number to a file, read it back,
|
||||
and the sensitivity label must not silently disappear
|
||||
([@JustinCappos](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711#issuecomment-2967516811)).
|
||||
This is a propagation property no single response annotation can guarantee on
|
||||
its own; it needs the taxonomy and `evidenceRef` stable first.
|
||||
- **Session-level propagation rules** — escalation semantics and the
|
||||
sequence-shape gap ("this was call N in a flagged sequence" has no response
|
||||
annotation surface today).
|
||||
**sequence-shape** gap: "this was call N in a flagged sequence" has no response
|
||||
annotation surface today ([marras0914](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1913)).
|
||||
A candidate `evidenceRef` scheme could carry a sequence assertion; tracked in
|
||||
[`schemes/README.md`](../schemes/README.md).
|
||||
|
|
|
|||
|
|
@ -15,10 +15,22 @@ annotation work. Several were surfaced in IG meetings (notably 2026-05-28).
|
|||
## Research
|
||||
|
||||
- **FIDES** — *Information-flow control for LLM agents.* [arXiv:2505.23643](https://arxiv.org/abs/2505.23643). Basis for the `ifc.fides.v1` scheme in [`schemes/`](../schemes/).
|
||||
- **Design Patterns for Securing LLM Agents** — IBM/Google/Microsoft. [arXiv:2506.08837](https://arxiv.org/abs/2506.08837). Plan-Then-Execute, Dual LLM, Map-Reduce, etc.
|
||||
- **Permissive Information-Flow Analysis for LLMs** — relaxes IFC join so a label propagates only when an input actually influences an output. [arXiv:2410.03055](https://arxiv.org/abs/2410.03055). Candidate `evidenceRef` scheme (per-result label), like FIDES.
|
||||
- **AirGapAgent** — contextual-integrity minimisation: restrict per-task data to what the context warrants. [arXiv:2405.05175](https://arxiv.org/abs/2405.05175). Candidate scheme: emits a contextual-integrity classification per result.
|
||||
- **CaMeL — Defeating Prompt Injections by Design** — capability-based control/data-flow extraction. [arXiv:2503.18813](https://arxiv.org/abs/2503.18813). A **host architecture**, not a data-label scheme (see note below); a capability token it issues could be referenced via `evidenceRef`, but the architecture itself is not a scheme.
|
||||
- **Design Patterns for Securing LLM Agents** — IBM/Google/Microsoft. [arXiv:2506.08837](https://arxiv.org/abs/2506.08837). Plan-Then-Execute, Dual LLM, Map-Reduce, etc. Also **host architectures**, not schemes.
|
||||
- **Trail of Bits** — prompt-injection via hidden content in GitHub issues. [blog](https://blog.trailofbits.com/2025/08/06/prompt-injection-engineering-for-attackers-exploiting-github-copilot/).
|
||||
- **OpenAI Auto Review** — https://alignment.openai.com/auto-review/ (shared in IG chat).
|
||||
|
||||
### Schemes vs. host architectures
|
||||
|
||||
The `evidenceRef` slot carries **data labels** — a per-result record a server
|
||||
can attach (FIDES, Permissive IFC, AirGapAgent, data-class, attestation
|
||||
envelopes). It does **not** carry **host architectures** — control-flow designs
|
||||
the *client/host* runs (CaMeL, the Design-Patterns catalogue, Dual-LLM). These
|
||||
are complementary: an architecture decides what to do with a label, the label is
|
||||
what a scheme produces. Only the former belong in [`schemes/`](../schemes/).
|
||||
|
||||
## Implementations & tooling
|
||||
|
||||
- [`kapil8811/mcp-trust-annotations`](https://github.com/kapil8811/mcp-trust-annotations) — reference Python SDK PoC for `trust-annotations`.
|
||||
|
|
|
|||
|
|
@ -137,6 +137,26 @@ are present, the content-block annotation refines the result-level one for that
|
|||
block; it MUST NOT *weaken* a result-level claim (union semantics — once
|
||||
`true`, stays `true`).
|
||||
|
||||
### Relationship to existing `*Hint` annotations
|
||||
|
||||
MCP tool definitions already carry an `openWorldHint` (alongside `readOnlyHint` /
|
||||
`destructiveHint` / `idempotentHint`). `untrusted` is deliberately **not** a
|
||||
synonym for `openWorldHint`:
|
||||
|
||||
- `openWorldHint` is a property of the **tool definition** — "this tool reaches
|
||||
an open, attacker-influenceable world" (e.g. a web fetch). It is known at
|
||||
registration time and does not vary per call.
|
||||
- `untrusted` is a property of a **specific result** — "*this* returned data
|
||||
originated from an open-world / untrusted source." A tool whose `openWorldHint`
|
||||
is `true` may still return trusted data on a given call, and a tool whose
|
||||
`openWorldHint` is `false` can surface untrusted data it read from storage.
|
||||
|
||||
The original SEP-1913 proposal reused `openWorldHint` for the result-level
|
||||
"untrusted source" signal and then drew exactly this distinction
|
||||
([issue #711](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/711)).
|
||||
Whether the two should share a name or vocabulary is an
|
||||
[open question](../../docs/open-questions.md).
|
||||
|
||||
### Lifecycle and `list_changed`
|
||||
|
||||
Trust annotations defined here are **response-level**: they describe a specific
|
||||
|
|
|
|||
Loading…
Reference in a new issue