ADR-085: Guardrail pipeline for provider responses
- Status
-
Accepted
- Date
-
2026-07-18
- Authors
-
Netresearch DTT GmbH
Context
nr-llm has good, targeted safety mechanisms (skill/tool egress, budgets, secret sanitisation on logged errors), but no general content-policy layer over a provider response. LLM output is untrusted content, and any editor-facing or autonomous use needs to be able to inspect it, rewrite it, block it, or route it for review — with a richer answer than a boolean.
Decision
A guardrail is a `GuardrailInterface` whose check returns a
Guardrail verdict — ALLOW, REDACT, RETRY, REQUIRE_APPROVAL, or DENY.
Guardrails are auto-collected through the nr_llm.guardrail DI tag (the same
pattern as nr_llm.tool), so a new guardrail is active simply by existing
under Classes/.
`GuardrailMiddleware` runs them in the existing ADR-026 provider pipeline, at
priority 115 — outermost, above Telemetry (110) (superseded — see the
2026-07-19 update under Consequences: the guardrail moved to priority 90, inside
the persistence layers). It screens every non-streaming Completion
after the downstream chain produces it:
- ALLOW passes on; REDACT rewrites the content and keeps screening (a later
guardrail may still deny); DENY throws
Guardrail; REQUIRE_APPROVAL throwsViolation Exception Guardrail; RETRY re-asks the provider once and re-screens the fresh response (capped at one retry).Approval Required Exception - It sits above Telemetry deliberately: a guardrail denial is a policy outcome, not a provider failure, so provider telemetry stays accurate.
- Non-
Completionresults (embeddings, vision, raw payloads) pass through untouched.Response
Two reference guardrails ship active: Secret (REDACT —
masks secret-shaped strings a model may have echoed, reusing
Error plus API-key/Bearer patterns) and
Provider (DENY — turns a silent content_filter
response into an explicit, catchable denial).
Consequences
Update 2026-07-19 — corrected pipeline placement (priority 115 → 90). Placing
the guardrail outermost meant it redacted the response only AFTER
Idempotency (105) had already serialised the unredacted
Completion into the nrllm_idempotency cache (24h, possibly a shared
backend) — so a model that echoed a secret leaked it into persistence even though
the caller saw the redacted value. The guardrail now runs at priority 90,
INSIDE both persistence layers (Idempotency 105, Cache 100) and above the
behavioural stack, so it redacts (or blocks) before anything is stored; a
DENY/REQUIRE_APPROVAL throws before the store, so a blocked response is never a
replayable result. Telemetry accuracy is preserved differently now that the
guardrail sits inside Telemetry (110): the two guardrail exceptions implement a
Guardrail marker, and Telemetry records such a policy
outcome as a successful provider run (the provider produced a response; the
guardrail refused to release it) — so a denial still does not distort the
provider failure-rate. A side benefit: a RETRY now genuinely re-runs the provider
instead of replaying the idempotency-cached response.
- Consumers get allow/redact/deny/retry/require-approval over model output, not
true/false. A denial is a typed, catchable exception (mirroring
Budget).Exceeded Exception - Behaviour change: a provider
content_filterresponse now raisesGuardrailinstead of returning silently — the degraded/empty response surfaces rather than passing through.Violation Exception - Output only, for now. This screens the response. Screening the prompt
(input redaction / injection detection) is a separate step: the prompt payload
is captured in the pipeline's terminal closure, not on the immutable
Provider, so input guardrails require threading the messages through the context first — a scoped follow-up, not built here.Call Context - Streaming is not covered.
Streamingbypasses the middleware pipeline (ADR-062), so streamed responses are not screened. Covering them is a separate integration in the streaming path.Dispatcher - The REQUIRE_APPROVAL verdict is the seam to human review: on a plain completion
it raises
Guardrail(distinct from a denial) so a consumer with a run/review context (the human-in-the-loop epic, ADR-084, or a review queue) can route it to approval instead of an error.Approval Required Exception