---
title: "ADR-021: Provider Fallback Chain"
manual: "TYPO3 LLM Extension"
version: "0.35"
permalink: "https://docs.typo3.org/permalink/netresearch/nr-llm:adr-021@0.35"
source: "Adr/Adr021ProviderFallbackChain.rst"
modified: "2026-09-16T22:09:16+00:00"
---

# ADR-021: Provider Fallback Chain

-   *Status:*

    Accepted (the streaming scope limitation is overtaken — see
    [Scope limitations (v1)](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-021-scope@0.35))

-   *Date:* 2026-04
-   *Authors:* Netresearch DTT GmbH

## Context

A single misbehaving provider (OpenAI rate-limit, Claude outage, local Ollama
daemon not running) previously bubbled up as an uncaught exception to every
consuming extension. Operators had no built-in way to degrade gracefully to a
second or third provider.

## Decision

A configuration's `fallback_chain` column stores an ordered JSON list of
other `LlmConfiguration` identifiers. On retryable failures during
`LlmServiceManager::chatWithConfiguration()` or
`completeWithConfiguration()`, `FallbackMiddleware` (a stage of the
provider middleware pipeline, [ADR-026](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-026@0.35)) walks the chain and
returns the first successful response — or throws
`FallbackChainExhaustedException` carrying every attempt error.

"Retryable" is narrowly defined: the request *might* succeed against a
different provider.

-   `ProviderConnectionException` — network / timeout / HTTP 5xx /
    retries exhausted
-   `ProviderResponseException` with HTTP code 429 — this provider is
    rate-limiting us, another might not be

Everything else (authentication, bad request, unsupported feature,
misconfiguration) bubbles up unchanged — a different provider won't help.

## Scope limitations (v1)

-   **Streaming is not wrapped.** Once the first chunk has been yielded, we
    cannot swap providers mid-stream. `streamChatWithConfiguration()`
    calls the primary adapter directly.

    > [!NOTE]
    > Overtaken. `StreamingDispatcher::openWithFallback()` walks the same
    > chain before the first chunk, so a primary that fails to open is no
    > longer fatal for a streamed call. The sentence that still holds is the
    > narrower one: no swap is possible *after* the first chunk has been
    > yielded, which is why the streaming chain is tried at open time only.
-   **Shallow only.** A fallback configuration's own chain is ignored. This
    prevents both cycles (`a -> b -> a`) and exponential blow-up of attempts.
-   **Inactive fallbacks are skipped**, not treated as failures.
-   **Missing identifiers are skipped** with a warning log, not treated as
    failures. Misconfiguration should not mask outages.

## Storage

The chain is stored as a single JSON column to keep the schema change
minimal and avoid an additional relation table. The
`Netresearch\NrLlm\Domain\DTO\FallbackChain` value object handles
serialization, deduplication, and order preservation.

TCA presents the field as a JSON textarea for v1. A richer UI (sortable
multi-select of available configurations) can replace the textarea without
schema or API change.

## Alternatives considered

-   **Fat middleware pipeline** (as in b13/aim). Rejected for this release —
    too invasive for a single-feature change. The middleware pattern remains
    on the roadmap as a v1.0 refactor; a fallback chain is the most valuable
    pipeline step users ask for and works fine as a standalone service.

    > [!NOTE]
    > Lifted. This rejection was scoped to one release ("for this release"),
    > and [ADR-026: Provider Middleware Pipeline](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-026@0.35) built the pipeline it deferred. Fallback runs as a
    > middleware step today. Cite this ADR against a *recursive* chain or a
    > *per-link* retry policy — the two rejections below, which stand
    > unconditionally — not against pipeline work as such.
-   **Recursive chain resolution** (fallback's fallback). Rejected as the
    cost (cycle detection, attempt amplification) outweighs the benefit;
    operators can always append to the primary's chain directly.
-   **Per-link retry policy** (per fallback: max retries, backoff, which
    exceptions). Rejected as over-engineered for the initial release.