--- title: "ADR-058: Telemetry Middleware" manual: "TYPO3 LLM Extension" version: "0.35" permalink: "https://docs.typo3.org/permalink/netresearch/nr-llm:adr-058@0.35" source: "Adr/Adr058TelemetryMiddleware.rst" modified: "2026-09-16T22:09:16+00:00" --- # ADR-058: Telemetry Middleware - *Status:* Accepted - *Date:* 2026-07 - *Amended:* 2026-08-11 by [ADR-153](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-153@0.35) - *Authors:* Netresearch DTT GmbH ## Context The middleware pipeline ([ADR-026: Provider Middleware Pipeline](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-026@0.35)) records cost and tokens through `UsageMiddleware`, but only for calls that *succeed*. Its own doc block names the gap: > The middleware never runs when `$next` throws: failed calls are not > tracked here. If failure-rate telemetry is needed later, a dedicated > middleware can wrap and record regardless of outcome. So today there is no durable record of: - how often a provider call *fails*, and with which exception type; - how long a call takes (latency), including the cache lookup; - whether a response was served from cache; - how many fallback configurations had to be tried before one answered. `UsageMiddleware` writes to `tx_nrllm_service_usage`, which is a **daily aggregate** keyed by service/provider/model/user — it cannot answer "which correlation id failed at 14:03 and why". Correlation ids exist on the `ProviderCallContext` and are already logged by `FallbackMiddleware`, but logs are not queryable telemetry. ## Decision Add a dedicated `TelemetryMiddleware` and a per-request log table. **1\. Table** `tx_nrllm_telemetry` — one immutable row per pipeline run (not an aggregate). Columns: `correlation_id`, `operation`, `provider`, `model`, `configuration_identifier`, `be_user`, `success`, `error_class`, `latency_ms`, `cache_hit`, `fallback_attempts`, `crdate`. No TCA / backend UI — it is a log read via SQL / analytics, like the other UI-less tables (`tx_nrllm_service_usage`, `tx_nrllm_tool_state`). **2.** `TelemetryMiddleware` **at priority 110** — the outermost layer, *outside* Cache. It measures wall-clock latency with `hrtime()` around `$next`, catches any `Throwable`, writes **exactly one** row on both success and failure, and re-throws the exception unchanged. Latency therefore includes the cache lookup, and a cache-served response still produces a row. **3\. Cache-hit signal.** The pipeline threads one immutable `ProviderCallContext` through every layer and the `$next` callable only forwards the `LlmConfiguration` — never a context. An inner middleware thus cannot hand a modified context back to an outer one. The one channel that survives the unwind is a mutable object reachable from the shared context: `TelemetrySignals`. The context default-constructs one per call. `CacheMiddleware` calls `recordCacheHit()` on a hit; the outer `TelemetryMiddleware` reads it. **4\. Fallback count.** `FallbackMiddleware` calls `recordFallbackAttempt()` on `TelemetrySignals` once per fallback configuration it actually dispatches (the primary attempt is not counted); the middleware reads it (`0` when it was never set). **5\. Attribution.** `be_user` is the caller-supplied `BudgetMiddleware::METADATA_BE_USER_UID` when present, else the ambient `backend.user` context aspect, else `0` — the same resolution `UsageTrackerService` uses. **6\. Persistence** goes through a narrow `TelemetryRepository` (Doctrine `ConnectionPool` directly, no Extbase repository), mirroring how `UsageTrackerService` writes. The service is **private** (ADR-028); nothing resolves it by class name from the container. **7\. Purge command** `nrllm:telemetry:purge` (`--days`, default 30) deletes rows older than the retention window. Registered via the native `#[AsCommand]` attribute (autoconfigured), like TYPO3 core commands. **8\. Deactivation.** The extension setting `telemetry.enabled` (default **on** — observability by default) turns the middleware into a verbatim pass-through when disabled. ## Consequences - **One row per request = growth.** Unlike the usage aggregate, the log grows with traffic. The `nrllm:telemetry:purge` command bounds it; run it from the scheduler. - **error_class, not message — a deliberate privacy trade-off.** Only the exception FQCN is stored. Exception messages can carry payload fragments (a prompt substring, a URL with a token), so they are never persisted here. No prompts and no responses are stored either. The central privacy model (retention tiers none/metadata/redacted/full) is a later workstream; this middleware is metadata-only by construction. - **Latency includes the cache lookup.** Because Telemetry sits outside Cache, `latency_ms` measures the whole pipeline as the caller experiences it, cache hits included. That is the intended semantic (end-to-end latency), not provider-only time. - **provider / model reflect the requested primary configuration.** Telemetry sits outside `FallbackMiddleware`, so it records the configuration the caller *asked for*. A fallback swap shows up as `fallback_attempts > 0`; the provider/model/cost of the configuration that actually *served* live in the usage table (`UsageMiddleware` sees the served config). Ad-hoc direct calls carry no attached model, so provider/model are empty and the provider is encoded in the `ad-hoc::` identifier. > [!NOTE] > **Overtaken.** The row no longer names only the requested configuration. > `served_configuration_identifier`, `served_provider` and > `served_model` name the one that answered, carried out of the pipeline > through the same `TelemetrySignals` scratchpad as the cache hit > (`recordServedBy()`, called only after the sibling's call *returned*). > `fallback_attempts` still counts the hops; it never said which > configuration answered, and an exhausted chain has hops but no swap. > The requested columns above are unchanged. - **Fail-soft.** A telemetry write error is logged and swallowed; it never breaks the call it observes. - **Streaming produces no telemetry row.** Streaming deliberately stays out of the pipeline ([ADR-026: Provider Middleware Pipeline](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-026@0.35)) — once the first chunk is emitted a provider cannot be swapped mid-stream. A streaming lifecycle (with its own telemetry) is a separate workstream. - **Mutable state on an "immutable" context.** `TelemetrySignals` is the one mutable object the otherwise-immutable `ProviderCallContext` carries. It holds only cross-cutting observability state, never payload, so it does not weaken ADR-026's "payload stays in the terminal closure" rule. A dedicated typed property (rather than a magic metadata key seeded by every caller) means every pipeline run — present and future entry points — captures cache/fallback signals with no per-caller wiring. ## Alternatives considered - **Extend** `UsageMiddleware` **to also record failures.** Rejected: usage is an aggregate keyed for cost roll-ups; per-request failure/latency rows have a different shape, lifetime (purged) and index set. Overloading one table would break both. - **Log-only (PSR-3), no table.** Rejected: logs are not queryable telemetry. Failure-rate and latency questions need a table an analytics view can group. - **Seed a mutable signal bag into the metadata map at each pipeline entry point.** Rejected: it pushes telemetry wiring into every caller and silently loses cache/fallback signals for any future entry point that forgets to seed. A default-constructed context property covers all runs by construction. - **Record provider/model from the response.** Rejected for the outer layer: there is no response on the failure path, and duplicating `UsageMiddleware`'s response-shape extraction at the outermost layer couples telemetry to every response type. Sourcing from the requested configuration is deterministic on success *and* failure. ## References - [ADR-026: Provider Middleware Pipeline](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-026@0.35) — Provider Middleware Pipeline (the pipeline and ordering this extends; `UsageMiddleware`'s open failure-telemetry note). - ADR-025 — Per-User AI Budgets (`be_user` attribution key reused here). - ADR-028 — Public Services Policy (the recorder stays private).