---
title: "Streaming support"
manual: "LLM Foundation"
version: "0.43"
permalink: "https://docs.typo3.org/permalink/netresearch/nr-llm:developer-streaming@0.43"
source: "Developer/Streaming.rst"
rendered: "2026-11-04T08:34:12+00:00"
---

# Streaming support {#developer-streaming}

Streaming allows you to receive LLM responses incrementally as they are
generated, rather than waiting for the complete response. This improves
perceived performance for long responses.

![Streaming: the prompt is screened, the model and adapter are resolved, the budget is checked, the dispatcher opens the provider stream with fallback, and each chunk passes a sliding redaction window before the Generator yields it to the caller.](../Images/diagram-streaming-flow.svg)

## Usage {#usage}

**Example: Streaming chat responses**

```php
$stream = $this->llmManager->streamChat($messages);

foreach ($stream as $chunk) {
    echo $chunk;
    ob_flush();
    flush();
}
```

The `streamChat` method returns a `Generator` that yields string chunks
as the provider generates them. Each chunk contains a portion of the response
text.

On a configuration a [term list](https://docs.typo3.org/permalink/netresearch/nr-llm:administration-term-lists-block@0.43) names,
the stream is held and checked as a whole: it delivers one chunk at the end, or
throws `GuardrailViolationException` on the first iteration. The exception
comes out of the `foreach`, not out of `streamChat()`, so wrap the loop.
An agent run that offers tools is not held: there the list checks what a writing
tool would store.

**Example: A stream a guardrail can stop**

```php
try {
    foreach ($this->llmManager->streamChat($messages) as $chunk) {
        echo $chunk;
    }
} catch (GuardrailPolicyException $e) {
    echo $e->getMessage();
}
```

## Audit and accounting {#audit-and-accounting}

After a successful stream, the dispatcher screens at most the first 50,000
raw response bytes for its guardrail audit. A large provider chunk cannot
extend this audit buffer. The audit cannot retract text already delivered;
live redaction uses its separate sliding window.

The audit limit does not truncate delivered chunks or consumption accounting.
Streaming token estimates count the complete raw output, including bytes
beyond that prefix and bytes masked by live redaction.

Providers that implement [StreamingCapableInterface](https://docs.typo3.org/permalink/netresearch/nr-llm:api-provider-streaming@0.43) support
streaming. Check provider capabilities before using:

**Example: Checking streaming support**

```php
use Netresearch\NrLlm\Provider\Contract\StreamingCapableInterface;

$provider = $this->llmManager->getProvider('openai');
if ($provider instanceof StreamingCapableInterface) {
    // Provider supports streaming
}
```

## Transport failures {#transport-failures}

The bundled adapters' streaming methods send each request once. They propagate
an exception raised by the PSR-18 transport without wrapping it as an nr_llm
exception. In particular, `Psr\Http\Client\NetworkExceptionInterface`
does not implement nr_llm's marker merely because it passed through an adapter.
This differs from the buffered adapters' transport-error normalization.

The manager's streaming dispatcher classifies a PSR-18 network exception as a
retryable connection failure. It can try the configured fallback before the
first chunk is yielded. After output has begun, it propagates the failure;
switching providers would combine two different answers. The dispatcher also
propagates the last retryable failure when no fallback can serve the call.
Callers that handle streaming transport failures should catch the PSR-18
network interface in addition to their nr_llm exception cases.