ADR-041: Playground live run streaming
- Status
-
Accepted
- Date
-
2026-07-06
- Authors
-
Netresearch DTT GmbH
Context
The playground inspector (ADR-040) ran the whole bounded agent loop server-side and returned the complete trace as one JSON document. The browser therefore showed nothing until the entire run — every model round-trip and tool execution — had finished, then rendered all at once. For a multi-round run against a slow local model that is several seconds of a blank pane, and it hides when each step happened.
The goal is a live inspector: each step appears the moment it is recorded.
Decision
Stream the run as newline-delimited JSON (NDJSON), one event per line, over the existing admin AJAX route:
- Opt-in. The client sends
stream=1; the controller then streams. Without it the controller keeps the batch path — oneresponddocument — which remains the no-JavaScript fallback and the shape the functional tests assert.Json () - Per-step callback.
Runtakes an optionalTrace onRecordclosure fired the instant eachRunis recorded. The streaming controller passes a closure that echoes oneStep {"event":"step","step":…}line and flushes; a final{"event":"done",…}(or{"event":"error",…}) line carries the summary. When no closure is passed (every production and test caller) the loop is byte-for-byte unchanged. - Beat the proxy buffer. A TYPO3 backend response is buffered by the reverse
proxy until a chunk clears its flush threshold, so small lines all arrive at
the end. The stream disables output buffering and zlib compression at runtime
and pads every line past 4 KB with trailing whitespace (ignored by
JSON.parse), which makes each event flush immediately. - NullResponse. Having written output directly, the controller returns
TYPO3CMSCore, whichHttp Null Response Abstractskips — TYPO3 emits nothing further, avoiding a double body or a headers-already-sent warning.Application:: send Response () - Same UTF-8 guard. Each line is encoded with
JSON_INVALID_UTF8_SUBSTITUTE(as ADR-040 established for the batch response), so a malformed byte substitutes rather than aborting the stream.
The client reads the response body stream, splits on newlines, parses each line, appends the step to a live trace and re-renders. If the browser cannot read a streaming body it falls back to the batch request.
Consequences
- Steps render as they happen; the summary strip fills in live from the
per-round token counts and finalises on
done. - The 4 KB padding is transfer overhead (a few KB per event) that never reaches the user — an accepted cost for reliable incremental flushing across proxies.
- Direct
echo/flushplusNullResponseis deliberately outside the PSR-7 body abstraction; it is confined to the one streaming method and the batch path stays a normal response. - A run whose final model round stops on
finishReason: lengthis now flagged with a truncation banner, and Max tokens/Temperature are exposed as per-run controls so the operator can lift the cap.