---
title: "ADR-040: Playground run trace and tool-path prompt augmentation"
manual: "TYPO3 LLM Extension"
version: "0.35"
permalink: "https://docs.typo3.org/permalink/netresearch/nr-llm:adr-040@0.35"
source: "Adr/Adr040PlaygroundRunTrace.rst"
modified: "2026-09-16T22:09:16+00:00"
---

# ADR-040: Playground run trace and tool-path prompt augmentation

-   *Status:* Accepted
-   *Date:* 2026-07-05
-   *Authors:* Netresearch DTT GmbH

## Context

The admin playground ([ADR-038](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-038@0.35)) ran the bounded agent loop
and returned only the final answer plus a flat list of executed tool calls
and a single total-token count. For an admin whose job is to understand *how*
a configuration behaves — and to debug a task, prompt or extension that
misbehaves — that is a black box. The information needed to reason about a run
was either discarded or never captured:

-   the exact messages sent to the provider each round (system prompt, injected
    skills, snippets, the growing tool dialog) were assembled and thrown away;
-   the raw provider response was parsed into `CompletionResponse` and
    discarded in every adapter except OpenRouter;
-   per-call latency was never measured, and the prompt/completion token split
    was summed away into one total;
-   the model's intermediate turns and `thinking` never surfaced.

Two capability gaps compounded this. Skill prose was injected for the
text-generation paths but **not** in the tool-loop path — the loop was the
sole caller of
`LlmServiceManager::chatWithToolsForConfiguration()` and applied no
skills. Prompt snippets had a composer but **no runtime caller at all**.

## Decision

Add an opt-in **run trace** and one-time **prompt assembly** to
`ToolLoopService`, consumed by the playground; production callers are
unaffected.

### Run trace

`Netresearch\NrLlm\Service\Tool\RunTrace` is a mutable recorder passed
into `runLoop()` as an optional, nullable argument. When present the loop
records a readonly `RunStep` per model round-trip (messages sent, tools
offered, content, thinking, finish reason, the prompt/completion/total token
split, estimated cost and the requested tool calls) and per executed tool
call (name, arguments, result, error flag). Timing is measured with
`hrtime()` around each provider call and each tool invocation — no new
middleware. When no `RunTrace` is passed — **every production caller** —
the loop records nothing and behaves exactly as before.

### One-time prompt assembly

The loop assembles the outgoing prompt once, before the first round:

-   **Configuration skills now inject into the tool path.** Because the loop is
    the sole caller of `chatWithToolsForConfiguration()` and re-sends its own
    accumulating message array (`augmentMessages()` returns a new list and
    never mutates the input), injecting once before the loop closes the gap
    without double-injecting. This is a behaviour change for the tool loop:
    configurations with attached skills now carry that prose into tool runs, as
    the text-generation paths already did.
-   A `RunAugmentation` adds the playground-only extras: **forced skills**
    (injected as additional task skills), **forced snippets** (added as separate
    leading system messages, one per snippet), and a **dry-run** mode that
    assembles and records the messages without calling the provider. When an
    augmentation is present the effective system prompt (a per-run override wins
    over the configuration's) is baked as the first message, so the snippet
    system messages cannot suppress it.

### Gated raw-response capture

`ToolOptions::withCaptureRaw()` sets a private `_capture_raw` directive
that flows through the call options to the adapters. When set, each adapter
stores the decoded provider body under `metadata['_raw']` via
`AbstractProvider::rawResponseMetadata()`. It is **off by default**, so
production calls never retain raw payloads — only the admin playground opts
in, and the module is admin-only.

## Consequences

-   The playground surfaces the full nr_llm ↔ LLM dialog: assembled prompt,
    per-round timing and token split, requested tool calls, thinking, the raw
    response (on demand) and the tool executions.
-   Skills attached to a configuration now influence tool runs, matching the
    text-generation paths; snippets have a first runtime consumer.
-   `RunTrace` and the augmentation collaborators are optional and
    autowired, mirroring the optional `SkillInjectionService` on
    `LlmServiceManager`; the production tool path and existing lean test
    wiring are unchanged.
-   Raw capture touches all seven adapters, but only along the gated path; the
    non-capture response is byte-for-byte identical.