Getting started 

For administrators 

Set up and manage AI providers, models, and configurations through the TYPO3 backend module.

For developers 

Build your TYPO3 extension on nr-llm — three lines of dependency injection, no API key handling.

[n] A Netresearch extension 

Professional TYPO3 development, AI integration, and enterprise consulting since 2002.

netresearch.de

Table of contents

What does it do? 

nr-llm is the shared AI foundation for TYPO3. It lets administrators configure LLM providers once in the backend — and every AI-powered extension on the site uses them automatically.

For extension developers, it eliminates the need to build provider integrations, manage API keys, or implement caching and streaming. Add AI capabilities to your extension with three lines of dependency injection.

For administrators, it provides a backend module tree to manage all AI connections, encrypted API keys, and provider configurations — plus a dedicated AI Tasks module in the Web area where editors run prepared tasks and decide pending approvals, gated by explicit permissions. Switch from OpenAI to Anthropic without touching any extension code.

For agencies, it means consistent AI architecture across client projects, no vendor lock-in, and a local-first option via Ollama for data-sensitive environments.

The extension enables developers to:

• Access multiple AI providers through a single, consistent API.
• Switch providers transparently without code changes.
• Leverage specialized services for common AI tasks (translation, vision, embeddings).
• Cache responses to reduce API costs and improve performance.
• Stream responses for real-time user experiences.
• Store API keys securely as nr-vault identifiers (envelope encryption).

Key features 

// Use database configurations for consistent settings
$config = $configRepository->findByIdentifier('blog-summarizer');
$adapter = $adapterRegistry->createAdapterFromModel($config->getModel());
$response = $adapter->chatCompletion($messages, $config->toOptions());

// Or use inline provider selection
$response = $llmManager->chat($messages, ['provider' => 'openai']);
$response = $llmManager->chat($messages, ['provider' => 'claude']);

foreach ($llmManager->streamChat($messages) as $chunk) {
    echo $chunk;
    flush();
}

$response = $llmManager->chatWithTools($messages, $tools);
if ($response->hasToolCalls()) {
    // Process tool calls
}

Use cases 

Requirements 

• PHP: 8.2 or higher.
• TYPO3: v13.4 LTS or v14.3 LTS.
• HTTP client: PSR-18 compatible (e.g., guzzlehttp/guzzle ).

Credits 

This extension is developed and maintained by:

Netresearch DTT GmbH

https://www.netresearch.de

Built with the assistance of modern AI development tools and following TYPO3 coding standards and best practices.

Quick start 

The recommended way to install this extension is via Composer:

composer require netresearch/nr-llm

After installation:

1. Activate the extension in Admin Tools > Extension Manager.
2. Configure providers and API keys in Admin Tools > LLM > Providers.
3. Define available models in Admin Tools > LLM > Models.
4. Create configurations in Admin Tools > LLM > Configurations.
5. Clear caches.

Composer installation 

Manual installation 

If you cannot use Composer:

1. Download the extension from the TYPO3 Extension Repository (TER).
2. Extract to typo3conf/ext/nr_llm.
3. Activate in Admin Tools > Extension Manager.
4. Configure API keys and settings.

Database setup 

The extension creates the following database tables automatically:

Run the database compare tool after installation:

vendor/bin/typo3 extension:setup nr_llm

Cache configuration 

The extension uses TYPO3's caching framework. Cache configuration is set up automatically — no backend is hardcoded. TYPO3 uses your instance's default cache backend, so Redis, Valkey, or Memcached work transparently if configured.

To override the cache backend specifically for nr-llm:

use TYPO3\CMS\Core\Cache\Backend\RedisBackend;

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']
    ['cacheConfigurations']['nrllm_responses']
    ['backend'] = RedisBackend::class;

Upgrading 

Uninstallation 

To remove the extension:

1. Deactivate in Admin Tools > Extension Manager.

2. Remove via Composer:

   Remove the extension

   composer remove netresearch/nr-llm

   Copied!

3. Clean up database tables if desired:

   Drop extension database tables

   DROP TABLE IF EXISTS tx_nrllm_provider;
   DROP TABLE IF EXISTS tx_nrllm_model;
   DROP TABLE IF EXISTS tx_nrllm_configuration;
   DROP TABLE IF EXISTS tx_nrllm_configuration_begroups_mm;
   DROP TABLE IF EXISTS tx_nrllm_task;
   DROP TABLE IF EXISTS tx_nrllm_prompttemplate;
   DROP TABLE IF EXISTS tx_nrllm_service_usage;

   Copied!
4. Remove any TypoScript includes referencing the extension.

The LLM backend module 

All AI management happens in Admin Tools > LLM; editors run prepared tasks and decide approvals in the separate Web > AI Tasks module (Backend user permissions). The Overview is a guided starting point:

• a usage & cost band across the top — 30-day cost, requests and tokens, the per-provider request mix, and a daily-requests sparkline (empty until the first request);
• a unified Set up & manage grid where each module card carries its own setup state — green when it is configured, a blue Next flag on the single recommended step, and Empty on an optional module with no entries yet — so the next action is always visible without a separate wizard. Each card links to its module;
• the Providers card shows a live, token-free reachability indicator per configured provider (a model-list/health ping, never a completion);
• a For developers section showing how to call the same configuration from PHP via LlmServiceManager.

The Overview's docheader carries a Governance tab — the read-only effective policy readout (ADR-140). It lists the four governance keys that carry a decision (privacy.level, privacy.retentionDays, tools.dataClassEnforcement, skills.minTrustLevel) with the value the runtime applies right now and the class that resolved it. Two things it deliberately does not do: it never changes a value — instance-wide keys are set in the Install Tool under Settings > Extension Configuration > nr_llm — and it never shows a value it did not get from a resolver, so a row that cannot be answered reads unknown rather than a default. Because the reads go through the runtime resolvers, the tab shows what is in force: a mistyped tools.dataClassEnforcement reads enforce, because the gate is fail-closed (ADR-113). The keys, their fallbacks and recommended settings are documented under Effective policy.

Two rows say more than their value alone:

• tools.dataClassEnforcement = observe is annotated as applying to built-in tools only. Tools reached through an MCP server are always enforced against the trust-zone ceiling (ADR-115), so an MCP tool can be dropped while this row reads observe.
• Every privacy.retention.<category> override that deviates from privacy.retentionDays gets its own row underneath it. Categories left at 0 resolve to the global window and are not listed — see Data retention & purge for the full set.

The admin module tree has fourteen sections accessible from the left-hand navigation:

• Overview — guided dashboard: usage & cost, per-module setup state, and the developer guide
• Providers — API connections
• Models — available LLM models
• Configurations — use-case presets
• Tasks — one-shot prompt templates
• Snippets — tagged reusable prompt fragments
• Get Started — pick a use case and install a matching pack of configuration, tasks and snippets (admin-only)
• Setup wizard — guided provider, model and configuration setup (admin-only)
• Skills — GitHub-hosted SKILL.md sources (admin-only)
• Tools — enable or disable the agent tools (admin-only)
• MCP servers — configure external MCP servers and import their tool catalogues (admin-only)
• Playground — run the agent tool loop interactively (admin-only)
• Agent runs — review and decide runs paused for approval or input (admin-only)
• Analytics — usage and cost dashboard (admin-only)

Editors do not use this tree: their surface is the separate Web > AI Tasks module, opened per backend group through the permission grants.

Adding a provider 

1. Navigate to Admin Tools > LLM > Providers.
2. Click Add Provider.

3. Fill in the required fields:

   Identifier

   A unique slug for programmatic access (e.g., openai-prod, ollama-local).

   Name

   A display name for the backend (e.g., OpenAI Production).

   Adapter Type

   Select the provider protocol. Available adapters: openai, anthropic, gemini, ollama, openrouter, mistral, groq, azure_openai, custom.

   API Key

   Your API key. Stored securely via nr-vault envelope encryption. Leave empty for local providers like Ollama.

4. Optionally set the endpoint URL, organization ID, timeout, and retry count.
5. Click Save.

Setting the key from the command line 

An unattended install cannot operate the wizard. nrllm:provider:set-key does the same job for a provider record that already exists, reading the key from STDIN:

printf '%s' "$OPENAI_API_KEY" | \
    vendor/bin/typo3 nrllm:provider:set-key openai

The key is never accepted as an argument — that would put it in the process list and the shell history. A terminal is refused rather than read, so a provisioning script fails visibly instead of hanging on a prompt.

Running it again for the same provider replaces the stored key and keeps the identifier, so anything already referring to that identifier — including providers.openai.apiKeyIdentifier in the extension configuration, which the speech and image services read — keeps working. See ADR-124.

Testing a connection 

After saving a provider, click Test Connection to verify the setup. The test makes an HTTP request to the provider API and reports:

• Connection status (success or failure).
• Available models (if the provider supports listing).
• Error details on failure.

$GLOBALS['TYPO3_CONF_VARS']['HTTP']['allowed_hosts'][] = 'ollama';

Editing and deleting providers 

• Click a provider row to edit its settings.
• Use the Delete action to remove a provider. Models linked to a deleted provider become inactive.

Adding a model manually 

1. Navigate to Admin Tools > LLM > Models.
2. Click Add Model.

3. Fill in the required fields:

   Identifier

   Unique slug (e.g., gpt-5, claude-sonnet).

   Name

   Display name (e.g., GPT-5 (128K)).

   Provider

   Select the parent provider.

   Model ID

   The API model identifier as the provider expects it (e.g., gpt-5.3-instant, claude-sonnet-4-6).

4. Optionally set capabilities (chat, completion, embeddings, vision, streaming, tools), context length, max output tokens, and pricing.
5. Click Save.

Fetching models from a provider 

Instead of adding models manually, use the Fetch Models action to query the provider API and auto-populate the model list:

1. Ensure the provider is saved and the connection test passes.
2. On the model list or model edit form, click Fetch Models.
3. The extension queries the provider API and creates model records with capabilities and metadata pre-filled.

This is the recommended approach — it ensures model IDs match the provider exactly and keeps your catalogue current as providers release new models.

Which capabilities the provider actually confirmed 

A capability badge in the model list says that the model has the capability, not who said so. Those are different claims, and the list separates them.

A grey badge means a live answer from the provider's own model endpoint declared that capability, and the row says when. A yellow badge with a question mark means nobody confirmed it: either an administrator ticked it by hand, or it came from the static model catalogue bundled with the extension. Hover the badge for the reason.

Expect the yellow badge for OpenAI, Anthropic, Groq and the Gemini models the extension knows by name, even against a reachable API. Those model endpoints list model ids and no capabilities, so the tokens come from the bundled catalogue on a live run exactly as on an unreachable one, and the badge says so rather than borrowing the provider's authority. Mistral, OpenRouter, Ollama and Gemini releases newer than this extension do report capabilities per model, and those confirm to grey.

Use the Confirm capabilities row action to ask the provider. It runs the same discovery the wizard uses, records what came back, and refreshes the confirmation date. A model the provider does not list is reported as such — that is not a confirmation, so nothing is stored.

Confirming never edits what you declared. A capability you ticked that the provider does not advertise stays on the model and simply stops borrowing the provider's authority for it. That matters most for a model created from the bundled catalogue: nothing about it was ever checked against the live API until you confirm it.

The three underlying fields — what was confirmed, when, and whether the answer was live or from the catalogue — are also visible read-only on the Capabilities tab of the model record.

Routing does not use this. Eligibility below still reads the declared capabilities, confirmed or not: making it depend on confirmation would silently drop every model an administrator declared by hand.

Which model a criteria-mode configuration picks 

A configuration set to Dynamic (Criteria) names no model. One is chosen per call from the models that are active, in two steps that never mix.

Eligibility is a hard yes or no. A model is considered only if it declares every capability the criteria require, uses a permitted adapter type, meets the minimum context length, stays within the cost ceiling, and — for the operation being run — does not declare capabilities that exclude it. A model refused here cannot come back: nothing about its speed, price or quality is even looked at.

Ranking orders what is left. The provider Priority you set decides first, always. Below it, the extension configuration option Routing policy (category routing) chooses what else counts:

Provider priority

The default. Nothing beyond your priority, the default-model flag and the sorting order, plus cost where the criteria set prefer lowest cost. This is what the extension has always done.

Balanced, Quality, Economy

Add measured signals — evaluation quality scores and recent provider health — weighted differently. Economy also weighs cost when the criteria did not ask for it.

The measured modes are opt-in on purpose: they change which model serves a call, so an upgrade never switches them on for you. Two rules make them safe to try. Your provider priority is never overruled by a measurement — a priority is an instruction, a score is evidence. And a model nobody has measured is not punished for it: an absent signal is skipped, not counted as zero.

If a criteria-mode configuration selects nothing, the cause is usually that every matching model declares it cannot serve that operation. The extension says so explicitly rather than letting the provider fail with an opaque error.

Adding a configuration manually 

1. Navigate to Admin Tools > LLM > Configurations.
2. Click Add Configuration.

3. Fill in the required fields:

   Identifier

   Unique slug for programmatic access (e.g., blog-summarizer).

   Name

   Display name (e.g., Blog Post Summarizer).

   Model

   Select the model to use.

   System Prompt

   The system message that sets the AI's behavior and context.

   Prompt snippet tags

   Optional. Active prompt snippets carrying any ticked tag are appended to the system prompt of every request made with this configuration.

4. Optionally adjust temperature (0.0-2.0), top_p, frequency/presence penalty, max tokens, and use-case type (chat, completion, embedding, translation).
5. Click Save.

Importing configuration presets 

Extensions consuming nr_llm can declare the configurations they need as presets (ADR-056). When at least one declared preset has not been imported yet, the configuration list shows a Pending presets panel above the records.

Each pending preset row shows the preset's name, identifier, description, and a requirement check:

• Satisfiable — an active model currently matches the preset's requirements; the model that would be used right now is named. Click Import to create the configuration record with one confirmation.
• Not satisfiable — no active model matches; the first missing requirement is named. The Import button stays disabled until you configure a matching provider and model.

Imported records are normal criteria-mode configurations: the model is resolved at runtime from the providers and models you configured, and you can edit or delete the record like any other. The panel disappears once no presets are pending.

If a consuming extension later changes its preset declaration, the imported configuration is flagged with a Preset changed badge in the list. The record is never updated automatically.

Next to the badge, Review update opens a dialog that lists, field by field, the record's current value against the changed declaration. Confirming with Apply update overwrites those record fields with the declared values and clears the badge. Your own settings are preserved: whether the configuration is active or the default, its backend-group assignment, and its fallback chain are never changed. If you switched the configuration to fixed model selection, or the changed requirements no longer match any active model, the update is refused with the reason shown.

Testing a configuration 

Click Test Configuration on any row. The test sends a short prompt to the model and shows the response, model ID, and token usage.

Editing configurations 

Click a configuration row to edit. Changes take effect immediately for any extension code that references this configuration's identifier — no code deployment needed.

Adding a task manually 

1. Navigate to Admin Tools > LLM > Tasks.
2. Click Add Task.

3. Fill in the required fields:

   Name

   Display name (e.g., Summarize Article).

   Configuration

   Select the LLM configuration to use.

   User Prompt

   The prompt template. Use {placeholders} for dynamic values.

4. Add a description so other admins understand what the task does.
5. Click Save.

Executing a task 

Click Run on any task to open the execution form. It shows the configuration, model, parameters, input field, and prompt template.

Example tasks:

• Summarize content — condense long articles.
• Generate meta descriptions — SEO optimization.
• Translate text — one-click translation.
• Extract keywords — pull key terms from content.

The editor module 

Editors do not need backend administrator rights to run tasks: the dedicated Web > AI Tasks module (ADR-131) lists every active task and opens a slim run form — without any of the management affordances of this admin module. Access takes both switches described under Backend user permissions: the module permission on the backend group AND the Execute AI tasks grant. Every run is pre-flighted against the user's own usage budget. Tasks with the table input type are not offered to editors — their database record picker stays admin-only.

Adding a snippet 

1. Navigate to Admin Tools > LLM > Snippets.
2. Click New Snippet.

3. Fill in the fields:

   Identifier

   Unique technical identifier (e.g., persona-friendly-expert).

   Name

   Display name (e.g., Friendly Expert).

   Tags

   Comma-separated tags consuming extensions search for (see below).

   Snippet text

   The prompt fragment itself.

   Metadata (JSON)

   Optional JSON object with extra settings.

4. Click Save.

Tag convention 

Tags are free-form, comma-separated strings. There is no fixed vocabulary — consuming extensions agree on tags with the editors. Matching is exact per tag and case-insensitive: the tag style does not match a snippet tagged lifestyle.

Established tags so far:

Persona snippets may carry a voice hint in their metadata so speech features can pick a matching text-to-speech voice:

{"voice": "nova"}

Attaching snippets to a configuration 

A configuration can select snippets by tag. Every request made with that configuration then carries them — chat, single-prompt completion, streaming and agent runs alike — without any extension code.

1. Navigate to Admin Tools > LLM > Configurations and edit a configuration.
2. Open the Parameters tab.
3. Tick the wanted tags under Prompt snippet tags. The list offers the tags the snippet records actually carry.
4. Click Save.

The active snippets carrying any ticked tag are appended to the configuration's System Prompt, each as a NAME: block separated by a blank line, in the order the tags are listed. A snippet carrying two ticked tags is added once. A tag no snippet carries adds nothing — there is no error, matching the free-tag model above.

You are a helpful assistant.

Nova persona:
You are Nova, a friendly expert.

Formal tone:
Use a formal, professional tone of voice.

Two limits are worth knowing:

• Only active snippets are composed; hiding a snippet removes it from every configuration that selects its tag.
• A caller that supplies its own system message replaces the configuration's system prompt for that call, and with it the snippet block. This is the documented per-call precedence and predates this field.

Using snippets from an extension 

Query snippets by tag through the public PromptSnippetRepository and compose the selected fragments with the PromptSnippetComposer:

$audiences = $this->promptSnippetRepository
    ->findActiveByTag('audience');
$tones = $this->promptSnippetRepository
    ->findActiveByTag('tone_of_voice');

$sections = $this->promptSnippetComposer->composeSections([
    'TARGET AUDIENCE' => $audiences[0] ?? null,
    'TONE OF VOICE' => $tones[0] ?? null,
]);

composeSections() renders each non-null snippet as a LABEL: block followed by the snippet text, joined by blank lines. Null entries and empty snippets are skipped.

findActiveByTag() filters on is_active only — like every repository here it ignores the enable fields, so hidden records are part of its result. Configurations drop them when they compose their prompt; an extension that queries directly has to skip isHidden() snippets itself if it wants the same behaviour.

See ADR-031 for the design rationale.

What an install creates 

Pick a use case, then a pack, and the plan screen lists every record the pack declares with its identifier and its current state — Would be created or Already there. Nothing is written until you press Create the missing records.

Editorial Starter declares:

The pack's configuration is a configuration preset, not a second kind of record, so it also appears in the Configuration module's Pending presets card and can be imported there instead.

The snippet-tag link, and what it pulls in 

A configuration composes the active snippets carrying any of its tags (ADR-031). Installing therefore adds the pack's snippet tags to its configuration's selection — without them the pack's snippets would exist and be read by nothing. Tags you selected yourself are kept; nothing is removed, and a tag already selected is written again by no install.

Because selection is by tag and not by owner, that link reaches in two directions, and the plan screen names both before you confirm:

Outwards

Existing configurations that already select one of the pack's tags compose the pack's new snippets into their prompts as well. They are listed by name and identifier.

Inwards

Existing snippets that already carry one of the added tags are composed into the pack's configuration. They are listed with their data class, because that is the part that can stop the configuration from working: input context is classified by the strictest data class among the composed snippets, so one confidential snippet raises the whole configuration, and an enforcing input-context gate then refuses every send through it.

To keep a snippet out, deactivate it or remove the tag from it before installing. To keep another configuration unchanged, edit its snippet-tag selection first, or rename the pack's tags afterwards.

What a pack recommends but never applies 

• Governance posture. The pack names the posture its content was written for (Editorial Starter: controlled cloud). Installing changes no governance value; the screen links to the governance readout where the posture in force is shown.
• Tool groups. The pack names the groups its tasks benefit from (Editorial Starter: content). Enabling a tool group stays an administrator decision in the Tools module.
• Nothing about providers or API keys. The pack states model requirements. When no active model satisfies them, the plan says which requirement is missing and links to the setup wizard.

A pack installs no skills. A skill carries provenance and a trust level from the source it was synced from, and a pack can produce neither; add a skill source instead.

Installing again 

"Already installed" means a record with that identifier exists — nothing more. The installer never overwrites and never compares contents, so:

• A task you renamed or whose prompt you rewrote is left exactly as it is.
• A record you disabled still counts as installed, so a second install cannot quietly resurrect what you switched off.
• A tag the configuration already selects is not written again.

A second install therefore reports records created: 0. The one case where it still has work is a configuration created by importing the preset in the Configuration module: its records exist but its snippet-tag selection does not yet include the pack's tags, so the confirm button stays offered and the success message names the tags it added.

A pack cannot be uninstalled. Nothing marks a record as pack-owned — that is what makes the installed records ordinary — so removing them is deleting records, one by one, like any other.

The full rationale is in ADR-163.

Source types 

A source has one of three types:

single_file

One SKILL.md at a fixed path in a repository. A single, explicit admin act — its skill may default to enabled.

repo

A whole repository. Every SKILL.md under the repo root, skills/<name>/, .claude/skills/<name>/ or <plugin>/skills/<name>/ is discovered. Discovered skills arrive disabled for review.

marketplace

An Anthropic marketplace.json index that lists plugins pointing at further repositories. Each entry is expanded with the repo flow. All discovered skills arrive disabled.

Adding a source 

1. Navigate to Admin Tools > LLM > Skills.
2. Click New Skill Source.

3. Fill in the fields:

   Title

   Display name for the source list.

   Type

   single_file, repo or marketplace (see above).

   URL

   The GitHub URL the type expects (the SKILL.md URL, the repository URL, or the marketplace.json URL).

   Ref

   A branch or tag (for example main or v1.2.0). It is resolved once to an immutable commit SHA at sync time; all bodies are then fetched by that SHA, never by the moving branch.

4. Click Save.

The pinned_sha, sync_status, sync_error and last_synced fields are managed by the sync run and shown read-only.

GitHub token and rate limits 

Unauthenticated GitHub API access is limited to 60 requests per hour, which is quickly exhausted by a repo or marketplace sync. Add a personal access token (a read-only, public-repo token is enough) to raise the limit and to read private repositories.

• The token is set through the Set token action on a source, not typed into a FormEngine field. It is stored as an nr-vault UUID (envelope-encrypted), mirroring provider API-key storage — never as plaintext in TCA, YAML or the database.
• When a sync hits the rate limit (HTTP 403 with no remaining quota), the source is set to sync_status = error carrying the reset time; state is not partially corrupted. Add a token and re-sync.

Host-allowlist prerequisite 

nr-llm enforces an app-level GitHub allowlist on every skill request: the scheme must be https and the host must be one of github.com, raw.githubusercontent.com, api.github.com or codeload.github.com. This is separate from, and in addition to, the nr-vault SSRF guard.

On hardened instances that restrict outbound HTTP through the global HTTP/allowed_hosts SSRF setting, those four GitHub hosts must be on that list, otherwise every sync fails closed. This is a deliberate prerequisite — nr-llm never silently bypasses the SSRF guard.

Syncing and the review flow 

1. On a source, click Sync. The source moves through never_synced → syncing → ok / partial / error. The syncing state also acts as a lock: a second concurrent sync on the same source is refused.
2. partial means the per-sync file-count or wall-time bound was reached (large marketplaces); the skills fetched so far are stored.
3. Discovered skills from repo and marketplace sources are created disabled by default. Review each one, then toggle it on with Enable.
4. Re-sync never silently changes an enabled skill. If a re-sync recomputes a different body_checksum for an enabled skill, nr-llm auto-disables it and surfaces a diff (Review changes) so you re-confirm before it is used again. Accepting the diff re-pins the SHA atomically.
5. A skill that disappeared upstream is marked orphaned and disabled, never silently dropped, so attachments (Plan 1b) do not vanish.

Deleting a source cascade-deletes its skills.

The partial support badge 

Each skill carries a support badge:

full

The skill is plain front-matter and prose.

partial

The body or front-matter references scripts, references/, assets/ or an allowed-tools declaration.

See ADR-035 for the full design and security rationale.

Attaching skills and injecting them into prompts 

Enabled, non-orphaned skills can be attached to a Task and/or an LLM configuration via the Skills field on those records (only enabled skills are offered). At execution time, for text-generation operations only — completion, translation and task execution; never embeddings, vision or speech — nr-llm composes the attached skills into a delimited block and prepends it to the user prompt. The configuration system_prompt is never modified.

Composition rules:

• Precedence. Configuration skills are the baseline, task skills are additive; the set is the union deduped by source + identifier (the configuration wins on a duplicate). The configuration block renders first.
• Budget. The block is bounded by a conservative character budget; when it is exceeded, task-additive skills are dropped before configuration-baseline skills and each drop is logged.
• Integrity. Each skill's body checksum is re-verified at injection time; a mismatch (tampering or a stale row) drops that skill — it is never injected.
• Untrusted output. Skill prose is third-party text; output produced under its influence is treated as untrusted and escaped/sanitized where it is stored or rendered. Message role is defense-in-depth, not a trust boundary.

See ADR-036 for the injection design.

Isolation controls: trust, fingerprint, injection scan, audit 

On top of the SHA-pin and checksum controls above, each source and skill carries the isolation controls introduced in ADR-061.

Publisher trust level. Every source is classified — untrusted (the default, for anonymous public GitHub content), community, verified or first_party (operator-controlled). This is provenance, independent of the partial support badge. Each synced skill denormalises its source's level, so re-classifying a source takes effect on the next sync. The instance-wide floor skills.minTrustLevel (extension configuration, default untrusted) gates use: a skill below the floor is dropped from both prompt injection and the allowed-tools union. Raising the floor to verified therefore hides every community/untrusted skill without deleting it. Trust is separate from the enabled = false default — an untrusted skill still needs an explicit enable. An unreadable or mistyped floor falls back to untrusted; the value actually in force is shown under Effective policy.

Skill-block byte budget. skills.maxBytes (extension configuration, default 24000) caps the composed block that is prepended to the user prompt. The measure is bytes, not tokens — a deliberate over-estimate, since no tokenizer is available. When the block exceeds the budget, skills are dropped from the tail first: task-additive skills go before the configuration baseline, and every drop is logged as a warning. Lower the value to reserve more of the model's context window for the conversation itself; raise it if a large configuration baseline is being trimmed. An empty, non-numeric or zero value falls back to 24000 — the cap cannot be switched off, so an emptied field never puts an unbounded block on the wire. The budget is instance-wide and independent of the model's context window; the per-request window bound is handled separately by the context-window manager (ADR-107).

The budget bounds the prompt block only, not the allowed-tools union (Gating tools). The union is computed over the effective skills before the block is assembled, so a skill dropped for the budget still grants its tools while its usage rules stay out of the prompt. That is deliberate — a budget-aware union would widen the gate, because dropping the last declaring skill removes the restriction entirely. Watch the drop warnings when lowering the value: they name every skill whose prose stopped shipping while its tools kept being offered.

Manifest fingerprint (optional). A source may declare an expected_fingerprint: the sha256 its whole skill set must hash to. When set, the digest is recomputed at sync and verified before anything is materialised; a mismatch fails closed (no skill is imported, the source goes to error) and leaves the last known-good skills untouched. Leave it empty to rely on the commit-SHA pin alone. This binds the reviewed bytes to a publisher-declared identity beyond "these bytes from this URL"; it is a declared digest, not a public-key signature.

Prompt-injection scan. Each body is scanned at ingest for known injection signatures. A high-confidence jailbreak marker (e.g. "ignore all previous instructions", role reset, chat-template control tokens) force-disables the skill at import — even a single-file source that would otherwise default enabled — and must be re-reviewed before enabling. Lower-confidence findings are recorded on the skill (Injection scan findings) for review without blocking.

Immutable audit trail. Every ingest, enable, disable and fail-closed rejection is written to tx_nrllm_skill_audit with who / when / source / SHA / checksum / trust level / scan result. The trail is append-only — the application never updates or deletes a row — so the provenance of any skill that can reach a prompt is reconstructable after the fact.

The built-in tools 

nr-llm ships forty-one read-only tools and five writing tools. Each is a reference implementation of the security contract: model-chosen arguments are validated and scoped, volumes are capped, and secret-bearing output is either redacted or gated behind a separate _raw variant. Thirty-eight ship enabled; the three unredacted _raw variants (get_env_raw, get_php_info_raw and list_be_users_raw) and all five writing tools (update_page_metadata, set_file_alternative_text, move_content_element, create_content_element_draft, create_translation_draft) ship disabled and must be enabled deliberately. Many require admin; the read-only structure, content and file tools (get_pagetree, get_tca, get_full_tca, get_table_schema, get_flexform_schema, fluid_resolve, search_records, get_page_content, read_records, get_record_history, resolve_url, validate_tca, list_fal_storages, browse_fal_folder, search_fal_files, get_fal_references, find_missing_files) are offered to non-admin backend users — those self-enforce the acting user's TYPO3 permissions (page-show rights, tables_select) inside the tool, so a non-admin only ever sees what the backend already grants them (see ADR-042).

The two tools below are the fullest illustrations of the contract:

fetch_logs

Returns the most recent sys_log entries, newest first, with an optional PSR level filter and a limit (default 20, hard-capped at 50). Personally-identifying fields — the client IP, the backend user id and the serialized payload — are redacted by omission, because the result egresses to the external provider.

read_fal_asset_meta

Returns read-only metadata (file name, MIME type, size, title, alternative text) for a single managed file (sys_file) by its uid. The uid is model-chosen and therefore injection-steerable, so the lookup is storage-scoped (default: the default storage). A uid in a non-permitted storage returns the same neutral "not found or not permitted" string as a missing uid — the model cannot enumerate arbitrary files.

The remaining tools follow the same pattern:

list_fal_storages

The file storages this run may touch (uid, name, driver, status flags). The effective set is the configured allow-list, intersected for non-admins with their file mounts; the server-side base path is never part of the output.

browse_fal_folder

One FAL folder: subfolders (with file count), then files with size and MIME type. Storage-relative identifiers only; anything unresolvable collapses into one neutral denial. Capped at 100 entries.

search_fal_files

Substring search over file name and metadata title/alternative within the accessible storages. %/_ in the query match literally; missing files are excluded.

get_fal_references

Where a file is used: sys_file_reference rows as `table:uid (field)`, hidden references marked. Soft references (RTE links, plain URLs) are not tracked — stated in the output so "no references" is never read as "safe to delete". Non-admins only see references from tables they may read.

find_missing_files

sys_file records whose physical file is gone (missing = 1) — the "broken image" diagnosis. The total count is always reported next to the capped listing.

get_env / get_env_raw

Process environment variables. get_env redacts secret-looking values (password, token, key, secret, salt, DSN, …); get_env_raw returns them unredacted (database password, encryption key) and ships disabled.

get_php_info / get_php_info_raw

PHP runtime configuration. get_php_info is redacted; get_php_info_raw returns the full, secret-bearing phpinfo detail and ships disabled.

get_pagetree

The backend page tree (uid, title, doktype) as a depth-indented outline; deleted and hidden pages are excluded — structure only, no content.

get_tca

The TYPO3 TCA schema: with no argument it lists the configured table names; with a table argument it returns that table's field definitions.

list_be_groups

The backend user groups (uid, title).

list_be_users / list_be_users_raw

Backend users. list_be_users omits credentials (password hashes and MFA secrets are never included); list_be_users_raw returns the full non-credential profile columns and ships disabled.

search_records

Full-text search across the tables that define TCA searchFields. Returns compact table:uid hits with a short excerpt around the match. Credential and nr-llm configuration tables are never searched; non-admins are limited to their tables_select tables and to hits on pages they may show.

get_page_content

One page's header data plus its content elements in column/sorting order (uid, colPos, CType, header, a short bodytext excerpt). Non-admins need page-show permission; only admins see hidden elements (marked [hidden]).

read_records

Generic equality-filtered read of one TCA table — never raw SQL. Fields are validated against the TCA and credential-like columns are silently dropped; the same table gates as search_records apply.

get_record_history

One record's change history from sys_history, newest first: when, which backend user, which action, and per modification the changed fields as old → new values. Values of credential-like fields are never rendered — only the fact that they changed. Same table gates as read_records, and non-admins additionally need page-show access on the record's page.

resolve_url

Map a URL (or path) of this instance to the page that serves it: site, language, page uid/title/slug and route arguments. Routing only — no request is sent; foreign hosts cannot match by construction. Non-admins need page-show permission on the resolved page.

get_typoscript

The resolved frontend TypoScript (setup or constants) effective on a page, with a dotted path drill-down and capped output. Admin-only — constants routinely carry API keys — and credential-like values render as [redacted] on top of that.

get_tsconfig

The rootline-merged Page TSconfig effective on a page, with the same path drill-down, output cap and redaction as get_typoscript. Admin-only.

get_last_exception

The newest exception/error from the TYPO3 file logs with its parsed stack trace and the surrounding source lines of the project frames inlined. index steps back through older errors, search filters by message, class or component. Admin-only.

read_source

A line-numbered range of one project source file. Paths must resolve inside the project root; dotfiles, var/* (except var/log), config/system, settings.php/additional.php, key material and credential paths are structurally unreadable. Admin-only.

search_code

Literal-substring (or opt-in regex) search across the project's source files, returning path:line hits. Vendor, var and dot directories are never searched; matched credential lines are value-redacted. Admin-only.

probe_url

One GET against a URL of this instance: status, key headers, timing and a short body excerpt — and on a 5xx the matching exception from the TYPO3 logs is appended automatically. Foreign hosts and non-http(s) schemes are denied; redirects are reported, not followed. Admin-only.

get_full_tca

The TCA index: the names and titles of all accessible tables, each with a pointer to get_table_schema. A navigation aid so the model can traverse the schema without the whole (multi-megabyte) TCA being sent at once. The same table gates as get_table_schema apply. Optional filter and extension narrow the list.

get_table_schema

One table's schema in a readable form: control settings plus, per field, its type and — for relational fields — the foreign table and relation kind (the value over get_tca). Sensitive tables are denied for every user; credential-like columns show name and type only.

get_flexform_schema

The data structure of a TCA FlexForm field, rendered as sheets and fields. When the field selects one of several structures by a pointer, the available keys are listed so a follow-up call can pass ds_pointer. Same table gates as get_table_schema.

fluid_resolve

Which physical Fluid file backs a template, partial or layout name in an extension: the candidate paths in override order with an exists flag and the winning path — to debug a wrong or missing template. Paths only. (Resolves an extension's own Resources/Private paths; TypoScript override root paths need a live rendering context and are not reflected.)

validate_tca

Structural TCA checks: ctrl.label/ctrl.type naming undefined columns, foreign_table references to unknown tables, showitem entries referencing undefined columns or palettes. One table or all accessible tables; findings name schema keys, never record data.

check_typoscript

Scans the TypoScript effective on a page (constants and setup) for syntax errors — invalid lines, unbalanced braces, @import matching no file — using the same core scanner as the backend's TypoScript module. Reports source and line number only, never the offending line's content (a constants line may carry an API key). Admin-only.

list_extensions

The installed (active) extensions: key, version, composer name and title — no package paths. Admin-only.

get_site_config

Without arguments the configured sites (identifier, base, root page); with identifier that site's configuration flattened to dotted key: value lines. Credential-like keys (camelCase included, e.g. apiKey) render as [redacted]. Admin-only.

list_scheduler_tasks

The scheduler tasks with next execution, disabled flag and a last-run-failed marker. The serialized task object is never unserialized; degrades gracefully when EXT:scheduler is absent. Admin-only.

get_system_status

One compact block: TYPO3/PHP/database versions, application context, composer mode, OS family, timezone — no paths, no hostnames. Admin-only.

list_deprecations

The newest distinct messages from the deprecation log, deduplicated with a ×count suffix and project paths relativized — the upgrade work list. Admin-only.

list_middlewares

A PSR-15 middleware stack (frontend or backend) in execution order with identifiers and classes. Admin-only.

site_rag_query

Curated evidence about the website's own public content for a question: source id, title, URL and match excerpt per source, retrieved from the best available search index — EXT:solr, ke_search, indexed_search or a database fallback — and labelled with the answering backend (ADR-049). Index-level filtering is always public-only (what the anonymous visitor could read).

site_fetch_source

The full indexed text behind a site_rag_query source id, capped at 8000 characters — for reading a promising source beyond its excerpt.

The writing tools 

Five tools change anything at all: update_page_metadata, set_file_alternative_text, move_content_element, create_content_element_draft and create_translation_draft. All five write through the TYPO3 DataHandler, as the acting backend user, in the live workspace only, on exactly one record per call (ADR-135, ADR-146).

What holds for all of them:

• They ship disabled, sit in their own editing group, and are not offered until both the group and the tool are enabled.
• Every call pauses for a human decision (ADR-134). The approval card names the record and the values, together with the values the write would REPLACE (ADR-136); nothing is written until somebody presses Approve, and the approver must themselves be permitted to run the tool (ADR-133).
• Permissions are enforced twice: the tool checks the acting user's own rights first, and the DataHandler enforces its own rules on top. A non-admin therefore writes only what the backend already lets them write.
• A field the user lacks the "exclude field" grant for is dropped by the DataHandler without an error. Every one of them re-reads the record afterwards and reports that as a failure rather than as a successful write.
• A call the tool would refuse is refused whole rather than applied in part, and a record the acting user may not reach is refused with the same words as a record that does not exist — so a refusal never confirms that a uid exists.

update_page_metadata

Sets a fixed set of descriptive fields on one page. Editable: title, subtitle, nav_title, abstract, description, keywords and — when EXT:seo is installed — seo_title, og_title, og_description, twitter_title, twitter_description. Anything else (slug, hidden, doktype, fe_group, perms_*, no_index, the image relations …) is refused. Authorised by the acting user's page-edit right; the DataHandler then enforces tables_modify and the field-level grants.

set_file_alternative_text

Sets the alternative text (sys_file_metadata.alternative) of one managed file, identified by its sys_file uid — the accessibility gap an editor most often leaves behind. It writes that one field and nothing else.

Authorised by the same storage allow-list and file mounts as the read-only FAL tools; core's own file-metadata permission check (a writable file mount) then applies inside the DataHandler, so a read-only mount is refused there.

Two limits worth knowing before enabling it:

• It never creates a metadata record. A file that carries none is refused, in the same words as a file the user may not reach — so the model cannot tell "not yours" from "not indexed".
• It writes the live, default-language record only and takes no language argument. A translation, and a draft version of the same record in a workspace, are both left alone. Translated alternative texts stay a backend job (ADR-135).

An empty string is accepted and is the correct value for a decorative image.

move_content_element

Moves one content element to a page and a column. The element keeps its uid, its content, its language, its history and its references — only its place changes, which is what makes it the least committal write in the set.

Both ends are authorised: the acting user needs content-edit rights on the source page as well as on the target, because moving an element out of a page edits that page's content too. An after_content_uid anchor must sit on the target page and in the same language; a wrong anchor is refused rather than silently corrected.

The destination column is always stated explicitly on the wire, so the element lands where the approval card said it would even when the anchor element sits in a column the caller did not expect.

create_content_element_draft

Creates one content element on a page — the first tool that brings a record into being. Three things bound it:

• It is always hidden. There is no argument to switch that off: publishing is a separate act with a separate audience, and the approval that let the tool run approved a draft.
• The content type is an allow-list (header, text, textmedia, bullets) intersected with what the installation's TCA actually declares. Types whose payload is configuration rather than prose — list (a plugin), html, shortcut — are out of reach.
• The field set is fixed: headline, body text, column, language, position. This is not a generic record API.

bodytext reaches the DataHandler and its RTE transformation exactly as an editor's input does. It is bounded in length and not otherwise filtered — an editor may write the same markup by hand, and a tool enforcing a stricter rule than the CMS would be enforcing a rule that does not exist.

create_translation_draft

Translates one page or content element into another language by running core's own localize command, so connected-mode translations, inline children and every localisation hook behave exactly as they do in the backend.

The result is hidden, which is the part core does not do: localize copies the source's visibility, so a translation of a live page would go live the moment it was created.

An existing translation stops the call and is named in the refusal. The overwrite argument is the only way past it: it deletes that translation first — recoverably (deleted = 1) and in sys_log — and the approval card says so on its own line. Whether the target language exists for the record's site is core's check, not a second implementation here.

Registering a tool 

A tool is a PHP class that implements Netresearch\NrLlm\Service\Tool\ToolInterface:

getSpec(): ToolSpec

Returns the declaration the model receives — a name, a description, and a JSON-Schema parameters block. Build it with ToolSpec::function($name, $description, $parameters).

execute(array $arguments): string

Runs the tool with the model-provided arguments and returns a plain string that is fed back into the conversation as a tool turn.

getGroup(): string

The tool's group — a short, stable identifier used to enable or disable whole families of tools at once. Built-ins use content, editing, structure, system, accounts and configuration; third-party tools declare their own group (recommended: the providing extension's key). See Tool groups.

The interface carries #[AutoconfigureTag('nr_llm.tool')], so a class is auto-registered simply by implementing it — no central registration file to edit. ToolRegistry collects every tagged tool through a DI iterator and indexes it by spec name; two tools with the same name is a developer error and fails fast at container build.

When you write a tool, honour the security contract: treat $arguments as attacker-influenced (the model is steerable by injected skill prose), validate and scope every input (cap volumes, scope identifier lookups), and never return secrets — the result leaves the instance.

Managing tools 

The Admin Tools > LLM > Tools module lists every registered tool with its global enable state and lets an admin toggle it. A disabled tool is refused on every run, everywhere — the runtime gate is fail-closed, so a disabled tool can never be offered to the model regardless of a skill's allowed-tools or the per-run selection in the playground. Some built-in tools (for example get_env_raw and get_php_info_raw) ship disabled by default because they return unredacted, secret-bearing output; enable them only deliberately.

A tool that carries an editor action declaration (ADR-152) reads differently in that list: it shows an icon, its translated name, one sentence written for a human, and the record types it addresses — instead of the wire name and the description written for the language model. All five writing tools declare one, and the wire name stays visible as the technical detail the toggle acts on. A read-only tool is unchanged.

The declaration is presentation only. It does not decide whether a tool writes — that is the tool's declared effect — and it changes nothing about how a call is fenced, approved or audited.

Tool groups 

Every tool belongs to a group (its getGroup() value). The built-in groups carry a translated name in the module header beside their identifier; a group a third-party extension brings has no translated name and shows its identifier alone. The built-in taxonomy:

Groups can be switched on three levels, and the result cascades fail-closed — a tool is offered only when every level permits it:

1. Centrally in the Tools module: each group header carries an Enable/Disable group toggle. A disabled group refuses all of its tools — including same-group tools installed later — and a per-tool override can not re-enable a tool inside a disabled group (predictable, fail-closed). Per-tool toggles keep working but take effect only once the group is enabled again.
2. Per configuration: the Allowed tool groups field on an LLM configuration restricts agent runs with that configuration to tools of the selected groups (empty = all groups). This intersects with a skill's allowed-tools declaration.
3. Per run in the playground: the tool checkboxes are grouped, and each group checkbox (de)selects its children.

Third-party extensions declare their own group per tool; the recommended value is the extension key, so an admin can disable an extension's whole tool family with one toggle. The design is recorded in ADR-043.

Network egress policy per group 

Network egress is governed per tool group and is fail-closed (ADR-061). Each group has a declared egress scope; a group with no declaration may make no outbound request:

Only the system group (which carries probe_url, the one built-in that fetches over the network) is granted own_site; every other group is none. There is no "any host" scope, so a newly installed or mis-declared tool group can never egress to an arbitrary target. The diagnostics tools that share the system group (get_env, fetch_logs …) never make a network request, so the grant does not loosen them.

Using the Tool Playground 

The playground lives in Admin Tools > LLM > Playground and is admin-only. It is a sibling of the Tools management module: the playground runs the loop, while the Tools module governs which tools exist and are enabled.

1. Pick an LLM configuration from the dropdown. Its vault-stored API key, model, temperature and system prompt are what the loop actually runs on — the playground never falls back to a default model.
2. Type a prompt. Optionally open the override panels to force-inject skills (added on top of the configuration's own), force-add snippets (inserted as leading system messages), override the system prompt, cap the max rounds, or tick capture raw provider response.
3. Click Run — or Dry run to assemble the prompt and inspect exactly what would be sent without calling the model.
4. Read the inspector — live from the moment you click Run. A summary strip reports rounds, tool calls, the prompt/completion token split, estimated cost, wall time and status. The step list is the nr_llm ↔ LLM dialog in order: each round opens with a Context budget step, then its outbound request (the messages sent and the tools offered) appears the instant it goes out, a waiting indicator shows while the model works, then the response and each tool execution stream in. Select a step to open its detail — requests carry Messages sent and Tools offered; responses carry Structured, Raw JSON and Thinking. The model's final answer closes the run.

5. The Context budget step says where the window went for the round that follows it, so you can act on the component that is yours to change rather than only learn that history was dropped. It has two tabs:

   Where the window went

   The window, the output reserve, the safety margin and the resulting budget, then four component lines — transcript, tool schema, system prompt (incl. snippets) and skills — that sum to the estimated total, plus what is left. On this surface the system-prompt line always reads counted in the transcript and the skills line always reads 0: the agent loop builds both into the transcript before the fit measures it, so their content is already on the transcript line. The table says so under the figures. A send whose reserve exceeds the whole window is handed to the provider unmeasured, and the step reports no accounting instead of a window of zero.

   Injected context

   Every snippet and skill this run injects, by name only, with the data class it declared (ADR-144) or not classified, and the strictest class across all of them. The list covers the snippets and skills you force-injected for this one run as well as the configuration's own — the input-context gate itself still answers for the configuration alone, so a forced source is shown here and is not gated.

The Tools available to this run list lets you narrow a single run to a subset of the globally-enabled tools (the full list and the global enable/disable controls live in the Tools module). Raw-response capture is off unless you tick it, so ordinary runs never retain the provider's raw payload. Every displayed string — tool arguments, tool results (which may include sys_log content), and the final answer — is rendered escaped; HTML is only ever shown inside a sandboxed preview, never injected into the page.

Each run is bounded by the iteration cap (default 5) and, when the configuration's backend user has a budget, by the per-iteration budget pre-flight. If the cap is hit with tools still pending, a final tool-free completion synthesises a closing answer and the run is marked truncated. The aggregated token usage is reported; the monetary cost is recorded in the usage table by the middleware pipeline.

Ollama model-capability dependency 

Tool calling depends on the model, not just the provider. For Ollama, only function-calling-capable models — for example llama3.1, mistral, qwen2.5 — return tool calls. A model without function-calling support simply answers the prompt directly and never calls a tool; the loop ends gracefully on the first plain answer. If a configured Ollama model never seems to use the available tools, verify it is one of the function-calling models for your Ollama version.

Gating tools with allowed-tools in a skill 

A skill's SKILL.md front-matter may carry an allowed-tools key that gates which tools the skills attached to a configuration (or task) grant for a run. The resolution is fail-closed on declaration, computed over the configuration's effective skills (enabled, non-orphaned, at or above the instance trust floor, deduped):

• Absent (no skill declares allowed-tools) — no opinion; all registered tools are offered.
• Declared list — the union of the declared lists across the effective skills; only those tools are offered (intersected with what is actually registered, so an unknown name is dropped).
• Declared empty (allowed-tools: []) — declares zero tools; if no other effective skill widens the set, the run gets no tools and is a single plain completion.

A disabled or orphaned skill never grants tools. The allow-list is enforced both when the tools are offered to the model and again when a tool call is executed, so a prompt injection cannot reach a tool the skills did not grant.

The effective set is not the same as what ends up in the prompt. The skill-block byte budget (skills.maxBytes, see Managing skills) is applied after the allow-list has been computed, while the block is assembled. A skill dropped for the budget therefore still contributes its allowed-tools, but its usage rules never reach the model. That order is deliberate: a budget-aware union would let the drop of the last declaring skill collapse the allow-list to "no restriction" — every registered tool — which is a looser gate, not a tighter one. Keep skills.maxBytes above the composed size of the skills you rely on if you want their prose to arrive alongside the tools it describes; every drop is logged as a warning.

See ADR-038 for the runtime design and security rationale.

How it works 

1. Configure a server — endpoint, authentication, the class of data its tools may see, and whether its tools need human approval. A server that declares no data class supplies nothing: there is no default anybody silently inherits (fail-closed, ADR-113).
2. Import its catalogue — an explicit action that fetches the tools the server advertises. Import is the only network call that happens outside an agent run; nothing talks to the server just because a page rendered. Tool input schemas are normalised into the supported subset on import; a tool whose schema cannot be expressed is skipped rather than silently weakened.
3. Enable individual tools — imported tools start disabled and are switched on one by one, exactly like the builtin tools in the Tools module.

Is the server alive? 

Test connection performs the MCP handshake and reports what came back: how long it took, which protocol revision the server chose and what the server calls itself. The report appears on the server's card and stays there until you leave the page — only the latency is stored, so the rest would be lost to a page reload, and this is the one action in the module that therefore does not reload. It writes no catalogue — no tool is added, removed or orphaned, and the import status and the last import error stay exactly as they were. Use it to check a server before enabling it, and to tell "the server is down" from "the server is fine and this tool is gone".

Each server card also shows Last successful contact. That is the last time this installation completed any round trip against the server — a tool call, an import or a connection test — together with how long that round trip took. It is deliberately not the same as Last import: a server that has been answering tool calls all month can still show an import from six weeks ago, and previously there was no way to see the difference.

A failed connection test replaces the report on the card with its reason, and is stored nowhere. Only a success moves the contact date.

Guard rails 

• Remote tools always require an administrator, and count as non-idempotent writes — unconditionally, whatever the server's catalogue says about them: they are never replayed on a retry, and never waved through the trust-zone gate in observe mode. That the classification cannot be argued down from the far side is what makes the setting below the only place the question is answered.
• Requires approval decides whether an agent run stops and waits for a person before it calls any tool of this server. It is on for a newly configured server: what a remote tool actually does cannot be inspected from here, and the server's own annotations do not get to answer the question about themselves (ADR-134). Switch it off per server once you know what its tools do. This holds for every server, including one configured before the setting existed: an update leaves it requiring approval, and nothing switches that off on your behalf.
• The number of remote calls one run may make is bounded (default: 20) — a remote call crosses the network while a backend user waits, and nothing else limits how many a model asks for at once.

What a remote answer can contain 

An MCP server may answer a tool call with several typed blocks: text, images, embedded resources. This client reads text only. When a server sends anything else, the blocks are dropped and the answer opens with a line saying how many were dropped and of which type — so a model reading a partial answer is told it is partial, and a run whose tool returned only an image is not told the tool returned nothing. The line comes first because a long answer is shortened before the model sees it, and a note at the end would be the part that is cut. If you need the image itself, the tool is not usable from here yet.

A call that fails does not fail the run. It comes back as a failed tool result naming the server, the model is told, and the run carries on. This covers both ways a call fails: the server not answering usefully — it is down, it refuses the credential, it sends something that is not JSON-RPC — and the server answering that the tool itself failed, which is the ordinary case of a missing page or a rejected argument. Both are recorded as failures in the run's event stream, so a server that is flaky is visible without reading transcripts.

See ADR-116 for the design rationale, ADR-154 for what liveness is measured on and why the connection test writes nothing, and ADR-161 for the conformance suite every supported connection is held to — including the one thing it does not do: cancelling a call that is already in flight. Cancelling a run stops it at the next step, but an outstanding remote call still runs to its 15-second timeout.

Setup wizard 

The setup wizard guides first-time configuration in five steps:

1. Connect — enter your provider endpoint and API key.
2. Verify — test the connection.
3. Models — fetch available models from the provider API.
4. Configure — create an initial configuration with system prompt and parameters.
5. Save — run a test prompt to confirm everything works.

Access it from the Dashboard when no providers are configured, or via the setup wizard link at any time.

Configuration wizard 

The configuration wizard generates a complete LLM configuration using AI. Instead of filling in each field manually, describe your use case in plain language and the wizard generates everything.

1. Navigate to Admin Tools > LLM > Configurations.
2. Click Create with AI.
3. Describe your use case (e.g., "summarize blog posts in three sentences").
4. The wizard generates: identifier, name, system prompt, temperature, and all other parameters.
5. Review and click Save.

Task wizard 

The task wizard creates a complete task setup — a task and a dedicated configuration — in one step.

1. Navigate to Admin Tools > LLM > Tasks.
2. Click Create with AI.
3. Describe the task (e.g., "extract the five most important keywords from an article").
4. The wizard generates: a task with prompt template, a configuration with system prompt and parameters, and a model recommendation.
5. Review and click Save.

Model discovery 

On the model edit form, use the Fetch Models button to query the provider API. This auto-populates available models with their capabilities, context length, and pricing metadata.

Recommended workflow 

For a fresh installation:

1. Run the Setup wizard from the dashboard to create your first provider, fetch models, and test a configuration.
2. Use the Configuration wizard to create additional use-case configurations (one per use case in your extensions).
3. Use the Task wizard to create reusable prompt templates for editors.
4. Share configuration identifiers with your extension developers — they reference them in code via $configRepository->findByIdentifier('...').

For ongoing maintenance:

• Add providers when you need additional AI services or separate prod/dev keys.
• Fetch models periodically to pick up new models from providers.
• Edit configurations to tune prompts and parameters — changes take effect immediately without code deployment.

What a budget caps 

Each row in tx_nrllm_user_budget binds to exactly one be_user and defines six independent ceilings. 0 on any axis means "unlimited on this axis".

Usage is aggregated on demand from tx_nrllm_service_usage — the same table the UsageTracker already writes to per request — so there is no second write per request and no way for a separate counter to drift away from the source of truth.

Creating a budget 

Budget records have rootLevel = -1, so admins can create them at the TYPO3 root (pid = 0) or on any regular page. Keeping them at the root is the convention because budgets are site-wide admin concerns, not page-scoped content; the recipe below follows that convention.

1. Open Web > List in the root (page UID 0) — or on the page where you keep other cross-site configuration records.
2. Click Create new record.
3. Choose LLM User Budget.
4. Pick the backend user, set the ceilings, toggle Enforce this budget on.
5. Save.

How the check runs 

Before dispatching a request the consuming extension calls NetresearchNrLlmServiceBudgetService::check(). The service:

1. Returns allowed when the user has no budget record, when Enforce this budget is off, or when every ceiling is 0.
2. Aggregates today's usage and this month's usage in a single database roundtrip.
3. Evaluates the daily window first; the monthly window only if the daily window passes.
4. Adds +1 request and +plannedCost to the usage figures before comparing, so a user at exactly the limit is still allowed one more call.

The returned BudgetCheckResult names which bucket was tripped (exceededLimit as a stable machine key, plus a human-friendly reason string suitable for log output or caller-side wrapping).

Budgets vs. configuration limits 

Both layers persist but cap different things:

See ADR-025: Per-User AI Budgets for the full design rationale, including the alternatives (counter table, group-level budgets, auto-throttling) we considered and why they were rejected.

Assigning grants 

Grants are ordinary TYPO3 custom permission options. In the backend group record (Access Lists tab, section AI (nr_llm) permissions), tick the grants the group should hold. Notes:

• Grants are group-scoped — a backend user without groups cannot hold a grant.
• Administrators hold every grant implicitly.
• Revoking a grant takes effect with the user's next request.

Available grants 

Execute AI tasks (tasks_use)

Run existing AI tasks and refresh their input data. What a task may read and which model and configuration it uses is defined by whoever manages the task — that is the trust boundary: task managers define, grant holders execute. Every run is pre-flighted against the user's own usage budget and attributed to them.

Since ADR-158 the same grant opens the Editor Action Center in that module. It does not decide which actions appear there: that is the tool gate's answer plus access to the default LLM configuration — a user outside its Allowed backend groups is offered nothing and can start nothing — and every writing tool is disabled until an administrator enables it. Starting an action always suspends for approval before anything is written.

Approve suspended AI runs (agent_approve)

Approve, deny or answer agent runs suspended for a human decision — including runs started by other users. Without this grant a user can only ever decide their own runs. Deliberately part of no recommended preset: granting it is an explicit trust decision.

Recommended presets 

Roles are named grant bundles, not code:

Approval (agent_approve) sits in no preset — add it deliberately where the organisation wants non-admin approvers.

Current reach 

Grant holders work in the dedicated AI Tasks module (web group, ADR-131). Reaching it takes BOTH switches: the module must be ticked in the group's module list (access => user) AND the grant must be held — the module switch alone never grants execution. Tasks with the table input type are not offered to editors (their record picker has no read boundary yet and stays admin-only). Everything else — configuration, providers, models, the playground, record browsing — remains administrator-only regardless of grants.

Which setting belongs to which service 

Translation 

Enter a text and a target language. Leaving the source language empty makes the service detect it.

The translator picker lists every registered translator and marks the ones without a usable credential. Leaving it on LLM (default) runs the LLM path, which needs a working provider but no specialized key — useful to confirm the page itself works before pointing at DeepL.

Selecting a translator routes to that one, which is the case worth running after entering its vault identifier. The result names the translator that answered, the detected source language, and either the characters billed (specialized translators) or the tokens used (the LLM path).

Image generation 

Enter a prompt and pick OpenAI (DALL·E) or FAL. Each service applies its own default model and size; the result reports which were used, and DALL·E additionally reports the prompt it rewrote yours into.

Reading the result 

A missing or unresolvable credential is reported as such, naming the Extension Configuration — that is the failure this page exists to surface. Any other failure of an otherwise-configured service is reported generically, with the detail in the system log, so that provider responses never reach the browser.

Both endpoints are admin-only and spend real provider quota on every run.

Speech 

Transcription and text-to-speech have no test surface. Transcription needs an audio upload and synthesis returns binary audio, which raises the same storage question the image test declines to answer, with less to gain from answering it. Their credential is the shared OpenAI identifier above, so a successful DALL·E test also confirms the key those two use.

Opening the module 

Open Admin Tools > LLM > Analytics. The submodule sits next to the other LLM sections in the left-hand navigation and is admin-only, like the rest of the module.

Choosing a date range 

A range switcher at the top of the page selects the reporting window. The range is a plain ?range= link, so changing it is an ordinary page reload — there is no AJAX. Four presets are available:

KPI tiles 

A row of tiles summarises the selected range:

• Total cost — the summed estimated cost across the window.
• Total requests — the number of AI requests recorded.
• Total tokens — prompt plus completion tokens consumed.
• Providers — how many distinct providers were used.
• Models — how many distinct models were used.

These are totals for the chosen range, not all-time figures.

Cost and requests trend 

A line chart plots daily estimated cost and daily request count across the range. Days with no usage are filled in as zero so the line is continuous rather than skipping gaps.

Breakdown charts 

Three bar charts split the window's usage along different axes:

• By provider — cost and requests per service_provider (OpenAI, Anthropic, Ollama, …).
• By model — cost and requests per model. This dimension is new: it relies on the model_uid / model_id columns added to the usage table, so it only reflects usage recorded after that change.
• By service — cost and requests per service type (chat, vision, translation, speech, image).

Per-user table 

A table lists usage grouped by backend user, ordered by cost. Each row shows the user's request count, token total, and estimated cost for the selected range, plus a monthly-budget bar that visualises how much of their per-user budget (see Per-user AI budgets) they have consumed.

Requests made without an authenticated backend user (CLI, scheduler, be_user = 0) are grouped under a system row.

Fallback rescues 

A table lists the runs a different configuration answered after the requested one failed — each line is one request the configuration you configured did not serve. It shows what was requested and what answered, each with its provider and model, how many configurations were tried, and how long the whole run took.

Unlike the rest of this module the list is read from the telemetry log (tx_nrllm_telemetry), not from the usage table, so it also covers runs that produced no billable usage.

Two things it deliberately does not show:

• Runs nobody served. A chain that was tried and exhausted names no serving configuration — it is a failure, not a rescue, and appears in the provider health scores instead.
• Runs recorded before this feature existed. Rows written by an older version carry no serving configuration and are left out rather than guessed at.

At most the 200 newest rescues of the period are listed. The limit counts rescues, not failed attempts, so a long outage — which writes one row per request — cannot push the rescues out of the list.

A configuration appearing here repeatedly is the signal to look at: its calls are being answered by a sibling, which may use a different provider, model and price than the one you selected.

Provider health and circuits 

A table lists every provider that is either configured and active or named by a run in the telemetry window, with its health score, the number of samples the score is based on, the window those samples were taken over, and the state of its circuit breaker.

Health and circuit state are both keyed by adapter type, not by provider record — two provider records on the same adapter share one score and one circuit, because it is the provider that is unhealthy, not the record.

Unlike the rest of this module the table ignores the date range selected above. Scores come from a rolling telemetry window (15 minutes by default, named on the page) and circuit state is live cache state — neither can be re-cut to a 90-day report period.

A provider with no telemetry in the window shows no data — not a score of zero. It was not called; that is not the same as failing.

A note on cost 

All cost figures are estimated. They are computed from the model pricing you configured (cents per 1M tokens, applied to the recorded prompt/completion token split), not billed back from the provider. Treat them as a planning and trend signal, not as an invoice. Costs are captured at call time, so they reflect the pricing in effect when each request ran. See ADR-029: Usage Analytics Dashboard for the design rationale.

Specialized services (DALL·E, text-to-speech, Whisper, DeepL) still record their requests and units, but their cost is currently shown as 0 — token-based pricing does not apply to them yet. Streaming responses are not recorded at all, because chunked output has no single terminal token count to price.

Usage columns in the list views 

The Providers, Models, Configurations, and Tasks list views each carry three extra columns — Cost (30d), Requests (30d) and Tokens (30d) — summarising the last 30 days of usage for that row, so you can spot the heavy hitters without leaving the list.

Two attribution notes:

• The Providers column aggregates by adapter type (the value stored on each usage row), not by individual provider record — two providers that share an adapter therefore show the same figures.
• The Tasks column relies on per-task tracking: each task execution records its task_uid so usage rolls up to the task that triggered it. Calls made outside a task (direct API/service use) are not attributed to any task row.

Demo data for local development 

To populate the module with something to look at during local development, run the dev-only DDEV command:

ddev seed-usage

It generates roughly 90 days of realistic historic usage across providers, models, services, and users so the trend line, breakdown charts, and per-user table all have content. This command is for local DDEV environments only — do not run it against production data.

The inbox 

The module shows two lists:

• Awaiting your decision — runs paused for an approval or for input, each rendered as a card with the controls to resolve it.
• Recent runs — a read-only table of the most recently finished runs (configuration, status, created, finished, cost when non-zero, and — on the runs you may open — a Timeline link to the run's detail page).

If the store cannot be read, the page shows a warning box rather than a silently empty inbox — an empty list therefore means "nothing waiting", not "load failed".

The run timeline 

Timeline opens one run end to end, read-only. It shows the run's summary — correlation, status and why it ended, configuration, rounds, tokens, cost — and below it a single time-ordered list of everything the run produced:

• Step — a recorded loop step (a request, a model answer, a tool execution).
• Provider call — a telemetry row: which provider and model served it, how long it took, whether a cache hit or a fallback was involved, and the error class when it failed. These are joined to the run because since ADR-153 every provider call a run makes carries the run's uuid as its correlation id.
• Governance — a decision taken during the run: a tool the gate withheld, a guardrail block, an approval requirement, an injected-context refusal.

You can only open your own runs; an administrator can open all of them. The approval grant lets you decide another user's waiting run, but not read its timeline — the Timeline link is therefore offered only on the rows you may open. Reaching a run you may not read directly by URL is indistinguishable from reaching one that does not exist: both send you back to the list.

A run started before this extension version has steps but no provider calls: its calls were traced individually and cannot be attributed retroactively.

Approving a tool call 

A run pauses for approval (status WAITING_FOR_APPROVAL) when the agent wants to call a tool that opts in to human approval, or when a guardrail demands it. The card lists every tool call in the pending turn — the tool name and, in a collapsible Arguments block, the exact arguments the model proposed. A call whose tool is no longer registered is flagged.

One Approve or Deny covers the whole pending turn, not a single call. Denying ends the run.

Providing input 

A run pauses for input (status WAITING_FOR_INPUT) when the agent asks for typed data against a declared schema. The card renders a form with one field per schema property (text, number, integer or checkbox, with the field description shown). Submitting validates and coerces the values against the current schema; invalid input re-renders the form in place, keeping what you typed and pointing at the error, rather than losing the run.

Running queued runs asynchronously 

By default a queued run executes in-process, synchronously, with no setup — suitable for interactive and small workloads. For genuinely asynchronous execution, route the queue message to the Doctrine transport and run a consumer:

// settings.php / additional.php
$GLOBALS['TYPO3_CONF_VARS']['SYS']['messenger']['routing']
    [\Netresearch\NrLlm\Service\Agent\Queue\AgentRunQueuedMessage::class] = 'doctrine';

vendor/bin/typo3 messenger:consume doctrine

Reclaiming stale runs 

When a run is executed asynchronously, the worker holds a 15-minute lease that it renews at every step. If the worker dies, the run is left RUNNING with a lease nobody renews. The reaper reclaims those runs — it puts them back on the queue, or, after three failed attempts, dead-letters them so they stop occupying the running set:

vendor/bin/typo3 nrllm:agent:reap

--limit (default 50) bounds how many stale runs one invocation handles. Schedule it from cron or the scheduler's Execute console commands task. It only concerns asynchronous runs — interactive runs hold no lease — and does nothing useful without a running consumer.

Retention and privacy 

A waiting run stores the transcript it needs to resume — the pending tool calls and the conversation so far — verbatim, so it can pick up exactly where it paused. Unlike the per-step event log, this resumable state is kept in full regardless of the configured privacy level, and is cleared when the run settles to a terminal status.

Nothing is deleted until a purge runs. Finished runs are removed on the privacy.retention.agentRun window; runs still waiting for a decision use the separate, deliberately longer privacy.retention.approval window, so a purge never destroys work an approver has not got to yet. Set the approval window generously if approvers may take days.

See Data retention & purge for the retention settings and the purge command that covers agent runs along with every other content-bearing table.

What is stored 

The extension configuration setting Content Privacy Level (privacy.level) decides how much request content is persisted:

none / metadata (default)

Content payloads are dropped. Metadata — timings, token counts, cost, tool names, sizes, error class — is kept. Agent-run steps are stored in this reduced form too, so persistence does not quietly build a prompt archive.

redacted

Content is stored with obvious credentials and email addresses masked and the length capped. A heuristic, not a guaranteed PII scrubber.

full

Content is stored verbatim. Choose this deliberately — for an agent run it means the whole transcript, including tool arguments and results.

Conversation messages are the exception: their content is the feature. A session replays its own history on every turn, so the transcript is always stored — and therefore governed by retention rather than by the level.

How long it is kept 

privacy.retentionDays (default 30) is the window for every category. Each category can override it under privacy.retention.* — set 0 to keep the default:

A zero, negative or non-numeric value never means "delete immediately" — it means "no override". When to set one at all, and how to read the window that is in force, is covered under The seven retention overrides.

Runs awaiting a decision are deliberately separate. A run suspended for an approval carries the state needed to resume it, so it is only deleted on the approval window. Give it a longer value than agentRun if approvers may take days.

Running the purge 

Nothing is deleted until a purge runs. Schedule the central command — it covers every content-bearing table in one pass:

vendor/bin/typo3 nrllm:privacy:purge

It reports the window applied and the rows deleted per category. --days=N overrides every category at once, which is useful for a one-off cleanup:

vendor/bin/typo3 nrllm:privacy:purge --days=7

Two single-table variants exist for operators who want separate schedules. They read the same policy, so they cannot drift from it:

vendor/bin/typo3 nrllm:session:purge
vendor/bin/typo3 nrllm:telemetry:purge

All three appear in the scheduler's Execute console commands task — no extra registration is needed.

Where the values are shown 

Admin Tools > LLM > Overview, docheader tab Governance (admin-only, like every other nr_llm admin surface). The table has three columns: the setting, the value the runtime applies right now, and the class that resolved it.

Two properties matter when reading it:

• The value is what happens, not what is stored. Each row is read through the same resolver the runtime uses, so the tab cannot drift from behaviour. A mistyped tools.dataClassEnforcement reads enforce, because that is what the gate applies to the next run.
• A row that cannot be answered reads unknown. It is never filled in from a shipped default. On a working installation every row is answered; the guarantee exists so a value shown here is always one the runtime would apply.

The tab changes nothing. All four keys are instance-wide and are set in Settings > Extension Configuration > nr_llm (why).

The four keys 

privacy.level (default metadata)

How much request content is written to the log tables — none and metadata drop the payload, redacted masks and caps it, full stores it verbatim. Read by Service\Privacy\PrivacyPolicy, which every content sink asks before it writes. An unrecognised or unreadable value resolves to metadata: the installation stores less than you asked for, never more. See What is stored.

privacy.retentionDays (default 30)

The window after which nrllm:privacy:purge deletes a row. Read by the same PrivacyPolicy. Empty, zero, negative or non-numeric falls back to 30 days — a window of 0 never means "delete immediately". Nothing is deleted until the purge command actually runs (Running the purge).

tools.dataClassEnforcement (default enforce)

Whether a tool whose data class exceeds the trust zone of the provider a run can reach is removed from that run (enforce) or still offered and merely recorded (observe). Read by Service\Tool\DataClassEnforcementResolver — the same object the gate itself asks. Only a literal observe observes; leading and trailing whitespace and letter case are ignored, everything else enforces (ADR-113).

skills.minTrustLevel (default untrusted)

The publisher-trust floor a skill's source must meet before the skill is injected into a prompt or may grant tools: untrusted, community, verified or first_party. Read by Service\Skill\SkillComposerFactory, which builds every skill composer. A missing, unreadable or unrecognised value resolves to untrusted — the lowest floor, at which every enabled skill passes. See Isolation controls: trust, fingerprint, injection scan, audit.

Two keys, opposite fallbacks 

tools.dataClassEnforcement and skills.minTrustLevel react to a broken value in opposite directions, and the difference is deliberate:

Both fall towards the outcome that cannot cause damage, but "safe" points the other way in each case. A broken enforcement value must not switch a security control off, so it enforces. A broken trust value must not raise a bar nobody set, because that would silently hide working skills from prompts with no error anywhere — so it drops to the floor at which nothing is hidden. Raising enforcement never grants a tool; raising the trust floor only ever removes skills.

For an operator this has one practical consequence: do not infer one key from the other. A typo in either key leaves the tab showing a plausible value, but a different one in each case:

• The tab reads enforce although you set observe — the stored value is mistyped, and the gate is removing tools.
• The tab reads untrusted although you set verified — the stored value is mistyped, and every enabled skill is passing.

In both cases the tab is right and the Install Tool field is wrong. Fix the field; the tab follows on the next request.

The seven retention overrides 

privacy.retentionDays has seven per-category overrides (privacy.retention.conversation, .agentRun, .approval, .telemetry, .evaluation, .skillAudit, .governance). They are not on the tab: all seven ship as 0, which means "no override", so on an untouched installation eleven rows would repeat the global window eight times.

0 never means "delete immediately" — neither does a negative or non-numeric value. Each category simply uses privacy.retentionDays until you give it a window of its own.

Set an override when one category genuinely needs a different window:

• Conversation transcripts are the most sensitive rows the extension stores, and unlike everything else their content is kept regardless of privacy.level. A shorter window here is the usual first override.
• Runs awaiting a decision (approval) need a longer window than finished runs. A run suspended for a human approval carries the state needed to resume it; purging it destroys work in flight. Give it more days than approvers realistically need.
• Telemetry and governance events carry no prompts or responses — only counts, reasons and the acting user. Keeping them longer than content rows buys capacity trends and denial evidence at no content risk.
• Skill audit is the append-only provenance trail for everything that reached a prompt. Keep it for as long as you may have to reconstruct that.

The full table of what each override covers is in How long it is kept.

Recommended settings 

The shipped defaults are a safe posture for a new installation. Nothing has to be set to reach it — this block only makes it explicit:

privacy.level              = metadata
privacy.retentionDays      = 30
tools.dataClassEnforcement = enforce
skills.minTrustLevel       = untrusted

For an installation that handles personal data, or that runs skills from sources it does not publish itself:

privacy.level                   = metadata
privacy.retentionDays           = 7
privacy.retention.conversation  = 3
privacy.retention.approval      = 30
tools.dataClassEnforcement      = enforce
skills.minTrustLevel            = verified

What changes against the default: rows are gone within a week and conversation transcripts within three days, while runs waiting for an approver survive a month; skills below verified are dropped from prompt injection and from the allowed-tools union without being deleted, so re-classifying a source brings them back.

privacy.level stays at metadata on purpose. none is not stricter: both drop the content payload and keep the metadata, and no code path treats them differently. Only redacted and full write content at all — pick either deliberately, and never on an installation that must not hold request content.

Telling whether it works 

The gate records every denial — and every observe-mode flag — as a governance event carrying the tool name, the reason, the trust zone and the ceiling. Two places show them:

• The Tool denials by reason dashboard widget. The Trust zone ceiling bar is the data-class axis. It counts a rolling 30 days and has no mode filter: an observe-mode flag and a real removal are the same row, with the same reason. Only the event's detail column separates them (observedOnly=1 marks a flag), and no view reads that column.
• The Governance blocks widget for the wider picture, including guardrail blocks and approvals.

The bar therefore answers "how often did the data-class axis fire in the last 30 days", not "how many tools would enforcement remove". Use it to see that the axis fires. Take the number you act on from the table.

The list you actually need is one row per configuration and tool, and it comes from tx_nrllm_governance_event:

SELECT configuration_identifier, tool_name, COUNT(*) AS events
FROM tx_nrllm_governance_event
WHERE decision = 'tool_denied'
  AND reason = 'trustZone'
  AND detail LIKE '%observedOnly=1%'
  AND crdate >= 1767225600
GROUP BY configuration_identifier, tool_name;

observedOnly=1 gives you what enforcement would remove; observedOnly=0 gives you what it did remove. crdate is Unix time — set it to the moment the period you care about started, the switch to observe or the upgrade.

Each row names one configuration and one tool the axis acted on, and that pairing is what you fix. Three ways to change it: raise the trust zone on the provider record, remove an external fallback that drags the configuration's reachable zone down to the external ceiling, or drop the tool from the configuration's allowed groups.

So moving a long-running installation to enforce goes like this: note the time, set observe, let a representative workload run, run the query for the observation window, fix the configurations it names, set enforce again. The bar is at its least trustworthy during exactly this procedure — while you observe it still carries the enforce-mode removals of the preceding 30 days, and after you switch back it carries the observe flags for another 30. The query is unaffected, because it filters on both the flag and the window.

Governance events are purged on privacy.retention.governance — make sure that window is longer than your observation period, or the evidence is gone before you read it.

Would this be allowed? 

Pick a configuration, a tool and — optionally — a backend user, then press Simulate. The tab runs that call past the four gates listed below and reports one verdict plus each gate's own answer (ADR-157).

The verdict is one of three:

Allowed

All four gates permit the call and it would run unattended.

Allowed, after a human approves

All four gates permit the call, and the tool is approval-bound (ADR-134): the run suspends and waits for a decision before it executes. Folding this into Allowed would hide the axis at exactly the moment it decides, so it is its own outcome.

Blocked

At least one of the four refuses. The table says which.

Four gates are asked, each through the service the runtime itself calls:

Only one axis is actor-scoped, and the table says so. Routing reads the model catalogue with enable-fields ignored and no user context, the input-context gate compares a configuration against a trust zone, and the approval requirement is a property of the tool's own declaration. A picker that implied four per-user answers where there is one would be worse than no picker.

Three things that can stop a real call are not asked here, so Allowed does not promise them.

Configuration access (ADR-070) is the one the picker makes easy to miss. ConfigurationResolver refuses a configuration whose backend groups the acting user is not a member of. The configuration selector lists every active configuration and applies no such filter. So a group-restricted configuration paired with a non-member reads Allowed on this tab and is refused at runtime. It is the second axis that reads the user's groups, and it is the one the tab does not ask.

The other two are the budget check and the guardrail pipeline. Both decide on the call itself — the remaining spend, the text of the prompt — and a picker supplies neither.

The actor picker is not impersonation. The selected backend user is resolved read-only through the same seam a queue worker uses to authorise for the user who queued its work (ADR-083): the uid is looked up, the fresh database record supplies the permission surface, and the gates are asked. No session is switched, nothing executes as that user, and nothing is written. Privilege comes from the record, so the picker cannot grant rights the account does not have — and a uid that no longer resolves, because the account was deleted or disabled, produces a stated refusal rather than a silent fall back to your own rights.

A simulation is not recorded. The runtime writes a governance event when it blocks a call; a simulation blocks nothing, so writing one would put rows into the audit for calls that never happened. The trade is deliberate and it has a cost: "who checked what, and when" cannot be answered from the audit. See ADR-157.

Observe mode is visible on both gates. A configuration the input-context gate refuses while tools.dataClassEnforcement is observe is reported as permitted and refused: the send proceeds and the refusal is recorded. Reading only "no exception" would have called that allowed.

Why this model? 

The same tab answers the other question an operator asks about a configuration: which model would actually serve a call through it, and why not one of the others. Pick a configuration, optionally the operation the call runs, optionally a policy mode to try, and press Explain (ADR-148).

The answer comes from Service\Routing\RoutingDecisionService — the decision point the runtime itself uses, not a second implementation of the ranking. It reports:

• the selected model, and the eligible candidates in the order they were ranked, each with its score and the per-signal values behind it;
• every refused candidate with the reason it was refused — a missing capability, an excluded adapter type, a context window below the minimum, a cost above the ceiling, or a declared capability set without the one the operation needs;
• the effective policy mode, and whether the operation-capability axis is enforcing or only observing.

Three things are worth knowing before reading it:

A fixed-mode configuration is not a decision. If the configuration names its model, nothing is chosen at call time. The tab says so instead of presenting the named model as the winner of a one-candidate ranking — there are no criteria to debug in that case.

A signal without data is not a zero. no data means nothing was measured for that model. It neither promotes nor demotes: the score is the weighted mean over the signals that do have data (ADR-142). In Provider priority mode no signal is collected at all, and the ordering falls through to provider priority and the established tiebreaks.

Trying a policy mode changes nothing. The mode selector evaluates a hypothetical for that one page view. routing.policyMode in the Install Tool is not written and not affected — the same read-only rule the rest of the tab follows.

Only operations that actually constrain the decision are offered. The others map to no required capability, so they would add nothing to the answer. Leaving the selector on No operation is answered as exactly that — the axis was not applied — and not as an operation that requires nothing.

An empty result is reported in two distinguishable ways, because they need opposite fixes: No candidates at all means the catalogue holds no active model, while a populated Refused, and why table means the criteria and the model records disagree.

Calls that were routed 

The readout above answers a hypothetical. Calls that were routed answers the same question about calls that already ran: the last seven days of runs whose model was chosen automatically, newest first, twenty at a time (ADR-156).

Each row names when the call ran and against which configuration, which model answered it, and the decision behind that: the policy mode, how many candidates were considered, which measured signals actually moved the ranking, and the distinct reasons that refused the rest.

Fixed-mode calls are absent, and that is the point. Nothing was chosen for them, so there is no decision to show. If the table is empty on a busy installation, the likely reasons are that every configuration names a fixed model, or that telemetry.enabled is off in the Install Tool.

"Signals used" means the signal moved this decision, not that the mode weighs it and not that the ranking collected it. A quality decision over a catalogue nobody has scored shows no signals used and ranks exactly as Provider priority would — the weights only apply to signals that have data. A signal the mode weighs at zero is not listed either: quality weighs cost at zero, so Prefer Lowest Cost on a quality configuration shows no cost signal, even though it still breaks ties between models that scored equally.

The candidate models are not stored per call. Which models exist and which lost is a catalogue question; read it off the live catalogue with the readout above. The row keeps the count and the reason set, which is what varies from request to request.

Rows are purged with the rest of the telemetry table by nrllm:telemetry:purge; a window shorter than your observation period deletes the evidence before you read it.

The complexity columns are observed, not applied 

The same rows carry a measurement of how involved each request was: a 0-100 structural score, the request shape (a single question, a conversation, or a tool-assisted transcript), the number of tool schemas on the wire, the payload size in bytes, the token estimate and how much of the model's context window it filled.

You see them for routed calls only. The measurement is taken on every configuration-driven send, fixed-mode ones included, but it is stored on the telemetry row and the table above shows only rows whose model was chosen automatically. An installation with no criteria-mode configuration collects these columns and displays none of them; the figures are in tx_nrllm_telemetry if you query it directly.

Nothing routes on any of it. There is no setting that turns it into a routing signal, and none is planned until three things have been shown on real traffic: that cheaper models hold for simple requests, that quality does not degrade, and that real cost drops by enough to be worth a permanent branch in the decision path (ADR-156 states the criteria in full). The columns exist so that question can be settled with data rather than opinion.

Two readings need care:

The score is uncalibrated. It is three capped terms — conversation turns, tool count, context utilisation — chosen to be defensible, not fitted to anything. Correlate against it; do not treat it as a threshold.

"window not measured" is not "empty". The token and utilisation figures come from the context fit (ADR-143). Where no fit ran they are stored as NULL, and the page says so rather than showing a zero nobody measured. The byte count is unaffected — it needs no fit — so a row that says "window not measured" still tells you how large the send was. A utilisation above 100 % is real: it is the overflow case, and it is deliberately not clamped.

A measured 0 % is a measurement. A short chat against a large window rounds to zero, and the page shows ~N tokens, 0% of the window for it rather than falling back to "not measured".

"complexity not measured" replaces the whole cell, and is a different statement from "window not measured". Some calls choose a model without ever sending a measurable payload through the context fit — an embeddings configuration in criteria mode is the usual one. Its row has a decision to show and nothing to measure, so the score, the shape, the tool count and the byte count are absent rather than shown as zeros.

Why there is no apply button 

The page is read-only on purpose (ADR-140), not unfinished. TYPO3 offers exactly one API for writing extension configuration, and it is marked internal, writes the whole merged array back at once rather than a single key, and is explicitly documented as unreliable when additional.php overrides a setting. An apply button would therefore report success while the next request still served the old value — the worst thing a governance page can do. It would also materialise every shipped default as an explicitly stored value, and the upgrade wizards read that distinction to tell "the operator chose this" from "nobody ever set it". The core synchronisation already erases it on its own the first time an admin enters the Install Tool, so the apply button would not cause that loss — it would make it unconditional.

The Install Tool owns the write, the synchronisation and the cache flush. The tab reports what is in force.

Required 

Optional 

Required 

Optional 

Required 

Optional 

Provider configuration 

Providers, models and configurations are database-backed and managed in the LLM backend module — not via TypoScript. nr-llm does not read plugin.tx_nrllm TypoScript settings; any such constants/setup have no effect (this is true for both classic TypoScript templates and site sets).

To make the generic chat() / complete() entry points work without pinning a provider per call, set up a default configuration:

1. Open the LLM backend module.
2. Create a Provider (e.g. OpenAI) and store its API key as an nr-vault identifier — see API key protection.
3. Create a Model for that provider.
4. Create a Configuration bundling the model, then mark it active and default.

The Setup Wizard in the module walks through these steps.

Without an active default configuration, generic calls throw "No provider specified and no default provider configured".

Environment variables 

# TYPO3 encryption key (used for API key encryption)
TYPO3_CONF_VARS__SYS__encryptionKey=your-key

Security 

// Strip markup and control characters from free-text input before it is
// sent to a provider. (GeneralUtility::removeXSS() was removed from the
// TYPO3 core and must not be used.)
$sanitizedInput = trim(strip_tags($userInput));

$response = $adapter->chatCompletion([
    ['role' => 'user', 'content' => $sanitizedInput],
]);

$response = $adapter->chatCompletion([
    ['role' => 'user', 'content' => $prompt],
]);

$safeOutput = htmlspecialchars(
    $response->content, ENT_QUOTES, 'UTF-8'
);

Logging 

use Psr\Log\LogLevel;
use TYPO3\CMS\Core\Log\Writer\FileWriter;

$GLOBALS['TYPO3_CONF_VARS']['LOG']
    ['Netresearch']['NrLlm'] = [
    'writerConfiguration' => [
        LogLevel::DEBUG => [
            FileWriter::class => [
                'logFileInfix' => 'nr_llm',
            ],
        ],
    ],
];

Log files: var/log/typo3_nr_llm_*.log

Caching 

The extension uses TYPO3's caching framework with cache identifier nrllm_responses.

No cache backend is specified — TYPO3 automatically uses the instance's default cache backend. If your instance has Redis, Valkey, or Memcached configured, nr-llm uses it transparently with zero configuration.

• Cache identifier: nrllm_responses
• Cache group: nrllm
• Default TTL: 3600 seconds (1 hour)
• Embeddings TTL: 86400 seconds (24 hours)

To override the backend for this cache specifically:

use TYPO3\CMS\Core\Cache\Backend\RedisBackend;

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']
    ['cacheConfigurations']['nrllm_responses']
    ['backend'] = RedisBackend::class;

Clear cache:

vendor/bin/typo3 cache:flush --group=nrllm

Core concepts 

┌─────────────────────────────────────────┐
│         Your Application Code           │
└────────────────┬────────────────────────┘
                 │
┌────────────────▼────────────────────────┐
│         Feature Services                │
│  (Completion, Embedding, Vision, etc.)  │
└────────────────┬────────────────────────┘
                 │
┌────────────────▼────────────────────────┐
│         LlmServiceManager               │
│    (Provider selection & routing)       │
└────────────────┬────────────────────────┘
                 │
┌────────────────▼────────────────────────┐
│           Providers                     │
│    (OpenAI, Claude, Gemini, etc.)       │
└─────────────────────────────────────────┘

use Netresearch\NrLlm\Service\LlmServiceManager;
use Netresearch\NrLlm\Service\Feature\CompletionService;
use Netresearch\NrLlm\Service\Feature\EmbeddingService;
use Netresearch\NrLlm\Service\Feature\VisionService;
use Netresearch\NrLlm\Service\Feature\TranslationService;

class MyController
{
    public function __construct(
        private readonly LlmServiceManager $llmManager,
        private readonly CompletionService $completionService,
        private readonly EmbeddingService $embeddingService,
        private readonly VisionService $visionService,
        private readonly TranslationService $translationService,
    ) {}
}

Using LlmServiceManager 

$messages = [
    ['role' => 'system', 'content' => 'You are a helpful assistant.'],
    ['role' => 'user', 'content' => 'What is TYPO3?'],
];

$response = $this->llmManager->chat($messages);

// Response properties
$content = $response->content;           // string
$model = $response->model;               // string
$finishReason = $response->finishReason; // string
$usage = $response->usage;               // UsageStatistics

use Netresearch\NrLlm\Service\Option\ChatOptions;

// Using ChatOptions object
$options = ChatOptions::creative()
    ->withMaxTokens(2000)
    ->withSystemPrompt('You are a creative writer.');

$response = $this->llmManager->chat($messages, $options);

// Or using array
$response = $this->llmManager->chat($messages, [
    'provider' => 'claude',
    'model' => 'claude-sonnet-4-6',
    'temperature' => 1.2,
    'max_tokens' => 2000,
]);

$response = $this->llmManager->complete('Explain recursion in programming');

// Single text
$response = $this->llmManager->embed('Hello, world!');
$vector = $response->getVector(); // array<float>

// Multiple texts
$response = $this->llmManager->embed(['Text 1', 'Text 2', 'Text 3']);
$vectors = $response->embeddings; // array<array<float>>

Response objects 

See the API reference for the complete response object documentation. Key classes:

• CompletionResponse — content, model, usage, finishReason, toolCalls
• EmbeddingResponse — embeddings, model, usage
• UsageStatistics — promptTokens, completionTokens, totalTokens

Error handling 

The extension throws specific exceptions:

use Netresearch\NrLlm\Provider\Exception\ProviderException;
use Netresearch\NrLlm\Provider\Exception\ProviderConfigurationException;
use Netresearch\NrLlm\Provider\Exception\ProviderConnectionException;
use Netresearch\NrLlm\Provider\Exception\ProviderResponseException;
use Netresearch\NrLlm\Provider\Exception\UnsupportedFeatureException;
use Netresearch\NrLlm\Exception\InvalidArgumentException;

try {
    $response = $this->llmManager->chat($messages);
} catch (ProviderConfigurationException $e) {
    // Invalid or missing provider configuration
} catch (ProviderConnectionException $e) {
    // Connection to provider failed
} catch (ProviderResponseException $e) {
    // Provider returned an error response
} catch (UnsupportedFeatureException $e) {
    // Requested feature not supported by provider
} catch (ProviderException $e) {
    // General provider error
} catch (InvalidArgumentException $e) {
    // Invalid parameters
}

Events 

Best practices 

1. Use feature services for common tasks instead of raw LlmServiceManager.
2. Enable caching for deterministic operations like embeddings.
3. Handle errors gracefully with proper try-catch blocks.
4. Sanitize input before sending to LLM providers.
5. Validate output and treat LLM responses as untrusted.
6. Use streaming for long responses to improve UX.
7. Set reasonable timeouts based on expected response times.
8. Monitor usage to control costs and prevent abuse.

Usage 

$stream = $this->llmManager->streamChat($messages);

foreach ($stream as $chunk) {
    echo $chunk;
    ob_flush();
    flush();
}

The streamChat method returns a Generator that yields string chunks as the provider generates them. Each chunk contains a portion of the response text.

Providers that implement streamingcapableinterface support streaming. Check provider capabilities before using:

$provider = $this->llmManager->getProvider('openai');
if ($provider instanceof StreamingCapableInterface) {
    // Provider supports streaming
}

Defining tools 

$tools = [
    [
        'type' => 'function',
        'function' => [
            'name' => 'get_weather',
            'description' => 'Get current weather for a location',
            'parameters' => [
                'type' => 'object',
                'properties' => [
                    'location' => [
                        'type' => 'string',
                        'description' => 'City name',
                    ],
                    'unit' => [
                        'type' => 'string',
                        'enum' => ['celsius', 'fahrenheit'],
                    ],
                ],
                'required' => ['location'],
            ],
        ],
    ],
];

Executing tool calls 

CompletionResponse::$toolCalls is a list of NetresearchNrLlmDomainValueObjectToolCall value objects — $toolCall->arguments is already a JSON-decoded associative array, so no manual json_decode() is needed. The two follow-up turns are built with the ChatMessage factories: ChatMessage::assistantToolCalls() echoes the assistant turn that carries the tool calls, and ChatMessage::toolResult() answers one call by its id.

use Netresearch\NrLlm\Domain\ValueObject\ChatMessage;

$response = $this->llmManager->chatWithTools($messages, $tools);

if ($response->hasToolCalls()) {
    // Echo the assistant turn (with all its tool calls) back first
    $messages[] = ChatMessage::assistantToolCalls($response->toolCalls, $response->content);

    foreach ($response->toolCalls as $toolCall) {
        // Execute your function — $toolCall->arguments is a decoded array
        $result = match ($toolCall->name) {
            'get_weather' => $this->getWeather($toolCall->arguments['location']),
            default => throw new \RuntimeException("Unknown function: {$toolCall->name}"),
        };

        // Answer the call by its id
        $messages[] = ChatMessage::toolResult($toolCall->id, json_encode($result, JSON_THROW_ON_ERROR));
    }

    // Ask the model to answer with the tool results in context
    $response = $this->llmManager->chat($messages);
}

Providers that implement toolcapableinterface support tool calling.

Registering your provider 

Register your provider in Services.yaml:

MyVendor\MyExtension\Provider\MyCustomProvider:
  arguments:
    $httpClient: '@Psr\Http\Client\ClientInterface'
    $requestFactory: '@Psr\Http\Message\RequestFactoryInterface'
    $streamFactory: '@Psr\Http\Message\StreamFactoryInterface'
    $logger: '@Psr\Log\LoggerInterface'
  tags:
    - name: nr_llm.provider
      priority: 50

What an adapter has to answer to 

Every bundled adapter passes one shared contract case, TestsUnitProviderContractAbstractAdapterContractTestCase (ADR-160: One adapter contract, and honest capability provenance). It is the readable statement of what an adapter is expected to do, so read it before writing one — and extend it in your own test suite if you want the same guarantees:

Identifier

getIdentifier() returns the stable registration key, and that same key travels on every CompletionResponse.

Capability declaration

The capability interfaces an adapter implements and the features it lists in $supportedFeatures must agree. The service layer reads the first, LlmServiceManager::supportsFeature() reads the second; a disagreement gives two callers opposite answers about the same adapter.

Error normalisation

401 becomes ProviderAuthenticationException, 429 becomes ProviderRateLimitException, any other 4xx becomes ProviderResponseException carrying the provider's own message, a 5xx and an undecodable 2xx body become ProviderConnectionException. Nothing leaves an adapter as a raw transport exception.

No credential, no request

An adapter that needs an API key throws ProviderConfigurationException before it builds a request, rather than sending without one and letting the provider answer 401. A keyless provider — a local Ollama — declares that with requiresApiKey(): false and the contract skips by name.

Timeout behaviour

A client-side timeout surfaces as a connection failure and is attempted exactly once. Retrying a timeout multiplies the caller's wait by the attempt count.

Usage reporting

Token counts come from the provider's own counters; the total is derived. A response with no usage block degrades to zero rather than failing, because cost accounting writes a row per call.

Tool calls and structured output

Where the adapter declares the capability: a provider tool call arrives as a typed ToolCall with a non-empty id, the declared tools reach the request, a strict schema is enforced through the provider's native shape, a schema the provider cannot enforce degrades instead of earning a 400, and a malformed structured answer is passed back untouched so CompletionService can run its one repair attempt.

A capability the adapter does not have is skipped by name in the run output. That is deliberate: it keeps "cannot" distinguishable from "not tested".

Preferred: the #[AsLlmProvider] attribute 

Add the attribute to any provider class that lives under the Netresearch\NrLlm\ namespace. The compiler pass auto-tags the service, sets it public (so backend diagnostics can resolve it by class name), and registers it with LlmServiceManager in priority order:

use Netresearch\NrLlm\Attribute\AsLlmProvider;
use Netresearch\NrLlm\Provider\AbstractProvider;

#[AsLlmProvider(priority: 85)]
final class MyProvider extends AbstractProvider
{
    public function getIdentifier(): string
    {
        return 'my-provider';
    }

    public function getName(): string
    {
        return 'My LLM Service';
    }

    // ... chatCompletion(), embeddings(), supportsFeature()
}

Priority is an ordering hint only. Providers are still resolved by their getIdentifier() at runtime. Higher priority wins when two providers otherwise tie.

Third-party fallback: yaml tagging 

Extensions that sit outside the Netresearch\NrLlm\ namespace still work via the original mechanism — declare a service with the nr_llm.provider tag:

services:
  Acme\MyExt\Provider\AcmeProvider:
    public: true
    tags:
      - name: nr_llm.provider
        priority: 85

When both yaml tagging AND the attribute are present on the same service, the yaml wins (the attribute pass skips already-tagged services). Treat this as an override hook rather than an additive mechanism.

Capability interfaces 

Priority governs registration order only; it says nothing about what a provider can do. Capabilities are advertised by implementing the relevant interface from NetresearchNrLlmProviderContract:

• VisionCapableInterface — image analysis
• StreamingCapableInterface — SSE streaming
• ToolCapableInterface — function / tool calling
• DocumentCapableInterface — PDF / structured document input

LlmServiceManager dispatches to a provider only when the caller's requested operation matches a capability the provider actually advertises. A provider that doesn't implement VisionCapableInterface can never be asked to describe an image, regardless of priority. See ADR-022: Attribute-Based Provider Registration for the attribute-discovery design decision and the Symfony registerAttributeForAutoconfiguration alternative we evaluated.

Configuring a chain 

The tx_nrllm_configuration.fallback_chain column stores a JSON object with a single key, configurationIdentifiers, whose value is the ordered array of target configuration identifiers:

{"configurationIdentifiers": ["claude-sonnet", "ollama-local"]}

Editors paste that JSON into the Fallback Chain tab in the backend form. The order is the retry order. Identifiers are matched case-insensitively against tx_nrllm_configuration.identifier. Using an object (rather than a bare top-level array) leaves room for future sibling fields — e.g. per-link retry policy — without a schema break.

Retryable vs. non-retryable errors 

Fallback only triggers for errors the next provider might actually recover from:

When every configuration in the chain trips a retryable error, NetresearchNrLlmProviderExceptionFallbackChainExhaustedException is thrown. It carries the per-attempt errors so consumers can surface the full failure sequence.

Scope limits 

v1 is deliberately narrow:

• No streaming. streamChatWithConfiguration() does not wrap the call. Once the first chunk has been yielded to the caller, mid-stream provider-switching would be detectable and surprising.
• No recursion. A fallback configuration's own chain is ignored. This avoids cycles (a -> b -> a) and unbounded attempt trees.
• Single primary-only chain is a no-op. If the configured chain contains only the primary's own identifier, the primary's original exception is rethrown verbatim rather than wrapped in FallbackChainExhaustedException.

Using the DTO directly 

For programmatic construction — e.g. a wizard that generates a configuration and also sets up fallback — use the NetresearchNrLlmDomainDTOFallbackChain value object:

use Netresearch\NrLlm\Domain\DTO\FallbackChain;

$chain = (new FallbackChain())
    ->withLink('claude-sonnet')
    ->withLink('ollama-local');

$configuration->setFallbackChainDTO($chain);

The DTO trims and lowercases identifiers on entry, deduplicates them, and silently rejects empty strings and non-string entries read from malformed JSON. See ADR-021: Provider Fallback Chain for the full design rationale and the alternatives we ruled out.

Declaring presets in your extension 

Implement ConfigurationPresetProviderInterface. The nr_llm.configuration_preset DI tag is applied automatically when your extension's Services.yaml has autoconfigure: true (the TYPO3 default):

<?php

declare(strict_types=1);

namespace Vendor\NrAiSearch\Integration;

use Netresearch\NrLlm\Domain\DTO\ModelSelectionCriteria;
use Netresearch\NrLlm\Service\Preset\ConfigurationPreset;
use Netresearch\NrLlm\Service\Preset\ConfigurationPresetProviderInterface;

final class AiSearchPresetProvider implements ConfigurationPresetProviderInterface
{
    public function getPresets(): array
    {
        return [
            new ConfigurationPreset(
                identifier: 'nr_ai_search.chat',
                name: 'AI Search Chat',
                description: 'Answers site-search questions with tool support.',
                criteria: new ModelSelectionCriteria(
                    capabilities: ['chat', 'tools'],
                    minContextLength: 8000,
                ),
                systemPrompt: 'You answer questions about this website.',
                temperature: 0.2,
                maxTokens: 2000,
            ),
            new ConfigurationPreset(
                identifier: 'nr_ai_search.embedding',
                name: 'AI Search Embeddings',
                description: 'Creates embeddings for the search index.',
                criteria: new ModelSelectionCriteria(
                    capabilities: ['embedding'],
                    preferLowestCost: true,
                ),
            ),
        ];
    }
}

Rules the value object enforces at construction time:

• The identifier is lowercase [a-z0-9_] segments separated by dots and must be namespaced with your extension key (nr_ai_search.chat) so presets from different extensions cannot collide. Duplicate identifiers across providers fail fast at container build time.
• The criteria must require at least one capability.

All other fields (system prompt, temperature, max tokens, the daily budget ceilings maxRequestsPerDay / maxTokensPerDay / maxCostPerDay, allowedToolGroups) are optional seeds; null keeps the column default of the created record.

At runtime, resolve the imported configuration by its identifier as usual, for example through LlmConfigurationServiceInterface.

Import flow 

1. The admin queries the pending presets (AJAX route nrllm_preset_list). Each entry carries a preflight result: whether the criteria currently match an active model (satisfiable + matchedModelLabel), or which requirement eliminates every candidate (missingRequirement).
2. The admin confirms one import (AJAX route nrllm_preset_import with the preset identifier). nr_llm creates an active, criteria-mode tx_nrllm_configuration record and stores the preset's checksum in preset_checksum.
3. The record is a normal configuration from then on — the admin can edit or delete it. A preset whose identifier already has a record is never offered again (and an import attempt is refused), so imports are idempotent; the stored checksum makes a changed declaration detectable.

Both endpoints are restricted to backend administrators (ADR-037). Admins normally go through the Configurations backend module, which renders the pending presets — including each preflight result — above the configuration records and imports one through nrllm_preset_import with a single click; see Importing configuration presets.

Change detection 

nrllm_preset_list additionally returns a drifted list: imported presets whose current declaration checksum no longer matches the preset_checksum stored at import time. Each entry carries identifier, name, configurationUid, and changedFields — the machine names of the fields an update would overwrite (an additive summary; may be empty when the declaration only dropped an optional seed). The Configurations module flags such records with a non-blocking "Preset changed" hint next to a Review update action.

nr_llm never updates an imported record automatically, but the admin can review and apply a changed declaration.

Update flow 

Two further admin-gated AJAX endpoints resolve drift:

1. nrllm_preset_diff (GET, identifier) returns the field-level changes an update would apply — each a field (machine name, e.g. temperature or criteria.capabilities), the record's current value, and the declared value. It refuses (422) when there is nothing to update: the record is up to date, was not imported from a preset, was switched to fixed model selection, or the changed criteria are currently unsatisfiable.
2. nrllm_preset_update (POST, identifier) applies a reviewed update after the admin re-confirmed, then returns the changedFields that were applied.

An update follows the declaration for name, description and criteria, and for each optional seed that carries a value; a seed the declaration left null does not reset the record. It leaves the admin-owned fields untouched — active state, default flag, backend groups, and the fallback chain — and re-stamps the stored checksum so the drift hint clears. See ADR-056 for the design.

When your extension changes a preset declaration, ship the change; admins see the drift hint and re-confirm the diff. Document only anything the flow cannot carry (for example a change that also needs a new provider or model configured first).

Declaring a golden set in your extension 

Implement GoldenPromptSetProviderInterface. The nr_llm.golden_prompt_set DI tag is applied automatically when your extension's Services.yaml has autoconfigure: true (the TYPO3 default):

<?php

declare(strict_types=1);

namespace Vendor\NrAiSearch\Evaluation;

use Netresearch\NrLlm\Service\Evaluation\Assertion;
use Netresearch\NrLlm\Service\Evaluation\GoldenPrompt;
use Netresearch\NrLlm\Service\Evaluation\GoldenPromptSet;
use Netresearch\NrLlm\Service\Evaluation\GoldenPromptSetProviderInterface;

final class AiSearchGoldenSetProvider implements GoldenPromptSetProviderInterface
{
    public function getGoldenPromptSets(): array
    {
        return [
            new GoldenPromptSet(
                identifier: 'nr_ai_search.faq',
                name: 'AI Search FAQ answers',
                description: 'Checks the model answers common FAQ prompts correctly.',
                prompts: [
                    new GoldenPrompt(
                        id: 'opening-hours',
                        prompt: 'When is the office open? Answer with the days.',
                        assertions: [
                            Assertion::contains('Monday'),
                            Assertion::regex('/9(:00)?\s*(am|–|-)/i'),
                        ],
                        reference: 'Monday to Friday, 9am to 5pm.',
                    ),
                    new GoldenPrompt(
                        id: 'contact-json',
                        prompt: 'Return the contact as JSON with an "email" field.',
                        assertions: [
                            Assertion::jsonSchema('{"type":"object","required":["email"],"properties":{"email":{"type":"string"}}}'),
                        ],
                    ),
                ],
            ),
        ];
    }
}

The identifier is namespaced (vendor_extension.set) so sets from different extensions cannot collide. Each prompt needs at least one assertion or a reference answer.

Assertion types 

The deterministic grader supports four assertion types; a prompt passes only when all of its assertions hold, and the score is the fraction satisfied.

The json_schema matcher is a lightweight structural check — a top-level type, object required keys, and recursive properties types. Extra keys are allowed. It is intentionally not a full JSON Schema draft validator.

Graders 

Two grading strategies sit behind GraderInterface:

• deterministic (default) — evaluates the assertions with no LLM call and no tokens.
• llm_judge (opt-in) — asks a judge model through CompletionService to score the response 0.0–1.0 with a justification. It uses the reference answer when one is declared. Because it spends tokens, it runs only when explicitly selected; an unknown grader name falls back to the deterministic grader.

Running an evaluation 

Use the nrllm:eval:run command:

# Deterministic grading against the configured default model
vendor/bin/typo3 nrllm:eval:run nr_ai_search.faq

# Evaluate a specific model with the LLM judge
vendor/bin/typo3 nrllm:eval:run nr_ai_search.faq --model gpt-5.2 --grader llm_judge

# Fail (non-zero exit) if quality regressed against the previous run — for CI
vendor/bin/typo3 nrllm:eval:run nr_ai_search.faq --fail-on-regression

The command prints the per-prompt gradings and the aggregate (pass rate, mean score), stores the run in tx_nrllm_eval_result, and reports whether the run regressed against the previous run for the same set and model. The regression tolerance is configurable with --max-pass-rate-drop and --max-mean-score-drop (both default to 0.1).

nr_llm ships an example set, nr_llm.smoke, so the command is runnable out of the box.

Quality-aware routing (opt-in) 

Stored evaluation results feed an opt-in routing hook. The existing cost/latency selection modes of ModelSelectionService are unchanged; nothing routes by quality unless you call the hook explicitly:

use Netresearch\NrLlm\Service\Evaluation\QualityAwareModelSelector;

// Inject QualityAwareModelSelector, then:
$model = $selector->selectByQuality(
    ['capabilities' => ['chat']],
    minQuality: 0.7,
);

selectByQuality() takes the candidates ModelSelectionService would return for the criteria and re-ranks them by measured quality score (latest run per set, averaged). Candidates without evaluation data keep their base order behind the scored ones; with minQuality set, candidates below it (or without data) are excluded. Making quality a first-class sort key inside ModelSelectionService is a planned follow-up (see ADR-060).

Retrieval evaluation 

The retrieval counterpart measures the retrieval step of a RAG pipeline — which documents surface for a question — with golden question sets and document-level top-1/top-3 hit rates. See ADR-072 for the design and the methodology it adopts.

A golden question carries the question text, its form (MATCH = the vocabulary overlaps the target document, GAP = an everyday rewording — the class retrieval problems live in), ALL document ids that answer it (any of them counts as a hit), an optional hard class for a per-class breakdown, and an optional answer gist documenting the label. A question with an empty expected-document list declares that nothing in the index answers it and scores as a hit only when the retriever returns nothing. nr_llm ships no golden questions — labels only mean something against a concrete corpus, so every set lives in the extension owning the content.

Declare a set by implementing GoldenQuestionSetProviderInterface (tag nr_llm.golden_question_set, applied automatically):

use Netresearch\NrLlm\Domain\Enum\QuestionForm;
use Netresearch\NrLlm\Service\Evaluation\GoldenQuestion;
use Netresearch\NrLlm\Service\Evaluation\GoldenQuestionSet;
use Netresearch\NrLlm\Service\Evaluation\GoldenQuestionSetProviderInterface;

final class BmdvGoldenQuestionSetProvider implements GoldenQuestionSetProviderInterface
{
    public function getGoldenQuestionSets(): array
    {
        return [
            new GoldenQuestionSet(
                identifier: 'nr_ai_search.bmdv',
                name: 'BMDV retrieval eval',
                description: 'Labeled questions over the BMDV corpus.',
                questions: [
                    new GoldenQuestion(
                        id: 'dialogforum-termin',
                        question: 'Wann findet das Dialogforum statt?',
                        form: QuestionForm::MATCH,
                        expectedDocumentIds: ['234_0', '309_0'],
                        hardClass: 'near-duplicate',
                    ),
                ],
            ),
        ];
    }
}

The retriever under test implements EvaluatableRetrieverInterface (tag nr_llm.evaluatable_retriever): a question string and a limit in, ranked document ids out. The adapter owns the mapping from its native results to document ids, which must use the same identity scheme as the set's labels. nr_llm ships LexicalSearchRetriever (nr_llm.lexical) over its own search cascade as the pattern to copy; a consumer wraps its vector retrieval the same way.

Run with the nrllm:eval:retrieval command:

# Measure the built-in lexical cascade against a labeled set
vendor/bin/typo3 nrllm:eval:retrieval nr_ai_search.bmdv nr_llm.lexical

# Fail (non-zero exit) if hit rates regressed — for CI
vendor/bin/typo3 nrllm:eval:retrieval nr_ai_search.bmdv nr_ai_search.vector \
    --fail-on-regression --max-top1-drop 0.05 --max-top3-drop 0.05

The command prints the per-question hits, the top-1/top-3 hit rates with by-form and by-hard-class breakdowns, stores the run in tx_nrllm_eval_result (grader retrieval_hit_rate; the stored pass rate is the top-1 hit rate and the stored mean score the top-3 hit rate), and reports whether the run regressed against the previous one for the same set and retriever.

Why build on nr-llm? 

When your extension calls an LLM API directly, it takes on responsibility for:

• HTTP client setup, authentication, and error handling per provider
• Secure API key storage (not in ext_conf_template.txt or $GLOBALS)
• Response caching to control costs
• Streaming implementation for real-time UX
• A configuration UI for administrators

nr-llm handles all of this. Your extension focuses on what to ask the AI, not how to reach it.

Step 1: Add the dependency 

composer require netresearch/nr-llm

Add the dependency to your ext_emconf.php:

'constraints' => [
    'depends' => [
        'typo3' => '13.4.0-14.99.99',
        'nr_llm' => '0.4.0-0.99.99',
    ],
],

Step 2: Inject the service 

All nr-llm services are available via TYPO3's dependency injection. Pick the service that matches your use case:

<?php

declare(strict_types=1);

namespace MyVendor\MyExtension\Service;

use Netresearch\NrLlm\Service\LlmServiceManagerInterface;

final readonly class MyAiService
{
    public function __construct(
        private LlmServiceManagerInterface $llm,
    ) {}

    public function summarize(string $text): string
    {
        $response = $this->llm->complete(
            "Summarize the following text in 2-3 sentences:\n\n" . $text,
        );

        return $response->content;
    }
}

No Services.yaml configuration needed — TYPO3's autowiring handles it.

Step 3: Use feature services for specialized tasks 

For common AI tasks, use the specialized feature services instead of raw chat:

use Netresearch\NrLlm\Service\Feature\TranslationService;

final readonly class ContentTranslator
{
    public function __construct(
        private TranslationService $translator,
    ) {}

    public function translateToGerman(string $text): string
    {
        $result = $this->translator->translate($text, 'de');
        return $result->translation;
    }
}

use Netresearch\NrLlm\Service\Feature\VisionService;

final readonly class ImageMetadataGenerator
{
    public function __construct(
        private VisionService $vision,
    ) {}

    public function generateAltText(string $imageUrl): string
    {
        return $this->vision->generateAltText($imageUrl);
    }
}

use Netresearch\NrLlm\Service\Feature\EmbeddingService;

final readonly class ContentRecommender
{
    public function __construct(
        private EmbeddingService $embeddings,
    ) {}

    /**
     * @param list<array{id: int, text: string, vector: list<float>}> $candidates
     * @return list<int> Top 5 most similar content IDs
     */
    public function findSimilar(string $query, array $candidates): array
    {
        $queryVector = $this->embeddings->embed($query);
        $results = $this->embeddings->findMostSimilar(
            $queryVector,
            array_column($candidates, 'vector'),
            topK: 5,
        );

        return array_map(
            fn(int $index) => $candidates[$index]['id'],
            array_keys($results),
        );
    }
}

Step 4: Handle errors gracefully 

nr-llm throws typed exceptions so you can provide meaningful feedback:

use Netresearch\NrLlm\Provider\Exception\ProviderConfigurationException;
use Netresearch\NrLlm\Provider\Exception\ProviderConnectionException;
use Netresearch\NrLlm\Provider\Exception\ProviderResponseException;

try {
    $response = $this->llm->complete($prompt);
} catch (ProviderConfigurationException) {
    // No provider configured — guide the admin
    return 'AI features require LLM configuration. '
         . 'An administrator can set this up in Admin Tools > LLM.';
} catch (ProviderConnectionException) {
    // Network issue — suggest retry
    return 'Could not reach the AI provider. Please try again.';
} catch (ProviderResponseException $e) {
    // Provider returned an error (rate limit, invalid input, etc.)
    $this->logger->warning('LLM provider error', ['exception' => $e]);
    return 'The AI service returned an error. Please try again later.';
}

Step 5: Use database configurations (optional) 

For advanced use cases, reference named configurations that admins create in the backend module. Resolve them through NetresearchNrLlmServiceConfigurationResolver::getActiveByIdentifier() — it works in user-less contexts (CLI, Symfony Messenger consumers, anonymous frontend requests) and applies the guards a raw repository lookup skips (ADR-070):

use Netresearch\NrLlm\Service\ConfigurationResolver;
use Netresearch\NrLlm\Service\LlmServiceManagerInterface;

final readonly class BlogSummarizer
{
    public function __construct(
        private ConfigurationResolver $configurationResolver,
        private LlmServiceManagerInterface $llm,
    ) {}

    public function summarize(string $article): string
    {
        // Uses the "blog-summarizer" configuration created by the admin
        // (specific model, temperature, system prompt, etc.)
        $config = $this->configurationResolver->getActiveByIdentifier('blog-summarizer');

        $response = $this->llm->chat(
            [['role' => 'user', 'content' => "Summarize:\n\n" . $article]],
            $config->toChatOptions(),
        );

        return $response->content;
    }
}

The method throws typed exceptions (all implementing NrLlmExceptionInterface): ConfigurationNotFoundException when no record with the identifier exists, ConfigurationInactiveException when it exists but is deactivated, and AccessDeniedException when it is restricted to backend groups (user-less callers cannot prove group membership; see ADR-070).

Step 6: Declare the configurations you need (optional) 

Instead of documenting "please create a configuration named myext.chat" in prose, declare it as a configuration preset: implement ConfigurationPresetProviderInterface (auto-tagged nr_llm.configuration_preset) and express your requirements as ModelSelectionCriteria — never a concrete provider, model, or API key. nr_llm lists your presets as pending and a backend admin imports them with one confirmation as criteria-mode configuration records.

See Configuration presets for the declaration example and import flow.

Testing your integration 

Mock the nr-llm interfaces in your unit tests:

use Netresearch\NrLlm\Domain\Model\CompletionResponse;
use Netresearch\NrLlm\Domain\Model\UsageStatistics;
use Netresearch\NrLlm\Service\LlmServiceManagerInterface;
use PHPUnit\Framework\TestCase;

final class MyAiServiceTest extends TestCase
{
    public function testSummarizeReturnsCompletionContent(): void
    {
        $llm = $this->createStub(LlmServiceManagerInterface::class);
        $llm->method('complete')->willReturn(
            new CompletionResponse(
                content: 'A short summary.',
                model: 'gpt-5.3-instant',
                usage: new UsageStatistics(50, 20, 70),
                finishReason: 'stop',
                provider: 'openai',
            ),
        );

        $service = new MyAiService($llm);
        self::assertSame('A short summary.', $service->summarize('Long text...'));
    }
}

Integration checklist 

1. composer.json — Added netresearch/nr-llm to require
2. ext_emconf.php — Added nr_llm to depends constraints
3. Services — Inject LlmServiceManagerInterface or feature services via DI
4. Error handling — Catch typed exceptions and show user-friendly messages
5. Testing — Mock LlmServiceManagerInterface in unit tests
6. Documentation — Tell your users they need to configure a provider in Admin Tools > LLM

Per-IP rate limiting 

TYPO3 core already depends on symfony/rate-limiter (core's own login rate limiting uses it) and ships a storage adapter that persists limiter state in the caching framework: TYPO3CMSCoreRateLimiterStorageCachingFrameworkStorage. The whole recipe is one service alias plus a  25-line class.

Require the component explicitly, since your code uses its classes directly:

composer require symfony/rate-limiter

Alias the Symfony storage interface to TYPO3's caching-framework-backed implementation:

services:
    _defaults:
        autowire: true
        autoconfigure: true
        public: false

    Symfony\Component\RateLimiter\Storage\StorageInterface:
        alias: TYPO3\CMS\Core\RateLimiter\Storage\CachingFrameworkStorage

The limiter itself keys on the client IP resolved through the normalizedParams request attribute — a NormalizedParams instance TYPO3 sets on every frontend and backend request — which honors TYPO3's reverse-proxy configuration ($GLOBALS['TYPO3_CONF_VARS']['SYS']['reverseProxyIP']) instead of trusting REMOTE_ADDR blindly. The raw REMOTE_ADDR fallback only covers requests that never passed through TYPO3's request handling, such as unit tests:

<?php

declare(strict_types=1);

namespace MyVendor\MyExtension\Infrastructure;

use Psr\Http\Message\ServerRequestInterface;
use Symfony\Component\RateLimiter\RateLimiterFactory;
use Symfony\Component\RateLimiter\Storage\StorageInterface;
use TYPO3\CMS\Core\Http\NormalizedParams;

final readonly class RequestRateLimiter
{
    private RateLimiterFactory $rateLimiterFactory;

    public function __construct(
        StorageInterface $storage,
        int $limitPerMinute = 30,
        string $limiterId = 'myext_frontend',
    ) {
        $this->rateLimiterFactory = new RateLimiterFactory(
            [
                'id' => $limiterId,
                'policy' => 'sliding_window',
                'limit' => $limitPerMinute,
                'interval' => '1 minute',
            ],
            $storage,
        );
    }

    public function tooManyRequests(ServerRequestInterface $request): bool
    {
        $normalizedParams = $request->getAttribute('normalizedParams');

        $remoteIp = $normalizedParams instanceof NormalizedParams
            ? $normalizedParams->getRemoteAddress()
            : (string)($request->getServerParams()['REMOTE_ADDR'] ?? '');

        return !$this->rateLimiterFactory
            ->create($remoteIp)
            ->consume()
            ->isAccepted();
    }
}