Queues and background answers
Running queued runs asynchronously
A consumer of the API that calls enqueue gets an in-process,
synchronous run by default, with no setup. For genuinely asynchronous execution,
route the queue message to the Doctrine transport and run a consumer, or leave
the routing alone and queue with defer, which
Background tasks from the command line and the scheduler describes:
// settings.php / additional.php
$GLOBALS['TYPO3_CONF_VARS']['SYS']['messenger']['routing']
[\Netresearch\NrLlm\Service\Agent\Queue\AgentRunQueuedMessage::class] = 'doctrine';
vendor/bin/typo3 messenger:consume doctrine
Warning
Once you route the message to doctrine you must run a consumer or
nrllm:. Without one, queued runs are never picked up, and the
stale-run reaper below has no worker to hand reclaimed runs back to.
Background tasks from the command line and the scheduler
An AI Task can run without anyone waiting for it. Two commands do that (ADR-234). Both appear in the scheduler under Execute console commands.
vendor/bin/typo3 nrllm:task:enqueue release-notes --input="the text to work on"
vendor/bin/typo3 nrllm:agent:work --limit=20 --time-limit=240
nrllm: stores one queued run and returns; no
provider is called. A task with manual input takes it from --. A task
that reads a source, such as the system log, reads it at this moment. To repeat
a task, schedule this command with the task's identifier as its argument.
nrllm: executes queued runs, oldest first. -- (default
20) and -- (seconds, default 240) bound one tick; a run that
has started finishes. Set the scheduler frequency above the time limit. Runs left
over wait for the next tick. A run that fails is listed as failed and the command
still succeeds; the command fails only when the queue cannot be read. A run that
fails for a temporary reason, such as a rate limit, returns to the queue and is
tried again at the next tick, three times at most.
On the command line no backend user is logged in, so nr_vault refuses the provider
key unless allow is on. Set worker.technicalBeUserUid to a backend user with read access to
the secret. Without either, every run fails on its first model call.
Before it claims a run, the command asks the budget. A run the user's budget or
the configuration's daily limit would refuse stays queued, and the runs behind it
continue, so a limit that resets at midnight delays work without losing it. A
Messenger consumer does not ask: routed to doctrine, a run it picks up meets
the refusal and ends there.
The worker and a Messenger consumer can run side by side: a run is claimed
atomically, so exactly one of them executes it. With the default routing no
consumer is involved. Routed to doctrine, a queued task also wakes the
consumer.
The run acts as a service account that may use the configuration
(ADR-110), not as the _cli_ user, and no per-user budget
applies to it; the configuration's own limits do. A task calls no tool, so it
writes nothing. Its configuration is the task's own or the active default. Its
usage appears under the task in the usage figures.
Background runs in the Agent Runs module lists these runs in every status, queued ones included, with what queued them. A run a service account queued has no owner: an administrator sees and opens it, a holder of the Approve suspended AI runs grant sees it without its detail page, and an editor sees neither.
An installation that runs only nrllm: has no consumer heartbeat.
Pass -- to nrllm:, and use
-- to learn that no tick is running.
Its detail page shows the answer. A background run keeps its answer in
tx_, encrypted like a queued request, until the purge
removes the row. Unlike the step log, this does not follow the default
privacy., because the answer is the result nobody watched
arrive: none keeps nothing and redacted keeps the redacted text. See
Data retention & purge. A run that went through an approval keeps no
answer, because what it did is the write.