ADR-001: Asynchronous Generation via Symfony Messenger
- Status
-
Accepted
- Date
-
2026-06-09
- Authors
-
Netresearch DTT GmbH
Context
A single generation run is long and I/O-bound: it fetches and analyzes a source (one or more nr-llm completion calls, possibly map-reduced), then synthesizes a multi-turn podcast (one TTS HTTP call per turn), generates several AI images, and drives headless Chromium and ffmpeg to render and stitch the results. End to end this is far longer than an acceptable backend HTTP request, and it must not block the editor who submitted the job.
The work also needs to be resumable and observable: an editor submits a job and then watches its progress in the list view, so the job's state has to live in the database, not in a request's memory.
Decision
Split submission from execution over a Symfony Messenger transport.
- The job row is the source of truth. The backend
createaction (viaJob) persists theSubmission Service Job, flushes to obtain its uid, and dispatches aGeneratethat carries only the uid. The worker re-reads every input from the database, so the message is immutable and minimal and there is no risk of a stale payload.Artifacts Message - A long-lived worker executes the pipeline. The message is routed to an
asynchronous transport (the doctrine transport in the dev setup); a
messenger:worker runsconsume Generation. The orchestrator updates the job'sOrchestrator:: process () status/progress/current_as it advances through the statusesstep queued → ingesting → analyzing → generating → done. The list view renders|partially_ done |failed statusandprogress; the detail view addscurrent_.step - The handler records failures instead of rethrowing. TYPO3 v14.3 Core
ships no retry/failure transport. So
Generatewraps the orchestrator in aArtifacts Handler try/: on a throwable it logs and callscatch Job, and deliberately does not rethrow. Rethrowing would let Messenger drop the message with no record, leaving the job stuck in a non-terminal state forever.Processing Repository:: mark Failed () - Processing is idempotent. The orchestrator returns early if the job is missing or already in a terminal status, so redelivery (or a manual re-dispatch) never reprocesses or duplicates artifacts.
The same orchestrator is reachable synchronously via the
nr_ CLI command, so ops and tests can run the exact
pipeline without a consumer.
Consequences
- Submission returns immediately; the editor watches progress in the list view while the worker does the work. Generation time is decoupled from the request lifecycle.
- Operating the extension requires running and supervising a worker process (restart loop, time/memory limits) on a host that has the rendering binaries — an explicit deployment requirement (see Run the generation worker).
- Because the handler never rethrows, Messenger's own retry/DLQ mechanisms are not exercised; failure handling lives entirely in the job row. This is a deliberate trade-off for v14.3 Core, which has no failure transport. If a later Core version adds one, the handler can be revisited.
- A crashed run always leaves a
failedjob with an error message rather than a silently lost message.