Message bus 

TYPO3 provides a message bus solution based on symfony/messenger. It has the ability to send messages and then handle them immediately (synchronous) or send them through transports (asynchronous, for example, queues) to be handled later.

For backwards compatibility, the default implementation uses the synchronous transport. This means that the message bus will behave exactly as before, but it will be possible to switch to a different (asynchronous) transport on a per-project base.

To offer asynchronicity, TYPO3 also provides a transport implementation based on the Doctrine DBAL messenger transport from Symfony and a basic implementation of a consumer command.

"Everyday" usage - as a developer 

Dispatch a message 

  1. Add a PHP class for your message object (which is an arbitrary PHP class)

    EXT:my_extension/Classes/Queue/Message/DemoMessage.php
    <?php
    
    declare(strict_types=1);
    
    namespace MyVendor\MyExtension\Queue\Message;
    
    final class DemoMessage
    {
      public function __construct(
        public readonly string $content,
      ) {}
    }
    
    Copied!
  2. Inject the MessageBusInterface into your class and call the dispatch() method

    EXT:my_extension/Classes/MyClass.php
    <?php
    
    declare(strict_types=1);
    
    namespace MyVendor\MyExtension;
    
    use MyVendor\MyExtension\Queue\Message\DemoMessage;
    use Symfony\Component\Messenger\MessageBusInterface;
    
    final class MyClass
    {
      public function __construct(
        private readonly MessageBusInterface $bus,
      ) {}
    
      public function doSomething(): void
      {
        // ...
        $this->bus->dispatch(new DemoMessage('test'));
        // ...
      }
    }
    
    Copied!

Register a handler 

Implement the handler class

EXT:my_extension/Classes/Queue/Handler/DemoHandler.php
<?php

declare(strict_types=1);

namespace MyVendor\MyExtension\Queue\Handler;

use MyVendor\MyExtension\Queue\Message\DemoMessage;
use Symfony\Component\Messenger\Attribute\AsMessageHandler;

#[AsMessageHandler]
final class DemoHandler
{
  public function __invoke(DemoMessage $message): void
  {
    // do something with $message
  }
}
Copied!

The message handler can be registered using the symfony PHP attribute \Symfony\Component\Messenger\Attribute\AsMessageHandler .

A Services.yaml entry is needed to use before / after for registration if you need to define an order:

EXT:my_extension/Configuration/Services.yaml
MyVendor\MyExtension\Queue\Handler\DemoHandler:
  tags:
    - name: 'messenger.message_handler'

# Define another handler which should be called before DemoHandler:
MyVendor\MyExtension\Queue\Handler\DemoHandler2:
  tags:
    - name: 'messenger.message_handler'
      before: 'MyVendor\MyExtension\Queue\Handler\DemoHandler'
Copied!

Bind a handler to a receiving transport with fromTransport 

Changed in version 14.3 Important: #107032 - AsMessageHandler "fromTransport" property is now honoured

Before, TYPO3 ignored fromTransport, and the handler ran for every message. Remove fromTransport from a handler that relied on this.

The fromTransport property of AsMessageHandler binds a handler to a receiving transport. The handler then runs only for messages that a worker received from this transport, for example with vendor/bin/typo3 messenger:consume doctrine:

EXT:my_extension/Classes/Queue/Handler/AsyncDemoHandler.php
<?php

declare(strict_types=1);

namespace MyVendor\MyExtension\Queue\Handler;

use MyVendor\MyExtension\Queue\Message\DemoMessage;
use Symfony\Component\Messenger\Attribute\AsMessageHandler;

// Runs only for messages that a worker received from "doctrine"
#[AsMessageHandler(fromTransport: 'async')]
final class AsyncDemoHandler
{
  public function __invoke(DemoMessage $message): void
  {
    // do something with $message
  }
}
Copied!

A message that is dispatched and handled synchronously still reaches the handler. A handler without fromTransport runs for every message, whatever the transport.

To bind a handler to several transports, repeat the attribute once for each transport. In the Services.yaml tag, fromTransport also takes a list of transport names:

EXT:my_extension/Configuration/Services.yaml
MyVendor\MyExtension\Queue\Handler\AsyncDemoHandler:
  tags:
    - name: 'messenger.message_handler'
      fromTransport: ['async', 'priority']
Copied!

"Everyday" usage - as a system administrator/integrator 

By default, TYPO3 will behave like in versions before TYPO3 v12. This means that the message bus will use the synchronous transport and all messages will be handled immediately. To benefit from the message bus, it is recommended to switch to an asynchronous transport. Using asynchronous transports increases the resilience of the system by decoupling external dependencies even further.

Currently, the TYPO3 Core provides an asynchronous transport based on the Doctrine DBAL messenger transport. This transport is configured to use the default TYPO3 database connection. It is pre-configured and can be used by changing the settings:

config/settings.php | config.additional.php
$GLOBALS['TYPO3_CONF_VARS']['SYS']['messenger']['routing']['*'] = 'doctrine';
Copied!

This will route all messages to the asynchronous transport (mind the * ).

Async message handling - the consume command 

To consume messages, run the command:

vendor/bin/typo3 messenger:consume <receiver-name>
Copied!
typo3/sysext/core/bin/typo3 messenger:consume <receiver-name>
Copied!

By default, you should run:

vendor/bin/typo3 messenger:consume doctrine
Copied!
typo3/sysext/core/bin/typo3 messenger:consume doctrine
Copied!

The command takes almost all options of the Symfony command of the same name.

Changed in version 14.2 Feature: #106261 - Align command line arguments of message consumer with Symfony original

The worker options below were added. Before, the command stopped after one hour on its own and reported that through an --exit-code-on-limit option. Both are gone: a worker now runs until you give it a limit.

A worker does not stop on its own. In continuous operation, configure a limit so that the PHP process is periodically recycled to release memory:

--time-limit, -t
Stop after this number of seconds.
--limit, -l
Stop after this number of messages.
--failure-limit, -f
Stop after this number of failed messages.
--memory-limit, -m
Stop when the worker uses more than this much memory, for example 128M.

A limit only takes effect after the current message is handled, so the worker never stops in the middle of one. These options select what the worker reads and how it waits:

--all
Consume from every configured receiver.
--bus, -b
Dispatch the received messages to this bus. Without it, TYPO3 picks the bus itself.
--queues
Consume only from these queues of the receiver. Repeat the option per queue.
--sleep
Wait this number of seconds before asking for messages again when none were found. The default is 1.
--keepalive
Keep the connection of the transport alive, if the transport implements it. The default interval is 5 seconds.

Whichever limit is reached, the worker process exits with code 0. Run it under a service manager like systemd that keeps the service active and automatically restarts a fresh worker process.

The following code provides an example for a service. Create the following file on your server:

/etc/systemd/system/typo3-message-consumer.service
[Unit]
Description=Run the TYPO3 message consumer
Requires=mariadb.service
After=mariadb.service

[Service]
Type=simple
User=www-data
Group=www-data
ExecStart=/usr/bin/php8.5 /var/www/myproject/vendor/bin/typo3 messenger:consume doctrine --time-limit=3600
# The command exits with 0 on its time limit, so restart it either way
Restart=always
RestartSec=1

[Install]
WantedBy=multi-user.target
Copied!

Where no service manager is available, the command can run as a scheduler task instead.

Database connections of a worker 

A worker keeps its database connections across messages. The database server can drop a connection that is idle for too long, for example after the MySQL wait_timeout. The next message then fails with "MySQL server has gone away".

To prevent this, two middlewares maintain the connections that are open in the ConnectionPool :

DoctrinePingConnectionMiddleware
Before a message is handled, pings each open connection and reconnects a stale connection.
DoctrineCloseConnectionMiddleware
After a message is handled, closes the open connections, also if the handler throws an exception.

The middlewares are active by default. They only act on messages that a worker consumes. A synchronous dispatch does not ping or close connections. SQLite connections are never pinged or closed.

To exclude a connection, use the events BeforeMessengerDoctrinePingEvent and BeforeMessengerDoctrineCloseEvent.

Advanced usage 

Configure a custom transport (senders/receivers) 

Transports are configured in the services configuration. To allow the configuration of a transport per message, the TYPO3 configuration (settings.php, additional.php on system level, or ext_localconf.php in an extension) is utilized. The transport/sender name used in the settings is resolved to a service that has been tagged with message.sender and the respective identifier.

config/settings.php | config/additional.php | EXT:my_extension/ext_localconf.php
<?php

$GLOBALS['TYPO3_CONF_VARS']['SYS']['messenger'] = [
  'routing' => [
    // Use "messenger.transport.demo" as transport for DemoMessage
    \MyVendor\MyExtension\Queue\Message\DemoMessage::class => 'demo',

    // Use "messenger.transport.default" as transport for all other messages
    '*' => 'default',
  ],
];
Copied!
EXT:my_extension/Configuration/Services.yaml | config/system/services.yaml
messenger.transport.demo:
  factory: [ '@TYPO3\CMS\Core\Messenger\DoctrineTransportFactory', 'createTransport' ]
  class: 'Symfony\Component\Messenger\Bridge\Doctrine\Transport\DoctrineTransport'
  arguments:
    $options:
      queue_name: 'demo'
  tags:
    - name: 'messenger.sender'
      identifier: 'demo'
    - name: 'messenger.receiver'
      identifier: 'demo'

messenger.transport.default:
  factory: [ '@Symfony\Component\Messenger\Transport\InMemory\InMemoryTransportFactory', 'createTransport' ]
  class: 'Symfony\Component\Messenger\Transport\InMemory\InMemoryTransport'
  arguments:
    $dsn: 'in-memory://default'
    $options: [ ]
  tags:
    - name: 'messenger.sender'
      identifier: 'default'
    - name: 'messenger.receiver'
      identifier: 'default'
Copied!

The TYPO3 Core has been tested with three transports:

  • \Symfony\Component\Messenger\Transport\Sync\SyncTransport (default)
  • \Symfony\Component\Messenger\Bridge\Doctrine\Transport\DoctrineTransport (using the Doctrine DBAL messenger transport)
  • \Symfony\Component\Messenger\Transport\InMemory\InMemoryTransport (for testing)

Add rate limiter 

Rate limiting can be applied to asynchronous messages processed through the consume command. This allows controlling message processing rates to:

  • Stay within external service limits (API quotas, mail sending thresholds)
  • Manage server resource utilization

Example: usage of a rate limiter 

Use the following configuration to limit the process of messages to max. 100 each 60 seconds:

EXT:my_extension/Configuration/Services.yaml | config/system/services.yaml
messenger.rate_limiter.limit_db_messages:
  class: Symfony\Component\RateLimiter\RateLimiterFactory
  arguments:
    $config:
      id: 'limit_db_messages'
      policy: 'sliding_window'
      limit: 100
      interval: '60 seconds'
    $storage: '@TYPO3\CMS\Core\RateLimiter\Storage\CachingFrameworkStorage'
  tags:
    - name: 'messenger.rate_limiter'
      identifier: 'doctrine'
Copied!

InMemoryTransport for testing 

The InMemoryTransport is a transport that should only be used while testing.

EXT:my_extension/Configuration/Services.yaml | config/system/services.yaml
messenger.transport.default:
  factory: [ '@Symfony\Component\Messenger\Transport\InMemory\InMemoryTransportFactory', 'createTransport' ]
  class: 'Symfony\Component\Messenger\Transport\InMemory\InMemoryTransport'
  public: true
  arguments:
    $dsn: 'in-memory://default'
    $options: [ ]
  tags:
    - name: 'messenger.sender'
      identifier: 'default'
    - name: 'messenger.receiver'
      identifier: 'default'
Copied!

Configure a custom middleware 

The middleware is set up in the services configuration. By default, these middlewares are registered, in this order:

  1. SendMessageMiddleware
  2. DoctrineCloseConnectionMiddleware
  3. DoctrinePingConnectionMiddleware
  4. HandleMessageMiddleware

See also the Custom middleware section in the Symfony documentation.

To add your own middleware, tag it as messenger.middleware and set the order using TYPO3's before and after ordering mechanism:

EXT:my_extension/Configuration/Services.yaml | config/system/services.yaml
Symfony\Component\Messenger\Middleware\SendMessageMiddleware:
  arguments:
    $sendersLocator: '@Symfony\Component\Messenger\Transport\Sender\SendersLocatorInterface'
    $eventDispatcher: '@Psr\EventDispatcher\EventDispatcherInterface'
  tags:
    - { name: 'messenger.middleware' }

Symfony\Component\Messenger\Middleware\HandleMessageMiddleware:
  arguments:
    $handlersLocator: '@Symfony\Component\Messenger\Handler\HandlersLocatorInterface'
  tags:
    - name: 'messenger.middleware'
      after: 'Symfony\Component\Messenger\Middleware\SendMessageMiddleware'
Copied!