Feature: #110227 - Database connection maintenance for long-running Messenger workers 

See forge#110227

Description 

Long-running Symfony Messenger workers started via typo3 messenger:consume <transport> keep a PHP process - and its database connections - alive across many messages. A connection that stays idle between two messages can be dropped by the database server (for example MySQL's wait_timeout ), so the next message handled by the worker fails with errors such as "MySQL server has gone away".

Two messenger middlewares now maintain the database connections of a worker:

  • \TYPO3\CMS\Core\Messenger\Middleware\DoctrinePingConnectionMiddleware pings each open connection with a lightweight dummy select before a message is handled and transparently closes and reconnects a connection that turned out to be stale.
  • \TYPO3\CMS\Core\Messenger\Middleware\DoctrineCloseConnectionMiddleware closes the open connections after a message has been handled - even when handling throws - so the next message starts with a fresh connection. It also flushes the runtime cache, which is meant to live for one request: a worker treats each message like a request, so nothing bound to a closed connection (such as a cached prepared statement) or stale from the previous message is kept.

The middlewares are TYPO3-native and operate on \TYPO3\CMS\Core\Database\ConnectionPool , so - unlike the equivalents in symfony/doctrine-bridge - they do not require Doctrine ORM or a ManagerRegistry .

Both middlewares are registered and active by default. No configuration is needed.

Only asynchronous handling is affected 

The middlewares act exclusively on messages consumed asynchronously by a worker, that is on envelopes carrying a \Symfony\Component\Messenger\Stamp\ConsumedByWorkerStamp . A synchronous dispatch - including the SyncTransport - is passed through untouched, so regular request handling never pings or closes database connections.

To avoid opening connections needlessly, only connections that are already open in the pool are pinged or closed; configured but unused connections are left alone.

Embedded SQLite connections are never pinged or closed: they are not server-based and cannot go stale, and closing an in-memory SQLite database would discard all its data.

Excluding individual connections 

Both middlewares dispatch a cancellable PSR-14 event per connection, allowing listeners to skip the ping or the close for a specific connection - for example to keep a dedicated connection open across messages:

  • \TYPO3\CMS\Core\Messenger\Event\BeforeMessengerDoctrinePingEvent
  • \TYPO3\CMS\Core\Messenger\Event\BeforeMessengerDoctrineCloseEvent

Each event exposes the affected getConnectionName() and a skip() method (with shouldSkip() ), and stops propagation once skipped:

use TYPO3\CMS\Core\Attribute\AsEventListener;
use TYPO3\CMS\Core\Messenger\Event\BeforeMessengerDoctrineCloseEvent;

final class KeepReportingConnectionOpen
{
    #[AsEventListener]
    public function __invoke(BeforeMessengerDoctrineCloseEvent $event): void
    {
        if ($event->getConnectionName() === 'Reporting') {
            // Keep the "Reporting" connection open across messages.
            $event->skip();
        }
    }
}
Copied!

Impact 

Database connections used by a messenger:consume worker are now kept alive (reconnected when stale) and released between messages automatically, removing a common cause of "server has gone away" failures in long-running workers. Projects that must keep a specific connection open across messages can opt out per connection via the two events.

Providing this feature on older TYPO3 versions 

The two middlewares and their events are self-contained and can be provided on an older TYPO3 version that does not ship them yet. There is one caveat: the middlewares rely on the new method \TYPO3\CMS\Core\Database\ConnectionPool::getOpenConnectionNames() , which is not available on older versions and - being a public API method on a core class - cannot be added from an extension. It has to be either patched into the core (Composer patch) or emulated in the copied middlewares via reflection.

Copy the middlewares into a local extension 

Copy the four classes into an extension, keeping their logic:

  • \TYPO3\CMS\Core\Messenger\Middleware\DoctrinePingConnectionMiddleware
  • \TYPO3\CMS\Core\Messenger\Middleware\DoctrineCloseConnectionMiddleware
  • \TYPO3\CMS\Core\Messenger\Event\BeforeMessengerDoctrinePingEvent
  • \TYPO3\CMS\Core\Messenger\Event\BeforeMessengerDoctrineCloseEvent

Register the two copied middlewares in the extension's Configuration/Services.yaml with the same ordering (close outermost, ping innermost), referencing the extension classes and the always-present Symfony core middlewares:

services:
  MyVendor\MyExtension\Messenger\Middleware\DoctrineCloseConnectionMiddleware:
    tags:
      - name: 'messenger.middleware'
        after: 'Symfony\Component\Messenger\Middleware\SendMessageMiddleware'
        before: 'MyVendor\MyExtension\Messenger\Middleware\DoctrinePingConnectionMiddleware'

  MyVendor\MyExtension\Messenger\Middleware\DoctrinePingConnectionMiddleware:
    tags:
      - name: 'messenger.middleware'
        after: 'MyVendor\MyExtension\Messenger\Middleware\DoctrineCloseConnectionMiddleware'
        before: 'Symfony\Component\Messenger\Middleware\HandleMessageMiddleware'
Copied!

In the copied middlewares, replace the call to $this->connectionPool->getOpenConnectionNames() with a private helper that reads the protected $connections property of the ConnectionPool via reflection (no setAccessible() call is needed on PHP 8.1 and newer):

use TYPO3\CMS\Core\Database\Connection;
use TYPO3\CMS\Core\Database\ConnectionPool;

/**
 * @return list<string>
 */
private function getOpenConnectionNames(): array
{
    $property = new \ReflectionProperty(ConnectionPool::class, 'connections');
    /** @var array<string, Connection> $connections */
    $connections = $property->getValue($this->connectionPool);

    return array_values(array_filter(
        array_keys($connections),
        static fn(string $connectionName): bool => $connectionName !== '',
    ));
}
Copied!

Because the middlewares only ever act on connections already open in the pool (and skip embedded SQLite connections), reading the currently instantiated connections is exactly what the core method returns.

Backport the change via a Composer patch 

Alternatively, apply the change to typo3/cms-core directly using a Composer patch tool such as cweagans/composer-patches:

{
    "require": {
        "cweagans/composer-patches": "^1.7"
    },
    "extra": {
        "patches": {
            "typo3/cms-core": {
                "Messenger Doctrine connection maintenance": "patches/messenger-doctrine-maintenance.patch"
            }
        }
    }
}
Copied!

The patch file should contain the new ConnectionPool::getOpenConnectionNames() method, the two middlewares and the two events, and register the middlewares in typo3/sysext/core/Configuration/Services.yaml. When the method is added this way, the reflection helper above is not needed and the copied middlewares can call getOpenConnectionNames() directly.