.. include:: /Includes.rst.txt .. _send-queue: ========== Send queue ========== Since version 4.3.0 a mail that could not be delivered to a recipient is not lost anymore. The recipient is put into a retry queue and the delivery is tried again later. Recipients that cannot be reached at all are marked as failed, so that you can see them in the backend, correct them and send again. .. contents:: :local: :depth: 2 .. _send-queue-how-it-works: How it works ============ When the sending command (:ref:`mail:mass-mailing `) delivers a mail to a recipient and the mail server refuses it or cannot be reached, the error is evaluated: * Temporary errors (SMTP reply code 4xx and errors without a code, for example a refused connection) are retried with increasing waiting times. * Permanent errors (SMTP reply code 5xx, for example a mailbox that does not exist) are not retried. The recipient is marked as failed right away. Recipients that were delivered at the first attempt are not touched. Only failed recipients are written to the database table :sql:`tx_mail_domain_model_queue`. .. _send-queue-retry-schedule: Retry schedule -------------- After a failed attempt the next attempt is scheduled with these waiting times: ============================= ================== Failed attempt Waiting time ============================= ================== 1st 1 minute 2nd 5 minutes 3rd 15 minutes 4th 1 hour 5th and every further attempt 4 hours ============================= ================== After :ref:`maxAttempts ` attempts (including the first one) a recipient is marked as failed for good. With the default of 20 attempts, a recipient is tried for about 61 hours (two and a half days). The retries are processed by the command :ref:`mail:retry-failed `. The command :ref:`mail:mass-mailing ` does this as well after each cycle, so the scheduler task for the mass mailing is enough for most installations. .. _send-queue-states: States of a queue entry ----------------------- Waiting for retry The delivery failed temporarily. The next attempt is shown in the backend. Sending The entry is claimed by a running command. If a run is interrupted, entries in this state are put back into the queue after 15 minutes. Sent The delivery succeeded at a later attempt. Failed The delivery failed permanently, or the maximum number of attempts is reached. Failed entries are not retried automatically. .. _send-queue-backend: Failed deliveries in the backend ================================ The module :guilabel:`Mail > Queue` shows the section :guilabel:`Deliveries to be retried or failed` as soon as at least one recipient is waiting for a retry or has failed. Failed entries are red, waiting entries yellow. .. include:: /Images/MailQueueFailedDeliveries.rst.txt For every entry you see the mail, the recipient, the status, the number of attempts, the time of the next attempt and the last error with its SMTP code. The status is shown as an icon: a red exclamation mark for a failed entry and a clock for an entry that waits for a retry. The text of the status appears as a tooltip. The complete error message is shown as a tooltip when you hover over the column :guilabel:`Last error`. You can * press :guilabel:`Send again` to try the delivery again. The entry is put back into the queue and delivered with the next run of the sending command. * press the delete button (trash icon) to drop the entry. You have to confirm this in a dialog, which names the recipient. The recipient does not receive the mail. * correct the email address before you send again, if you enabled this function, see :ref:`below `. .. include:: /Images/MailQueueDeleteConfirmation.rst.txt .. _send-queue-email-correction: Correcting the email address ---------------------------- If a delivery failed because the address is wrong, for example because of a typo, the address can be corrected in the queue module before the mail is sent again. This function is disabled by default. Enable the checkbox :ref:`enableEmailCorrection ` in the extension configuration of mail (tab :guilabel:`feature`) to use it. .. include:: /Images/ExtensionConfigurationFeature.rst.txt As soon as it is enabled, the column :guilabel:`Recipient` is an input field. Correct the address there and press :guilabel:`Send again`. The entry is put back into the queue with the corrected address and delivered with the next run of the sending command. .. include:: /Images/MailQueueFailedDeliveriesCorrection.rst.txt If the function is disabled, the address is shown as text. The server refuses requests that try to change the address anyway. .. note:: The correction only applies to this delivery. The recipient record itself (for example in tt_address or fe_users, or a csv list) is not changed. If a corrected address should be applied to the record as well, listen to the :ref:`RecipientEmailCorrectedEvent `, which is dispatched after a correction. .. _send-queue-refresh: Automatic refresh ----------------- The list is refreshed automatically. The interval is set by :ref:`undeliveredRefreshRate ` (5 seconds by default, 0 turns it off). It is independent of the refresh of the delivery progress bars. You can change both intervals in the configuration dialog of the queue module, which opens with the gear icon at the top right. .. include:: /Images/MailQueueConfiguration.rst.txt Every cycle only asks the server whether something has changed. The list is reloaded only if it has, so a retry that runs in the background shows up without reloading the module. The list is not replaced while you edit an email address. .. _send-queue-empty: Empty queue ----------- .. include:: /Images/MailQueueEmpty.rst.txt If there is no mail in the queue and no failed delivery, the module shows an info box "The mail queue is empty". The box disappears automatically as soon as a mail is scheduled, also by another user, or a delivery fails and is waiting for a retry. When the queue is empty again, the box is shown again. Mails are checked every :ref:`refreshRate ` seconds, failed deliveries every :ref:`undeliveredRefreshRate ` seconds. Finished mailings leave the list at once. .. note:: If a mail is deleted in the :guilabel:`Queue` or :guilabel:`Reports` module, all its queue entries are deleted with it. If a mail record is deleted another way, for example in the page tree, its queue entries are removed silently with the next run of the sending command. No failure is reported for them. .. _send-queue-log: Reports ======= A recipient that failed for good gets an additional entry with the response type "failed" in the mail log. The SMTP code is stored as return code, so that the failure shows up in the :ref:`mail report ` together with the returned mails found by the bounce analysis. If a failed recipient is delivered later, this entry is removed again and the delivery is counted as sent. .. _send-queue-lock: Parallel runs ============= The commands that deliver mails (:ref:`mail:mass-mailing ` and :ref:`mail:retry-failed `) share a lock. If a run is still active when the next one starts, the new run ends immediately with the note "Another send run is active. Skipped." This prevents mails from being delivered twice, for example if a scheduler task takes longer than its interval. .. _send-queue-alerting: Alerting ======== There are two ways to get informed about problems. Both are configured in the :ref:`extension configuration `. Email If :ref:`opsNotificationEmail ` is set, an email is sent for each recipient that failed for good. It contains the number of the mail, the recipient, the SMTP code and the error. Webhook If :ref:`alertWebhookUrl ` is set, the queue health is checked after each run. The queue counts as unhealthy as soon as the oldest recipient that is waiting or failed has been in the queue for at least :ref:`alertThresholdMinutes ` minutes. The webhook is called once when the queue turns unhealthy and, if :ref:`notifyOnRecovery ` is active, once when it is healthy again. While the state stays the same, nothing is sent. The request is a JSON POST. Only HTTPS addresses are called. Slack incoming webhooks work as they are, because Slack uses the field `text` and ignores the others. .. code-block:: json :caption: Example of the request body { "text": "EXT:mail send queue: deliveries are failing for 7 minutes.", "state": "red", "oldestUndeliverableSince": 1793381440 } The field `state` is `red` or `recovered`. `oldestUndeliverableSince` is the Unix timestamp of the oldest recipient that could not be delivered, or `null` when the queue is healthy again. .. warning:: Failed entries keep the queue unhealthy until they are deleted, either manually or by the cleanup. If you get a webhook "red" message because of recipients that failed permanently, delete the failed entries in the backend or with :ref:`mail:queue-purge ` to get the "recovered" message. .. _send-queue-cleanup: Cleanup ======= Entries stay in the queue table after they are processed. The command :ref:`mail:queue-cleanup ` removes sent and failed entries after the retention time set in the extension configuration (:ref:`sentRetentionDays ` and :ref:`failedRetentionDays `). Add it as a daily scheduler task.