---
title: "Testing"
manual: "Microsoft Exchange 365 Mailer"
version: "main"
permalink: "https://docs.typo3.org/permalink/oliverkroener/ok-exchange365-mailer:testing@main"
source: "Development/Testing.rst"
rendered: "2026-10-04T20:03:13+00:00"
---

# Testing {#testing}

The extension ships an automated **cross-version test matrix**. It provisions one
throwaway TYPO3 installation per supported major, wires the working tree in as a
Composer path repository, and runs the test layers against each one.

## Requirements {#testing-requirements}

-   [DDEV](https://ddev.com/) 1.24 or newer, and a running Docker daemon.
-   Roughly 6 GB of free disk — each lab is about 600–900 MB of `vendor/`
    plus a database volume.
-   Git, because three of the labs test a different branch of this repository.
-   For the live checks: Node.js 18 or newer (the browser check installs
    `playwright-core` into `Build/testing/browser/` on first use).

## Running the matrix {#testing-running}

From the extension directory:

```bash
make test-matrix
```

A cold run provisions six labs and takes roughly 30–60 minutes. Warm re-runs reuse
the existing labs and take a few minutes per lab.

Useful variants:

```bash
make test-matrix                        # all majors, offline layers only
make test-matrix-live                   # additionally the live checks, see below
make test-matrix-clean                  # delete all labs and worktrees
make install-hooks                      # gate tag pushes on a green matrix

Build/Scripts/runTests.sh --versions=13,14   # a subset
Build/Scripts/runTests.sh --keep-labs        # never delete, even when green
Build/Scripts/runTests.sh --layers=unit      # one layer only
Build/Scripts/runTests.sh --status           # what state is each lab in?
```

The layer names are `resolve`, `install`, `canary`, `unit`,
`functional`, `phpstan` and `cgl`.

## What is tested where {#testing-matrix}

The matrix is defined in `Build/matrix.json`. Three of the six labs check out
a **different branch**, because the extension is maintained as one release line per
TYPO3 major — see [Version Compatibility](https://docs.typo3.org/permalink/oliverkroener/ok-exchange365-mailer:compatibility@main).

| TYPO3 | Branch tested | PHP | Source wired into the lab |
| --- | --- | --- | --- |
| 14.3 | `main` | 8.2 | the working tree |
| 13.4 | `main` | 8.2 | the working tree |
| 12.4 | `main` | 8.1 | the working tree |
| 11.5 | `feature-typo3-11` | 8.1 | a `git worktree` |
| 10.4 | `feature-typo3-10` | 7.4 | a `git worktree` |
| 9.5 | `feature-typo3-9` | 7.4 | a `git worktree` |

Coding standards are checked once per branch, in its newest lab: the older labs
resolve an older `typo3/coding-standards` whose rules contradict the newer ones,
and style is a property of the source, not of the TYPO3 version.

## The test layers {#testing-layers}

Each lab runs these in order — cheapest and most likely to fail first:

| Layer | What it proves |
| --- | --- |
| Composer resolve | The declared constraints actually resolve against that TYPO3 major. A conflict here is a **finding**, not an error — it means the advertised compatibility is wrong. |
| TYPO3 install | A real, installed TYPO3 site comes up on that major. |
| Wiring canary | The lab is testing **your working tree** and not a copy fetched from Packagist. See the warning below. |
| PHPUnit unit | Configuration precedence, the empty-value guard, the `saveToSentItems` exemption, credential validation, the sender resolution order, client/token reuse and error wrapping. On the 1.x and 2.x lines, the complete send path including retries runs against a mocked HTTP client. |
| PHPUnit functional | The extension activates, and TYPO3 really selects this transport when `MAIL.transport` is set to the fully-qualified class name. |
| PHPStan | Static analysis at level 8 against that major's core API. |
| Coding standards | TYPO3 CGL via `php-cs-fixer`. |
| Live checks | Skipped unless explicitly enabled. See below. |

> [!WARNING]
> The **wiring canary** is not a formality. It writes a temporary file into the
> extension source on the host and checks that it appears inside the container. If
> the path repository is not wired correctly, Composer silently installs a release
> copy from Packagist instead, and every green result in that lab is meaningless.
> A failed canary marks the lab `wiring` and invalidates its results.

> [!NOTE]
> TYPO3 11 and 12 are **ELTS**. Every freely published patch of those majors
> carries security advisories, so Composer's audit would refuse to load any of
> them and the major could not be tested at all. The harness therefore sets
> `policy.advisories.block=false` *inside the lab*. That is safe for a
> throwaway installation, but it means a lab is **not** a security-current
> install — the report's environment header says so on every run.

## Known coverage gaps {#testing-coverage-gaps}

> [!IMPORTANT]
> On the 3.x and 4.x lines, the call into the Microsoft Graph SDK itself is only
> executed by the live checks. Everything around it — configuration, sender
> resolution, client reuse and the error wrapping — has offline tests through
> the `createGraphServiceClient()` seam.

## The live checks {#testing-live}

With `--live` (`make test-matrix-live`), each lab additionally runs four checks
against the real Microsoft Graph API. They use a lab-only fixture from
`Build/testing/` that is mounted into the lab and never shipped.

| Check | What it proves |
| --- | --- |
| `live-cli` | A real mail is accepted by Graph in **CLI context**, with the credentials in `TYPO3_CONF_VARS`. |
| `browser` | A headless Chromium logs into the backend, opens *System > Configuration* and finds the client secret **masked** — never in plain text. Screenshots are stored with the report. |
| `live-frontend` | A real mail is accepted by Graph in **frontend context**, while `TYPO3_CONF_VARS` hold **no** credentials: they come only from `:= getEnv(...)` in the root template's setup and constants. This proves the documented [getEnv() configuration](https://docs.typo3.org/permalink/oliverkroener/ok-exchange365-mailer:frontend-getenv@main) on that TYPO3 version. |
| `getenv-missing` | A `getEnv()` of a variable that does not exist leaves the send failing cleanly with `missing required field: clientSecret`. |

Every message has the subject `[ex365-matrix] <run> v<major> <context>`, so a run
can be traced in the mailbox. A full run sends two messages per lab.

1.  Create the credentials file **outside** the repository:

    ```bash
    mkdir -p ~/.config/ok-ex365
    cp Build/testing/.env.test.dist ~/.config/ok-ex365/test.env
    chmod 600 ~/.config/ok-ex365/test.env
    ```

    Fill in a test tenant. `EXCHANGE365_TEST_RECIPIENT` defaults to the sender
    address. Set `OK_EX365_TEST_ENV` to use a different file.
1.  Run the matrix with the flag:

    ```bash
    make test-matrix-live
    ```

The credentials are passed to the lab's web container through
`.ddev/.env.web`, so they are in the real process environment where
`getenv()` sees them, and that file is deleted again after the checks. No
secret is written into a PHP or TypoScript file.

> [!NOTE]
> The live checks surfaced two network failure modes that production servers
> share: dead IPv6 routes to Microsoft (DNS returns IPv6 addresses the host
> cannot reach) and connections that stall after connecting. All transports
> therefore use a 10 s connect and 30 s total timeout — for the Graph call
> instead of the SDK's 100 s, and for the OAuth token request, which the
> SDK's OAuth library otherwise sends with **no timeout at all** (a stalled
> token request would block a scheduler run forever). A request that never connected is retried once; a request that
> stalled after connecting is **not** retried, because Graph may already have
> accepted the mail and a retry could send it twice.

## Release gate {#testing-release-gate}

```bash
make install-hooks
```

enables `.githooks/pre-push` for this clone and all its worktrees. Pushing a
**tag** then runs the matrix for every TYPO3 major of the branch the tag is on —
including the live checks when the credentials file exists — and refuses the push
unless everything is green. The checkout being tested must be clean and at the
tagged commit, so a green gate always means "this exact release was tested".
Branch pushes are not affected. `OK_EX365_SKIP_MATRIX=1 git push --tags` bypasses
the gate in an emergency.