---
title: "Evidence collection"
manual: "nr-vault"
version: "1.0"
permalink: "https://docs.typo3.org/permalink/netresearch/nr-vault:auditor-evidence-collection@1.0"
source: "Auditor/EvidenceCollection.rst"
rendered: "2026-09-18T07:37:50+00:00"
---

# Evidence collection {#evidence-collection}

The commands to run and the artefacts to keep. Everything here is read-only and
safe on a production system; the destructive checks live in
[Verification procedures](https://docs.typo3.org/permalink/netresearch/nr-vault:auditor-verification-procedures@1.0).

Run every command from the TYPO3 project root. Redirect output to a file — the
artefact is the evidence, not the terminal.

> [!NOTE]
> Several of these commands assert an operation permission that the
> unattributed CLI actor holds only when `allowCliAccess = 1` **and** the
> operation appears in [cliAllowedOperations](https://docs.typo3.org/permalink/netresearch/nr-vault:confval-ext-nrvault-cliallowedoperations@1.0):
> `vault:retrieve` (`secret.reveal`), `vault:rotate-master-key`
> (`master_key.rotate`), `vault:audit` — `audit.view` to list and to
> `--verify`, `audit.export` for `--export`, `vault.configure` for
> `--reset-anchor` — and the two scheduler-task wrappers,
> `vault:audit-verify` (`audit.view`) and `vault:audit-anchor`
> (`vault.configure`).
> The cleanest arrangement for an assessment is a named technical actor
> holding exactly those, so every evidence command appears in the audit trail
> under its own identity rather than as the anonymous CLI operator.
>
> A refusal is not evidence about the vault's contents: the command exits 1
> before doing any work, and writes no audit row. Record it as a gap in the
> evidence run, then re-run with the grant in place.

## Configuration and posture {#configuration-and-posture}

**Machine-readable posture snapshot**

```bash
vendor/bin/typo3 vault:doctor --format=json > evidence/doctor.json

# And, if the installation claims to be hardened, the policy assertion:
vendor/bin/typo3 vault:doctor --profile=hardened --format=json \
    > evidence/doctor-hardened.json; echo "exit=$?" >> evidence/doctor-hardened.json
```

Exit code is the verdict: `0` every control passed, `1` warnings only,
`2` at least one critical finding. Record it — a JSON body without the exit
code loses half the signal. Severity is **worst-wins**, so a long list of
passes can never average a critical finding away.

Two behaviours that matter when this runs as a pipeline gate: an unusable
`--profile` value and an internal crash **both** exit `2`, deliberately
coinciding with "critical" so a gate that could not actually check something is
never readable as "checked and fine".

> [!NOTE]
> **\`\`--profile\`\` changes the question, never the configuration.**
> `--profile=hardened` on a standard installation answers *"would this pass
> if we hardened it?"* and writes nothing. For an assessment that is the more
> useful run: it produces the real finding list for the un-migrated system
> without anyone flipping a switch on production. Short forms are `-p` and
> `-f`; `--format` defaults to `text`.

The JSON body carries `profile`, `configuredProfile`,
`profileOverridden`, `auditReady`, `highestSeverity`, `exitCode`, a
`summary` object (`total`, `pass`, `warning`, `critical`) and a
`findings` array.

**\`\`findings\`\` lists every control, including the ones that passed** — 24 are
always emitted, plus four conditional families:

-   `cli.access_groups` and `cli.allowed_operations`, only when
    `allowCliAccess` is on. `cli.frontend_placeholder_legacy` is **not** in
    this family despite the shared `cli.` prefix — it reports a setting that
    is independent of `allowCliAccess` and is therefore always emitted;
-   `provider.key_permissions`, only for the `file` master-key provider;
-   `audit.sink_state.<sink>`, one per enabled audit sink;
-   `audit.sink_probe.<sink>`, one per enabled sink and only under
    `--active-probes` — or the single literal `audit.sink_probe.none` when
    no sink is enabled.

A run with fewer findings than another is therefore not a smaller
installation; check which families were in scope before comparing two runs.

Each entry carries a stable dotted `id` plus
`severity` (`pass` | `warning` | `critical`), `summary`, `risk`,
`remediation`, `docsUrl` and `details`; for a pass, `risk` and
`remediation` are empty strings rather than absent. The `id` is what lets
you diff two runs months apart instead of re-reading prose.

> [!WARNING]
> **Check for \`\`error\`\` before \`\`findings\`\`.** On a rejected `--profile`
> value, or a run that could not start at all, the payload is
> `{error, exitCode}` *instead* — there is no `findings` key. A parser
> that reads `findings` unconditionally will crash on exactly the runs
> where something was wrong.

> [!IMPORTANT]
> **\`\`vault:doctor\`\` is a gate, not the authoritative verifier.** It runs in
> a pipeline and on a backend page load, so two of its audit controls are
> deliberately bounded:
>
> -   `audit.hash_chain` verifies only the **newest 1000 entries**;
> -   `audit.anchor` checks only that the chain has **not shrunk** below
>     the anchored sequence. It does **not** re-compare the anchored row's
>     tip hash, and it does **not** check the anchor's age.
>
> `vault:audit-verify` remains the authoritative full-range verifier — it
> walks the whole chain and performs the tip-hash comparison. For an
> assessment, run both and keep both artefacts; do not accept a green
> `vault:doctor` as evidence that the full chain verifies.

> [!NOTE]
> **A default standard installation has zero criticals by design**, so read
> a green run in context. Eight controls branch on the target profile:
> `profile.admin_override`, `environment.production_context`,
> `provider.configured`, `cli.access`, `audit.reads_logged`,
> `audit.external_sink`, `audit.anchor` and
> `audit.sink_state.<sink>`.
>
> Two of them shift from *pass* to *critical*: `audit.external_sink` and a
> missing `audit.anchor` are passes under `standard` (sinks and
> anchoring are opt-in there, matching `NO_EXTERNAL_SINK`'s documented
> semantics) and criticals under `hardened`. `provider.configured` and
> `audit.reads_logged` go from warning to critical. This is why the
> `--profile=hardened` run is the informative one even on a standard
> installation.

Finding ids worth citing directly in an assessment:

| Area | Ids |
| --- | --- |
| Profile and administrative override | `profile.valid`, `profile.admin_override` |
| Master-key custody | `provider.known`, `provider.configured`, `provider.available`, `provider.master_key_readable`, `provider.key_permissions` (`file` provider only) |
| Audit evidence | `audit.hash_chain`, `audit.hmac_epoch`, `audit.db_anchor`, `audit.anchor`, `audit.external_sink`, `audit.sink_delivery`, `audit.sink_state.<sink>`, `audit.reads_logged`, `audit.retention` |
| CLI exposure | `cli.access`, `cli.access_groups`, `cli.allowed_operations`, `cli.frontend_placeholder_legacy` |
| Emergency access | `breakglass.window_open` |
| Secret hygiene | `secrets.never_rotated`, `secrets.expired`, `secrets.dead` |
| Platform | `version.extension`, `version.typo3_supported`, `environment.production_context`, `environment.backend_lock_ssl` |
| Check failed to run | `check.crashed` — treat as unknown, never as a pass |

**Administrative override state**

```bash
vendor/bin/typo3 vault:break-glass --status > evidence/break-glass-status.txt
```

The important field is `adminOverrideDisabledEffective` (`yes` / `no`),
reported alongside the raw `disableAdminOverride` setting. A raw `1` with an
effective `no` means the flag is inert because the profile is `standard` — a
finding, and a common one.

Also capture, by inspection rather than by command:

-   the extension configuration (`securityProfile`, `masterKeyProvider`,
    `masterKeySource`, `auditHmacEpoch`, `allowCliAccess`,
    `auditReads`, the `auditSink*` block);
-   `config/system/additional.php` — which values are **pinned** out of
    the backend's reach;
-   `$GLOBALS['TYPO3_CONF_VARS']['HTTP']['allowed_hosts']`, if a webhook
    sink is configured;
-   the `tx_nrvault:*` grants per backend user group
    (**Backend Users > Groups**, **Custom module options**) —
    the authorization model as actually deployed;
-   file ownership and mode of the master-key file or wrapped blob, and of the
    NDJSON audit and anchor files.

## Audit chain integrity {#audit-chain-integrity}

**Chain plus anchor comparison — the primary integrity artefact**

```bash
vendor/bin/typo3 vault:audit-verify --format=json > evidence/audit-verify.json
echo "exit=$?" >> evidence/audit-verify.json
```

This is the one command that checks **both** halves: the internal hash chain
*and* the comparison against the last published chain-tip anchor. Findings
carry the stable reason codes from
[Alert reason codes](https://docs.typo3.org/permalink/netresearch/nr-vault:security-audit-evidence-reasons@1.0).

**Chain-only verification**

```bash
vendor/bin/typo3 vault:audit --verify > evidence/audit-chain.txt
```

`vault:audit --verify` verifies the chain and reports the in-database tip
anchor (`Tip anchor: …`), but performs no comparison against the *external*
anchor. Useful for isolating a finding: a chain that verifies here but fails
`vault:audit-verify` points at the external anchor, not at the rows.

> [!NOTE]
> `vault:audit-verify --tamper-only` suppresses `NO_EXTERNAL_SINK` and
> `SINK_FAILURE`. For an assessment, run **without** it — those two codes
> are exactly the ones that tell you whether independent evidence exists at
> all.

### Anchor inspection {#anchor-inspection}

The anchor file is NDJSON, one record per line. Read it directly; do not take
the application's word for it:

**The anchor with the highest sequence is the effective baseline**

```bash
jq -c 'select(.type=="anchor") | .anchor' < /var/log/typo3/nr-vault-anchors.ndjson \
    | tail -20 > evidence/anchors-recent.json

# The effective baseline — the reader takes the MAXIMUM sequence, not the last line.
jq -s 'map(select(.type=="anchor") | .anchor) | max_by(.sequence)' \
    < /var/log/typo3/nr-vault-anchors.ndjson > evidence/anchor-effective.json
```

Each anchor carries `sequence`, `chainTip`, `timestamp` and
`hmacEpoch`. Three things to check **by inspecting the file yourself** —
none of these is covered by a `vault:doctor` control:

1.  **Freshness.** Compare the newest `timestamp` against the configured
    anchoring interval. A stale anchor means the detection baseline is old even
    though nothing reported an error. **No automated control checks anchor
    age** — `audit.anchor` covers presence and shrinkage only — so the blind
    window is the anchoring *interval*, and confirming it is a manual step.
1.  **Continuity.** Sequences should rise across the file. A long flat stretch
    means anchoring was not running.
1.  **Epoch.** `hmacEpoch` should equal the configured `auditHmacEpoch`. A
    lower value in recent anchors means the protection level was reduced.

> [!WARNING]
> An anchor file stored only on the host whose database it protects is weak
> evidence: whoever can truncate the audit table can usually truncate the
> file. **Prefer the off-host copy** — the syslog archive or the SIEM — and
> record which source you used. This distinction decides what the anchor
> actually proves.

## Audit log export {#audit-log-export}

**Period export, with the hash columns**

```bash
# --export asserts audit.export, which is a permission of its own: the
# exported copy leaves the tamper-evident storage behind.
vendor/bin/typo3 vault:audit \
    --since="2026-01-01" --until="2026-06-30" \
    --format=json --limit=100000 \
    --export=evidence/audit-2026-H1.json
```

**Useful narrower slices**

```bash
# One secret's full history.
vendor/bin/typo3 vault:audit --identifier=<identifier> --format=json \
    --export=evidence/audit-secret.json

# Every denial in the period — reconnaissance and broken integrations.
vendor/bin/typo3 vault:audit --action=access_denied --format=json \
    --export=evidence/audit-denials.json

# Break-glass activations and closures.
vendor/bin/typo3 vault:audit --identifier=__break_glass__ --format=json \
    --export=evidence/audit-break-glass.json

# Master-key lifecycle.
vendor/bin/typo3 vault:audit --identifier=__master_key__ --format=json \
    --export=evidence/audit-master-key.json
```

`--format` accepts `table`, `json` and `csv`; `--limit` defaults to
`50`, so raise it explicitly for an export or you will silently truncate the
evidence. Other filters: `--action`, `--actor`, `--success`.

> [!WARNING]
> **Export the hash columns.** `uid`, `previous_hash`, `entry_hash` and
> `hmac_key_epoch` are what make the export evidence rather than a log.
> Without them nobody can re-check the links later.
>
> And note the honest limit: an export has **no hash chain of its own, no
> retention policy and no further access control**. It is why
> `audit.export` is a separate permission from `audit.view`, and it means
> the export must itself be handled as sensitive material. It also contains
> personal data (`actor_username`, `ip_address`, `user_agent`).

> [!NOTE]
> Pair every export with the `vault:audit-verify` output from the **same
> session**. Authenticity is demonstrable only while the master key is
> available; the verification run is the artefact that records the moment it
> was demonstrated. See
> [Retention obligations versus deletion](https://docs.typo3.org/permalink/netresearch/nr-vault:operations-decommissioning-retention@1.0) for why this matters at
> end of life.

## Plaintext exposure elsewhere {#plaintext-exposure-elsewhere}

```bash
vendor/bin/typo3 vault:scan > evidence/secret-scan.txt
```

Scans database content for values that look like unprotected secrets. It
evidences a different question from everything above: not "is the vault
sound?" but "is the vault actually being used?" A hardened vault next to
API tokens sitting in a content field is a finding about the deployment, not
about the extension.

## Development and release evidence {#development-and-release-evidence}

Verified against this repository's workflows. Do not assume a typical setup —
the list below is what is actually declared here.

| Evidence | Where | What it shows |
| --- | --- | --- |
| Compatibility test matrix | `.github/workflows/ci.yml` | PHP 8.2, 8.3, 8.4, 8.5 × TYPO3 `^13.4` and `^14.3`; unit and functional tests; coverage uploaded |
| Dependency vulnerability audit | `.github/workflows/checks.yml` → shared `security.yml` | `composer audit` with abandoned packages reported |
| SAST | `checks.yml` → shared `security.yml`, `opengrep` job | **Opengrep**, not Semgrep. Default arguments `--config auto --error --severity WARNING` block CI on WARNING-or-worse; SARIF uploaded to code scanning under category `opengrep`. nr-vault passes no `opengrep-config` override, so the default applies |
| Static application security testing | `checks.yml` → `codeql.yml` | CodeQL results in the repository's security tab |
| Secret scanning | `checks.yml` → shared `gitleaks.yml` | Scans every pull request against the in-repo `.gitleaks.toml`; findings reported to code scanning |
| Workflow-definition audit | `checks.yml` → shared `zizmor.yml` | Audits `.github/workflows/` itself — injection-prone expressions, over-broad permissions, dangerous triggers — configured by `.github/zizmor.yml` |
| Fuzz testing | `checks.yml` → `fuzz.yml` | Fuzz suite on every push, pull request, merge group and weekly schedule |
| Dependency review | `checks.yml` → `dependency-review.yml` | Runs on pull requests |
| Licence compliance | `checks.yml` → `license-check.yml` | Dependency licences checked in CI |
| OpenSSF Scorecard | `checks.yml` → `scorecard.yml` | Weekly and on default-branch pushes; supply-chain posture score |
| Release provenance attestations | `.github/workflows/release.yml` | Tag-triggered release requesting `id-token: write` and `attestations: write`; verify the published attestation with `gh attestation verify` |
| Static analysis and code style | `phpstan.neon`, `.php-cs-fixer.dist.php`, `phpat.neon`, `rector.php` | PHPStan level and baseline; architecture rules pinning the HTTP client ([ADR-028: PHPat architectural lock for HTTP client construction](https://docs.typo3.org/permalink/netresearch/nr-vault:adr-028-phpat-http-client-lock@1.0)) |
| Mutation testing | `infection.json5`, `Documentation/Developer/mutation-baseline.md` | Infection configuration and the recorded baseline. Run locally with `make test-mutation` |
| Coverage and quality gates | `codecov.yml`, `.sonarcloud.properties` | Codecov thresholds; SonarCloud project configuration |
| SBOMs, signatures, checksums | Shared reusable `release-typo3-extension.yml`, job `build-and-sign` | SPDX + CycloneDX SBOMs, Sigstore bundles and `checksums.txt` attached to every tagged release — see below |
| Vulnerability disclosure policy | `SECURITY.md` | Private reporting through GitHub security advisories |

> [!WARNING]
> **One config file in this repository is not wired into CI.**
> `semgrep.yml` exists at the repository root, but no workflow,
> `composer` script or Makefile target references it — verified by
> grepping the whole tree. The shared `security.yml` runs Opengrep with
> `--config auto`, i.e. registry rules, **not** the in-repo ruleset.
>
> Do not credit it as an enforced control. Its presence is the kind of thing
> that reads as "custom SAST rules are applied in CI" when nothing applies
> them. Treat it as local or historical tooling unless someone wires it up.
>
> `.gitleaks.toml` **is** wired: the `gitleaks` job in
> `.github/workflows/checks.yml` scans every pull request against it
> and reports to code scanning.

> [!IMPORTANT]
> **Supply-chain controls are one level up — follow the delegation, do not
> infer from the call site.** This repository's `release.yml` contains
> no SBOM, signing or checksum steps of its own. They live in the shared
> reusable `netresearch/typo3-ci-workflows/.github/workflows/release-typo3-extension.yml`,
> job `build-and-sign`, which produces for every tagged release:
>
> -   `<prefix>-<version>.sbom.spdx.json` and `.sbom.cdx.json` —
>     SPDX and CycloneDX, via `anchore/sbom-action` (gated on
>     `include-sbom`, default `true`);
> -   `<file>.sigstore.json` for **every** file in `dist/` — keyless
>     Sigstore signing via `sigstore/cosign-installer` and
>     `cosign sign-blob --bundle` (gated on `sign-artifacts`, default
>     `true`);
> -   `checksums.txt` — `sha256sum` over the whole `dist/` directory;
> -   a build-provenance attestation over the `.zip` and `.tar.gz` via
>     `actions/attest-build-provenance` — **ungated**.
>
> nr-vault's call site passes only `archive-prefix`, `package-name` and
> `extension-key`, so it opts out of neither gate and both defaults hold.
>
> Record **which** source you verified this against and at which revision:
> the reusable is referenced at `@main`, so its content can change without
> any commit in this repository.

**Verifying a published release**

```bash
gh release download v<version> -R netresearch/t3x-nr-vault -D release-assets
cd release-assets

# Integrity.
sha256sum -c checksums.txt

# Signature (keyless — identity and issuer must match the release workflow).
cosign verify-blob \
    --bundle nr-vault-<version>.zip.sigstore.json \
    --certificate-identity-regexp 'https://github\.com/netresearch/.+' \
    --certificate-oidc-issuer https://token.actions.githubusercontent.com \
    nr-vault-<version>.zip

# Build provenance.
gh attestation verify nr-vault-<version>.zip -R netresearch/t3x-nr-vault
```

Third-party actions in that release path are pinned to full commit SHAs.
Netresearch-owned reusables are deliberately referenced at `@main` so
upstream fixes propagate; an assessment should record that as policy rather
than flag it as an oversight, and should pin the *observed revision* in its own
evidence instead.

## Assembling the evidence package {#assembling-the-evidence-package}

**A complete package**

```text
evidence/
├── doctor.json                  # posture + exit code
├── doctor-hardened.json         # policy assertion + exit code
├── break-glass-status.txt       # adminOverrideDisabledEffective
├── audit-verify.json            # chain + anchor, with reason codes
├── audit-chain.txt              # chain only
├── anchor-effective.json        # highest-sequence anchor
├── anchors-recent.json          # recent anchor history
├── audit-<period>.json          # entry sequence WITH hash columns
├── audit-denials.json
├── audit-break-glass.json
├── audit-master-key.json
├── secret-scan.txt
├── config-snapshot.txt          # settings, pins, allowed_hosts, grants
├── file-permissions.txt         # key file, NDJSON files
└── ci/
    ├── workflow-runs.txt        # run URLs for ci.yml / checks.yml / release.yml
    ├── scorecard.json           # OpenSSF Scorecard result
    ├── coverage.txt             # Codecov / SonarCloud summary
    ├── sbom.spdx.json           # SBOM as published with the release
    ├── sbom.cdx.json
    ├── checksums-verify.txt     # sha256sum -c output
    ├── cosign-verify.txt        # cosign verify-blob output
    └── attestation-verify.txt   # gh attestation verify output
```

Record for the package as a whole: **when** it was collected, **from which
environment**, **by whom**, under **which extension, TYPO3 and PHP versions**,
and — for the anchor — **from which storage** the copy came. An evidence
package without that provenance cannot be re-checked, which is the only thing
it was collected for.