Evidence collection
The commands to run and the artefacts to keep. Everything here is read-only and safe on a production system; the destructive checks live in Verification procedures.
Run every command from the TYPO3 project root. Redirect output to a file — the artefact is the evidence, not the terminal.
Note
Two of these commands need a permission that a CLI operator only holds when
allowCliAccess = 1 (vault:retrieve and vault:rotate-master-key).
Everything in this section works without it. If a read-only command reports
access denied, that is itself a finding worth recording — with an
access_denied audit row to match.
Configuration and posture
vendor/bin/typo3 vault:doctor --format=json > evidence/doctor.json
# And, if the installation claims to be hardened, the policy assertion:
vendor/bin/typo3 vault:doctor --profile=hardened --format=json \
> evidence/doctor-hardened.json; echo "exit=$?" >> evidence/doctor-hardened.json
Exit code is the verdict: 0 every control passed, 1 warnings only,
2 at least one critical finding. Record it — a JSON body without the exit
code loses half the signal. Severity is worst-wins, so a long list of
passes can never average a critical finding away.
Two behaviours that matter when this runs as a pipeline gate: an unusable
--profile value and an internal crash both exit 2, deliberately
coinciding with "critical" so a gate that could not actually check something is
never readable as "checked and fine".
Note
``--profile`` changes the question, never the configuration.
--profile=hardened on a standard installation answers "would this pass
if we hardened it?" and writes nothing. For an assessment that is the more
useful run: it produces the real finding list for the un-migrated system
without anyone flipping a switch on production. Short forms are -p and
-f; --format defaults to text.
The JSON body carries profile, configuredProfile,
profileOverridden, auditReady, highestSeverity, exitCode, a
summary object (total, pass, warning, critical) and a
findings array.
``findings`` lists every control, including the ones that passed — 24 are always emitted, plus four conditional families:
cli.access_groupsandcli.allowed_operations, only whenallowCliAccessis on.cli.frontend_placeholder_legacyis not in this family despite the sharedcli.prefix — it reports a setting that is independent ofallowCliAccessand is therefore always emitted;provider.key_permissions, only for thefilemaster-key provider;audit.sink_state.<sink>, one per enabled audit sink;audit.sink_probe.<sink>, one per enabled sink and only under--active-probes— or the single literalaudit.sink_probe.nonewhen no sink is enabled.
A run with fewer findings than another is therefore not a smaller installation; check which families were in scope before comparing two runs.
Each entry carries a stable dotted id plus
severity (pass | warning | critical), summary, risk,
remediation, docsUrl and details; for a pass, risk and
remediation are empty strings rather than absent. The id is what lets
you diff two runs months apart instead of re-reading prose.
Warning
Check for ``error`` before ``findings``. On a rejected --profile
value, or a run that could not start at all, the payload is
{error, exitCode} instead — there is no findings key. A parser
that reads findings unconditionally will crash on exactly the runs
where something was wrong.
Important
``vault:doctor`` is a gate, not the authoritative verifier. It runs in a pipeline and on a backend page load, so two of its audit controls are deliberately bounded:
audit.hash_chainverifies only the newest 1000 entries;audit.anchorchecks only that the chain has not shrunk below the anchored sequence. It does not re-compare the anchored row's tip hash, and it does not check the anchor's age.
vault:audit-verify remains the authoritative full-range verifier — it
walks the whole chain and performs the tip-hash comparison. For an
assessment, run both and keep both artefacts; do not accept a green
vault:doctor as evidence that the full chain verifies.
Note
A default standard installation has zero criticals by design, so read
a green run in context. Eight controls branch on the target profile:
profile.admin_override, environment.production_context,
provider.configured, cli.access, audit.reads_logged,
audit.external_sink, audit.anchor and
audit.sink_state.<sink>.
Two of them shift from pass to critical: audit.external_sink and a
missing audit.anchor are passes under standard (sinks and
anchoring are opt-in there, matching NO_EXTERNAL_SINK's documented
semantics) and criticals under hardened. provider.configured and
audit.reads_logged go from warning to critical. This is why the
--profile=hardened run is the informative one even on a standard
installation.
Finding ids worth citing directly in an assessment:
| Area | Ids |
|---|---|
| Profile and administrative override | profile.valid, profile.admin_override |
| Master-key custody | provider.known, provider.configured,
provider.available, provider.master_key_readable,
provider.key_permissions (file provider only) |
| Audit evidence | audit.hash_chain, audit.hmac_epoch,
audit.db_anchor, audit.anchor,
audit.external_sink, audit.sink_delivery,
audit.sink_state.<sink>,
audit.reads_logged, audit.retention |
| CLI exposure | cli.access, cli.access_groups,
cli.allowed_operations,
cli.frontend_placeholder_legacy |
| Emergency access | breakglass.window_open |
| Secret hygiene | secrets.never_rotated, secrets.expired,
secrets.dead |
| Platform | version.extension, version.typo3_supported,
environment.production_context,
environment.backend_lock_ssl |
| Check failed to run | check.crashed — treat as unknown, never as a pass |
vendor/bin/typo3 vault:break-glass --status > evidence/break-glass-status.txt
The important field is adminOverrideDisabledEffective (yes / no),
reported alongside the raw disableAdminOverride setting. A raw 1 with an
effective no means the flag is inert because the profile is standard — a
finding, and a common one.
Also capture, by inspection rather than by command:
- the extension configuration (
securityProfile,masterKeyProvider,masterKeySource,auditHmacEpoch,allowCliAccess,auditReads, theauditSink*block); config/— which values are pinned out of the backend's reach;system/ additional. php $GLOBALS, if a webhook sink is configured;['TYPO3_ CONF_ VARS'] ['HTTP'] ['allowed_ hosts'] - the
tx_nrvault:*grants per backend user group (Backend Users > Groups, Custom module options) — the authorization model as actually deployed; - file ownership and mode of the master-key file or wrapped blob, and of the NDJSON audit and anchor files.
Audit chain integrity
vendor/bin/typo3 vault:audit-verify --format=json > evidence/audit-verify.json
echo "exit=$?" >> evidence/audit-verify.json
This is the one command that checks both halves: the internal hash chain and the comparison against the last published chain-tip anchor. Findings carry the stable reason codes from Alert reason codes.
vendor/bin/typo3 vault:audit --verify > evidence/audit-chain.txt
vault:audit --verify verifies the chain and reports the in-database tip
anchor (Tip anchor: …), but performs no comparison against the external
anchor. Useful for isolating a finding: a chain that verifies here but fails
vault:audit-verify points at the external anchor, not at the rows.
Note
vault:audit-verify --tamper-only suppresses NO_EXTERNAL_SINK and
SINK_FAILURE. For an assessment, run without it — those two codes
are exactly the ones that tell you whether independent evidence exists at
all.
Anchor inspection
The anchor file is NDJSON, one record per line. Read it directly; do not take the application's word for it:
jq -c 'select(.type=="anchor") | .anchor' < /var/log/typo3/nr-vault-anchors.ndjson \
| tail -20 > evidence/anchors-recent.json
# The effective baseline — the reader takes the MAXIMUM sequence, not the last line.
jq -s 'map(select(.type=="anchor") | .anchor) | max_by(.sequence)' \
< /var/log/typo3/nr-vault-anchors.ndjson > evidence/anchor-effective.json
Each anchor carries sequence, chainTip, timestamp and
hmacEpoch. Three things to check by inspecting the file yourself —
none of these is covered by a vault:doctor control:
- Freshness. Compare the newest
timestampagainst the configured anchoring interval. A stale anchor means the detection baseline is old even though nothing reported an error. No automated control checks anchor age —audit.anchorcovers presence and shrinkage only — so the blind window is the anchoring interval, and confirming it is a manual step. - Continuity. Sequences should rise across the file. A long flat stretch means anchoring was not running.
- Epoch.
hmacEpochshould equal the configuredauditHmacEpoch. A lower value in recent anchors means the protection level was reduced.
Warning
An anchor file stored only on the host whose database it protects is weak evidence: whoever can truncate the audit table can usually truncate the file. Prefer the off-host copy — the syslog archive or the SIEM — and record which source you used. This distinction decides what the anchor actually proves.
Audit log export
vendor/bin/typo3 vault:audit \
--since="2026-01-01" --until="2026-06-30" \
--format=json --limit=100000 \
--export=evidence/audit-2026-H1.json
# One secret's full history.
vendor/bin/typo3 vault:audit --identifier=<identifier> --format=json \
--export=evidence/audit-secret.json
# Every denial in the period — reconnaissance and broken integrations.
vendor/bin/typo3 vault:audit --action=access_denied --format=json \
--export=evidence/audit-denials.json
# Break-glass activations and closures.
vendor/bin/typo3 vault:audit --identifier=__break_glass__ --format=json \
--export=evidence/audit-break-glass.json
# Master-key lifecycle.
vendor/bin/typo3 vault:audit --identifier=__master_key__ --format=json \
--export=evidence/audit-master-key.json
--format accepts table, json and csv; --limit defaults to
50, so raise it explicitly for an export or you will silently truncate the
evidence. Other filters: --action, --actor, --success.
Warning
Export the hash columns. uid, previous_hash, entry_hash and
hmac_key_epoch are what make the export evidence rather than a log.
Without them nobody can re-check the links later.
And note the honest limit: an export has no hash chain of its own, no
retention policy and no further access control. It is why
audit.export is a separate permission from audit.view, and it means
the export must itself be handled as sensitive material. It also contains
personal data (actor_username, ip_address, user_agent).
Note
Pair every export with the vault:audit-verify output from the same
session. Authenticity is demonstrable only while the master key is
available; the verification run is the artefact that records the moment it
was demonstrated. See
Retention obligations versus deletion for why this matters at
end of life.
Plaintext exposure elsewhere
vendor/bin/typo3 vault:scan > evidence/secret-scan.txt
Scans database content for values that look like unprotected secrets. It evidences a different question from everything above: not "is the vault sound?" but "is the vault actually being used?" A hardened vault next to API tokens sitting in a content field is a finding about the deployment, not about the extension.
Development and release evidence
Verified against this repository's workflows. Do not assume a typical setup — the list below is what is actually declared here.
| Evidence | Where | What it shows |
|---|---|---|
| Compatibility test matrix | .github/ | PHP 8.2, 8.3, 8.4, 8.5 × TYPO3 ^13.4 and ^14.3; unit and
functional tests; coverage uploaded |
| Dependency vulnerability audit | .github/ → shared
security.yml | composer audit with abandoned packages reported |
| SAST | checks.yml → shared security.yml,
opengrep job | Opengrep, not Semgrep. Default arguments
--config auto --error --severity WARNING block CI on
WARNING-or-worse; SARIF uploaded to code scanning under category
opengrep. nr-vault passes no opengrep-config override, so
the default applies |
| Static application security testing | checks.yml → codeql.yml | CodeQL results in the repository's security tab |
| Secret scanning | checks.yml → shared gitleaks.yml | Scans every pull request against the in-repo
.gitleaks.; findings reported to code scanning |
| Workflow-definition audit | checks.yml → shared zizmor.yml | Audits .github/ itself — injection-prone
expressions, over-broad permissions, dangerous triggers —
configured by .github/ |
| Fuzz testing | checks.yml → fuzz.yml | Fuzz suite on every push, pull request, merge group and weekly schedule |
| Dependency review | checks.yml → dependency-review.yml | Runs on pull requests |
| Licence compliance | checks.yml → license-check.yml | Dependency licences checked in CI |
| OpenSSF Scorecard | checks.yml → scorecard.yml | Weekly and on default-branch pushes; supply-chain posture score |
| Release provenance attestations | .github/ | Tag-triggered release requesting id-token: write and
attestations: write; verify the published attestation with
gh attestation verify |
| Static analysis and code style | phpstan.,
.php-,
phpat.,
rector. | PHPStan level and baseline; architecture rules pinning the HTTP client (ADR-028: PHPat architectural lock for HTTP client construction) |
| Mutation testing | infection.,
Documentation/ | Infection configuration and the recorded baseline. Run locally with
make test-mutation |
| Coverage and quality gates | codecov.,
.sonarcloud. | Codecov thresholds; SonarCloud project configuration |
| SBOMs, signatures, checksums | Shared reusable
release-typo3-extension.yml, job build-and-sign | SPDX + CycloneDX SBOMs, Sigstore bundles and checksums.txt
attached to every tagged release — see below |
| Vulnerability disclosure policy | SECURITY. | Private reporting through GitHub security advisories |
Warning
One config file in this repository is not wired into CI.
semgrep. exists at the repository root, but no workflow,
composer script or Makefile target references it — verified by
grepping the whole tree. The shared security.yml runs Opengrep with
--config auto, i.e. registry rules, not the in-repo ruleset.
Do not credit it as an enforced control. Its presence is the kind of thing that reads as "custom SAST rules are applied in CI" when nothing applies them. Treat it as local or historical tooling unless someone wires it up.
.gitleaks. is wired: the gitleaks job in
.github/ scans every pull request against it
and reports to code scanning.
Important
Supply-chain controls are one level up — follow the delegation, do not
infer from the call site. This repository's release. contains
no SBOM, signing or checksum steps of its own. They live in the shared
reusable netresearch/typo3-ci-workflows/.github/workflows/release-typo3-extension.yml,
job build-and-sign, which produces for every tagged release:
<prefix>-<version>.sbom.spdx.jsonand.sbom.cdx.json— SPDX and CycloneDX, viaanchore/sbom-action(gated oninclude-sbom, defaulttrue);<file>.sigstore.jsonfor every file indist/— keyless Sigstore signing viasigstore/cosign-installerandcosign sign-blob --bundle(gated onsign-artifacts, defaulttrue);checksums.txt—sha256sumover the wholedist/directory;- a build-provenance attestation over the
.zipand.tar.gzviaactions/attest-build-provenance— ungated.
nr-vault's call site passes only archive-prefix, package-name and
extension-key, so it opts out of neither gate and both defaults hold.
Record which source you verified this against and at which revision:
the reusable is referenced at @main, so its content can change without
any commit in this repository.
gh release download v<version> -R netresearch/t3x-nr-vault -D release-assets
cd release-assets
# Integrity.
sha256sum -c checksums.txt
# Signature (keyless — identity and issuer must match the release workflow).
cosign verify-blob \
--bundle nr-vault-<version>.zip.sigstore.json \
--certificate-identity-regexp 'https://github\.com/netresearch/.+' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
nr-vault-<version>.zip
# Build provenance.
gh attestation verify nr-vault-<version>.zip -R netresearch/t3x-nr-vault
Third-party actions in that release path are pinned to full commit SHAs.
Netresearch-owned reusables are deliberately referenced at @main so
upstream fixes propagate; an assessment should record that as policy rather
than flag it as an oversight, and should pin the observed revision in its own
evidence instead.
Assembling the evidence package
evidence/
├── doctor.json # posture + exit code
├── doctor-hardened.json # policy assertion + exit code
├── break-glass-status.txt # adminOverrideDisabledEffective
├── audit-verify.json # chain + anchor, with reason codes
├── audit-chain.txt # chain only
├── anchor-effective.json # highest-sequence anchor
├── anchors-recent.json # recent anchor history
├── audit-<period>.json # entry sequence WITH hash columns
├── audit-denials.json
├── audit-break-glass.json
├── audit-master-key.json
├── secret-scan.txt
├── config-snapshot.txt # settings, pins, allowed_hosts, grants
├── file-permissions.txt # key file, NDJSON files
└── ci/
├── workflow-runs.txt # run URLs for ci.yml / checks.yml / release.yml
├── scorecard.json # OpenSSF Scorecard result
├── coverage.txt # Codecov / SonarCloud summary
├── sbom.spdx.json # SBOM as published with the release
├── sbom.cdx.json
├── checksums-verify.txt # sha256sum -c output
├── cosign-verify.txt # cosign verify-blob output
└── attestation-verify.txt # gh attestation verify output
Record for the package as a whole: when it was collected, from which environment, by whom, under which extension, TYPO3 and PHP versions, and — for the anchor — from which storage the copy came. An evidence package without that provenance cannot be re-checked, which is the only thing it was collected for.