HashiCorp Vault storage
With storage the encryption envelope of every new secret
lives in HashiCorp Vault (KV version 2) instead of in the TYPO3 database. The
decision and its trade-offs are in ADR-053: HashiCorp Vault storage adapter.
Table of contents
What lives where
| Data | TYPO3 database | Vault KV v2 |
|---|---|---|
| Identifier, description, context, metadata | yes | no |
| Owner, group tiers, expiry, counters, availability | yes | no |
Pointer to the Vault key (external_) | yes | no |
| Encrypted value, wrapped data key, nonces, checksum | no (columns are empty) | yes |
| Master key | no | no (see Key custody) |
The value is encrypted by nr-vault before it is sent. Vault stores ciphertext it
cannot read; the master key never reaches Vault, except wrapped, with the
transit provider.
Existing secrets are not migrated. A secret stored locally stays local and keeps
working after you switch; only new secrets go to Vault. Roll back by switching
the setting back: secrets already in Vault become unreadable until you switch to
hashicorp again, so do not remove them from Vault.
Prepare Vault
The adapter needs a KV v2 mount (development servers mount one at secret/)
and a policy that is limited to the extension's prefix:
path "secret/data/typo3/*" { capabilities = ["create", "read", "update"] }
path "secret/metadata/typo3/*" { capabilities = ["read", "delete"] }
vault policy write typo3 typo3-policy.hcl
hashicorp. is written as {mount}/, so the policy
above matches the default secret/. A nested mount such as
team/ is supported: the first path segment named data
separates the mount from the prefix.
Choose an authentication method
Set the method in hashicorp.. The storage adapter implements all
three. The transit master key provider still implements token only.
Token
vault token create -policy=typo3 -period=24h
Export the token to the PHP process as VAULT_ (or the name set in
hashicorp.). A token stored in hashicorp. also works but
is readable in the Install Tool and lands in configuration exports. A token has
no fallback: when it expires or is revoked, requests fail until you replace it.
Use a periodic token that something renews.
AppRole
vault auth enable approle
vault write auth/approle/role/typo3 token_policies=typo3 token_ttl=1h
vault read auth/approle/role/typo3/role-id
vault write -f auth/approle/role/typo3/secret-id
Put the role id in hashicorp.. Export the secret id as
VAULT_ (or the name in hashicorp.). The secret id
is read from the environment only; there is no setting that stores it. The login
token lives for the request or CLI run, is refreshed before its lease ends, and
is replaced once by a new login if Vault rejects it with 403.
Kubernetes
vault auth enable kubernetes
vault write auth/kubernetes/config kubernetes_host=https://kubernetes.default.svc
vault write auth/kubernetes/role/typo3 \
bound_service_account_names=typo3 \
bound_service_account_namespaces=web \
token_policies=typo3 token_ttl=1h
Set hashicorp. to typo3. The adapter reads the pod's
service account token from hashicorp. (default
/var/) on every login.
Configure TYPO3
$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['nr_vault']['storageAdapter'] = 'hashicorp';
$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['nr_vault']['hashicorp'] = [
'address' => 'https://vault.example.com:8200',
'path' => 'secret/data/typo3',
'authMethod' => 'approle',
'roleId' => 'b2c1…',
];
Every setting is described under Extension configuration. The adapter reads its configuration on each request; nothing is cached across requests.
Network and TLS
- The address must use
https.hashicorp.allowsallow Insecure Http httpfor local development; with it on, the token and every envelope cross the network unencrypted. - The address must be a bare origin: no credentials, path, query or fragment.
-
Requests use the same client as every other outbound call of the extension. It refuses loopback, private, link-local and metadata addresses, follows no redirects and logs no requests. A Vault on a private network therefore needs a literal entry (a wildcard does not count) in
$GLOBALS:['TYPO3_ CONF_ VARS'] ['HTTP'] ['allowed_ hosts'] config/system/additional.php$GLOBALS['TYPO3_CONF_VARS']['HTTP']['allowed_hosts'][] = 'vault.internal.example.com';Copied! - A whole transfer is bounded to ten seconds.
Behaviour you should know
- Failure
- A Vault outage or a rejected token makes the secrets stored in Vault unreadable and new secrets unwritable; secrets stored locally are unaffected. On store, Vault is written first: a failure leaves no row behind.
- Paths that already hold data
- A new secret claims its KV path with a create-only write, in one request. If
the path holds anything that is not an nr-vault write for that identifier,
the store is refused (code
1794400012), the data is left unchanged and no row is written; a key created by someone else at the same moment is refused the same way. Use a dedicatedhashicorp.so paths cannot collide with other users of the mount. The policy needspath createon the data path (the one above has it). An unreferenced key that an interrupted nr-vault write of this installation left behind is recognised and reused; a key written by another installation is not, so give each installation its ownhashicorp.. A row whose stored location differs from the one the currentpath hashicorp.resolves to is refused (codepath 1794400015). If the audit write of a create then fails, a reused key is put back as it was; a key the call created is removed. A rollback that is skipped because somebody wrote to the key since is logged as a warning. - Half-finished writes
- A crash between the Vault write and the row write leaves an unreferenced key
in Vault, which the next store of that identifier overwrites. A row whose
Vault key is missing is reported as an error when the value is read (code
1794400009), never returned as an empty secret.vault:anddelete vault:still reach the row, so you can remove it or write a new value.rotate - Delete
- Deleting removes the Vault key with all of its versions, then the row. Run
vault:again after an interrupted delete.delete - Master-key rotation
vault:refuses while any secret is stored outside the local database. It names the first one it meets and changes nothing. Data keys held in Vault are not re-wrapped by this release.rotate- master- key - Audit
- Every operation is written to the nr-vault audit log, and when the audit
write fails the previous state is restored. Unlike local secrets, the Vault
write and the database write are not one transaction: a crash after the
Vault write and before the audit write persists the change without an audit
entry. Vault's own audit device records every access to the KV paths, so
monitor it for writes and deletes that have no matching nr-vault entry.
Vault has no audit device until an operator enables one (
vault audit enable file file_), so enable one before you rely on this.path=/ var/ log/ vault_ audit. log - Changing
hashicorp.path - The Vault key of a secret is derived from the current setting. Existing Vault-backed secrets are not found under a new path, so move the keys in Vault first (or keep the old path) before you change it.
- Backup
- The database no longer contains the envelopes. A restore needs the database dump, the master key and a Vault that still holds the KV data; back all three up, and keep them apart (Backup and restore).
Troubleshooting
| Message | Likely cause |
|---|---|
address must use https | The address is http. Use https, or set
hashicorp. on a development machine. |
could not be sent with a blocked-host message | The address is private or loopback and not listed in
allowed_. |
was rejected with HTTP 403 | The token or the policy does not allow the path. Compare the policy
with hashicorp.. Under token there is no retry. |
login was rejected | Wrong role id, spent secret id, or a Kubernetes role that does not bind the pod's service account. |
login credential is missing | The environment variable or service account file named in the message is not visible to the PHP process. |
holds no value for it | The Vault key was deleted outside nr-vault, or the row points at another Vault. Restore the key from a Vault snapshot. |