ADR-054: AWS Secrets Manager storage adapter
Table of contents
Status
Accepted
Date
2026-11-13
Context
The storage setting listed aws while the value behaved like
local. ADR-053: HashiCorp Vault storage adapter settled how an external store
fits the storage contract: the database row stays the registry of a secret, and
only the encryption envelope moves. Operators who run on AWS want that envelope
in AWS Secrets Manager, under IAM and CloudTrail, with the same guarantees.
Three things differ from the HashiCorp case. AWS has no plain HTTP API that is
cheap to call by hand: every request is signed with Signature Version 4, and the
usual way to produce that signature is the AWS SDK for PHP. The SDK's default
credential lookup asks the instance metadata service at 169.,
which the extension's SSRF defence refuses. And with two external adapters, a
row can now be bound to an adapter other than the configured one.
Decision
Apply the design of ADR-053 to a second store, and decide the three differences.
Same as ADR-053
- What is stored where. The row keeps identifier, owner, group tiers,
metadata, version, counters and timestamps, plus
adapter = awsandexternal_. Its envelope columns are empty. Secrets Manager holds one JSON document per secret with the encrypted value, wrapped data key, both nonces, checksum, envelope version and algorithm marker, base64-encoded where binary. The value is still encrypted by nr-vault before it leaves the process.reference - Per-row routing. Each row routes by its own
adaptercolumn. A secret stored locally before the switch stays local; a new secret, and a row that exists without a value, goes to AWS. Nothing migrates existing secrets. - Order of writes. AWS is written first, then the row. A failed first insert removes the AWS secret it just created, unless a row for the identifier exists by now. A failed row update puts the previous envelope back as a new version. On delete, AWS is emptied first, then the row, so a half-finished delete can be repeated.
- Audit. Unchanged.
Vaultaudits every operation at its own boundary. When the audit write fails after an AWS change, the existing compensation restores the previous envelope for create, rotate and delete.Service - Gate classification and master-key rotation. Both already test for "not
stored locally" and not for a named adapter, so they hold for
awswithout change: a row that points at AWS but has no loadable envelope is an existing secret and keeps the rotate and policy gates, andvault:refuses while any secret lives outside the local database.rotate- master- key
The SDK is a suggestion, not a requirement
aws/ is listed in suggest and require-, never in
require. A TYPO3 installation that does not select aws does not install
it and boots without it: no class outside
Aws and
Aws names an SDK type, no autowired constructor takes
one, and the factory builds the client on first use.
Selecting aws without the SDK fails with a
Configuration
(code 1794500002) that names the Composer command. It does not fall back to
local storage. The failure comes on the first operation that needs AWS, not when
the setting is read, so secrets stored locally stay readable on an installation
whose SDK went missing.
ADR-053 declined a client library because it would bring its own transport and
bypass the SSRF and DNS-pin middleware. The SDK is used here for what is not worth
rewriting, Signature Version 4 signing, request building and response parsing,
and its transport is replaced: the AWS client is given the client that
Secure builds, through the SDK's http_ option.
Every request therefore passes the SSRF range check and the DNS pin, follows no
redirect, logs nothing and is bounded to ten seconds. A functional test pins this
with the same refusal ADR-053 has: an unlisted loopback endpoint is refused.
That client is built with http_ off, so that the vault HTTP client can
audit error statuses. Handed to the SDK as it is, a 403 or 500 arrives as an
ordinary response and is parsed as an empty successful result: a rejected write
would read as stored. The AWS client therefore sets http_ back on for
its own requests, and a unit test builds the client the production way to pin it.
Credentials are explicit
The SDK is given one credentials object. That means its default provider chain, including the instance metadata call, never runs, and ambient credentials of the host are not picked up.
- The access key id is read from an environment variable whose name is
aws., withaccess Key Id Env Var aws.as a stored fallback (it identifies, it does not authenticate).access Key Id - The secret access key is read from the environment only, from the variable
named in
aws.. No setting stores it.secret Access Key Env Var - An optional session token, from
aws., makes temporary credentials usable.session Token Env Var - The default variable names are
NR_, not the SDK'sVAULT_ AWS_* AWS_*, so credentials that belong to something else on the host are not taken by accident. - Shared config and credentials files are not read, and
AWS_variables are ignored. OnlyENDPOINT_ URL aws.redirects the adapter.endpoint
Assuming a role is not part of this decision. It needs an STS call and a second endpoint to harden, and temporary credentials can already be supplied through the session token by whatever assumes the role outside PHP.
Transport and names
- Region is required and validated. An optional
aws.(a VPC endpoint, a compatible service) must be a bare origin and must useendpoint httpsunlessaws.is set, the same mechanism asallow Insecure Http hashicorp.. At most one retry, so an unreachable endpoint cannot hold a request for minutes.allow Insecure Http - The AWS secret name is
{prefix}/. AWS allows letters, digits and{identifier} /_+=.@-; an identifier may contain:(thesite:<namespace>:<name>form), which AWS refuses, so the colon is written as+. The mapping is injective because+is not in the identifier alphabet. The prefix is normalised, so the defaulttypo3/yieldstypo3/, notname typo3//. The registry row storesname <region>:<name>. - Delete uses
Force. A recovery window would keep the last envelope alive for up to thirty days and block the name; without it no version survives.Delete Without Recovery - A write adds a version (
Put) and creates the secret when AWS says it does not exist (Secret Value Create); if a concurrent create wins in between, the conflict is raised (codeSecret 1794500012) and nothing is written into that writer's secret. A compensation that cannot apply because the secret changed since keeps the row and logs a warning.
Errors
Aws names the operation, the HTTP status and the AWS
error code, a fixed identifier such as Resource. It is not
chained to the SDK exception, which holds the signed request and the command
parameters, and those include the encrypted envelope. Transport failures carry a
redacted client message.
A secret of somebody else is never overwritten or deleted
AWS secret names are shared with everything else in the account. For a new secret
(or a row without a value) the adapter therefore creates the AWS secret with
Create, which fails when the name exists, instead of writing a version.
When it exists, the adapter reads it: only a document that is an nr-vault write
for this very identifier and this installation (format marker, identifier and an
HMAC of the master key stored in the document), an unreferenced leftover of an
earlier crash, is adopted and overwritten; a document of another installation is
not, so each installation needs its own prefix. Anything else is
refused with code 1794500011 and left untouched. Compensation follows the
same rule: a failed row write destroys only a secret this call created, and
puts an adopted one back. An existing AWS-bound row owns its secret at the
location it recorded: when the stored reference differs from the one the current
region and prefix resolve to, store, delete and read are refused (code
1794500014) instead of touching an unrelated secret; the document must carry
the row's identifier and no other installation's marker before it is overwritten
or destroyed. A document without a marker (an earlier write) is accepted for a
row bound to it, because such a row holds no envelope to compare, and the next
write stamps the marker. A document that is not a JSON object does not make the
row undeletable.
Taking a write back does not compare payloads. Create and
Put return the Version they created, and the rollback (put
the previous document back, or delete the secret this call created) reads the
AWSCURRENT version id with Get and applies only if it is still
the one the write returned. AWS has no conditional put or delete, so the id is
read immediately before the write or delete and a write that lands in that gap
is not detected; the read is retried a few times so a read that has not caught
up with the write is not mistaken for a concurrent writer, and a rollback that is
skipped is logged as a warning. A failed audit write is compensated by
Vault: an adapter implementing
Create destroys a secret the call created and
puts back one it adopted, then removes the row. The prefix typo3/ stays the default; a
dedicated prefix keeps the names apart from other users of the account.
A row bound to another external adapter is refused
With two external adapters a row can name one that is not configured.
Hashi and
Aws therefore refuse to
store, retrieve or delete a row bound to a different external adapter
(
Configuration, code 1794500020), instead of treating its
empty envelope as "new" and rebinding it, which would write a second copy and
orphan the first. The administrative lookup still returns the row, so it stays
listable. Select the adapter that owns the row to work with it; nothing moves
secrets between adapters.
Test infrastructure
Functional tests run against Moto (motoserver/, an exact tag) through
Testcontainers, not LocalStack, whose current releases need a licence token. Moto
emulates the five Secrets Manager calls the adapter makes (create, put value,
get, describe, delete without recovery) and, with its IAM enforcement on, rejects
an unknown key id and a wrong secret with HTTP 403.
Consequences
Positive
- IAM policies and CloudTrail apply to every envelope.
- Listing, access control, expiry, analytics and the audit trail work unchanged, because the registry row stays.
- An installation without the SDK is unaffected, and one that selects
awswithout it is told so. - The SDK's signing is reused without giving up the extension's transport defences.
Negative
- A secret has two stores and they are not updated atomically. A crash between the writes leaves an unreferenced AWS secret (overwritten by the next store of that identifier) or a row whose metadata lags the envelope by one write. A crash between the AWS delete and the row delete leaves a row whose envelope is gone; that value is lost and comes back only from a backup. A compensated delete recreates the secret with its latest value only (ADR-036: Mutation and audit are all-or-nothing).
- AWS stages versions. After a rotation the previous envelope stays in AWS
labelled
AWSPREVIOUS(measured on Moto 5.2.5; the real service was not measured), until the secret is deleted. The previous envelope is ciphertext and needs the master key, as does the current one. - The AWS SDK and its dependencies are a large package. It is optional, but an
installation that selects
awscarries it. - A secret in AWS makes every read a network call, and AWS bills per secret and per API call.
- Master-key rotation is unavailable while AWS-backed secrets exist.
- The registry rows are still in the database: names, owners, groups and metadata remain visible to anyone who can read it.
- Assume-role is not supported; supply temporary credentials through the session token.
Not verified against real AWS
The functional suite runs against Moto only. Three things it cannot show:
- Resource-scoped IAM. Moto evaluates a policy's
Resourceagainst*for Secrets Manager calls instead of the secret's ARN: a policy scoped to the prefix denied every call (measured on 5.2.5). The test policy is therefore action-scoped, which proves that the four actions the adapter calls are enough and that no other is needed, and says nothing about the ARN pattern on the operations page. - Name reuse after a forced delete. Whether AWS keeps a force-deleted name unavailable for a short time was not measured; Moto deletes immediately. A create or a compensating restore inside that window is rejected with a request error; it cannot store a wrong value.
- Colons in names. Moto accepts a colon in a secret name and AWS does not. The mapping is pinned by unit tests on the name, not by the emulation.
Alternatives considered
Sign the requests by hand over the PSR-18 client
Rejected. It keeps the extension free of the SDK, but Signature Version 4 is exactly the code that is wrong in quiet ways (canonical headers, payload hash, clock skew, session tokens), the SDK already does it, and the SDK's transport can be replaced. The cost of the SDK is the package size, which the decision to make it optional contains.
Make the SDK a requirement
Rejected by the repository owner: every installation would carry it for a feature most do not select.
Use the SDK's default credential chain
Rejected. It calls the instance metadata service, which the SSRF defence blocks, and it picks up credentials the operator never meant for nr-vault.
Fail when the setting is read, not on first use
Rejected. The adapter is built whenever the vault service is, so failing there would make locally stored secrets unreadable on an installation whose SDK is missing, for a setting that only concerns new secrets.
Move or migrate a row between adapters
Rejected, as in ADR-053: nothing migrates secrets. The adapters refuse the foreign row instead of guessing.