Design: Improve / Rewrite the ToArray conversion
Status: IMPLEMENTED (all phases done), 2026-08-26 — see git history on
feature/. This is a historical design record; where wording differs from the code, the code wins.Improve To Array
Date: 2026-08-26 Scope: Classes/ (rewritten into Classes/), Content, Container
1. Goal
The core job of this extension: convert complex, rich TYPO3 domain objects (Record, File, Lazy, Typolink, ...) into plain PHP arrays so they can be serialized to JSON for headless responses.
Goals of the rewrite:
-
Replaceable architecture instead of one giant
switchblock.(true) - Use TYPO3 Core APIs (Schema API, Record API) instead of ContentBlocks internals where possible.
- Full control over the JSON contract (key mapping, field ordering, value shaping) — that is our product, not a side effect.
- Keep the PSR-14 extension point.
- Keep (or improve) the current 100 % test coverage as a safety net.
Non-goals:
- No generic serializer framework (we only serialize TYPO3 domain objects).
- No denormalization (JSON → objects).
- We do not change the headless extension itself.
2. Current situation
ContentBlocksJsonDataProcessor / ContainerJsonDataProcessor
└─ RecordToArray(Record) → toArray(), strips system fields
└─ ArrayRecursiveToArray(array) → THE switch, per-value dispatch
├─ RecordToArray → recursion
├─ TypolinkParameterToArray
├─ FileReferenceToArray → crop + public URL (ImageService)
├─ LazyFileReferenceCollectionToArray
├─ LazyFolderCollectionToArray
├─ LazyRecordCollectionToArray → recursion per item
└─ LazyRecordCollectionSysCategoryToArray
└─ event: ModifyArrayRecursiveToArrayEvent → PSR-14 per field/value
└─ headless.php → per Content Block PHP post-processing
Pain points:
| # | Pain | Where |
|---|---|---|
| 1 | Monolithic switch with instanceof chains; unknown types silently dropped (default case commented out) | Array |
| 2 | Heavy dependency on ContentBlocks internals: Table, Tca, 15 *Field instanceof checks | everywhere |
| 3 | Manual relation-target parsing (foreign_, allowed splitting) | get |
| 4 | Version hacks property_ etc. — only needed because we construct ContentBlocks internals in tests | tests |
| 5 | General everywhere instead of DI → hard to mock, hidden dependencies | all ToArray classes |
| 6 | String field handling (password blanking, richtext, passthrough) mixed into the same switch | process |
| 7 | No way to configure output (absolute URLs? include system fields? per-block options) — hard-coded | various |
3. Research: What do TYPO3 13 / 14 and others offer?
3.1 Core Record API (since 13.0, enhanced 13.3)
Record already converts raw DB rows into rich objects (Feature #103581, "Automatically transform TCA field values for record objects"):
- relations →
Record,Lazy,Record Collection File,Reference Lazy,File Reference Collection Folder,LazyFolder Collection - datetime →
DateTime Immutable - link →
TypolinkParameter - flex →
FlexForm Field Values - json → array
Insight: We already feed Record objects into our converter. The Record API does the object resolution — our job is only the array shaping. Nothing to gain from replacing this part; it is the right input.
3.2 Core Schema API (since 13.0!) — the important one
TYPO3CMSCore is the Core-native replacement for the ContentBlocks definitions we use:
| ContentBlocks (now) | Core Schema API (13+14) |
|---|---|
Table | Tca |
Table | Tca |
Tca instanceof *Field | Field:: → Table enum (Category etc.) |
manual foreign_ / allowed parsing | Relational → Active |
Content (CType → definition) | Tca / has |
field | Field:: |
Key finding: The Schema API exists in both TYPO3 13 and 14. A rewrite based on it does not need a "v13 legacy / v14 only" split! The property_ version hacks exist only because we construct ContentBlocks internals — if we stop doing that, the hacks disappear.
3.3 Core PSR-14 events in the Record pipeline
Record— fired when aCreation Event Recordis created from a row. Tempting, but rejected as main mechanism: modifications there affect every consumer of the Record API (Fluid templates, other processors), not only the headless JSON output. Side effects we do not want.
3.4 TYPO3 14 specifics worth using
lib./parse Func lib.are always provided by ext:frontend (since 13.2, default removed from fluid_styled_content in 14 — Breaking-107438). Our richtext path can rely on them being present.parse Func_ RTE - System Resource API (Feature-107537) — URL generation, not needed for FileReference public URLs, but relevant if we ever support
EXT:resources. - No breaking changes in Schema/Record API that affect this rewrite.
3.5 What we still need ContentBlocks for
Two things have no Core equivalent:
-
Identifier mapping: ContentBlocks maps DB column
tx_→ Content Block field identifierext_ my_ field my_. Core TCA only knows the column name. Dropping this would change every consumer's JSON contract (breaking).field - headless.php: resolving the Content Block folder via
Content.Block Registry:: get Content Block Ext Path ()
- headless.php: resolving the Content Block folder via
→ ContentBlocks stays a dependency, but only behind two narrow interfaces.
3.6 Real-world usage analysis (cms-netzbewegung-v5-2025, production site)
Investigated the example project (/var/) and its JSON response (https://).
Setup:
- TYPO3 13.4.32, content-blocks 1.6.1, headless 4.7.3, b13/container 3.1.12 → the main consumer is on v13. Confirms V2 (unified 13/14); a v14-only rewrite would not serve production.
- Wiring lives in
nb_(TypoScript set):frontend_ api datachains per block,Processing nb-for container columns (colPos 201/202 →container- json left/right), extbase plugins asUSER_(separate path).INT - JSON wrapper
{"id":…,"type":"netzbewegung_is built by nb_frontend_api/headless around our processor output.headline","col Pos": 0,"data": {…}}
headless.php reality check: 13 headless.php files exist, all 13 do the same thing: generate responsive image variants (mobile, mobile2x, desktop, desktop2x, … webp, widths like 883c) via a static Thumbnail (which internally abuses the Fluid Image!). Per block, per mode — dozens of hardcoded width configs.
→ Strongest finding for the rewrite: built-in, declarative image processing on File/FileReference fields would eliminate all headless.php files. ext:headless File + Processing already implements exactly this (TypoScript-style strings like width=883c,file).
Contracts confirmed in production JSON (must not break):
- Link fields:
url,target,type,title,config,attr(+nullfor empty links) — ourTypolinkoutputParameter To Array - FileReference:
id,alt,title,public(+Url thumbnailsadded per block in headless.php) - Collections (
items,buttons) withprefixidentifiersField: false - Basics includes (
Netzbewegung/),Appearance use(Existing Field space_), custom collection tables (after_ class tx_),nb_ buttons prefix→Type: full tx_columnsnb_* - Alphabetically sorted keys in
data(ourksort— consumers see it) - richtext
bodytextrendered with custom parseFunc classes (p-)text … processis actively used (Additional Data Processors nb-etc. via TypoScriptjob- cards- dynamic data)Processing. 20 { as = cards }
3.7 How others do "object → array/JSON"
| Approach | Example | Takeaway for us |
|---|---|---|
| Normalizer registry | Symfony Serializer (Normalizer + chaining) | Extensible, each normalizer small & testable. No new dependency needed — we implement the pattern natively (symfony/serializer is not installed) |
| Resource/Transformer classes per type | Laravel API Resources, League Fractal | Explicit, discoverable; but one class per Content Block is too much boilerplate for us |
| Serializer with context | Symfony groups / JMS Serializer | The context idea is good: per-call options (absolute URLs, include system fields, field subsets) |
| JsonSerializable on wrappers | various | Elegant, but lazy — key mapping/sorting/event hooks get awkward; harder to debug |
| DataProcessors per type | ext:headless itself (Files, Gallery + File) | Confirms the DataProcessor entry point; their Processing string syntax is a nice idea for per-field image processing options |
4. Options
Option A — Normalizer registry (Symfony-Serializer-style, hand-rolled) ★ recommended
RecordNormalizer (entry, uses RecordFactory output)
└─ NormalizerChain::normalize($value, Context)
iterates registered Normalizers, first `supports()` wins
Normalizers (one per type, DI-registered, tagged):
ScalarNormalizer null/int/string/float/array-plain
DateTimeNormalizer → W3C (configurable format)
RecordNormalizer → recursion + system-field stripping + key mapping
FileReferenceNormalizer → crop/processing + publicUrl (reuses ext:headless FileUtility?)
FileReferenceCollectionNormalizer
FolderCollectionNormalizer
RecordCollectionNormalizer → per item, resolves target schema via ActiveRelation
CategoryCollectionNormalizer
FlexFormNormalizer
TypolinkNormalizer
UnknownTypeNormalizer → explicit: log + null (no silent drops)
- Field-level shaping (password blanking, richtext parseFunc) becomes a separate FieldValueTransformer phase driven by Schema field type + config — no longer mixed into the type dispatch.
Content+Blocks Identifier Mapper Headlessbehind interfaces.Php Processor - Normalizers get real DI (no
make).Instance
Pros: testable units, open for site-packages to register own normalizers (the actual headless use case), no version hacks. Cons: more classes ( 12), slight indirection.
Option B — Schema-driven single converter
One Schema service: walks Record->to, asks Tca for each field's Table + configuration and converts in one (still large) method. Replaces instanceof chains with the enum, but stays monolithic and hard to extend by third parties.
Pros: compact. Cons: extensibility only via events; the switch survives.
Option C — Hook into Core RecordCreationEvent
Modify values at Record creation so Record->to is already JSON-ready.
Pros: least own code. Cons: global side effects on all Record consumers, no per-response shaping. Rejected.
Option D — JsonSerializable value wrappers
Wrap rich objects in lazy Json adapters; json_ triggers conversion.
Pros: elegant. Cons: key mapping/k_sort/strip/events fight the pattern; hard stack traces. Rejected.
Cross-cutting decision — ContentBlocks usage
| Variant | Description |
|---|---|
| C1 keep as-is | current state, all pain points stay |
| C2 narrow behind interfaces ★ | Core Schema API for field metadata + relations; ContentBlocks only for identifier mapping + headless.php |
| C3 drop completely | breaking JSON contract (column names instead of identifiers), headless.php gone — only for a future 3.0 |
Version strategy
| Variant | Description |
|---|---|
| V1 v13 legacy + v14 new | two code paths, double maintenance |
| V2 unified via Schema API ★ | Schema/Record API exist in 13+14 → one implementation, version hacks gone |
| V3 v14-only | unnecessary once V2 is chosen; only interesting if we wanted 14-only Core features |
5. Evaluation
| Criterion | A + C2 + V2 | B + C2 + V2 | current |
|---|---|---|---|
| Extensibility (3rd-party normalizers) | ++ | o (events only) | o (event only) |
| Testability | ++ |
|
|
| TYPO3 13 & 14, no hacks | ++ | ++ | − |
| JSON contract stable | ++ (covered by characterization tests) | ++ | ++ |
| Migration risk |
|
| – |
| Code size | +6 classes | − | – |
6. Recommendation
Option A (normalizer registry) + C2 (narrow ContentBlocks) + V2 (unified 13/14).
No split into v13/v14 versions — the Core APIs we need ship in both. This is also not a big-bang rewrite: phases below keep the suite green.
7. Proposed architecture
Classes/
├── DataProcessing/
│ ├── ContentBlocksJsonDataProcessor.php (entry, stays; slimmer)
│ └── ContainerJsonDataProcessor.php (stays)
├── Normalization/
│ ├── NormalizerInterface.php supports($value, Context): bool
│ ├── NormalizerChain.php DI-tagged 'nb_headless.normalizer'
│ ├── Context.php request, TcaSchema, options, eventDispatcher
│ ├── Normalizer/
│ │ ├── ScalarNormalizer.php
│ │ ├── DateTimeNormalizer.php
│ │ ├── RecordNormalizer.php recursion, strips system fields
│ │ ├── RecordCollectionNormalizer.php target schema via ActiveRelation
│ │ ├── CategoryCollectionNormalizer.php
│ │ ├── FileReferenceNormalizer.php crop, publicUrl, declarative image variants (à la ext:headless ProcessingConfiguration)
│ │ ├── FileReferenceCollectionNormalizer.php
│ │ ├── FolderCollectionNormalizer.php
│ │ ├── FlexFormNormalizer.php
│ │ ├── TypolinkNormalizer.php
│ │ └── UnknownTypeNormalizer.php explicit null + debug log
│ └── Event/
│ ├── BeforeFieldNormalizationEvent.php (replaces ModifyArrayRecursiveToArrayEvent)
│ └── AfterRecordNormalizationEvent.php (whole record, for headless.php alternatives)
├── FieldTransformer/
│ ├── FieldValueTransformerInterface.php
│ └── String/ PasswordBlanker, RichtextParser (driven by TableColumnType + TCA config)
└── ContentBlocks/
├── IdentifierMapperInterface.php uniqueIdentifier → identifier (passthrough fallback)
└── HeadlessPhpProcessor.php current headless.php include
Details worth discussing:
-
Context object carries: PSR-7 request, current
Tca, options (Schema absolute,Urls include,System Fields date, per-processor config from TypoScriptTime Format options.).- FileReference output: today
id/. Candidate: reuse or align with ext:headlessalt/ title/ public Url File/Utility Processingso consumers get consistent URLs across processors. Contract change → decide.Configuration - Events: keep
Modify(deprecated alias) for one release; fire the new field-level event with the same payload.Array Recursive To Array Event - Services.yaml: normalizers as tagged services;
Normalizerreceives them via tagged iterator — noChain makeleft in the normalization path.Instance
- FileReference output: today
8. Migration plan (incremental, suite stays green)
| Phase | Step | Safety net |
|---|---|---|
| 0 | Freeze the contract: convert current functional tests into characterization tests (assert full JSON per fixture record) | they must stay green until the end |
| 1 | Introduce Normalization/ namespace with Normalizer + first normalizers; Array internally delegates, still the public entry | unit tests per normalizer |
| 2 | Replace ContentBlocks field-metadata lookups with Tca (incl. Active for relation targets); delete get parsing | characterization tests |
| 2b | Add declarative image variant processing to File (config source per open question 6); migrate the 13 production headless.php files as pilot | characterization tests |
| 3 | Move string shaping into Field; move identifier mapping behind Identifier | unit tests |
| 4 | Wire DI (tagged services), remove make in the path; deprecate Modify | – |
| 5 | Cleanup: remove old classes, remove version hacks from tests, update AGENTS.md | full suite + CGL + PHPStan on 13 & 14 |
9. Decisions (2026-08-26)
Strategy: Option A (normalizer registry) + C2 (narrow ContentBlocks) + V2 (unified 13/14) — confirmed.
| # | Topic | Decision |
|---|---|---|
| 1 | Key ordering | Keep ksort — consumers already receive alphabetically sorted keys; no contract change |
| 2 | FileReference shape | Keep id/; add opt-in declarative image variants (thumbnails) to replace handwritten headless.php generators |
| 3 | Unknown value types | Emit null + debug log (no silent drops, never break the response) |
| 4 | headless.php API | Optional 2nd parameter Context $context — function — backwards compatible |
| 5 | Context options | TypoScript options. passthrough in v1, minimal set (absolute, date) |
| 6 | Image-variant config source | Both: headless. in the Content Block (default, versioned with the block) + TypoScript override (per site); TypoScript wins on conflict |
headless.yaml sketch (decision 2 + 6)
# ContentBlocks/ContentElements/image/headless.yaml
fields:
image:
processing:
mobile: "width=883c,fileExtension=webp"
mobile2x: "width=1766c,fileExtension=webp"
desktop: "width=1564c,fileExtension=webp"
desktop2x: "width=3128c,fileExtension=webp"
# per-site override (nb_frontend_api / site package)
tt_content.netzbewegung_image.fields.data.dataProcessing.10 {
options {
processing {
image {
desktop = width=1600c,fileExtension=webp
}
}
}
}
Processing strings reuse the ext:headless File/Processing syntax, so output URLs match what headless consumers already get from Files etc.
10. Remaining open points (non-blocking, decide during implementation)
- Exact
Contextoption names and defaults (v1:absolute,Urls=true date)Time Format=Date Time Interface:: W3C - Whether
Unknownlog channel isType Normalizer nb_or the site's default core logheadless_ content_ blocks - Deprecation window length for
Modify(proposal: one minor release, keep firing it until then)Array Recursive To Array Event
11. Additional goal: zero GeneralUtility::makeInstance()
End state (after Phase 4/5): no make anywhere in Classes/ — all dependencies via constructor injection. Normalizers never inject the chain (they recurse via Context::), so no circular DI problem. Intentional non-DI leftovers: the headless. include (receives Context as parameter) and static utility calls (General etc.).