Importer architecture 

Several upstream shapes, one relation set 

Upstream rarely expresses such a property one way. schema:keywords arrives as an @id reference to a vocabulary term, as a typed literal naming an ontology term by CURIE, or as free text an editor typed. All three resolve to the same internal entry — identity, title, parent — and land in one relation set. Detection belongs in a small reader class; the resolver should not branch on shape.

Identity must be derived so that repeated imports reuse rather than accumulate. A URI is already an identity; free text has none, so one is derived from the value (lowercased, mb_* throughout — strtolower() is byte-wise and splits Ölmühle from ölmühle into two records). Prefix the identifier by source and shape so two shapes can never collide on one stored row.

The property is collected run-scoped, never handed to the payload 

Resolution collects entries onto ResolverContext; it does not stage them into DataHandlerPayload. A single flush after the last root writes them.

This is the part most easily got wrong. Two independent reasons:

• A relation field is submitted as the complete set, and the framework replaces what is stored with what is submitted. Anything missing from the submitted set is thereby removed. Staging during resolution submits an incomplete set, so entries resolved later in the run are wiped by the earlier write.
• Targets are shared heavily across roots — one vocabulary term is referenced by hundreds of objects. Resolution runs once per root URL, so a per-root write means one root removing what another had just written.

A collector is therefore run-scoped, guarded by a first-claim key of table|ownerKey|field|identity. Owner and field belong in that key: the same target claimed by two records must yield two relations, and only a repeat by the same owner collapses.

Removal falls out of submitting the complete set 

Because submission replaces, a target upstream no longer supplies loses its relation with no deletion code — provided the submitted set is complete, which is what the deferred flush guarantees. Only relations are removed; the shared target record itself stays, since editors may still use it.

The corollary is the dangerous half: an entry that failed to resolve is also missing from the set, and would therefore be removed. A technical failure is indistinguishable from an upstream deletion, so the run records which owner/field had a failure and carries that owner's stored targets forward into the submitted set. Only upstream positively reporting a target absent (404, 410) may cost a relation — see FetchFailureVerdict. Every other failure keeps what is stored, because a credential, rate-limit or server fault arrives for every target on that host at once and would otherwise strip a whole run.

One hole is known and shared by every property built this way: an owner that collects nothing never enters the flush loop, so its relations survive even when upstream dropped all of them. It is recorded in the project backlog and wants a pattern-level fix, not a per-property one.

Each property gets its own everything 

Sharing an established path is the tempting shortcut and the wrong one. A new relation-set property gets its own transient bucket, its own configuration anchor, its own collector, its own run-scoped dedup map, and its own relation column. Where two properties both borrow sys_category, that shared table is an implementation detail with no semantic meaning: they must not share an anchor, a relation field, an identifier, or a dedup bucket.

Two properties sharing a dedup map hand each other staged keys, and the trees silently merge. Two sharing an anchor put one property's records under the other's root, where a rootline-scoped lookup then finds the wrong row.

Where a target's storage location varies by table, the owning entity declares it — EntityInterface::KEYWORD_FIELD and MEDIA_FIELDS are read from the entity, never assumed by the resolver. The resolver sees only the payload, so such a declaration travels in the bucket entry alongside the reference it belongs to.

Boundaries not to be modified 

• The ``@type`` category path — applyCategoryMapper(), the _categories bucket, wireCategories(). It looks like a general "category relations" mechanism and is not: it stamps the category anchor and dedups through the category map. A property placed there silently acquires both. A diff touching applyCategoryMapper() while adding a new property is the sign the boundary was crossed.
• The payload's per-row transient harvest — routing a relation set through it splits one record's set across two places at flush, which the completeness requirement above cannot work with.
• ResolverContext::promoteNewKeys() — it must learn every new run-scoped key map. A map left out is looked up by its NEW… placeholder in the next persistence round, misses, and stages a second row for a target that already exists. Silently: nothing errors.

Tripwires 

Each of these failures is invisible in ordinary testing, so each wants a test that fails when the boundary is crossed:

• Two properties whose targets carry deliberately identical titles, asserting each tree holds exactly its own members and that same-titled rows are distinct records with distinct identifiers.
• Two roots referencing the same target, asserting one stored record and two relations — this is what catches dedup state living in a local instead of on the context.
• A re-import dropping one target of several, asserting the relation is gone and the target record remains.
• A failed fetch alongside a surviving entry, asserting nothing is removed. Note this needs a surviving entry: an owner whose every entry fails never reaches the flush, so the guard is inert and the test proves nothing.
• Ancestors or grouping records, if the property has them, asserting they are not related to the owner — only the target the record actually cites is a relation.

Why one field per table 

"Any place" cannot be one relation. Extbase resolves a relation through a single concrete, table-mapped class — its only polymorphism is a recordType column selecting a subclass within one table (DataMapper::getTargetType()). A property typed across several tables produces a query against a table named after the class and fails. The core's own multi-table group fixture (blog_example's tx_blogexample_domain_model_tag.items) shows the same conclusion from the other side: the owning record never maps that property, and every readable side is typed to one concrete class.

So each target table gets its own field, and TouristAttraction::getContainedInPlaces() merges them back into one list for templates. Adding a place table to the import means adding it to the map and adding its field to the TCA of every owner table — the map is also the allowlist.

Every relation here is multi-value: a record can belong to more than one town (an airport serving two cities) and to places of several kinds at once.

Probing and reporting 

Because a bucket may name several tables, the pre-fetch lookup probes them in map order and takes the first hit. remote_id is unique per record, so one hit settles the question; the order only decides how many queries run first. Commonest kind first.

A reference whose record imported into a table the bucket has no field for is logged as referenceUnrelatable at info: the record exists and only the relation was dropped, which is upstream data drift rather than a fault. A reference that produced no record at all — a type this extension does not model — is not reported, because there was never a relation to lose. That distinction is the point of the report; a change that makes it fire for everything, or for nothing, has broken it.

What the provisioner guarantees 

• Reuse by identifier, so an editor's rename survives and no re-import duplicates a record. The match is guarded by the anchor's rootline, so a record belonging to another tree is never taken.
• Movement, not replacement. A stored record whose parent changed is re-parented in place. sys_category uids appear in plugin flexforms; a replacement looks identical in the tree and is wrong everywhere it is referenced.
• Translations for the languages the site configures, and no others.
• Skipping. A term with no default-language title is not created — a record an editor cannot read is worse than none — and its children attach to the nearest ancestor that was created.

Adding a field 

1. Add the relation column to the owner table's TCA and list it in that entity's RELATION_FIELDS.
2. Give the consumer its own SysCategoryAnchor — own parent, storage pid, identifier prefix — and its own SysCategoryProvisioningState .
3. Resolve titles through TitleResolver if the values come from a vocabulary; pass them directly if they do not.
4. Call provision() per term, parents before children, and relate the owner only to the terms it actually names.

Bind the state to a map on \WerkraumMedia\ThueCat\Import\ResolverContext where keys must survive between DataHandler passes: promoteNewKeys() rewrites NEW… placeholders to real uids there, and a state still holding placeholders stages a second record on the next round.

Two decisions the service does not make 

What a title means. TitleResolver asks upstream per language, treats a label carrying no language as English, and falls back to the mapper's titleMap for the default language only. That reading suits the class vocabularies and not keyword terms, whose untagged labels are German — the same JSON-LD shape means different things in different vocabularies, so one consumer is always served wrongly. Consulting the map for the default language is also what puts a value in the import report.

Which parent a value hangs from, where its source offers several. See Building the @type hierarchy.

Where it comes from 

Two vocabularies, fetched whole and merged into one index by \WerkraumMedia\ThueCat\Import\Vocabulary\VocabularyProvider :

• https://schema.org/version/latest/schemaorg-current-https.jsonld
• https://thuecat.org/ontology/thuecat/1.0/?format=jsonld

Whole documents rather than per-type lookups: the per-type endpoints are rate limited, and one climb would need a request per ancestor. ThueCat extends schema.org, so chains cross between them and the index must hold both to resolve one.

\WerkraumMedia\ThueCat\Import\Vocabulary\VocabularyIndexCache keeps the distilled index for 14 days, measured from a fetchedAt timestamp it stores itself rather than from the cache backend — TYPO3 cannot read an entry past its lifetime, and an expired entry is exactly what a failed refresh falls back on. A refresh is all or nothing: pairing a fresh vocabulary with a stale one drops the failed one's classes and breaks every chain crossing between them.

Building one chain 

\WerkraumMedia\ThueCat\Import\SysCategory\ChainBuilder walks upward from the type and returns the classes to create, ancestors first.

• Cut-off. No category for schema:Thing or schema:Place: every imported record belongs to them, so they distinguish nothing. A type left without ancestors becomes a root.
• Redundant parents. A class naming both CivicStructure and Museum, where Museum is itself a CivicStructure, has named one chain and not a fork. The nearer parent wins; the restated ancestor keeps its own level further up.
• Genuine branches. Where the remaining parents do not meet, one is chosen — a tree cannot have two. Which one depends on what the record is, so \WerkraumMedia\ThueCat\Import\SysCategory\ParentStrategies holds a \WerkraumMedia\ThueCat\Import\SysCategory\ParentStrategy per owner table: attractions prefer a branch reaching TouristAttraction, then Place; events prefer Event; anything else takes the deepest branch. A branch reaching no preferred root is logged at warning severity, because no rule fits it and a person has to look.

Preferred roots steer without appearing. TouristAttraction and Place sit in the mappers' ignoredValues() as structural supertypes an editor should never see: the strategy uses them to choose a branch, the cut-off declines to create them, and the chain lands on the configured anchor.

Only the types a record names become relations. Ancestors exist to give the tree its levels.