Configuration 

Where a seed set lives 

A seed set is a directory Configuration/DataFactory/<name>/ inside any active extension, with config.yml as its entry file:

packages/my_extension/Configuration/DataFactory/demo/
├── config.yml               entry file, and the only mandatory one
├── Scenario/
│   ├── Pages.yaml           the records, named by "scenarios"
│   └── Content.yaml
├── Files.yaml               optional, pulled in through "imports"
├── Files/
│   └── placeholder.svg      resources named by "files"
└── Sites/
    └── main/
        ├── config.yaml      site configuration template
        └── settings.yaml    optional site settings

Every relative path inside a set - a scenarios entry, a file source, a site template - is resolved against the directory holding the entry file, not against the file declaring it. A set can therefore be moved or renamed without touching a single path inside it. EXT: paths are accepted everywhere a path is, and an absolute path is taken as it stands.

An imports resource is the one exception: it never reaches this extension, it is resolved by the TYPO3 YAML loader relative to the file that declares it - see Splitting a set over several files.

A set is found because the extension providing it is installed and activated. There is no path to configure and nothing to register. A directory below Configuration/DataFactory/ without a config.yml is not a set and is passed over, which is what lets a set keep partials next to itself.

The set descriptor 

identifier: demo
title: 'Demo page tree'
description: 'Pages, content elements, a file and the site they are reachable through.'

imports:
  - { resource: Files.yaml }

scenarios:
  - Scenario/Pages.yaml
  - Scenario/Content.yaml

files:
  - identifier: placeholder
    source: 'Files/placeholder.svg'
    folder: 'demo'

references:
  - file: placeholder
    table: tt_content
    uid: 2000
    field: assets

sites:
  - identifier: main
    rootPage: 1000
    template: 'Sites/main'

This list is closed: any other key at this level is refused, naming the known ones. That is deliberate - scenario: instead of scenarios: would otherwise be an import that reports success and writes nothing.

The descriptor carries no entitySettings and no entities. One rule, no ambiguity: config.yml describes the set, a scenario file describes records. Nothing this extension invents is mixed into a scenario file, and nothing a scenario file declares has to be understood by the descriptor.

The three metadata keys have to be declared in config.yml itself. They cannot be pulled in through imports, because listing the sets of an installation reads them without following imports.

The scenario format 

A scenario file is a map of at most three keys. Anything else at the top level is refused, naming the file:

entitySettings:
  '*':
    nodeColumnName: 'pid'
    columnNames: {id: 'uid', language: 'sys_language_uid'}
    defaultValues: {pid: 0}
  page:
    isNode: true
    tableName: 'pages'
    parentColumnName: 'pid'
    languageColumnNames: ['l10n_parent', 'l10n_source']
    columnNames: {type: 'doktype', root: 'is_siteroot'}
    defaultValues: {hidden: 0, doktype: 1}
    valueInstructions:
      shortcut:
        first: {shortcut: 0, shortcut_mode: 1}
  content:
    tableName: 'tt_content'
    languageColumnNames: ['l18n_parent', 'l10n_source']
    columnNames: {title: 'header', type: 'CType'}
    defaultValues: {hidden: 0, CType: 'text', colPos: 0}
  category:
    tableName: 'sys_category'

entitySettings:
  '*':
    nodeColumnName: 'pid'
    columnNames: {id: 'uid'}
  category:
    tableName: 'sys_category'

page:
  valueInstructions:
    shortcut:
      first: {shortcut: 0, shortcut_mode: 1}

entities:
  page:
    - self: {id: 1000, title: 'Demo', root: true, slug: '/'}
      entities:
        content:
          - self: {id: 2000, title: 'A frontend to look at', type: 'header'}
          - self: {id: 2001, bodytext: '<p>Seeded, not clicked together.</p>'}
      children:
        - self: {id: 1100, title: 'About', slug: '/about'}
          entities:
            content:
              - self: {id: 2100, title: 'About us'}
        - self: {id: 1200, title: 'Contact', slug: '/contact'}
    - self: {id: 1900, title: 'Storage', type: 254, slug: '/storage'}
      entities:
        category:
          - self: {id: 3000, title: 'News'}

entitySettings:
  '*':
    nodeColumnName: 'pid'
    columnNames: {id: 'uid', language: 'sys_language_uid'}
    defaultValues: {pid: 0}
  page:
    isNode: true
    tableName: 'pages'
    parentColumnName: 'pid'
    languageColumnNames: ['l10n_parent', 'l10n_source']
    columnNames: {type: 'doktype', root: 'is_siteroot'}
    defaultValues: {hidden: 0, doktype: 1}
  content:
    tableName: 'tt_content'
    languageColumnNames: ['l18n_parent', 'l10n_source']
    columnNames: {title: 'header', type: 'CType'}
    defaultValues: {hidden: 0, CType: 'text', colPos: 0}

entities:
  page:
    - self: {id: 1000, title: 'EN: Demo', root: true, slug: '/'}
      children:
        - self: {id: 1100, title: 'EN: About', slug: '/about'}
          languageVariants:
            - self: {id: 1101, title: 'DE: Über uns', language: 1, slug: '/ueber-uns'}
            - self: {id: 1102, title: 'FR: À propos', language: 2, slug: '/a-propos'}
          entities:
            content:
              - self: {id: 2100, title: 'EN: About us'}
                languageVariants:
                  - self: {id: 2101, title: 'DE: Über uns', language: 1}

entitySettings:
  workspace:
    tableName: 'sys_workspace'

entities:
  workspace:
    - self: {id: 1, title: 'Draft'}
  page:
    - self: {id: 1000, title: 'Demo', root: true, slug: '/'}
      children:
        - version: {id: 1300, title: 'EN: Coming soon', slug: '/soon', workspace: 1}

entities:
  page:
    - self: {id: 1100, title: 'EN: About', slug: '/about'}
      versionVariants:
        - version: {title: 'EN: About us, revised', workspace: 1}

entities:
  page:
    - self: {id: 1000, title: 'Demo', root: true, slug: '/'}
      children:
        - self: {id: 1100, title: 'About', slug: '/about'}
        - self: {id: 1200, title: 'Archive', slug: '/archive'}
          versionVariants:
            # In workspace 1 the page was moved below "About".
            - version: {workspace: 1}
              actions:
                - {action: 'move', type: 'toPage', target: 1100}
        - self: {id: 1300, title: 'Draft only', slug: '/draft'}
          versionVariants:
            # In workspace 1 the page was deleted.
            - version: {workspace: 1}
              actions:
                - {action: 'delete'}

Declared uids, and what makes a seed reproducible 

Every record of a scenario has a uid before it is written. Either the item declares one as id, or the seeder hands out a dynamic one from 10000 upwards. Both are suggested to DataHandler, so a seeded page tree is the same page tree in every installation - a site configuration, a TypoScript condition or a test may refer to a page by its number.

Declare the uid of everything that is referred to from outside the set: a site root, a shortcut target, a storage folder. Let the rest be dynamic.

TYPO3 treats a suggested uid as a suggestion. If the uid is already used in its table, the record is not written elsewhere - the insert fails. The import therefore checks every suggested uid up front and refuses when one is taken, naming the records in the way:

[ERROR] The seed set "demo" suggests 2 uids this installation already uses.

 ---------- ----- -------------------
  Table      Uid   Occupied by
 ---------- ----- -------------------
  pages      1000  Company site
  pages      1100  Products
 ---------- ----- -------------------

Nothing is written in that case. --force imports anyway: every record of a table something collides in is written with a free uid instead of the suggested one, a table nothing collides in keeps its uids, and nothing that is in the way is deleted or changed.

A deleted record occupies its uid as much as any other one - the row is still there.

Suggested uids are honoured for an administrator backend user only. DataHandler ignores them silently for anybody else, so the import refuses to run as a non-admin rather than write a set whose uids are not the ones it declares.

Relations between records 

A relation needs no key of its own, in config.yml or anywhere else. Because every record has a uid before it is written, the record holding the relation can name the records on the other end of it by writing their uids into its relation field, exactly as a backend form submits them:

entitySettings:
  '*':
    nodeColumnName: 'pid'
    columnNames: {id: 'uid'}
    defaultValues: {pid: 0, hidden: 0}
  page:
    isNode: true
    tableName: 'pages'
    parentColumnName: 'pid'
    defaultValues: {doktype: 1}
  content:
    tableName: 'tt_content'
    columnNames: {title: 'header', type: 'CType'}
  item:
    tableName: 'tx_myext_item'

entities:
  page:
    - self: {id: 1, title: 'Root', slug: '/', is_siteroot: 1}
      entities:
        content:
          - self: {id: 21, title: 'List', type: 'my_itemlist', tx_myext_items: '32,31'}
        item:
          - self: {id: 31, title: 'One'}
          - self: {id: 32, title: 'Two'}

That is the whole declaration. The relation itself is described where it is always described, in the TCA of the field:

'tx_myext_items' => [
    'config' => [
        'type' => 'inline',
        'foreign_table' => 'tx_myext_item',
        'foreign_field' => 'parentid',
        'foreign_table_field' => 'parenttable',
        'foreign_sortby' => 'sorting_foreign',
    ],
],

The children are an entity of their own in the same scenario, and the parent declares its relation field as the comma separated list of the ids they declare. Nothing else is needed: DataHandler resolves that list like any other relation and writes the columns tying a child to its parent by itself. Which columns those are is read from the TCA of the parent field - its foreign_field, foreign_table_field and foreign_sortby, which are parentid, parenttable and sorting_foreign above. A scenario therefore never names them, and a relation whose TCA calls them something else needs nothing different here. The relation field of the parent ends up holding the number of children, the way TYPO3 stores an inline relation.

Two consequences are worth stating, because they are what an integrator will actually rely on:

• The order of the relation is the order of the declared list, not the order of the uids. The example writes '32,31', and item 32 comes first.
• It works more than one level deep. A child that is itself the parent of a relation declares its own list the same way, and the second level is resolved from the TCA of the child's field. Nothing about it is special cased.

The children live on the page the scenario declares them under, like every other record: the relation ties them to the parent record, not to its page.

Files 

files:
  - identifier: placeholder             # required, unique among the files
    source: 'Files/placeholder.svg'     # required; relative to the set, or EXT:
    folder: 'demo'                      # optional, default the storage root
    name: 'placeholder.svg'             # optional, default the source basename
    storage: 1                          # optional, default the default storage

This list is closed as well: any other key on a file is refused, naming the known ones - foldr: instead of folder: would otherwise put the file in the storage root and report success.

Files are copied before the records are written, through the file storage API, which is what indexes them - a file copied into fileadmin/ by hand exists on disk and does not exist for TYPO3. A missing target folder is created, an existing file of the same name is replaced, and the source file in the extension is left where it is.

Placing a file is one thing and attaching it to a record is another, and the second is what references below does.

File references 

references:
  - file: placeholder                 # required, an identifier declared under "files"
    table: tt_content                 # required, the table of the record it hangs on
    uid: 2000                         # required, the uid the scenario declares as "id"
    field: assets                     # required, the file relation column of that record
    values:                           # optional, the fields of the sys_file_reference row
      title: 'A placeholder'
      alternative: 'Nothing to see here'

This list is closed as well: any other key on a reference is refused, naming the known ones.

A scenario record carries no symbolic name, so a reference names its record by uid - the same rule rootPage follows, and for the same reason. That uid is resolved against what the run actually wrote rather than trusted. A reference naming a record no entity of the scenario declares is refused before anything is written:

[ERROR] The seed set "demo" declares a file reference to "placeholder" on the
        record tt_content:2001, which no entity of its scenario declares as
        its "id".

Five columns of the sys_file_reference row are structural and are written by the seeder: uid_local, uid_foreign, tablenames, fieldname and pid. Declaring one of them under values does not change it - the seeder's value wins, because a definition may not detach a reference from the record it declares it on. pid follows the convention TYPO3 uses for a file relation: a reference belongs to the page its record is on, and for a record that is a page, to that page itself.

Several references on one field come out in the order they are declared, which is what sorting_foreign is written from - so a multi image field is a gallery in the declared order rather than in whatever order the database returns. The same file may be referenced from several records, or from none.

References are written in a pass of their own, through DataHandler, after every record of the set exists. That is what puts the relation into the reference index, and it is why a mistyped uid is caught up front rather than after the whole tree has been written.

identifier: demo
title: 'A content element with two images'

scenarios:
  - Scenario.yaml

files:
  - identifier: landscape
    source: 'Files/landscape.jpg'
    folder: 'demo'
  - identifier: portrait
    source: 'Files/portrait.jpg'
    folder: 'demo'

references:
  - file: landscape
    table: tt_content
    uid: 2000
    field: assets
    values: {title: 'Landscape', alternative: 'The wide one'}
  - file: portrait
    table: tt_content
    uid: 2000
    field: assets
  - file: landscape
    table: pages
    uid: 1000
    field: media

entitySettings:
  '*':
    nodeColumnName: 'pid'
    columnNames: {id: 'uid'}
    defaultValues: {pid: 0, hidden: 0}
  page:
    isNode: true
    tableName: 'pages'
    parentColumnName: 'pid'
    defaultValues: {doktype: 1}
  content:
    tableName: 'tt_content'
    columnNames: {title: 'header', type: 'CType'}

entities:
  page:
    - self: {id: 1000, title: 'Demo', slug: '/', is_siteroot: 1}
      entities:
        content:
          - self: {id: 2000, title: 'Teaser', type: 'textmedia'}

Site configurations 

sites:
  - identifier: main                    # required, the directory in config/sites/
    rootPage: 1000                      # required, the uid of a seeded page
    template: 'Sites/main'              # optional, default Sites/<identifier>
    base: 'https://example.com/'        # optional, overrides the template

A site configuration is written from a template: a directory holding a config.yaml and optionally a settings.yaml, which is exactly the shape of a site below config/sites/. A template is therefore produced by copying a working site out of an installation.

This list is closed too: a site is configuration rather than a record, so nothing on it is written verbatim and an unknown key can only be a mistake.

A scenario record carries no symbolic name, so its stable handle is the uid it declares - which is why rootPage is a number here and not a name. A site naming a uid that no pages entity of the scenario declares is refused before anything is written, rather than after the whole tree exists.

Three rules apply to the result:

• The root page always wins. rootPageId is taken from the uid the declared page was actually written with, whatever the template says.
• An existing site identifier is refused. TYPO3 merges an incoming site configuration into an existing one, so seeding over it would produce neither the template nor the previous configuration but a mixture of both. Remove the site first if the seed is meant to replace it.
• A minimal template needs almost nothing. base and languages are worth declaring, because their defaults are a site on / in "Default / en_US.UTF-8". dependencies - the site sets a site pulls in - is written unchanged, and is a TYPO3 v13 key: TYPO3 v12 has no site sets, so a template declaring it seeds a configuration whose dependencies v12 simply ignores.

Placeholders such as %env(...)% inside a template are not resolved while seeding. They are written as they stand, so the installation resolves them every time it reads the file - which is what they are for.

A complete seed set 

Everything above, in one set. It is the set this extension's own functional tests import, and a test compares the three files printed here against the files on disk - so what is shown is what runs.

packages/my_extension/Configuration/DataFactory/complete/
├── config.yml
├── Scenario.yaml
├── Files/
│   └── placeholder.svg
└── Sites/
    └── main/
        └── config.yaml

identifier: import-documented
title: 'A complete seed set'
description: 'Pages with content and translations, a file, a file reference and a site.'

scenarios:
  - Scenario.yaml

files:
  - identifier: placeholder
    source: 'Files/placeholder.svg'
    folder: 'documented'

references:
  - file: placeholder
    table: pages
    uid: 1000
    field: media
    values:
      title: 'The placeholder'
      alternative: 'A grey rectangle'

sites:
  - identifier: documented-main
    rootPage: 1000
    template: Sites/main

entitySettings:
  '*':
    nodeColumnName: 'pid'
    columnNames: {id: 'uid', language: 'sys_language_uid'}
    defaultValues: {pid: 0, hidden: 0}
  page:
    isNode: true
    tableName: 'pages'
    parentColumnName: 'pid'
    languageColumnNames: ['l10n_parent', 'l10n_source']
    defaultValues: {doktype: 1}
  content:
    tableName: 'tt_content'
    languageColumnNames: ['l18n_parent', 'l10n_source']
    columnNames: {title: 'header', type: 'CType'}

entities:
  page:
    - self: {id: 1000, title: 'EN: Demo', slug: '/', is_siteroot: 1}
      children:
        - self: {id: 1100, title: 'EN: About', slug: '/about'}
          languageVariants:
            - self: {id: 1101, title: 'DE: Über uns', language: 1, slug: '/ueber-uns'}
          entities:
            content:
              - self: {id: 2100, title: 'EN: About us', type: 'header'}
                languageVariants:
                  - self: {id: 2101, title: 'DE: Über uns', language: 1, type: 'header'}

rootPageId: 0
base: 'https://documented.example.org/'
languages:
  -
    title: English
    enabled: true
    languageId: 0
    base: /
    locale: en_US.UTF-8
  -
    title: Deutsch
    enabled: true
    languageId: 1
    base: /de/
    locale: de_DE.UTF-8
errorHandling: []
routes: []

Importing it with vendor/bin/typo3 data-factory:import import-documented writes, in this order:

• the file placeholder.svg into fileadmin/documented/, indexed as a sys_file;
• the pages 1000, 1100 and its translation 1101, and the content elements 2100 and its translation 2101, each with the uid it declares;
• a sys_file_reference attaching the file to pages.media of page 1000, carrying the declared title and alternative;
• the site config/sites/documented-main/config.yaml, from the template, with rootPageId replaced by 1000.

Two details of the example are deliberate and worth copying. The entities declare nodeColumnName, which is what puts the translated page onto the page its original sits on rather than out of the tree, and hidden sits on the '*' entry while doktype sits on page - a key declared on both sides is merged into a list, not overridden.

Workspaces are the one construct the example leaves out. They need a sys_workspace record and have a section of their own; a set that seeds a workspace is not what a first set looks like.

Splitting a set over several files 

A set splits in two independent ways, and they are not interchangeable.

imports:
  - { resource: Files.yaml }
  - { resource: 'EXT:my_extension/Configuration/DataFactory/shared/Sites.yaml' }

scenarios:
  - Scenario/Pages.yaml
  - Scenario/Content.yaml
  - 'EXT:my_extension/Configuration/DataFactory/shared/Categories.yaml'

What this version does not do 

Two boundaries of the format, named here rather than left to be discovered:

• A file reference reaches only the records of its own set. The uid of a references entry is resolved against what the run writes, so it cannot attach a file to a record that is already in the installation. Naming one is refused as an undeclared record, not answered with a database lookup.
• A file reference cannot be declared in a scenario file. The scenario format has no concept of a file, and this extension does not give it one - that is the point of references living in config.yml. A scenario file of a seed set stays a file that could be lifted into a functional test unchanged.

The commands 

vendor/bin/typo3 data-factory:list
vendor/bin/typo3 data-factory:list -v

vendor/bin/typo3 data-factory:import demo
vendor/bin/typo3 data-factory:import demo --dry-run -v
vendor/bin/typo3 data-factory:import demo --base='https://example.com/'

What the seeder writes by itself 

Everything else in a record comes from its self or version, from the defaultValues of its entity, or from the TCA of the installation where neither declares anything. This extension adds no default of its own - in particular, a page is created hidden unless the scenario says otherwise, because pages is the one table whose TCA sets 'hidden' => ['config' => ['default' => 1]]. tt_content and the other tables of an installation default to 0. That is why every example above declares defaultValues: {hidden: 0}: it costs nothing on the tables that already default to 0, and it is what keeps a seeded page tree visible.

The same holds for doktype, l10n_parent and sys_language_uid - they come from the TCA of the installation unless the scenario declares them, so a set that wants the same records in two installations declares them.

See also 

• Introduction
• Installation
• Changelog