Data compiling
This is the first step of FormEngine. The data compiling creates an array containing all data the rendering needs to come up with a result.
A basic call looks like this:
$formDataGroup = GeneralUtility::makeInstance(TcaDatabaseRecord::class);
$formDataCompiler = GeneralUtility::makeInstance(FormDataCompiler::class);
$formDataCompilerInput = [
'request' => $request, // the PSR-7 request object
'tableName' => $table,
'vanillaUid' => (int)$theUid,
'command' => $command,
];
$formData = $formDataCompiler->compile($formDataCompilerInput, $formDataGroup);
Deprecated since version 12.4
Using the FormEngine data provider to render records without passing the
current request object is deprecated. Additionally,
when instantiating the backend FormEngine-related Form
,
the constructor argument Form
should be omitted,
the form data group should be provided as second argument to compile
instead. Failing to do so will stop working with TYPO3 v13.
The above code is a simplified version of the relevant part of the Edit
. This controller
knows by its GET
or POST
parameters which record ("vanillaUid") of which specific table ("tableName")
should be edited (command="edit") or created (command="new"), and sets this as init data to the DataCompiler. The
controller also knows that it should render a full database record and not only parts of it, so it uses the
Tca
data provider group to trigger all data providers relevant for this case. By calling ->compile
on this data group, all providers configured for this group are called after each other, and form
ends up
with a huge array of data record details.
So, what happens here in detail?
- Variable
$form
maps input values to keys specified byData Compiler Input Form
as "init" data.Data Compiler - FormDataCompiler returns a unified array of data. This array is enriched by single data providers.
- A data provider group is a list of single data providers for a specific scope and enriches the array with information.
- Each data provider is called by the DataGroup to add or change data in the array.
The variable $form
roughly consists of this data after calling $form
:
- A validated and initialized list of current database row field variables.
- A processed version of
$TCA
containing only those column fields a current user has access to.['given Table'] - A processed list of items for single fields like select and group types.
- A list of relevant localizations.
- Information of expanded
inline
record details if needed. - Resolved flex form data structures and data.
- A lot more
Basic goal of this step is to create an array in a specified format with all data needed by the render-part of FormEngine.
A controller initializes this with init data, and then lets single data providers fetch additional data and write it
to the main array. The deal is here that the data within that array is not structured in an arbitrary way, and each single
data provider only adds data the render part of FormEngine understands and needs later. This is why the main array keys are restricted:
The main array is initialized by Form
, and each Data
can only add data to sub-parts of that array.
Note
The main data array is prepared by Form
, each key is well documented in this class. To find out
which data is expected to reside in this array, those comments are worth a look.
Data Groups and Providers
So we have this empty data array, pre-set with data by a controller and then initialized by Form
,
which in turn hands over the data array to a specific Form
. What are these data providers now? Data providers are
single classes that add or change data within the data array. They are called in a chain after each other. A Form
has the responsibility to find out, which specific single data providers should be used, and calls them in a specific order.
Why do we need this?
- Which data providers are relevant depends on the specific scope: For instance, if editing a full database based record,
one provider fetches the according row from the database and initializes
$data
. But if flex form data is calculated, the flex form values are fetched from table fields directly. So, while the['database Row'] Database
data provider is needed in the first case, it's not needed or even counter productive in the second case. TheEdit Row Form
's are used to manage providers for specific scopes.Data Group - FormDataGroups know which providers should be used in a specific scope. They usually fetch a list of providers from some global configuration array. Extensions can add own providers to this configuration array for further data munging.
- Single data providers have dependencies to each other and must be executed in a specific order. For instance, the
page TSconfig of a record can only be determined, if the rootline of a record has been determined, which can only happen
after the pid of a given record has been consolidated, which relies on the record being fetched from the database.
This makes data providers a linked list and it is the task of a
Form
to manage the correct order.Data Group
Main data groups:
- TcaDatabaseRecord
- List of providers used if rendering a database based record.
- FlexFormSegment
- List of data providers used to prepare flex form data and flex form section container data.
- TcaInputPlaceholderRecord
- List of data providers used to prepare placeholder values for
type=input
andtype=text
fields. - InlineParentRecord
- List of data providers used to prepare data needed if an inline record is opened from within an Ajax call.
- OnTheFly
- A special data group that can be initialized with a list of to-execute data providers directly. In contrast to the others, it does not resort the data provider list by its dependencies and does not fetch the list of data providers from a global config. Used in the Core at a couple of places, where a small number of data providers should be called right away without being extensible.
Note
It is a good idea to set a breakpoint at the form data result returned by the DataCompiler and to have a look at the data array to get an idea of what this array contains after compiling.
Let's have a closer look at the data providers. The main Tca
group consists mostly of three parts:
Main record data and dependencies:
- Fetch record from DB or initialize a new row depending on
$data
being "new" or "edit", set row as['command'] $data
['database Row'] - Add user TSconfig and page TSconfig to data array
- Add table TCA as
$data
['processed Tca'] - Determine record type value
- Fetch record translations and other details and add to data array
Single field processing:
- Process values and items of simple types like
type=input
,type=radio
,type=check
and so on. Validate theirdatabase
values and validate and sanitize theirRow processed
settings.Tca - Process more complex types that may have relations to other tables like
type=group
andtype=select
, set possible selectable items in$data
of the according fields, sanitize their TCA settings.['processed Tca'] - Process
type=inline
andtype=flex
fields and prepare their child fields by using new instances ofForm
and adding their results toData Compiler $data
.['processed Tca']
Post process after single field values are prepared:
- Execute display conditions and remove fields from
$data
that shouldn't be shown.['processed Tca'] - Determine main record title and set as
$data
['record Title']
Extending Data Groups With Own Providers
The base set of DataProviders for all DataGroups is defined within typo3/
in section ['SYS']
, and ends up in variable $GLOBALS
after Core
bootstrap. The provider list can be read top-down, so the Dependency
typically does not resort this
list to a different order.
Adding an own provider to this list means adding an array key to that array having a specification where the new data provider
should be added in the list. This is done by the arrays depends
and before
.
As an example, the extension "news" used an own data provider in
a past version
to do additional flex form data structure preparation. The Core internal
flex preparation is already split into two providers: Tca
determines the data structure and parses
it, Tca
uses the prepared data structure, processes values and applies defaults if needed. The data provider
from the extension "news" hooks in between these two to add some own preparation stuff. The registration happens with this
code in ext_
:
<?php
declare(strict_types=1);
use GeorgRinger\News\Backend\FormDataProvider\NewsFlexFormManipulation;
use TYPO3\CMS\Backend\Form\FormDataProvider\TcaFlexPrepare;
use TYPO3\CMS\Backend\Form\FormDataProvider\TcaFlexProcess;
defined('TYPO3') or die();
// Inject a data provider between TcaFlexPrepare and TcaFlexProcess
$GLOBALS['TYPO3_CONF_VARS']['SYS']['formEngine']['formDataGroup']['tcaDatabaseRecord'][NewsFlexFormManipulation::class] = [
'depends' => [
TcaFlexPrepare::class,
],
'before' => [
TcaFlexProcess::class,
],
];
This is pretty powerful since it allows extensions to hook in additional stuff at any point of the processing chain, and it does not depend on the load order of extensions.
Limitations:
- It is not easily possible to "kick out" an existing provider if other providers have dependencies to them - which is usually the case.
- It is not easily possible to substitute an existing provider with an own one.
Note
It may happen that the Core splits or deletes the one or the other DataProvider in the future. If then an extension
has a dependency to a removed provider, the Dependency
, which takes care of the sorting, throws
an exception. There is currently no good solution in the Core on how to mitigate this issue.
Note
Data providers in general should not know about render
, but only about type
. Their goal is to prepare
and sanitize data independent of a specific render
. At the moment, the Core data provider just has one
or two places, where specific render
's are taken into account to process data, and those show that these areas
are a technical debt that should be changed.
Adding Data to Data Array
Most custom data providers change or add existing data within the main data array. A typical use case is an additional
record initialization for specific fields in $data
or additional items somewhere within
$data
. The main data array is documented in Form
.
Sometimes, own DataProviders need to add additional data that does not fit into existing places. In those cases they
can add stuff to $data
. This key is not filled with data by Core DataProviders and serves as a place
for extensions to add things. Those data components can be used in own code parts of the rendering later. It is advisable
to prefix own data in $data
with some unique key (for instance the extension name) to not collide
with other data that a different extension may add.
Disable Single FormEngine Data Provider
Single data providers used in the FormEngine data compilation step can be disabled to allow extension authors to substitute existing data providers with their solutions.
As an example, if editing a full database record, the default Tca
could be removed by setting
disabled
in the tca
group in an extension's ext_
file:
<?php
declare(strict_types=1);
use TYPO3\CMS\Backend\Form\FormDataProvider\TcaCheckboxItems;
defined('TYPO3') or die();
$GLOBALS['TYPO3_CONF_VARS']['SYS']['formEngine']['formDataGroup']
['tcaDatabaseRecord'][TcaCheckboxItems::class]['disabled'] = true;
Extension authors can then add an own data provider, which depends
on the disabled one and is configured as before
the
next one. Therefore effectively substituting single providers with their
solution if needed.