General TCA configuration¶
Here is an example of a typical “ctrl” section syntax:
$extensionPath = \TYPO3\CMS\Core\Utility\ExtensionManagementUtility::extPath('externalimport_tut');
$GLOBALS['TCA']['tx_externalimporttut_departments'] = array(
'ctrl' => array(
'title' => 'LLL:EXT:externalimport_tut/locallang_db.xml:tx_externalimporttut_departments',
...
'external' => array(
0 => array(
'connector' => 'csv',
'parameters' => array(
'filename' => $extensionPath . 'res/departments.txt',
'delimiter' => "\t",
'text_qualifier' => '"',
'skip_rows' => 1,
'encoding' => 'latin1'
),
'data' => 'array',
'referenceUid' => 'code',
'priority' => 10,
'description' => 'Import of all company departments'
)
)
),
);
The external
property is an indexed array. The available properties
are described below.
Properties¶
Property | Data type | Scope/Step |
---|---|---|
additionalFields | string | Read data |
clearCache | string | Clear cache |
connector | string | Read data |
customSteps | array | Any step |
data | string | Read data |
dataHandler | string | Handle data |
description | string | Display |
disabledOperations | string | Store data |
disableLog | boolean | Store data |
enforcePid | boolean | Store data |
group | string | Sync process |
minimumRecords | integer | Validate data |
namespaces | array | Handle data (XML) |
nodetype | string | Handle data (XML) |
parameters | array | Read data |
pid | integer | Store data |
priority | integer | Display/automated import |
referenceUid | string | Store data |
useColumnIndex | string or integer | Configuration |
whereClause | string | Store data |
connector¶
- Type
- string
- Description
Connector service subtype.
Must be defined only for pulling data. Leave blank for pushing data. You will need to install the relevant connector extension. Here is a list of available extensions and their corresponding types:
Type Extension csv svconnector_csv json svconnector_json sql svconnector_sql feed svconnector_feed - Scope
- Read data
parameters¶
- Type
- array
- Description
Array of parameters that must be passed to the connector service.
Not used when pushing data.
- Scope
- Read data
data¶
- Type
- string
- Description
- The format in which the data is returned by the connector service. Can
be either
xml
orarray
. - Scope
- Read data
dataHandler¶
- Type
- string
- Description
- A class name for replacing the standard data handlers. See the Developer’s Guide for more details.
- Scope
- Handle data
group¶
- Type
- string
- Description
- This can be any arbitrary string of characters. All External Import configurations having the same value for the “group” property will form a group of configurations. It is then possible to execute the synchronization of all configurations in the group in one go, in order of priority. Group synchronization is available on the command line and in the Scheduler task.
- Scope
- Sync process
nodetype¶
- Type
- string
- Description
- Name of the reference nodes inside the XML structure, i.e. the children of these nodes correspond to the data that goes into the database fields (see also the description of the field attribute).
- Scope
- Handle data (XML)
referenceUid¶
- Type
- string
- Description
Name of the column where the equivalent of a primary key for the external data is stored.
Important
This is the name of a field in the TYPO3 CMS database, not in the external data! It is the field where the reference (or primary) key of the external data is stored.
- Scope
- Store data
priority¶
- Type
- integer
- Description
A level of priority for execution of the synchronization. Some tables may need to be synchronized before others if foreign relations are to be established. This gives a clue to the user and a strict order for scheduled synchronizations.
Not used when pushing data.
- Scope
- Display/Automated import process
pid¶
- Type
- string
- Description
- ID of the page where the imported records should be stored. Can be ignored and the general storage pid is used instead (see Configuration).
- Scope
- Store data
enforcePid¶
- Type
- boolean
- Description
If this is set to true, all operations regarding existing records will be limited to records stored in the defined pid (i.e. either the above property or the general extension configuration). This has two consequences:
- when checking for existing records, those records will be selected only from the defined pid.
- when checking for records to delete, only records from the defined pid will be affected
This is a convenient way of protecting records from operations started from within the external import process, so that it won’t affect e.g. records created manually.
- Scope
- Store data
useColumnIndex¶
- Type
- string or integer
- Description
In a basic configuration the same index must be used for the general TCA configuration and for each column configuration. With this property it is possible to use a different index for the column configurations. The “ctrl” part has to exist with its own index, but the columns may refer to another index and thus their configuration does not need to be defined. Obviously the index referred to must exist for columns.
The type may be a string or an integer, because a configuration key may also be either a string or an integer.
- Scope
- Configuration
customSteps¶
- Type
- array
- Description
As explained in the process overview, the import process goes through several steps, depending on its type. This property makes it possible to register additional steps. Each step can be placed before or after any existing step (including previously registered custom steps).
The configuration is a simple array, each entry being itself an array with two properties: “class” referring to the PHP class containing the custom step code and “position” stating when the new step should happen. The syntax for position is made of the keyword
before
orafter
, followed by a colon (:
) and the name of an existing step class.Example:
'customSteps' => array( array( 'class' => \Cobweb\ExternalimportTest\Step\EnhanceDataStep::class, 'position' => 'after:' . \Cobweb\ExternalImport\Step\ValidateDataStep::class ) ),
If any element of the custom step declaration is invalid, the step will be ignored. More information is given in the Developer’s Guide.
- Scope
- Any step
whereClause¶
- Type
- string
- Description
SQL condition that will restrict the records considered during the import process. Only records matching the condition will be updated or deleted. This condition comes on top of the “enforcePid” condition, if defined.
Warning
This may cause many records to be inserted over time. Indeed if some external data is imported the first time, but then doesn’t match the
whereClause
condition, it will never be found for update. It will thus be inserted again and again. Whenever you make use of thewhereClause
property you should therefore watch for an unexpectedly high number of inserts.- Scope
- Store data
additionalFields¶
- Type
- string
- Description
Comma-separated list of fields from the external source that should be made available during the import process, but that will not be stored in the internal table.
This is usually the case for fields which you want to use in the transformation step, but that will not be stored eventually.
- Scope
- Read data
namespaces¶
- Type
- array
- Description
Associative array of namespaces that can be used in XPath queries. The keys correspond to prefixes and the values to URIs. The prefixes can then be used in XPath queries.
Example
Given the following declaration:
'namespaces' => array( 'atom' => 'http://www.w3.org/2005/Atom' )
a Xpath query like:
atom:link
could be used. The prefixes used for XPath queries don’t need to match the prefixes used in the actual XML source. The defaut namespace has to be registered too in order for XPath queries to succeed.
- Scope
- Handle data (XML)
description¶
- Type
- string
- Description
- A purely descriptive piece of text, which should help you remember what this particular synchronization is all about. Particularly useful when a table is synchronized with multiple sources.
- Scope
- Display
disabledOperations¶
- Type
- string
- Description
Comma-separated list of operations that should not be performed. Possible operations are insert, update and delete. This way you can block any of these operations.
- insert
- The operation performed when new records are found in the external source.
- update
- Performed when a record already exists and only its data needs to be updated.
- delete
- Performed when a record is in the database, but is not found in the external source anymore.
See also the column-specific property disabledOperations.
- Scope
- Store data
minimumRecords¶
- Type
- integer
- Description
- Minimum number of items expected in the external data. If fewer items are present, the import is aborted. This can be used – for example – to protect the existing data against deletion when the fetching of the external data failed (in which case there are no items to import).
- Scope
- Validate data
disableLog¶
- Type
- integer
- Description
- Set to
true
to disable logging by TCEmain. This setting will override the general “Disable logging” setting (see Configuration for more details). - Scope
- Store data
clearCache¶
- Type
- string
- Description
- Comma-separated list of caches identifiers for caches which should be cleared at the end of the import process. See Clearing the cache.
- Scope
- Clear cache