General TCA configuration¶

Connector service subtype.

Must be defined only for pulling data. Leave blank for pushing data. You will need to install the relevant connector extension. Here is a list of available extensions and their corresponding types:

Array of parameters that must be passed to the connector service.

Not used when pushing data.

Pointer to a sub-array inside the incoming external data, as a list of keys separated by some marker. The sub-array pointed to will be used as the source of data in the subsenquent steps, rather than the whole structure that was read during the ReadDataStep.

For more details on usage and available options, see the dedicated page.

When the special * segment is used in an arrayPath, the resulting structure is always an array. If the arrayPath target is actually a single value, this may not be desirable. When arrayPathFlatten is set to true, the result is preserved as a simple type.

Name of the column where the equivalent of a primary key for the external data is stored.

Records for which this data does not exist are skipped (since version 6.1). This is tested with PHP's isset() function. If you think your data may contain empty values and you wish to skip them too, use the isEmpty transformation property with the invalidate option set to true.

A level of priority for the execution of the synchronization. Some tables may need to be synchronized before others if foreign relations are to be established. This gives a clue to the user and a strict order for scheduled synchronizations (either when synchronizing all configurations or when synchronizing a group).

The lowest priority value goes first.

If priority is not defined, a default value of 1000 is applied (defined by class constant \Cobweb\ExternalImport\Importer::DEFAULT_PRIORITY).

Not used when pushing data.

If this is set to true, all operations regarding existing records will be limited to records stored in the defined pid (i.e. either the above property or the general extension configuration). This has two consequences:

1. when checking for existing records, those records will be selected only from the defined pid.
2. when checking for records to delete, only records from the defined pid will be affected

This is a convenient way of protecting records from operations started from within the external import process, so that it won't affect e.g. records created manually.

In a basic configuration the same index must be used for the general TCA configuration and for each column configuration. With this property it is possible to use a different index for the column configurations. The general configuration part has to exist with its own index (say "index A"), but the columns may refer to another index (say "index B") and thus their configuration does not need to be defined. Obviously the index referred to ("index B") must exist for columns.

The type may be a string or an integer, because a configuration key may also be either a string or an integer.

Since version 6.1, it is possible to define specific configurations for selected columns using the index from the general configuration ("index A"). It will not be overridden by the configuration corresponding to the index referred to with useColumnIndex property ("index B").

Example:

'stable' => [
    'connector' => 'feed',
    'parameters' => [
        'uri' => 'EXT:externalimport_test/Resources/Private/ImportData/Test/StableProducts.xml',
        'encoding' => 'utf8'
    ],
    'group' => 'Products',
    'data' => 'xml',
    'nodetype' => 'products',
    'referenceUid' => 'sku',
    'priority' => 5120,
    'useColumnIndex' => 'base',
    ...
],

This general configuration makes reference to the "base" configuration. This means that all columns will use the "base" configuration, unless they have a configuration using specifically the "stable" index. So the "sku" column will use the configuration from the "base" index:

'sku' => [
    'exclude' => false,
    'label' => 'SKU',
    'config' => [
        'type' => 'input',
        'size' => 10
    ],
    'external' => [
        'base' => [
            'xpath' => './self::*[@type="current"]/item',
            'attribute' => 'sku'
        ],
        'products_for_stores' => [
            'field' => 'product'
        ],
        'updated_products' => [
            'field' => 'product_sku'
        ]
    ]
],

However, the "name" column has a specific configuration corresponding to the "stable" index, so it will be used, and not the configuration from the "base" index:

'name' => [
    'exclude' => false,
    'label' => 'Name',
    'config' => [
        'type' => 'input',
        'size' => 30,
        'eval' => 'required,trim',
    ],
    'external' => [
        'base' => [
            'xpath' => './self::*[@type="current"]/item',
        ],
        'stable' => [
            'xpath' => './self::*[@type="current"]/item',
            'transformations' => [
                10 => [
                    'userFunction' => [
                        'class' => \Cobweb\ExternalimportTest\UserFunction\Transformation::class,
                        'method' => 'caseTransformation',
                        'parameters' => [
                            'transformation' => 'upper'
                        ]
                    ]
                ]
            ]
        ],
        'updated_products' => [
            'field' => 'name'
        ]
    ]
],

By default, columns (regular columns or additional fields) are handled in alphabetical order whenever a loop is performed on all columns (typically in the \Cobweb\ExternalImport\Step\TransformDataStep class). This can be an issue when you need a specific column to be handled before another one.

With this property, you can define a comma-separated list of columns, that will be handled in that specific order. It is not necessary to define an order for all columns. If only some columns are explicitly ordered, the rest will be handled after the ordered ones, in alphabetical order. The order is visually reflected in the backend module, when viewing the configuration details.

As explained in the process overview, the import process goes through several steps, depending on its type. This property makes it possible to register additional steps. Each step can be placed before or after any existing step (including previously registered custom steps).

The configuration is a simple array, each entry being itself an array with three properties:

• class (required): name of the PHP class containing the custom step.
• position (required): states when the new step should happen. The syntax for position is made of the keyword before or after, followed by a colon (:) and the name of an existing step class.
• parameters (optional): array which is passed as is to the custom step class when it is called during the import process. Inside the step, it can be accessed using $this->parameters.

Example:

'customSteps' => [
        [
                'class' => \Cobweb\ExternalimportTest\Step\EnhanceDataStep::class,
                'position' => 'after:' . \Cobweb\ExternalImport\Step\ValidateDataStep::class
        ]
],

If any element of the custom step declaration is invalid, the step will be ignored. More information is given in the Developer's Guide.

SQL condition that will restrict the records considered during the import process. Only records matching the condition will be updated or deleted. This condition comes on top of the "enforcePid" condition, if defined.

Associative array of namespaces that can be used in XPath queries. The keys correspond to prefixes and the values to URIs. The prefixes can then be used in XPath queries.

Example

Given the following declaration:

'namespaces' => array(
   'atom' => 'http://www.w3.org/2005/Atom'
)

a Xpath query like:

atom:link

could be used. The prefixes used for XPath queries don't need to match the prefixes used in the actual XML source. The defaut namespace has to be registered too in order for XPath queries to succeed.

Comma-separated list of operations that should not be performed. Possible operations are insert, update and delete. This way you can block any of these operations.

insert

The operation performed when new records are found in the external source.

update

Performed when a record already exists and only its data needs to be updated.

delete

Performed when a record is in the database, but is not found in the external source anymore.

See also the column-specific property disabledOperations.