Custom process steps¶

Besides all the events, it is also possible to register custom process steps. How to register a custom step is covered in the Administration chapter. This section describes what a custom step can or should do and what resources are available from within a custom step class.

Parent class¶

A custom step class must inherit from abstract class \Cobweb\ExternalImport\Step\AbstractStep. If it does not, the step will be ignored during import. The parent class makes a lot of features available some of which are described below.

If you want to use Dependency Injection in your custom step class, just remember to declare it as being public in your service configuration file.

Available resources¶

A custom step class has access to the following member variables:

data: Instance of the object model encapsulating the data being processed (\Cobweb\ExternalImport\Domain\Model\Data).
importer: Back-reference to the current instance of the \Cobweb\ExternalImport\Importer class.
parameters: Array of parameters declared in the configuration of the custom step.

See the API chapter for more information about these classes.

Furthermore, the custom step class can access a member variable called abortFlag. Setting this variable to true will cause the import process to be aborted after the custom step. Any such interruption is logged by the \Cobweb\ExternalImport\Importer class, albeit without any detail. If you feel the need to report about the reason for interruption, do so from within the custom step class:

$this->getImporter()->addMessage(
        'Your message here...',
        FlashMessage::WARNING // or whatever error level
);

In general, use the getters and setters to access the member variables.

Custom step basics¶

A custom step class must implement the run() method. This method receives no arguments and returns nothing. All interactions with the process happens via the member variables described above and their API.

The main reason to introduce a custom step is to manipulate the data being processed. To read the data, use:

// Read the raw data or...
$rawData = $this->getData()->getRawData();
// Read the processed data
$records = $this->getData()->getRecords();

Note

Depending on when you custom step happens, there may not yet be any raw nor processed data available.

If you manipulate the data, you need to store it explicitely:

// Store the raw data or...
$this->getData()->setRawData();
// Store the processed data
$this->getData()->setRecords();

Another typical usage would be to interrupt the process entirely by setting the abortFlag variable to true, as mentioned above.

The rich API that is available makes it possible to do many things beyond these. For example, one could imagine changing the External Import configuration on the fly.

In general the many existing Step classes provide many examples of API usage and should help when creating a custom process step.

Preview mode¶

It is very important that your custom step respects the preview mode. This has two implications:

If relevant, you should return some preview data. For example, the TransformDataStep class returns the import data once transformations have been applied to it, the StoreDataStep class returns the TCE structure, and so on. There's an API for returning preview data:
```
$this->getImporter()->setPreviewData(...);
```
The preview data can be of any type.
Most importantly, you must respect the preview mode and not make any persistent changes, like saving stuff to the database. Use the API to know whether preview mode is on or not:
```
$this->getImporter()->isPreview();
```

Example¶

Finally here is a short example of a custom step class. Note how the API is used to retrieve the list of records (processed data), which is looped over and then saved again to the Data object.

In this example, the "name" field of every record is used to filter acceptable entries.

Warning

Note the call to array_values() to compact the array again once records have been removed. This is very important to avoid having empty entries in your import.

This used to be done automatically when using hooks "preprocessRawRecordset" and "preprocessRecordset". When replacing these hooks by custom steps, make sure to call array_values() if needed.

<?php

declare(strict_types=1);

namespace Cobweb\ExternalimportTest\Step;

use Cobweb\ExternalImport\Step\AbstractStep;

/**
 * Class demonstrating how to use custom steps for external import.
 *
 * @package Cobweb\ExternalimportTest\Step
 */
class TagsPreprocessorStep extends AbstractStep
{

    /**
     * Filters out some records from the raw data for the tags table.
     *
     * Any name containing an asterisk is considered censored and thus removed.
     */
    public function run(): void
    {
        $records = $this->getData()->getRecords();
        foreach ($records as $index => $record) {
            if (strpos($record['name'], '*') !== false) {
                unset($records[$index]);
            }
        }
        $records = array_values($records);
        $this->getData()->setRecords($records);
        // Set the filtered records as preview data
        $this->importer->setPreviewData($records);
    }
}