Introduction

This extension is designed to fetch data from external sources and store them into tables of the TYPO3 CMS database. The mapping between this external data and the TYPO3 CMS tables is done by extending the syntax of the TCA. A backend module provides a way to synchronize any table manually or to define a scheduling for all synchronizations. Synchronizations can also be run using the command-line interface. Automatic scheduling can be defined using a Scheduler task.

The main idea of getting external data into the TYPO3 CMS database is to be able to use TYPO3 CMS standard functions on that data (such as enable fields, for example, if available).

Connection to external applications is handled by a class of services called "connectors", the base of which is available as a separate extension (svconnector).

Data from several external sources can be stored into the same table allowing data aggregation.

The extension also provides an API for sending it data from some other source. This data is stored into the TYPO3 CMS database using the same mapping process as when data is fetched directly by the extension.

This extension is quite flexible, thanks to the possibility of calling user functions to transform incoming data, listening to events to react to some part of the process or adding custom steps at any point in the process. It is also possible to create custom connectors for reading from a specific external source. Still this extension was not designed for extensive data manipulation. It is assumed that the data received from the external source is in a "palatable" format. If the external data requires a lot of processing, it is probably better to put it through an ETL or ESB tool first, and then import it into TYPO3 CMS.

Please also check extension externalimport_tut which provides a tutorial to this extension.

More examples can be found in extension "externalimport_test", which is used for testing purposes. The setup is not documented, but can be interesting to look at. This extension is distributed only via Github: https://github.com/cobwebch/externalimport_test

Alternatives

There exists several extensions for importing data into TYPO3, including the system extension "impexp". Extension "impexp" is specifically designed to export data from a TYPO3 installation and import it again into TYPO3, using a specific file format ("T3D"). When the need is to move around data that is already in a TYPO3 installation, "impexp" is the logical choice. External Import differs by being designed to import data into TYPO3 from a large variety of sources outside TYPO3.

There are other extensions available, like xlsimport and importr, which were released years after External Import and - as such - I never really looked into them since I had all the tools I needed. So it is hard to compare their features.

"xlsimport" is designed for one-time import of data in Excel and CSV format. It cannot be automated. An interface is provided for the mapping configuration, but it cannot be saved. The process is definitely quicker and lighter to set up than External Import, but is limited if you need to import the same data on a regular basis.

"importr" seems to come quite close to External Import in terms of features, although maybe with less flexibility in the data handling and less import sources (import resources can probably be added). It is probably easier to set up than External Import, since it allows for simply pointing to an Extbase model, plus a simple mapping of fields to import.

Questions and support

If you have any questions about this extension, use the dedicated channel in the TYPO3 Slack workspace (#ext-external_import) or the issue tracker on GitHub (https://github.com/cobwebch/external_import/issues).

Please also check the Troubleshooting section in case your issue is already described there.

Keeping the developer happy

Every encouragement keeps the developer ticking, so don't hesitate to send thanks or share your enthusiasm about the extension.

If you appreciate this work and want to show some support, please check https://www.monpetitcoin.com/en/support-me/.

Participating

This tool can be used in a variety of situations and all use cases are certainly not covered by the current version. I will probably not have the time to implement any use case that I don't personally need. However you are welcome to join the development team if you want to bring in new features. If you are interested use GitHub to submit pull requests.

Sponsoring

You are very welcome to support the further development of this extension. You will get mentioned here.

Credits

The icon for the log table records is derived from an icon made by iconixar from www.flaticon.com.