Importing events from a sync channel 

The import configuration type Sync API channel (events) (syncChannel) imports the events of one channel of the ThueCat sync API on https://widgets.thuecat.org. It is the only type that imports licence-restricted ("private") event images: the sync API serves each event as one merged record that carries the private image in place of the public one.

The type imports events only. Places, organisations and trails keep coming from the other Import configuration types.

On this page

Fields 

Setting Meaning
storagePid (required) Target folder for the imported events.
fileFolder (required) Target folder for images and other media files.
syncChannelId (required) ID of the channel, as ThueCat issued it.
apiKey (required) The key ThueCat issued for this channel.
runBudget, fetchCacheLifetime Optional. Override the Extension Configuration values, see Import tuning.

The type offers no apiDomain: the host is always https://widgets.thuecat.org. An API domain stored while the record had another type is ignored. Neither does it offer fetchLastXDays; the window of changes is derived, see The window of changes.

The key is presented in the form each host expects:

  • The sync API receives it as an Authorization header.
  • Media downloads from cdb.thuecat.org receive it as api_key query parameter.
  • No other host receives it. https://thuecat.org/resources/ needs no key.

Imported events keep https://thuecat.org as their source_url, as events imported by the other types do.

The window of changes 

Each run asks the channel for the events created, updated, removed or expired since a point in time. That point is computed when the run starts:

  • The storage folder holds no event: five years before the run. Deleted events and translations do not count.
  • Otherwise: one day before the creation time of the most recent successful run of this configuration. The day of overlap absorbs clock skew and upstream changes indexed late; re-importing an unchanged event changes nothing.
  • No successful run is logged: five years as well.

A run is successful unless its import log holds an entry of type runAborted, runFailed or fetchingError. Skipped references, mapping errors, DataHandler errors and the empty-folder warning do not count against it: they concern single records, and letting them reset the window would fetch the whole history again for one bad event. A run that is not successful leaves the window where it was, so the next run asks for the same changes again.

Every event the change list names is imported, unless the channel also lists it as removed or expired. Each event is fetched on its own. An event the channel lists but answers with an empty response is logged as fetchingError, and the run goes on with the next one.

If the change list or the expiry list cannot be read, the run fails before it fetches any event, and nothing is deleted.

Removal applies to the whole site 

After the import, every event the channel lists as removed or expired is deleted, in the whole site the storage folder belongs to. It does not matter which import configuration imported the event or which folder of the site holds it. Events of other sites are not touched.

This follows from record identity: a site holds one event per ThueCat ID, whichever configuration wrote it, see Records, identity and languages. Deleting only this configuration's events would leave an event the channel declares invalid in place.

Events are deleted through the DataHandler, so their translations, dates and file references go with them. A deletion that fails is logged as dataHandlerError; the event stays listed and is deleted by a later run, as long as the channel still reports it within the window.

Nothing is deleted when the run was aborted by its run budget, or when either list could not be read.

References are asked of the channel first 

Events refer to other records by https://thuecat.org/resources/<id> URLs: media, authors, keywords. On a syncChannel run, each such reference is first requested from the sync channel:

  • The sync channel serves the ID: its record is used, including a private image it carries. thuecat.org is not asked.
  • The sync channel answers with an empty response: the reference is fetched from the URL it carries, as on the other types. This is not logged.

Either way, the related record keeps the URL the data carries as its identity; only the source of its content changes. References to other URLs, such as vocabulary terms, are fetched from their URL directly. Each reference is asked of the sync channel once per run.

The channel comes first because a record fetched from thuecat.org carries only the public image. Fetching a reference there would import the public image, or replace a private one already stored.

Empty storage folder warning 

If the storage folder holds no event after a run, the import log carries a storageFolderEmpty entry with severity warning, shown in the backend module ThueCat - Imports.

The usual cause: every event of the channel already exists in another folder of the site, so the run updated those records instead of creating new ones in its own folder. Check whether another import configuration of the site imports the same events.

The warning does not count against the run's success, see The window of changes.