Importing records from a sync channel
The import configuration type Sync API channel (sync) imports the records of
one channel of the ThueCat sync API on https://: events, places or trails,
whatever the channel lists. It is the only type that imports licence-restricted ("private") images:
the sync API serves each record as one merged record that carries the private image in place of the
public one.
Each configuration reads one channel. To import a channel of events and a channel of places,
configure one sync configuration for each.
On this page
Fields
| Setting | Meaning |
|---|---|
storage (required) | Target folder for the imported records. |
file (required) | Target folder for images and other media files. |
sync (required) | ID of the channel, as ThueCat issued it. |
api (required) | The key ThueCat issued for this channel. |
run, fetch | Optional. Override the Extension Configuration values, see Import tuning. |
The type offers no api: the host is always https://. An API domain
stored while the record had another type is ignored. Neither does it offer fetch; the
window of changes is derived, see The window of changes.
The key is presented in the form each host expects:
- The sync API receives it as an
Authorizationheader. - Media downloads from
cdb.receive it asthuecat. org api_query parameter.key - No other host receives it.
https://needs no key.thuecat. org/ resources/
Imported events keep https:// as their source_, as events imported by the other
types do.
The window of changes
Each run asks the channel for the records created, updated, removed or expired since a point in time. That point is computed when the run starts:
- The storage folder holds no record: five years before the run. Records of every kind count: events, places of every table, organisations, towns, meeting rooms and trails. Deleted records and translations do not count; inline children such as addresses or opening hours are no records of their own.
- Otherwise: one day before the creation time of the most recent successful run of this configuration. The day of overlap absorbs clock skew and upstream changes indexed late; re-importing an unchanged record changes nothing.
- No successful run is logged: five years as well.
A run is successful unless its import log holds an entry of type run, run or
fetching. Skipped references, mapping errors, DataHandler errors and the
empty-folder warning do not count against it: they concern single
records, and letting them reset the window would fetch the whole history again for one bad record.
A run that is not successful leaves the window where it was, so the next run asks for the same
changes again.
Every record the change list names is imported, unless the channel also lists it as removed or
expired. Each record is fetched on its own. A record the channel lists but answers with an empty
response is logged as fetching, and the run goes on with the next one.
Channels of places and trails report removed records only; their expiry list stays empty.
If the change list or the expiry list cannot be read, the run fails before it fetches any record, and nothing is deleted.
Removal applies to the whole site
After the import, every record the channel lists as removed or expired is deleted, in the whole site the storage folder belongs to. It does not matter which import configuration imported the record, which folder of the site holds it, or which kind of record the configuration imports. A ThueCat ID stored in more than one table is deleted from each. Records of other sites are not touched.
This follows from record identity: a site holds one record per ThueCat ID, whichever configuration wrote it, see Records, identity and languages. Deleting only this configuration's records would leave a record the channel declares invalid in place.
Records are deleted through the DataHandler, so their translations, inline children (addresses,
opening hours, a trail's way types, locations and conditions), dates and file references go with
them. A deletion that fails is logged as data; the record stays listed and is deleted
by a later run, as long as the channel still reports it within the window.
Nothing is deleted when the run was aborted by its run budget, or when either list could not be read.
Warning
A sync configuration deletes records that another configuration of the same site
imported. Keep this in mind before adding one to a site that also imports records by other means.
References are asked of the channel first
Records refer to other records by https:// URLs: media, authors,
keywords, organisations. On a sync run, each such reference is first requested from the sync channel:
- The sync channel serves the ID: its record is used, including a private image it carries.
thuecat.is not asked.org - The sync channel answers with an empty response: the reference is fetched from the URL it carries, as on the other types. This is not logged.
Either way, the related record keeps the URL the data carries as its identity; only the source of its content changes. References to other URLs, such as vocabulary terms, are fetched from their URL directly. Each reference is asked of the sync channel once per run.
The channel comes first because a record fetched from thuecat. carries only the public image.
Fetching a reference there would import the public image, or replace a private one already stored.
Empty storage folder warning
If the storage folder holds no record after a run, the import log carries a storage
entry with severity warning, shown in the backend module ThueCat - Imports.
The usual cause: every record of the channel already exists in another folder of the site, so the run updated those records instead of creating new ones in its own folder. Check whether another import configuration of the site imports the same records.
The warning does not count against the run's success, see The window of changes.