Important: #82363 - Make Extbase translation handling consistent with TypoScript 

Description 

Extbase now renders the translated records in the same way TypoScript rendering does. The new behaviour is controlled by the Extbase feature switch consistentTranslationOverlayHandling.

config.tx_extbase.features.consistentTranslationOverlayHandling = 1

The new behaviour is enabled by default in TYPO3 v9. The feature switch will be removed in TYPO3 v10, so there will be just one way of fetching records. You can override the setting using normal TypoScript.

Impact 

Users relying on the old behaviour can disable the feature switch.

The change modifies how Extbase interprets the TypoScript settings config.sys_language_mode and config.sys_language_overlay and the Typo3QuerySettings properties languageOverlayMode and languageMode.

Changes in the rendering:

1. Setting Typo3QuerySettings->languageMode does not influence how Extbase queries records anymore. The corresponding TypoScript setting config.sys_language_mode is used by the core to decide what to do when a page is not translated to the given language (display 404, or try page with different language). Users who used to set Typo3QuerySettings->languageMode to strict should use Typo3QuerySettings->setLanguageOverlayMode('hideNonTranslated') to get translated records only.

   The old behavior was confusing, because languageMode had different meaning and accepted different values in TS context and in Extbase context.

2. Setting Typo3QuerySettings->languageOverlayMode to true makes Extbase fetch records from default language and overlay them with translated values. So e.g. when a record is hidden in the default language, it will not be shown. Also records without translation parents will not be shown. For relations, Extbase reads relations from a translated record (so it’s not possible to inherit a field value from translation source) and then passes the related records through $pageRepository->getRecordOverlay(). So e.g. when you have a translated tt_content with FAL relation, Extbase will show only those sys_file_reference records which are connected to the translated record (not caring whether some of these files have l10n_parent set).

   Previously Typo3QuerySettings->languageOverlayMode had no effect. Extbase always performed an overlay process on the result set.

3. Setting Typo3QuerySettings->languageOverlayMode to false makes Extbase fetch aggregate root records from a given language only. Extbase will follow relations (child records) as they are, without checking their sys_language_uid fields, and then it will pass these records through $pageRepository->getRecordOverlay(). This way the aggregate root record's sorting and visibility doesn't depend on default language records. Moreover, the relations of a record, which are often stored using default language uids, are translated in the final result set (so overlay happens).

   For example: Given a translated tt_content having relation to 2 categories (in the mm table translated tt_content record is connected to category uid in default language), and one of the categories is translated. Extbase will return a tt_content model with both categories. If you want to have just translated category shown, remove the relation in the translated tt_content record in the TYPO3 Backend.

Note that by default Typo3QuerySettings uses the global TypoScript configuration like config.sys_language_overlay and $GLOBALS['TSFE']->sys_language_content (calculated based on config.sys_language_uid and config.sys_language_mode). So you need to change Typo3QuerySettings manually only if your Extbase code should behave different than other tt_content rendering.

Setting setLanguageOverlayMode() on a query influences only fetching of the aggregate root. Relations are always fetched with setLanguageOverlayMode(true).

When querying data in translated language, and having setLanguageOverlayMode(true), the relations (child objects) are overlaid even if aggregate root is not translated. See QueryLocalizedDataTest->queryFirst5Posts().

Following examples show how to query data in Extbase in different scenarios, independent of the global TS settings:

1. Fetch records from the language uid=1 only, with no overlays. Previously ( consistentTranslationOverlayHandling = 0):

   It was not possible.

   Now ( consistentTranslationOverlayHandling = 1):

$querySettings = $query->getQuerySettings();
$querySettings->setLanguageUid(1);
$querySettings->setLanguageOverlayMode(false);

2. Fetch records from the language uid=1, with overlay, but hide non-translated records Previously ( consistentTranslationOverlayHandling = 0):

 $querySettings = $query->getQuerySettings();
 $querySettings->setLanguageUid(1);
 $querySettings->setLanguageMode('strict');

Now (:typoscript:`consistentTranslationOverlayHandling = 1`):

$querySettings = $query->getQuerySettings();
$querySettings->setLanguageUid(1);
$querySettings->setLanguageOverlayMode('hideNonTranslated');

Summary of the important code changes 

1. DataMapper gets a Query as a constructor parameter. This allows to use aggregate root QuerySettings (language) when fetching child records/relations. Later, in a separate patch we can pass other settings too e.g. setIgnoreEnableFields to fix issue around this setting. See DataMapper->getPreparedQuery method.
2. DataMapper is passed to LazyLoadingProxy and LazyObjectStorage, so the settings don't get lost when fetching data lazily.
3. Query object gets a new property parentQuery which is useful to detect whether we're fetching aggregate root or child object.
4. Extbase model for FileReference uses _localizedUid for fetching OriginalResource
5. DataMapper forces child records to be fetched using setLanguageOverlayMode(true).
6. When getRespectSysLanguage is set, DataMapper uses aggregate root language to overlay child records to correct language.
7. The where clause used for finding translated records in overlay mode (true, hideNonTranslated) has been fixed. It filters out the non translated records on db side in case hideNonTranslated is set. It allows for filtering and sorting by translated values. See Typo3DbQueryParser->getLanguageStatement()

Most important known issues (this patch doesn't solve) 

• Persistence session uses the same key for default language record and the translation - https://forge.typo3.org/issues/59992
• Extbase allows to fetch deleted/hidden records - https://forge.typo3.org/issues/86307

For more information about rendering please refer to the TypoScript reference.