Important: #82363 - Make Extbase translation handling consistent with TypoScript¶
See forge#82363
Description¶
Extbase now renders the translated records in the same way TypoScript rendering does.
The new behaviour is controlled by the Extbase feature switch consistentTranslationOverlayHandling
.
config.tx_extbase.features.consistentTranslationOverlayHandling = 1
The new behaviour is enabled by default in TYPO3 v9. The feature switch will be removed in TYPO3 v10, so there will be just one way of fetching records. You can override the setting using normal TypoScript.
Impact¶
Users relying on the old behaviour can disable the feature switch.
The change modifies how Extbase interprets the TypoScript settings
config.sys_language_mode
and config.sys_language_overlay
and the
Typo3QuerySettings
properties languageOverlayMode
and languageMode
.
Changes in the rendering:
Setting
Typo3QuerySettings->languageMode
does not influence how Extbase queries records anymore. The corresponding TypoScript settingconfig.sys_language_mode
is used by the core to decide what to do when a page is not translated to the given language (display 404, or try page with different language). Users who used to setTypo3QuerySettings->languageMode
tostrict
should useTypo3QuerySettings->setLanguageOverlayMode('hideNonTranslated')
to get translated records only.The old behavior was confusing, because
languageMode
had different meaning and accepted different values in TS context and in Extbase context.Setting
Typo3QuerySettings->languageOverlayMode
totrue
makes Extbase fetch records from default language and overlay them with translated values. So e.g. when a record is hidden in the default language, it will not be shown. Also records without translation parents will not be shown. For relations, Extbase reads relations from a translated record (so it’s not possible to inherit a field value from translation source) and then passes the related records through$pageRepository->getRecordOverlay()
. So e.g. when you have a translatedtt_content
with FAL relation, Extbase will show only thosesys_file_reference
records which are connected to the translated record (not caring whether some of these files havel10n_parent
set).Previously
Typo3QuerySettings->languageOverlayMode
had no effect. Extbase always performed an overlay process on the result set.Setting
Typo3QuerySettings->languageOverlayMode
tofalse
makes Extbase fetch aggregate root records from a given language only. Extbase will follow relations (child records) as they are, without checking theirsys_language_uid
fields, and then it will pass these records through$pageRepository->getRecordOverlay()
. This way the aggregate root record's sorting and visibility doesn't depend on default language records. Moreover, the relations of a record, which are often stored using default language uids, are translated in the final result set (so overlay happens).For example: Given a translated
tt_content
having relation to 2 categories (in the mm table translated tt_content record is connected to category uid in default language), and one of the categories is translated. Extbase will return att_content
model with both categories. If you want to have just translated category shown, remove the relation in the translatedtt_content
record in the TYPO3 Backend.
Note that by default Typo3QuerySettings
uses the global TypoScript configuration like
config.sys_language_overlay
and $GLOBALS['TSFE']->sys_language_content
(calculated based on config.sys_language_uid
and config.sys_language_mode
).
So you need to change Typo3QuerySettings
manually only if your Extbase code should
behave different than other tt_content
rendering.
Setting setLanguageOverlayMode()
on a query influences only fetching of the aggregate root. Relations are always
fetched with setLanguageOverlayMode(true)
.
When querying data in translated language, and having setLanguageOverlayMode(true)
, the relations
(child objects) are overlaid even if aggregate root is not translated.
See QueryLocalizedDataTest->queryFirst5Posts()
.
Following examples show how to query data in Extbase in different scenarios, independent of the global TS settings:
Fetch records from the language uid=1 only, with no overlays. Previously (
consistentTranslationOverlayHandling = 0
):It was not possible.
Now (
consistentTranslationOverlayHandling = 1
):
$querySettings = $query->getQuerySettings();
$querySettings->setLanguageUid(1);
$querySettings->setLanguageOverlayMode(false);
Fetch records from the language uid=1, with overlay, but hide non-translated records Previously (
consistentTranslationOverlayHandling = 0
):
$querySettings = $query->getQuerySettings();
$querySettings->setLanguageUid(1);
$querySettings->setLanguageMode('strict');
Now (:typoscript:`consistentTranslationOverlayHandling = 1`):
$querySettings = $query->getQuerySettings();
$querySettings->setLanguageUid(1);
$querySettings->setLanguageOverlayMode('hideNonTranslated');
QuerySettings property |
old behaviour |
new behaviour |
default value (TSFE|Extbase) |
---|---|---|---|
languageUid |
same |
0 |
|
respectSysLanguage |
same |
|
|
languageOverlayMode |
not used |
values: |
0 | |
languageMode |
documented values: |
not used |
|
Identifiers¶
Domain models have a main identifier uid
and two additional properties _localizedUid
and _versionedUid
.
Depending on whether the languageOverlayMode
mode is enabled (true
or 'hideNonTranslated'
) or disabled (false
),
the identifier contains different values.
When languageOverlayMode
is enabled then uid
property contains uid
value of the default language record,
the uid
of the translated record is kept in the _localizedUid
.
Context |
Record in language 0 |
Translated record |
---|---|---|
Database |
uid:2 |
uid:11, l10n_parent:2 |
Domain Object values with |
uid:2, _localizedUid:2 |
uid:2, _localizedUid:11 |
Domain Object values with |
uid:2, _localizedUid:2 |
uid:11, _localizedUid:11 |
See tests in extbase/Tests/Functional/Persistence/QueryLocalizedDataTest.php
.
The $repository->findByUid()
(or $persistenceManager->getObjectByIdentifier()
) method takes current
rendering language into account (e.g. L=1). It does not take defaultQuerySetting
set on the repository into account.
This method always performs an overlay.
Values in braces show previous behaviour (disabled flag) if different than current.
The bottom line is that with the feature flag on, you can now use findByUid()
using translated record uid to get
translated content independently from language set in global context.
L=0 |
L=1 |
||||
---|---|---|---|---|---|
repository method |
property |
Overlay |
No overlay |
Overlay |
No overlay |
findByUid(2) |
title |
Post 2 |
Post 2 |
Post 2 - DK |
Post 2 - DK |
uid |
2 |
2 |
2 |
2 |
|
_localizedUid |
2 |
2 |
11 |
11 |
|
findByUid(11) |
title |
Post 2 - DK (Post 2) |
Post 2 - DK (Post 2) |
Post 2 - DK |
Post 2 - DK |
uid |
2 |
2 |
2 |
2 |
|
_localizedUid |
11 (2) |
11 (2) |
11 |
11 |
Note
Note that $repository->findByUid()
internally sets respectSysLanguage(false)
so it behaves differently
than a regular query by an uid
like $query->matching($query->equals('uid', 11));
The regular query will return null
if passed uid
doesn't match
the language set in the $querySettings->setLanguageUid()
method.
Filtering & sorting¶
When filtering by aggregate root property like Post->title
,
both filtering and sorting takes translated values into account and you will get correct results, also with pagination.
When filtering or ordering by child object property, then Extbase does a left join between aggregate root
table and child record table.
Then the filter is applied as where clause. This means that filtering or ordering by child record property
only takes values from child records which uids are stored in db (in most cases its default language record).
See TranslationTest::fetchingTranslatedPostByBlogTitle()
This limitation also applies to Extbase with feature flag being disabled.
Summary of the important code changes¶
DataMapper
gets aQuery
as a constructor parameter. This allows to use aggregate rootQuerySettings
(language) when fetching child records/relations. Later, in a separate patch we can pass other settings too e.g.setIgnoreEnableFields
to fix issue around this setting. SeeDataMapper->getPreparedQuery
method.DataMapper
is passed toLazyLoadingProxy
andLazyObjectStorage
, so the settings don't get lost when fetching data lazily.Query
object gets a new propertyparentQuery
which is useful to detect whether we're fetching aggregate root or child object.Extbase model for
FileReference
uses_localizedUid
for fetchingOriginalResource
DataMapper
forces child records to be fetched usingsetLanguageOverlayMode(true)
.When getRespectSysLanguage is set,
DataMapper
uses aggregate root language to overlay child records to correct language.The
where
clause used for finding translated records in overlay mode (true
,hideNonTranslated
) has been fixed. It filters out the non translated records on db side in casehideNonTranslated
is set. It allows for filtering and sorting by translated values. SeeTypo3DbQueryParser->getLanguageStatement()
Most important known issues (this patch doesn't solve)¶
Persistence session uses the same key for default language record and the translation - https://forge.typo3.org/issues/59992
Extbase allows to fetch deleted/hidden records - https://forge.typo3.org/issues/86307
For more information about rendering please refer to the TypoScript reference.