DEPRECATION WARNING

This documentation is not using the current rendering mechanism and is probably outdated. The extension maintainer should switch to the new system. Details on how to use the rendering mechanism can be found here.

EXT: l10nmgr

Author:Kasper Skårhøj
Created:2002-11-01T00:32:00
Changed:2012-01-22T17:20:30
Author:Daniel Zielinski
Email:dev@loctimize.com
Info 3:Loctimize GmbH
Info 4:

EXT: l10nmgr

Extension Key: l10nmgr

Copyright 2000-2011, Daniel Zielinski, <dev@loctimize.com>

This document is published under the Open Content License

available from http://www.opencontent.org/opl.shtml

The content of this document is related to TYPO3

- a GNU/GPL CMS/Framework available from www.typo3.com

Table of Contents

EXT: l10nmgr 1

Introduction 1

What does it do? 1

Localization of TYPO3 websites 2

Support and automation of the localization process 2

Translation workflow without Localization Manager 2

Translation workflow with Localization Manager 2

Features 3

XML format 4

Screenshots 4

User manual 4

Installation 4

Creating a Localization Manager configuration 6

Online translation 6

Export 7

Download configuration files 8

Import 9

FAQs 10

Administration 10

Prerequisites 10

Rights and permissions 10

Database operations 10

FAQ 10

Configuration 10

TYPO3 – multi-language setup 12

Other helpful extensions: 12

FAQ 12

Tutorial 12

Hooks 12

Support 12

Known problems 13

To-Do list 13

Changelog 13

Introduction

What does it do?

The Localization Manager (l10nmgr) is a localization management extension for TYPO3 supporting a variety of online and offline translation workflows.

Online translation

Online translation in the TYPO3 backend can be done in a two column view with additional information on changes in the source text.

Offline translation

For offline translation workflows the l10nmgr allows for export of translatable content into XML and MS Excel format. The exported XML files have a localization-friendly format and can be translated with any state of the art translation tool such as across , DéjàVu , MemoQ , SDL TRADOS TagEditor and SDL PASSOLO . Configuration files for major translation tools are provided and can be downloaded in the backend. MS Excel files can be translated in OpenOffice Calc or Microsoft Excel. After translation the target files can be reimported into TYPO3. All translation data is stored automatically in the right place – no manual copying anymore.

Automation

Exports for different parts of the website and for multiple target languages at the same time can be automated and exported files can be uploaded automatically to the FTP server of a language service provider or send by email.

Translated XML files (XML or ZIP) can be automatically imported into TYPO3 by an import service.

Localization of TYPO3 websites

According to the adopted localization concept , new localizations of a web page and their respective content elements are mainly realized either with language overlays or with a one-tree-per-language approach , where the whole source page tree is copied (sometimes onto a different TYPO3 installation).

Although TYPO3 comes with all necessary localization features, localization of larger TYPO3 web sites is very often a tedious and time-consuming task.

Translation of pages and content elements is often done inside TYPO3 by manually localizing (translating) a web page and copying the default content elements, which are then overwritten with the translation. This requires translators to have access to and knowledge about TYPO3 (how to access TYPO3, how to use the RTE, how to localize or translate a page, etc.). While translating within TYPO3, translators do not have support from professional translation tools – such as translation memories, terminology databases or spell-checkers – which usually guarantee consistency of translations and terminology, as well as the efficiency of the translation process. This makes translators and translation service providers (as well as TYPO3 administrators) somewhat unhappy...

The other possible route to localizing a TYPO3 website is to dump the database tables containing the localizable information ( pages , tt_content ,...) e. g. into *.csv format, then to translate the content and import it into another TYPO3 installation. This approach enables the use of professional translation tools but requires the database tables to be translated as a whole, as no subset can be exported. Furthermore, one has to cope with lowlevel problems such as choosing the right separator. Little programming or SQL knowledge is required. However, this approach works only for the one-tree-per- language approach.

Support and automation of the localization process

Given this situation, the main idea behind developing Localization Manager was to realize a tool that supports and automates the localization process in order to make it more userfriendly. The user managing the website localization (e.g. administrator or project manager) should be able to export the localizable data so that it can be sent or assigned to translators. After translators have finished their work (using their CAT tools), translations can be sent back to the responsible person, who re-imports then the files into TYPO3. On import, all data in localized pages and content elements and the relations between them are automatically inserted into the database.

Translation workflow without Localization Manager

img-1 Figure 1: Translation workflow without Localization Manager

Translators work directly in TYPO3. They do not have support from translation tools, translation memories and terminology databases or spell checkers. For each page to be translated the following steps have to be performed: (1) Access the page to be translated, (2) Translate the web page into $LANGUAGE, (3) Insert translations for all fields in the new translation form, (4) Copy default content elements , (5) Open each content element to overwrite the source text with the translation. To translate e.g. 50 web pages containing 300 content elements you would need at least 800 mouse clicks – not counting the typing of the translation.

Translation workflow with Localization Manager

img-2 Figure 2: Translation workflow with Localization Manager

Translators do not have to access TYPO3. Translation is done in integrated translation tools with translation memories, terminology databases, spell checkers, etc. Translators can work offline and only need to translate the exported XML files.

Features

In its current version Localization Manager comprises the following features:

General features:

Support for all contents (pages, content elements) including contents from internationalised extensions (e.g. tt_news) and flexible content elements

Workspace support: Allows for translation workflows in offline workspaces

TemplaVoilà support

Export features:

Fine grained export configurations (via Localization Manager Configuration record)

Export of localizable content (pages and page content) into

  • a localization-friendly XML format
  • MS Excel XML (for translation in MS Excel or OpenOffice Calc)

Selection of localizable page and content element types as well as localizable database table fields

Check for already localized pages and content elements

Options

  • export only new/changed content
  • skip export of hidden content elements
  • single page export, recursive export
  • exclude selected elements from export
  • include selected elements from export
  • check encoding (UTF-8)
  • check XML for wellformedness
  • select source languages other than the default language (important for relay translations)

Predefined configuration files for across, DéjàVu, memoQ, SDL TRADOS TagEditor, SDL Trados Studio 2009, and SDL PASSOLO, which avoids time- consuming manual setup

Automation of exports via command line scripts (CLI) with

  • multiple Localization Manager configurations
  • for multiple target languages
  • upload of source language XML files to remote server via FTP
  • sending source language XML files via email
Import features

Re-import of localized XML files

Automatic insertion of localized pages and page content into TYPO3 database

Options:

  • Overwrite existing translations (pages and content elements)
  • Generate preview links (e.g. if translations are imported in a workspace)
Workflow features

Localization overview (very basic)

XML format

The XML format used by the l10nmgr is a simple XML format that can be used with all state-of-the-art localisation and translation tools. The Localization Manager ships with the required settings files that specify translatable and non-translatable elements. Currently settings files are available for the following tools: across, DéjàVu, SDL Trados and SDL Passolo. Settings files for other tools can easily be set up.

The file format for the export & import of translatable data of a TYPO3 website is a localization-friendly XML format. The default encoding of XML files is UTF-8.

Context information

One important feature of the XML format is that the translatable contents (<data> elements) are grouped according to the context of the page on which they occur (<pageGrp>).

To get a preview of the source web page, the URL can be reassembled by combining the <baseURL> with the ID attribute of the <pageGrp> element and the ID of the source language (<sysLang>):

http://<baseURL>/index.php?id=pageGrp:id&L=<sysLang>

Structure
Element name

a

Element name

b

Semantics

<TYPO3L10N>

a

<TYPO3L10N>

b

Root element

<t3_l10ncfg>

a

<t3_l10ncfg>

b

IDof Localization Manager configuration used for the export.

<t3_sysLang>

a

<t3_sysLang>

b

ID of target language (equal toIDsofwebsite languages)

<t3_sourceLang>

a

<t3_sourceLang>

b

ISO-639-1 language code for source language

<t3_targetLang>

a

<t3_targetLang>

b

ISO-639-1 language code for target language

<t3_baseURL>

a

<t3_baseURL>

b

Base URL of TYPO3-Website to be translated

<t3_workspaceId>

a

<t3_workspaceId>

b

ID of workspace from which has been exported

<t3_count>

a

<t3_count>

b

Number of exported data sets

<t3_wordCount>

a

<t3_wordCount>

b

Word count of source text

<t3_internal>

a

<t3_internal>

b

Internal messages

<t3_skippedItem>

a

<t3_skippedItem>

b

Elements skipped during export

<t3_description>

a

<t3_description>

b

Error messages

<t3_key>

a

<t3_key>

b

Key of skipped element

<t3_formatVersion>

a

<t3_formatVersion>

b

Version numberof XML format

<pageGrp>

a

<pageGrp>

b

Grouping element that embraces all translatable elements of a page. The id-attribute indicates the page ID. The sourceUrl attribute indicates the url of the source language webpage.

<data>

a

<data>

b

Translatable contents. Used attributes:

table = database table from which contents has been exported

elementUid = UID of localizable data set

key = Trigger for localization command. Syntax:

table:NEW/ t3_sysLang/elementUid:fieldname (Initial localization)

table:elementUid:fieldname (Update localization, elementUidis ID of data set to be updated)

((Unknown Property))

a

b

<data> can further contain all kind HTML elements. Number and type of elements can vary for every TYPO3 installation.

Screenshots

See section user manual.

User manual

Installation

Download the extension from the TYPO3 extension repository or from forge.typo3.org. Then install it with the extension manager. If you already use an older version of l10nmgr, make a backup of the extension directory. In case something fails, you'll have the possibility of re-establishing the original installation state from where you started the update.

Please note that you first need to install the extension static_info_tables .

Enabling hooks for processing of custom tags

Extensions may add custom tags to RTE-enabled fields. First and foremost is the DAM, which adds tag <media>. Upon export to CAT XML, this tag must be interpreted (as when loaded into the RTE) otherwise it is exported with < and > transformed to HTML entities, and reimported as such, breaking all links to RTE files.

Because there might be many different custom tags used in a TYPO3 installation we decided against including default implementations. If you want to use e. g. the hook to transform the DAM media tag you can add the following PHP code to your localconf:

$TYPO3_CONF_VARS['EXTCONF']['l10nmgr']['transformation']['txdam_media'] = 'EXT:dam/binding/mediatag/class.tx_dam_rtetransform_mediatag.php:&tx_dam_rtetransform_mediatag';
$TYPO3_CONF_VARS['EXTCONF']['l10nmgr']['transformation']['ts_links'] = 'EXT:dam/binding/mediatag/class.tx_dam_rtetransform_ahref.php:&tx_dam_rtetransform_ahref';
Platforms and versions supported
  • Linux/Unix Server, Windows
  • Apache, MySQL,
  • PHP5 (recommended 5.2 or higher)- compiled with --enable-pcntl for automation via /services- compiled with --enable-zip for zip generation (v4)

Creating a Localization Manager configuration

The Localization Manager first requires a Localization Manager configuration record in which you define the scope of translation or export/import of localizable contents. The Localization Manager configuration is a record that has to be created at the starting point in the page tree from which you want to start your translation or export/import.

To create a Localization Manager configuration record, proceed as follows:

Go to the backend and click on Web > List

Select the first page in the page tree from where you want to start your translation

In the right window click create new record and then select Localization Manager Configuration

Fill in the fields as shown in the following screenshot

Save the configuration

img-3 Figure 3: Creation of localization manager configuration

Online translation

To translate online in the TYPO3 backend do the following:

Go to the backend and click on Web > Localization Manager .

Select your previously created localization configurationfrom the table and click on the name of the configuration. This opens the Localization Manager and loads settings from the configuration.

Select the target language for your export from the drop-down list.

Select Online translation from the drop-down list and fill in the translations into the form,

Upon completion click Save to store the translations to the database.

img-4 Figure 4: Online translation

Export

The export function allows for exporting pages and their content into a) a localization-friendly XML format or b) the MS Excel XML format.

In order to export pages and page content to be localized, proceed as follows:

Go to the backend and click on Web > Localization Manager .

Select your previously created localization configurationfrom the table and click on the name of the configuration. This opens the Localization Manager and loads settings from the configuration.

Select the target language for your export from the drop-down list.

Select XML Export/Import or MS Excel Export/Import from the drop-down list and click on the Export tab. For translation with professional translation tools choose XML. For translation with MS Excel or OpenOffice Calc choose Excel.

img-5 Figure 5: XML export settings

img-6 Figure 6: Excel export settings

  • New/changed content only: Check this option if you only want to export new or changed content for translation. The comparison is always based on the default language version of an element.
  • No hidden content : Check this option if you do not want to export hidden content elements (often content elements that are still not ready to be published).

Check for already exported content : Check this option if you want to make sure that you don't export the same translation job twice.

Do not check XML : Select this option if you don't want to use the inbuilt XML check. In that case Localization Manager includes content that is not XML well-formed as CDATA elements. Otherwise this contents is not exported and you will find error messages indicating the Ids of the skipped elements.

  • Check encoding (UTF-8) : Check this option if you want to make sure that the output XML file consistently used UTF-8 encoding.
  • Force source language : Select a language from the drop-down list to choose a source language for translation other than the default language (relay translations)
  1. Click on Export to start the export.
  2. The data is now retrieved and packed into the desired file format.
  3. Save the export file.

Download configuration files

Localization Manager comes with predefined configuration files for Across, DéjàVu, memoQ, SDL TRADOS TagEditor, SDL Trados Studio 2009 and SDL PASSOLO . Configuration files or settings files contain information about the localization format, e.g. which tags are internal or external, which tags or attributes are translatable or not, etc. With this information translation tools are able to extract only translatable information from the file and protect non-translatable information from being touched. Download the configuration files and send them to the translator or translation service provider together with the XML.

To download the configuration files, proceed as follows:

Go to Web>Localization Manager in the backend.

Select a localization configurationfrom the table and click on the name of the configuration. This opens the Localization Manager and loads settings from the configuration.

Select the target language for your export from the drop-down list.

Select XML Export/Import from the drop-down list and click on the Settings tab.

Download the configuration file for your translation tool with a right-click on the desired configuration > Save target as

img-7 Figure 7: Download settings files for translation tools

Import

To import the translated XML or MS Excel files, proceed as follows:

Go to Web>Localization Manager in the backend.

Select your previously created localization configurationfrom the table and click on the name of the configuration. This opens the Localization Manager and loads settings from the configuration.

Select the target language for your export from the drop-down list.

Select XML Export/Import or MS Excel Export/Import from the drop-down list and click on the Import tab.

Set the import options:

  • Generate preview links : Check this option to generate preview links for all imported pages. You can then send these links to translators or reviewers so they can proofread the translations.
  • Delete previous localizations before import : Check this option to delete old translations before importing new ones.

Select the file to be imported (XML) from your local hard drive.

Click on Upload to start the import. Now the localized data is written to the appropriate database tables. Upon completion you will see a message on the bottom telling you how many data sets have been imported.

img-8 Figure 8: XML import settings

Note: Make sure the file to be imported is well-formed XML. If setup correctly translation tools will not modify or corrupt the XML. Anyway, sometimes translators insert non-standard entities into the XML, e.g. &reg; for ® leading to XML parsing errors. Just substitute the entities with their original character before import and everything works fine.

FAQs

  • Q: When I download the localization package and open the XML file, my XML editor fails (Error: encoding is not correct UTF-8). What can I do?A: The reason might be a database and/or configuration problem. Check the TYPO3 configuration for UTF-8 database support. You might also want to set TYPO3_CONF_VARS [BE][forceCharset] = utf-8, and check [SYS][multiplyDBfieldSize] and [SYS][UTF8filesystem].(See http://wiki.typo3.org/index.php/UTF-8_support for more information). Localization Manager assumes that you are using UTF-8 (which is the most highly recommended for multilingual websites).
  • Q: When I try to export localization data, I get a database error and the message: No source language given .A: Ensure that you have installed static_info_tables and that you have correctly set up your website languages.
  • Q: I get the error message “Call-time pass-by-reference has been deprecated...” under PHP5. What can I do?A: Add “php_value allow_call_time_pass_reference 1” to your .htaccess file.
  • Q: My exported files are not valid XML. What can I do?A: Problem is mainly due to nasty bodytext fields or other database fields where editors are allowed to insert HTML code. Try to use Tidy to clean the contents of theses fields. Or use an XML editor and fix the errors manually ;-)

Administration

Prerequisites

  • Install the extension static_info_tables to display the correct ISO language codes.
  • Precondition is a proper multi-language setup of TYPO3 and correct UTF8 environment is suggested.
  • The key concept of the extension is that you need to set up the required “L10N Configuration” Records. These records define what content should be included for the translation work. (See section Creating a Localization Manager Configuration)

Rights and permissions

The basics about the Rights and Permissions concepts of TYPO3 can be found in the “Getting started” document:

http://typo3.org/documentation/document- library/tutorials/doc_tut_quickstart/0.1.0/view/1/12/#id2856932

For some advanced options, see the “MOD” section in “doc_core_tsconfig”:

http://typo3.org/documentation/document- library/references/doc_core_tsconfig/4.0.0/view/1/3/#id2798076

Note that for exporting/importing localized data the user needs write access to the affected database tables e.g. pages, pages_language_overlay and tt_content. Furthermore, s/he needs access to the languages to be imported.

Database operations

The extension uses the TYPO3 API for all database operation to insure data integrity and consistency.

FAQ

No questions yet

Configuration

Configuration options available during installation of extension:

  • Enable hidden languages
  • img-9 Send email notification
  • img-10 Enable FTP upload
  • img-11 L10N Manager configurations: List uids of configuration records for automation of export via CLI script
  • img-12 L10N Manager target languages: List uids of sys_languages for automated export for multiple target languages via CLI
  • img-13 img-14 img-15 img-16 Email address: for notification emails in CLI mode
  • FTP server details
  • img-17 img-18 CLI options

TYPO3 – multi-language setup

  • There needs to be a connection between the default language and their translation, means all the translation needs to have a kind of pointer to their original default element. Therefore use these hints:
  • Use “one tree” concept for your page
  • Translation of normal content elements:
    • they need to use the “l18n_parent” feature: This means you should translate elements in the PAGE module or in LIST module with the check box “Localization view”
    • So you always have a default record overlayed with a translation.
  • TemplaVoilà: setup of Flexible Content Elements (FCE):
    • The localization mode needs to be: langChildren=enabled
    • To translate FCE also with normal overlay elements (introduced with extension languagevisibility) you need to add <langDatabaseOverlay type="integer">1</langDatabaseOverlay> to the DS of the FCE.

For more information read the Frontend Localization Guide in the TYPO3 core documentation.

Other helpful extensions:

  • Language visibility (languagevisibility): Enables multilevel fallback and exception in visibility of content elements.
  • Language detection (rlmp_language_detection): Detects the visitors preferred language and redirects to the localized pages accordingly.
  • Flag icons for sys_languages (syslanguage_flags) :Adds 200 flag icons to sys_language flag selection.

FAQ

No questions yet

Tutorial

See podcasts on http://typo3.org

Hooks

savePreProcess

Name

savePreProcess

Method

processBeforeSaving

savePostProcess

Name

savePostProcess

Method

processAfterSaving

exportCatXmlPreProcess

Name

exportCatXmlPreProcess

Method

processBeforeExportingCatXml

exportExcelXmlPreProcess

Name

exportExcelXmlPreProcess

Method

processBeforeExportingExcelXml

transformation

Name

transformation

Method

transform_rte

transformation

Name

transformation

Method

transform_db

Support

The extension is maintained by a team:

  • Kasper Skårhøj
  • Daniel Pötzinger (AOE media GmbH)
  • Daniel Zielinski (Loctimize GmbH)
  • Andreas Otto (SMA Solar Technology)

Please use http://forge.typo3.org/projects/show/extension-l10nmgr for support issues and feature requests.

Known problems

Performance issues in the context of very large websites during export & import of translations. Temporary solution is to increase the maximum execution time of PHP scripts (php.ini) and Apache processes (httpd.conf). Another option is to split the export/import files into smaller chunks. Just make sure that you have the head section in the XML and that you don't cut the <pageGrp> element.

To-Do list

  • Performance issues (will be fixed in version 4)
  • Usability

Changelog

See SVN repository on http://forge.typo3.org/projects/show/extension- l10nmgr .

img-19 EXT: l10nmgr - 13