https://img.shields.io/github/v/release/meine-krankenkasse/typo3-search-algolia?sort=semver https://img.shields.io/github/license/meine-krankenkasse/typo3-search-algolia https://github.com/meine-krankenkasse/typo3-search-algolia/actions/workflows/ci.yml/badge.svg

typo3-search-algolia 

A TYPO3 extension that integrates Algolia search into your website by indexing TYPO3 content for lightning-fast, relevant search results.

Features 

  • Seamless integration with TYPO3 CMS
  • Indexing of various content types:

    • Pages
    • Content elements
    • News articles
    • Files (including PDF content extraction)
  • Configurable indexing services
  • Customizable field mapping
  • Backend module for managing indexing
  • Context menu integration for direct indexing
  • Support for excluding specific content from search

Requirements 

  • TYPO3 v12.4
  • PHP >=8.3 and <8.5
  • Algolia account with API credentials

Quick Start 

  1. Install the extension via composer: composer require meine-krankenkasse/typo3-search-algolia
  2. Configure your Algolia API credentials in additional.php
  3. Create a data folder for your search configuration
  4. Set up a search engine and indexing services
  5. Start indexing your content

Table of contents 

  • Documentation/Requirements
  • Documentation/Configuration
  • Documentation/Indexers

    • Documentation/PageIndexer
    • Documentation/ContentElementIndexer
    • Documentation/NewsIndexer
    • Documentation/FileIndexer

Optional Features 

Workspace Support 

To enable automatic reindexing when publishing workspace records, install the workspaces extension:

composer require typo3/cms-workspaces
Copied!

Without this extension, the search indexer will still work but won't automatically queue records when publishing from workspaces.

Releasing 

  1. Bump the version in ext_emconf.php
  2. Add a CHANGELOG.md entry (# X.Y.Z heading)
  3. Commit and push
  4. Create and push a tag matching the version, no v prefix, e.g.:

    git tag -s 1.3.3 -m 1.3.3
    git push origin 1.3.3
    
    Copied!
  5. A GitHub Release is created automatically from the tag, which publishes the new version to TER

To re-publish an already-tagged version (e.g. to fix the TER upload comment), trigger the "Publish new extension version to TER" workflow manually via workflow_dispatch with that tag as input.

Configuration 

Update database structure 

Use the "Analyze Database Structure" in the "Maintenance" Admin Tools section to update the database structure.

Web service 

API endpoint 

To access the Algolia API, store the corresponding configuration in the file additional.php within the global structure "TYPO3_CONF_VARS" under "EXTENSIONS" and "typo3_search_algolia" (note the spelling) of your TYPO3 installation.

// The universal messenger API endpoint
$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['typo3_search_algolia'] = array_merge( 
    $GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['typo3_search_algolia'] ?? [], 
    [ 
        'appId' => 'YOUR-APP-ID', 
        'apiKey' => 'YOUR-API-KEY', 
    ]
);
Copied!
Field Description
appId Your Algolia application ID.
apiKey An API key with the necessary permissions to make the request. The required access control list (ACL) to make a request is listed in each endpoint's reference.

You can find your application ID and API key in the Algolia dashboard.

Basics 

Next, create a data directory to store the configured index services, search engines, and, if applicable, file collections. To do this, switch to the list view in the TYPO3 backend and add a new "Folder" element to the page tree, then select the created data folder.

Setting up index services and search engine 

Search engine 

Create a new record of type "Search Engine".

configuration-search-engine-1

Fig. 1: Create a new record of type "Search Engine"

Add a title, an optional description to the new record, and select the search engine service to use (the default available one is "Algolia Search Service"). To create a custom search engine service, refer to: Custom Search Engine

Add a search engine dependent configuration into the "index_name" field. In the case of Algolia searches, this is the name of an index within Algolia in which the indexed data will ultimately be stored.

configuration-search-engine-2

Fig. 2: Configuration TCA of type "Search Engine"

Indexing Service 

In the next step, the respective indexing services are created. These define which data will be indexed.

To do this, create a new record of type "Indexing Service".

configuration-indexing-service-1

Fig. 3: Create a new record of type "Indexing Service"

Give the indexing service a meaningful title, an optional description, and select the type of indexer to be used. By default, the following are available:

  • Content elements
  • Pages
  • Messages
  • Files

The display updates depending on your selection and offers additional configuration options.

Then select one of the previously created search engine configurations.

configuration-indexing-service-1

Fig. 4: Configuration TCA of type "Indexing Service"

Page Indexer Configuration 

The following additional options are available for the page indexer:

  • Include content elements [include_content_elements]

    • Enable this option to index the page's content elements in addition to the page properties. Alternatively, you can use the content element indexer, especially if the search engine has a record size limit.
    • With this option, the content element fields stored in the ContentElementIndexer.md#custom-fields module.tx_typo3searchalgolia.indexer.tt_content.fields are read and stored directly in the page's index entry in the content attribute. A separate indexing service for the content elements is then no longer necessary. However, a data set for a page can become very large, so it's important to check whether the search engine supports this.
  • Page type [pages_doktype]

    • Select the page types by which the pages selected below for indexing should be filtered. If no selection is made here, all page types will be included.
  • Single pages [pages_single]

    • Select individual pages to be indexed here. Indexing will only occur for pages whose "Include in Search" option is enabled in the page properties.
  • Pages (recursively) [pages_recursive]

    • Select the pages you want to recursively index, including all their subpages. Indexing will only occur for pages whose "Include in Search" option is enabled in the page properties.

Content Element Indexer Configuration 

The following additional options are available for the content item indexer:

  • Single pages [pages_single]

    • Select individual pages to be indexed here. Indexing will only occur for pages whose "Include in Search" option is enabled in the page properties.
  • Pages (recursively) [pages_recursive]

    • Select the pages you want to recursively index, including all their subpages. Indexing will only occur for pages whose "Include in Search" option is enabled in the page properties.

File Indexer Configuration 

The following additional options are available for the file indexer:

  • File collections [file_collections]

    • Select the file collections that should be used for indexing. Indexing will only occur for files whose "Include in Search" option is enabled in the metadata properties.

Content Element Indexer 

The Content Element Indexer is a specialized component of the TYPO3 Search Algolia extension that focuses on indexing individual content elements from TYPO3 pages. This allows for more granular search results, enabling users to find specific content elements rather than just pages.

How It Works 

The Content Element Indexer processes content elements based on the configuration in your indexing service. When triggered, it:

  1. Retrieves content elements from the pages specified in your configuration
  2. Extracts relevant data from each content element, including headers, text content, and other configured fields
  3. Creates individual search records for each content element
  4. Sends the processed data to Algolia for indexing

This approach is particularly useful for websites with large pages containing multiple content elements, as it allows for more precise search results.

Indexed Fields 

Standard Fields 

In addition to the Indexers.md#standard-indexed-fields, the following fields are indexed by default for content elements:

Field Description
site The domain name of the page.
url The absolute URL to the content element on the page (only included if available).

Custom Fields 

Additional fields to be indexed from the content element properties can be defined using the TypoScript configuration module.tx_typo3searchalgolia.indexer.tt_content.fields. This is set by default as follows:

module {
    tx_typo3searchalgolia {
        indexer {
            tt_content {
                fields {
                    header = title
                    subheader = subTitle
                    bodytext = description
                }
            }
        }
    }
}
Copied!

Usage and Best Practices 

When to Use the Content Element Indexer 

Consider using the Content Element Indexer when:

  • Your pages contain many different content elements
  • You want users to find specific content within pages
  • You need more granular search results than page-level indexing provides
  • Your pages are large and would exceed Algolia's record size limits if indexed as a whole

Best Practices 

  1. Content Element Types: Consider which content element types should be indexed. Text-based elements like text, textpic, and textmedia are typically most relevant for search.
  2. Field Mapping: Customize the field mapping to ensure the most important content from your elements is properly indexed. Different content element types may need different field mappings.
  3. URL Generation: Ensure that the URL generation for content elements includes proper anchors to direct users to the specific element within a page.
  4. Complementary Indexing: Consider using both Page Indexer and Content Element Indexer together - pages for general content and content elements for specific details.
  5. Performance Considerations: Be mindful of the number of content elements being indexed, as this can significantly increase the size of your index and affect performance.

Custom Indexer 

TBD

Custom Search Engine 

TBD

File Indexer 

The File Indexer is a powerful component of the TYPO3 Search Algolia extension that indexes files stored in TYPO3's File Abstraction Layer (FAL). It extracts metadata and content from files, making them searchable through Algolia. This is particularly useful for websites with document repositories, downloadable resources, or media libraries.

How It Works 

The File Indexer processes files based on the configuration in your indexing service. When triggered, it:

  1. Retrieves files from the file collections specified in your configuration
  2. Filters files based on their properties (file extension, no_search flag, etc.)
  3. Extracts metadata from each file
  4. For supported file types (currently PDF), extracts the actual content
  5. Sends the processed data to Algolia for indexing

This allows users to search not only for file names and metadata but also for text contained within supported documents.

File Properties 

The "sys_file_metadata" table has been expanded to include the "no_search" field, which behaves identically to the existing field in the "pages" table and, if set, allows a file to be excluded from indexing. By default, this field is not set for all files, meaning they will be indexed.

To set the property, click on the desired file in the file list and switch to the new "Behavior" tab. Here you can adjust the value for the "no_search" property accordingly.

metadata-no-search

Fig. 1: Exclude file from indexing

Indexed Fields 

Standard Fields 

In addition to the Indexers.md#standard-indexed-fields, the following fields are indexed by default for files:

Field Description
extension The file extension.
mimeType The MIME type of the file.
name The name of the file.
size The size of the file in bytes.
url The relative URL to the file (only included if available).
content The content of the file (only included for supported file types, currently only PDF).

Custom Fields 

Additional fields to be indexed from the file metadata can be defined using the TypoScript configuration module.tx_typo3searchalgolia.indexer.sys_file_metadata.fields. This is set by default as follows:

module {
    tx_typo3searchalgolia {
        indexer {
            sys_file_metadata {
                fields {
                    title = title
                    description = description
                    alternative = alternative
                    creator = author
                }

                # Comma-separated list of allowed file extensions
                extensions = pdf
            }
        }
    }
}
Copied!

Context menu 

A single file can also be directly enqueued using the TYPO3 context menu in the file list:

file-context-menu

Fig. 2: Directly enqueue a file

Usage and Best Practices 

Supported File Types 

Currently, the File Indexer fully supports:

  • PDF files (including content extraction)
  • Other file types (metadata only)

The supported file extensions can be configured in the TypoScript settings as shown in the Custom Fields section.

Triggering File Indexing 

Files are indexed when:

  • The indexing is triggered through the backend module
  • A file is directly enqueued via the context menu
  • A scheduled task runs the indexing process

Best Practices 

  1. File Collections: Organize your files into logical file collections to make indexing management easier. This allows you to selectively index different types of files for different purposes.
  2. Metadata Quality: Ensure your files have proper metadata (title, description, etc.) as this significantly improves search quality. Well-described files are more likely to appear in relevant search results.
  3. File Size Considerations: Be mindful of very large PDF files, as content extraction can be resource-intensive. Consider setting size limits for content extraction in custom implementations.
  4. File Extensions: Only enable content extraction for file types that contain searchable text. Adding non-text file types to the extensions list won't provide useful search content.
  5. Security Awareness: Remember that indexed file content becomes searchable. Ensure that sensitive documents are either excluded from indexing or properly access-restricted in your search implementation.

Indexers 

Indexers are the core components of the TYPO3 Search Algolia extension that extract and process data from various TYPO3 content types for search indexing. They serve as the bridge between your TYPO3 content and the Algolia search service, ensuring that your content is properly structured and optimized for search.

General 

An indexer provides the necessary knowledge for processing the associated data, i.e., it knows exactly which data needs to be read, processed, and indexed, how, and where. Processing can be customized and controlled via an additional configuration (indexing service).

There can only ever be one indexer for a data type. However, there can be multiple indexing services that configure the respective indexer.

To add additional indexers, see: Custom Indexer

Available Indexers 

The following indexers are already implemented:

Standard Indexed Fields 

Depending on the indexer, certain fields of a record are indexed. The following fields are common to all:

Field Description
uid The UID of the record.
pid The parent ID of the record.
type The type of the indexed record. Correlates with the table name.
indexed The timestamp at which indexing took place.
created The timestamp at which the record was created (only included if available).
changed The timestamp at which the record was last modified (only included if available).

News Indexer 

The News Indexer is a specialized component of the TYPO3 Search Algolia extension that indexes news articles from the TYPO3 news extension (ext:news). This enables users to search for news content through your Algolia-powered search, making your news archive more accessible and useful.

How It Works 

The News Indexer processes news articles based on the configuration in your indexing service. When triggered, it:

  1. Retrieves news articles from the TYPO3 database
  2. Extracts relevant data from each news article, including title, teaser, content, and metadata
  3. Creates individual search records for each news article
  4. Sends the processed data to Algolia for indexing

This integration allows your site visitors to find relevant news articles quickly through search, improving the discoverability of your news content.

Configuration 

To use the News Indexer, create a new indexing service record and select "News" as the indexer type. For detailed instructions on setting up an indexing service, see the Configuration documentation.

News Properties 

By default, all news articles will be indexed. If you want to exclude specific news articles from indexing, you can implement this through a custom extension of the News Indexer.

Indexed Fields 

Custom Fields 

Additional fields to be indexed from the news properties can be defined using the TypoScript configuration module.tx_typo3searchalgolia.indexer.tx_news_domain_model_news.fields. This is set by default as follows:

module {
    tx_typo3searchalgolia {
        indexer {
            tx_news_domain_model_news {
                fields {
                    title = title
                    abstract = teaser
                    bodytext = description
                    author = author
                    author_email = authorEmail
                    keywords = keywords
                }
            }
        }
    }
}
Copied!

Usage and Best Practices 

Triggering News Indexing 

News articles are indexed when:

  • They are created or updated
  • The indexing is triggered through the backend module
  • A scheduled task runs the indexing process

Best Practices 

  1. News Record Quality: Ensure your news articles have proper titles, teasers, and content. Well-written news articles with descriptive titles and comprehensive content will yield better search results.
  2. Metadata Usage: Make good use of the author, keywords, and categories fields in your news records. These fields can enhance search relevance and enable filtering in search results.
  3. News Archive Structure: Consider how your news categories and archive structure should be reflected in search results. You might want to include category information in the indexed data for better context.
  4. Pagination Handling: For news archives with many articles, consider implementing pagination in your search results to handle large result sets efficiently.
  5. Date-Based Relevance: Consider configuring your search to factor in the publication date of news articles, potentially giving more weight to newer articles in search results.
  6. Related Content: If your news articles relate to other content on your site, consider how to link between search results for news and other content types.

Page Indexer 

The Page Indexer is a core component of the TYPO3 Search Algolia extension that indexes TYPO3 pages for search. It extracts relevant information from pages, including titles, metadata, and optionally content elements, making them searchable through Algolia.

How It Works 

The Page Indexer processes pages based on the configuration in your indexing service. When triggered, it:

  1. Retrieves pages according to your configuration (single pages or recursive page trees)
  2. Filters pages based on their properties (page type, no_search flag, etc.)
  3. Extracts relevant data from each page
  4. Optionally includes content elements if configured
  5. Sends the processed data to Algolia for indexing

Page Properties 

Indexing depends on the page properties. Depending on the status of the page properties, the pages and/or subpages can be indexed or not.

To actively exclude a page from indexing, deactivate the "no_search" property in the page properties. This is located in the "Behavior" tab by default.

page-no-search

Fig. 1: Exclude page from indexing

Indexed Fields 

Standard Fields 

In addition to the Indexers.md#standard-indexed-fields, the following fields are indexed by default for pages:

Field Description
site The domain name of the page.
url The absolute URL to the page (only included if available).
content The page content (only included if the content elements of a page are indexed). See also: Configuration.md#page-indexer-configuration

Custom fields 

Additional fields to be indexed from the page properties can be defined using the TypoScript configuration module.tx_typo3searchalgolia.indexer.pages.fields. This is set by default as follows:

module {
    tx_typo3searchalgolia {
        indexer {
            pages {
                fields {
                    title = title
                    subtitle = subTitle
                    nav_title = navTitle
                    description = description
                    abstract = teaser
                    author = author
                    keywords = keywords
                }
            }
        }
    }
}
Copied!

Usage and Best Practices 

Triggering Page Indexing 

Pages are automatically indexed when:

  • They are created or updated
  • The indexing is triggered through the backend module
  • A scheduled task runs the indexing process

Best Practices 

  1. Selective Indexing: Only index pages that are relevant for search to keep your index size manageable.
  2. Page Types: Consider which page types should be indexed. Typically, you'll want to index standard content pages but exclude special page types like folders, shortcuts, or backend modules.
  3. Content Elements: Decide whether to include content elements directly in the page index. For sites with many content elements per page, it might be better to use a separate Content Element Indexer to avoid hitting Algolia's record size limits.
  4. Metadata: Ensure your pages have proper metadata (titles, descriptions, keywords) for better search results.
  5. Hierarchical Structure: Consider how your page hierarchy should be reflected in search results. You might want to include parent page information in child pages for context.

Requirements & Installation 

System Requirements 

  • TYPO3 v12.4
  • PHP >=8.3 and <8.5
  • Composer
  • Algolia account with API credentials

Dependencies 

  • The extension requires access to the Algolia API for indexing and searching content
  • For indexing news articles, the TYPO3 news extension (ext:news) must be installed

Installation 

The extension should be installed via composer:

composer require meine-krankenkasse/typo3-search-algolia
Copied!

After installation:

  1. Update the database structure using the "Analyze Database Structure" tool in the TYPO3 backend
  2. Configure your Algolia API credentials (see Configuration)
  3. Set up your indexing services and search engine (see Configuration)

Uninstallation 

To remove the extension from your TYPO3 installation:

composer remove meine-krankenkasse/typo3-search-algolia
Copied!

After uninstallation, you may want to:

  1. Remove any remaining database tables
  2. Clean up any configuration in your additional.php file
  3. Delete any indexed data from your Algolia account

Development & Testing 

The extension includes several tools for development and testing:

# Install dependencies
composer install    

# Check coding guidelines
composer ci:cgl

# Run all tests
composer ci:test

# Run specific tests
composer ci:test:php:phplint    # Check PHP syntax
composer ci:test:php:phpstan    # Static analysis
composer ci:test:php:rector     # Code quality checks
composer ci:test:php:fractor    # Code quality checks
Copied!

These commands help ensure code quality and compatibility with TYPO3 coding standards.