Faceted Search

Extension key: ke_search
Package name: tpwd/ke_search
Version: main
Language: en
Author: ke_search Dev Team
License: This document is published under the Open Content License.
Rendered: Fri, 04 Jul 2025 11:31:08 +0000

Extension to search and filter content of the TYPO3 installation. It indexes directly from the database with build-in indexers and allows to add custom indexers. Provides indexers, plugins for searching and filtering, a backend module and a command-line interface.

Table of Contents

Introduction

ke_search is a TYPO3 search engine. It allows to search content stored in the TYPO3 database (pages, content elements, news and other extension content) and files.

It offers faceting possibilities to enhance the search experience. Faceting means you can narrow down your search results by selecting certain categories, called facets or filter options. By using faceting you can also create applications which are not related directly to fulltext search but make more use of filtering the content. Good examples would be a product finder for companies or a study finder for universities.

By writing your own indexer you can put any content you want into the index.

ke_search does not use frontend crawling but fetches content elements and data records directly from the database. This approach has the advantage that content will only be stored once in the index, even if it shows up on multiple pages of the website. For each type of content (pages, news, files ...) there has to be a dedicated indexer available. That means there may not be an indexer already available for the content type you want to index. On the other hand, it's quite easy for a programmer to write custom indexers for custom data records. A set of indexers for common content types comes bundled together with ke_search (including pages, news and pdf files).

ke_search uses the MySQL fulltext search algorithm, so it does not need any additional tools installed on the server. But you will need to install tools if you want to use file indexing (PDF, XLS, DOC files).

System requirements

ke_search requires MySQL / MariaDB, since it uses the MATCH … AGAINST function.
For file indexing additional tools are required: pdftotext, pdfinfo, catdoc, catppt, xls2csv.

Source code

The source code is managed at

https://github.com/tpwd/ke_search

Examples

Indexer configuration

Start indexing in the backend

The indexing progress is shown (since version 5.4.0).

Basic result list

Live examples

Here are examples on how a search result list can be used to display either a more classical list of search results or results in form of tiles. It is even possible to use only the faceting possibilities and skip the fulltext search.

Quick start

Follow the steps below to set up a simple fulltext search for your pages. In order to use the faceting feature see "Faceting".

Download and installation

Install the extension ke_search via extension manager or via composer (recommended):

composer require tpwd/ke_search

You can find the current version (and older ones) at

https://extensions.typo3.org/extension/ke_search

Include TypoScript

Site Sets

New in version TYPO3 v13.1 / ke_search v6.3.0

If you are working with TYPO3 v12.4, use TypoScript sets (the traditional way).

The extension ships some TypoScript code which can be included in the site configuration via Site sets:

Got to backend module Site Management > Sites.
Edit the configuration of your site.
On the first tab go to Sets for this Site.
Include the set Faceted Search (ke_search).

TypoScript sets (the traditional way)

If you are not using Site Sets you can add the TypoScript via the TypoScript module (formerly known as "Include static template"):

Got to backend module Site Management > TypoScript.
Select Edit TypoScript record --> Edit the whole TypoScript record.
On the first tab go to Include TypoScript sets.
Include the set Faceted Search (ke_search).

Create pages

Create a new page called "Search" (or similar) and a sysfolder called "Search data" (or similar).

Configure Plugins

You need to create two plugins: The searchbox and the resultlist.

Faceted Search plugin in new content element wizard

Create a plugin Faceted Search: Show searchbox and filters on the page Search

Fill in the field Record Storage Page in the Tab Plugin > General with the folder that you created in step 2 (our example: Search data).

Hint

It is useful to give the plugin Searchbox and Filters a header (our example: Searchbox, can also set to hidden):

That makes it easier to identify the correct content element in the next step.
Create a plugin Faceted Search: Show resultlist on the page Search

In the field Load FlexForm config from this search box fill in the Search-Plug-In that you created in the previous step (our example: Searchbox).

After this steps, you should have two plugins on your search page.

Create the indexer configuration

Use the List module to create an indexer configuration on the page Search data.

Choose a title.
Set the Type to Pages.
Set the Storage to your folder Search data.
Choose the pages you wish to index. You can decide whether the indexing process runs on all pages recursively or if only one page will be indexed. You can combine both fields.

Start Indexer

Open the backend module Web > Faceted Search and start the indexing process.

You're done!

Open the Search page in the frontend and start finding ...

Indexing

ke_search fetches content from pages, news, files etc. and stores it into an index table. This process is called "indexing". For each content type (pages, news etc.) ke_search needs an "indexer" (see below, Available indexers).

Whenever the content in your website changes, the indexing process needs to be started to reflect that changes in the search results.

Create indexer configurations

You will have to create an indexer configuration for each content type you want to index (pages, news, ...). You may create more than one indexer configuration of one type, eg. two page indexer configurations for different page trees.

Use the list module an open your search storage page and create an "indexer configuration" record.

Configure the indexer configuration

Set the title, this field is used only internally.
Set the storage page for the index. Set this page to your search storage folder.
If you are working with filters, you can define that every index entry gets a tag automatically if it has been indexed by this indexer by setting Add tag to all indexed elements to a filter option you like. By doing so you can for example create a filter by content type (news, page, event, ...) if you have an indexer for each content type. Please remember that you will have to create the filter options first.

The other indexer configurations options differ from type to type.

Full indexing and incremental indexing

Since version 3.8.0 there are two ways of indexing the data: full and incremental.

Note

Incremental indexing is a lot faster than full indexing!

Full indexing goes through all the data which should be indexed (records, files, custom indexers) and checks for each record if it has been changed or if it is new. In this case the data will be updated or stored to the index. After that a cleanup process is started and all old data will be deleted.

The incremental indexing process fetches only the records which have been added, changed or deleted since the last indexing process took place. It only adds, updates or deletes those records from the index.

Incremental indexing has a few drawbacks:

Not every indexer has incremental indexing capabilities. The indexer class needs to implement the method startIncrementalIndexing. If this method does not exist, a full indexing is started for this indexer even if the indexing process is started in incremental mode. Since there is no cleanup in incremental mode, old entries won't be deleted in incremental mode if the indexer does not support incremental indexing. The indexing report will mention if no incremental indexing is available ("Incremental indexing is not available for this indexer, starting full indexing.")
Changes to files won't be recognized if you use the page or content element indexer, only if you use the dedicated file indexer.

Therefore it is recommended to run the full indexing process once in a while (like once a day or once a week) and run the incremental indexer more often (like once an hour). You can do so by creating two scheduler tasks, one for the full indexing and one for the incremental indexing.

After adding or removing an indexer configuration you should always run a full indexing process.

Starting the indexing process manually

You can start the indexing process in the Faceted Search backend module.

You can also start the indexer using the command line

vendor/bin/typo3 ke_search:indexing

Or if you want to use the incremental mode

vendor/bin/typo3 ke_search:indexing --indexingMode=incremental

Starting the indexing process automatically

For keeping the index up-to-date it is recommended to use the TYPO3 scheduler.

You can execute the console command via scheduler.

Create a task, choose "Execute console commands".
Choose ke_search:indexing in the dropdown Schedulable command.
After saving the form you can choose whether you want to do full indexing (default) or incremental indexing by setting the option indexingMode to either full or incremental.
Deactivate the Allow Parallel Execution option (default).

Available indexers

ke_search comes with indexers for the most important content types.

Pages

The page indexer indexes standard TYPO3 pages.

All content elements on a page will be grouped and written to the index in one index entry. That means if your search word appears in two different text elements on a page, you will get only one search result for the page these two elements belong to.

There is a set of default content element types which are indexed without further configuration needed. If you created your own content elements (e.g. with EXT:mask or EXT:content_blocks) you can add them to the configuration as described below.

Configuration

Set the type of the indexer configuration to Pages.
Set a title (only for internal use).
Set the field Storage to the folder where you want to store your search data.
Set Startingpoints (recursive) to the pages you want to index recursively.
Set Single Pages to the pages you want to index non-recursively.

Content element types

By default the page indexer indexes content element of the following types which are delivered by the TYPO3 core:

Text (CType text)
Text with image / Media element (CTypes textmedia and textpic)
Bullet list (CType bullets)
Table (CType table)
Plain HTML (CType html)
Header (CType header)
File lists (CType uploads)
Referenced content elements (CType shortcut)

Additionally it indexes content elements provided by the EXT:bootstrap_package:

Accordion (CType accordion)
Tab (CType tab)
Carousel (CTypes carousel, carousel_fullscreen, carousel_small)
Icon Group (CType icon_group)
Card Group (CType card_group)
Timeline (CType timeline)

File indexing

Files linked in RTE text fields will be detected and indexed. Also files linked in fields of type "link" (e.g. header_link) will be detected.

Container and Gridelements

If content elements are placed inside a container or gridelement, the indexer will index these content elements as well, even if they are nested inside other containers or gridelements. If a container is inserted via the "shortcut" content element on another page, the content will be indexed as well (after version 6.3.0), but not recursively.

Abstract

In the page properties there's the field Abstract for search result in the tab Search. Here you can enter a short description of the page, this text will be used as an abstract in the search result list. If this field is empty, it falls back to the field Description in the Metadata tab of the page properties. In the FlexForm of the search plugin you can define the "Text for used for search result (preview text)" and set it to "Always show the abstract if set" or "Show abstract if it contains searchword, otherwise show an excerpt from the content".

Advanced options

If you set Index content elements with restrictions to yes, content elements will be indexed even if they have frontend user group access restrictions. This function may be used to "tease" certain content elements in your search and then tell the user that he will have to log in to see the full content once he clicks on the search result.
If you created custom page types which you want to index, you can add them in Page types which should be indexed set the page types you want to index.
in Content element types which should be indexed you can add your own content element types. For example those created with EXT:mask or EXT:content_blocks. If you are not sure what to enter here, have a look a the table tt_content in the column CType or activate TYPO3 backend debug mode.
(since version 5.3.0) In Additional tables for content elements you can define tables which hold additional content. That is used for example by EXT:bootstrap_package, EXT:mask or EXT:content_blocks. See below ("Index content from additional tables") for details.
In tt_content fields which should be indexed you can define custom fields which should be indexed. Default is here "bodytext,subheader, header_link" which is used for the default content elements. This is useful if you added your custom content elements for example using EXT:mask or EXT:content_blocks.
Using the field Comma separated list of allowed file extensions you can set the allowed file extension of files to index. By default this is set to pdf,ppt,doc,xls,docx,xlsx,pptx. For pdf, ppt, doc and xls files you need to install external tools on the server.
Using the field tt_content fields which should be indexed for file references you can add fields from tt_content which hold file references and for which the attached files should be indexed.
You can choose to add a tag to all index entries created by this indexer.
You can choose to add that tag also to files indexed by this indexer.

Index content from additional tables (eg. mask, bootstrap_package, content_blocks)

Some extension Some extension like the widely used mask and bootstrap_package and content_blocks extensions store content not in the tt_content table but in additional tables which hold a reference to the record in tt_content.

Since version 5.3.0 it is possible to index those tables without the need for a 3rd party extension or custom indexer. In the field Additional tables for content elements you can configure those tables. The ini configuration format is used here.

Since version 5.6.0 / 6.1.0 it is possible to index sub-elements of additional tables. If you have repeating elements in a mask element which themselves have repeating elements you can define the parent table for the sub-elements here. Indexing will be done recursively.

You need to define the table name, the field which holds the reference to the tt_content table and the fields which should be indexed.

Options

first line (eg. [custom_element]): The content type, stored as CType in the table tt_content. You will also have to add this to Content element types which should be indexed. If your content element has multiple additional tables, you can have multiple configurations for the same CType by adding a dot and an index, e.g. "my_ctype.1", "my_ctype.2" which then will all internally be mapped to the configuration for "my_ctype".
table: This is the table that holds the content.
referenceFieldName: This is the field that holds the relation to the tt_content record (the UID of the record). In EXT:bootstrap_package it is named tt_content, in EXT:mask it is named parentid, in EXT:content_blocks it is named foreign_table_parent_uid.
parentTable: (since version 5.6.0 / 6.1.0) The parent table is an optional setting. It's only necessary if you want to index sub-elements of EXT:mask. For example If you have repeating elements in a mask element which themselves have repeating elements. You can define the parent table for the sub-elements here (see example below). Indexing will be done recursively. If set the database query will contain a "WHERE parenttable = ..." condition. This column exists in content elements from EXT:mask but not in content elements from EXT:bootstrap_package.
fields[]: A list of database fields which should be indexed. If the field is configured as type "file" in the TCA the indexer will check if it links to a file and index that file. Otherwise the field will be treated as a text field and will be indexed like other fields, e.g. the bodytext field in content elements. Links to files will also be resolved here and the files will be indexed.

Examples

Bootstrap Package

Add this to Additional tables for content elements to index the bootstrap package element "accordion" (remember to also add accordion to Content element types which should be indexed:

[accordion]
table = tx_bootstrappackage_accordion_item
referenceFieldName = tt_content
fields[] = header
fields[] = bodytext

Mask

Add this to Additional tables for content elements to index mask elements (remember to also add mask_list and mask_mytest to Content element types which should be indexed:

[mask_list]
table = tx_mask_content
referenceFieldName = parentid
fields[] = tx_mask_content_item

Mask with multiple additional tables

This is an example how to add multiple additional tables for the same CType.

[mask_mytest]
table = tx_mask_repeating1
referenceFieldName = parentid
fields[] = tx_mask_name

[mask_mytest.1]
table = tx_mask_repeating2
referenceFieldName = parentid
fields[] = tx_mask_title

Mask with sub-elements in additional tables

This is an example how to index sub-elements of additional tables (note the parentTable configuration line).

[mask_mytest]
table = tx_mask_repeating1
referenceFieldName = parentid
fields[] = tx_mask_name

[mask_mytest.1]
table = tx_mask_repeating2
parentTable = tx_mask_repeating1
referenceFieldName = parentid
fields[] = tx_mask_title

More Mask examples

This is an example for a some mask elements:

The element mask_custom_text_element adds a field tx_mask_customtext to the tt_content table.
The element mask_custom_file_download adds a file download field tx_mask_file to the tt_content table.
The element mask_list stores content in the table tx_mask_content.

Example for indexing a custom elements created with mask 1/2

Example for indexing a custom elements created with mask 2/2

Content Blocks

In order to index content elements from EXT:content_blocks you need to add

the name of the CType to Content element types which should be indexed.
the name of the tt_content field to tt_content fields which should be indexed.
(optionally) the configuration for the additional table(s) to Additional tables for content elements.

In order to find out the correct names you can activate the TYPO3 backend debug mode.

Finding the field name for a content element made with content blocks

Content Elements

The content element indexer is much like the pages indexer with one important difference: The content element indexer allows you to index single content element and every content element will be written to a single entry in the index.

That means you will be able to directly point the visitor to the matching content element but on the other side, you will have multiple index entries for one page.

In most cases, the pages indexer is more suitable than the content element indexer because in most cases you only want to have one index record per page.

The content element indexer can be configured in the same way as the pages indexer.

The access restrictions for content elements are fully taken into account (see pages indexer).

Every content element inherits the access restrictions from the page it belongs to!

Note

There's another important difference:

Indexing your content on element basis instead of page basis means on the one hand you will get two results if you have two text elements containing your search word on one page, which may be unwanted in most cases. On the other hand, all access restrictions are taken fully into account, while by using the pages indexer only the page access restrictions will be taken into account, not the content element access restrictions.

In the result list, the result will link to the content element directly (via an anchor link).

News (EXT:news)

With this indexer you can index news from the extension "news" (extension key "news", not "tt_news").

The following fields will be indexed:

title
teaser
content
author
author email
keywords

Access limitations will be taken into account. You can either index all news or news from certain categories.

An image will be shown in the result list if you activate that setting in the ke_search searchbox plugin. The first assigned image will be shown.

Configuration

In order to index news, create a new indexer configuration an configure it as follows:

Title: just for internal use
Storage: the folder all your search data is saved in.
Target page: The page your news detail view plugin is placed on.
Type: has to be "News (news)".
Record storage page(s) single/recursive: Folders with news data to index.
Category selection: Here you can define if you want to index all news or just news from selected categories. Important: This selection is not recursive. You have to select each category individually if you don't select "all".
Add tag(s) of parent folder: If you added a tag to the folder containing news, this / these tag(s) will be added to the news index entry.
File indexer: Define which extensions of related files will be indexed.
Attached files should be handled as: Define if the contents of related files will be indexed as separate index records or as content of the news index record.
Add tag to all indexed elements: You can select an already existing filter option / tag to add it to all indexed elements.

Alternative single view page from category

In the indexer configuration you can set a target page. When a news appears in the result list, the link will go to that page. On that page there has to be a news plugin showing the detail view.

If a news has a category assigned to it which has an alternative single view page, this page will be used as target page.

Filtering with system categories

The extension "news" allows you to categorize with categories, keywords and tags.

In order to use news with faceting, indexed news get tagged automatically. But you will still have to create a filter option with the corresponding tag.

These tags are generated as follows:

from news-categories (system-categories)
from news-keywords
from news-tags

Example: If you want to filter for seasons, you can use the news categories "summer" and "winter". In ke_search you will then have to add the filter options eg. "All about the summer" and "Winter stuff" with the tags corresponding to the categories names: "summer" and "winter". Make sure the categories and the ke_search tags are spelled the same. For tags and keywords it's the same principle.

Tags will be generated by applying the rules for tags (no spaces and special characters). For example, if you have category "Blue cars!", a tag named "Bluecars" will be created.

For categories there will also be tags which are based on the string "syscat" and the id of the category, eg. "syscat123". This tags don't change if the category name changes.

To automatically create and update filters for these tags, see Using system categories for filtering.

File indexing

Files attached to news records will be indexed. You can specify in the indexer configuration whether to include the content of the files into the news record search result, that means they will appear as one result, or to index files separately, making them show up as a individual result.

You can also specify which files should be indexed by defining a comma-separated list of file extensions. If you leave this field empty, no files will be indexed.

News (EXT:tt_news)

With this indexer you can index news from the extension "news" (extension key "tt_news", not "news").

The following fields will be indexed:

title
short
bodytext
author
author_email
keywords

Access limitations will be taken into account.

An image will be shown in the result list if you activate that setting in the ke_search searchbox plugin. The first assigned image will be shown.

Configuration

In order to index news, create a new indexer configuration an configure it as follows:

Title: just for internal use
Storage: the sysfolder all your search data is saved on.
Target page: The page your news detail view plugin is placed on.
Type: has to be "News (tt_news)".
Pages/folders recursive/single: Sysfolders with news data to index.
Add tags of parent pages/folders: If you added a tag to the sysfolder containing news, these tags will be added to the news index entry.
Add tag to all indexed elements: You can select an already existing filter option / tag to add it to all indexed elements.

Alternative single view page from category

If a news record has a news category assigned which has a Single-view page for news from this category [single_pid] set, this will be used as single view and will overwrite the setting Target page of the indexer configuration. This is the same behaviour as if the corresponding tt_news constant set.

plugin.tt_news.useSPidFromCategory = 1

Note

This constant has no effect in ke_search. In ke_search this category is always used.

File indexing

You can also specify which files should be indexed by defining a comma-separated list of file extensions. If you leave this field empty, no files will be indexed.

Automated tagging

Tags will be automatically generated from keywords and from assigned categories by applying the rules for tags (no spaces and special characters). For example, if you have category "Blue cars!", a tag named "Bluecars" will be assigned to the index record. You can use these tags to create filters. Just use the same tag ("Bluecars") as tag in the filter option you are creating.

Addresses (EXT:tt_address)

The Address indexer allows you to index tt_address entries.

The following fields are indexed (if they are filled):

name
first_name
middle_name
last_name
company
address
zip
city
country
region
email
phone
fax
mobile
www

If set, the description is used as an abstract (search result list teaser).

Please notice that there is no single view in tt_address, the parameter tt_address[showUid] is nevertheless set. If you need another parameter – e.g. for use with another extension that handles tt_address records – you will have to modify the indexer content by using your own hook.

Files

The files indexer allows you to index content of files from the file system. Currently the indexer supports indexing of the following files: PDF, DOC, PPT, XLS, DOCX, PPTX, XLSX.

Note

Until version 1.2 only PDF and PPT files could be indexed.

There are two ways to index files:

Use the page or content element indexer. It will automatically detect links to files and index those files.
Use directory based file indexer. You can specify folder which should be indexed. You can either choose to use FAL (File Abstraction Layer), that will also index the metadata of files. Or you can decide not to use FAL. That will only index file contents and no file metadata.

An incremental indexer is available. It will detect changes by comparing the file modification time against the last indexing time and re-index those files.

Since version 4.6.0 it is possible to exclude files from indexing by deactivating the "Include in search" checkbox in the new "Search" tab of the file metadata.

System requirements

For PDF indexing you will need to have the external tools "pdfinfo" and "pdftotext" installed (in Ubuntu Linux they come with the package "poppler-utils").
For PPT indexing you will need to have the external tool "catppt" installed (in Ubuntu Linux it comes with the package "catdoc").

Please use the extension configuration to tell ke_search the file paths where to find these tools.

Directory based file indexer

You can specify the folders ke_search should index. Since the files are indexed directly from the file system, there's no access check! Please make sure only public content is in the folders you make searchable.

You can select a FAL storage from where you want to index files. If you do so, FAL metadata will be indexed. Categories will be used to generate tags (like in the pages and news indexer), this makes it possible to do faceting over files, see also Using system categories for filtering.

Configuration of the file indexer

Set a title (only for internal use).
Set the Record storage page of search data your search data folder.
Set the type to Files.
Select FAL storage or select Don't use FAL.
Define one or more directories which contain your files to be indexed. If you selected a FAL storage, the directories must be subdirectories of the selected storage, e. g. my_directory. If you selected Don't use FAL, you need to specify the folders including the storage path, e. g. fileadmin/my_directory. Multiple directories can be entered comma-separated. If you want to index all files in the given storage, just enter a dot (".") in the field Directories.
Enter the list of allowed file extensions. Only files with extensions the indexer supports will be indexed. If you use FAL indexing, you can also provide other filetypes, eg. JPG. From these files the metadata will be indexed.

The indexer will go recursively into the defined directories and index files in there.

FAL metadata will be indexed if you select a FAL storage (title, alternative text and description). Tags will be generated from categories (like in the news and pages indexer), see also Using system categories for filtering.

Content based file indexer

This indexer detects files while indexing pages and content elements and indexes the files automatically. Supported content element types are Text, Text with image and Filelinks.

Just create an indexer configuration of type "pages" or "content elements". Indexing of files will take place automatically.

File types which should be indexed can be specified in the indexer configuration. Leaving the field empty will have the effect that no files will be indexed.

Content restrictions from the linking content elements will be taken into account.

FAL metadata will be indexed (title, alternative text and description). Tags will be generated from categories (like in the news and pages indexer), see also Using system categories for filtering.

Language handling

When a file is linked to a content element or page, the language of that content element or page is used to determine the language of the file. The file metadata language is then indexed for that specific language. This means that if a file is linked to a page in English, the file will be indexed with the English metadata. If the file is linked to a content element in German, the file will be indexed with the German metadata.

If the file is not linked to a content element or page (meaning the file is indexed by a dedicated "Files" indexer and not linked from a content element), ke_search tries to detect the language of the file by looking at the file metadata. If no language is set in the file metadata, the file is indexed for all langauges (value -1). The file metadata of the default or "all" language is indexed (language values [0, -1], see findByFileUid in cms-core/Classes/Resource/Index/MetaDataRepository.php).

Custom Indexer

You may write your own indexer and plug it into ke_search.

Feel free to use that extension as a kickstarter for your own custom indexer:

https://github.com/tpwd/ke_search_hooks

Hint

Make sure you fill $pid, $type, $language and (important) $additional_fields['orig_uid']. These fields are needed for the check if a content element already has been indexed. If you don't fill them, it may happen that only one content element of your specific type is indexed because all the elements are interpreted as the same record.
You don't need to fill $tags if you don't use faceting.
You don't need to fill $abstract, it will then generated automatically from $content.
You will have to fill $params if you want to link to a extension which expects a certain parameter, e.g. &tx_myextension_pi1[showUid]=123

Extending existing indexers

The indexers shipped with ke_search have hooks built in which allow you to modify the indexed data without writing a custom indexer. For example the page indexer provides the hook modifyPagesIndexEntry.

Adding hidden content

It is possible since version 4.5.0 to add content to the index which is searched but not shown in the result list. This is e.g. useful for Synonyms, different spellings, additional keywords and so on. The event ModifyFieldValuesBeforeStoringEvent in the class IndexerRunner is used for that.

Indexing custom content fields

If you added fields to the tt_content table in order to use them with your own content elements, you can index these fields with the default page indexer, too.

Note

The hooks described below are not needed anymore to index custom fields in tt_content since ke_search version 4.5.0. Since then you can simply configure those fields in the indexer configuration. see indexer for pages.

Two hooks are needed:

One adds the new field to the list of fields fetched from the tt_content table,
the other one adds the field to the content written to the index.

You need to register the hooks in your ext_localconf.php as follows:

// Register hooks for indexing additional fields.
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['ke_search']['modifyPageContentFields'][] =
   \MyVendor\KeSearchHooks\AdditionalContentFields::class;

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['ke_search']['modifyContentFromContentElement'][] =
   \MyVendor\KeSearchHooks\AdditionalContentFields::class;

Hook class

class AdditionalContentFields {

   public function modifyPageContentFields(&$fields, $pageIndexer)
   {
      // Add the field "subheader" from the tt_content table, which is normally
      // not indexed, to the list of fields.
      $fields .= ",subheader";
   }

   public function modifyContentFromContentElement(string &$bodytext, array $ttContentRow, $pageIndexer)
   {
      // Add the content of the field "subheader" to $bodytext, which is, what
      // will be saved to the index.
      $bodytext .= strip_tags($ttContentRow['subheader']);
   }

}

Example

You can find an example in the extension "ke_search_hooks": https://extensions.typo3.org/extension/ke_search_hooks

Backend

Backend module

In the backend you can use the dedicated module to start the indexing process, view the indexed content or statistical data.

Scheduler task

A scheduler task is provided to start the indexing process automatically (see Indexing).

Dashboard widgets

There are also widgets for the dashboard which show some information at a glance.

Filters / Faceted Search

ke_search comes with a faceted search feature which allows the website visitors to narrow down the search result list by using filters.

Filters are either based on tags (also known as "filter options") or on a date.

For tag based filters you'll have to create "filters" and "filter options". Filter options appear in the searchbox in the frontend and at the same time you can set them as "tags" for pages (or other content) in the backend.

New in version ke_search v6.4.0

Index tag titles as hidden content: If activated in the extension settings, tag titles will be indexed as hidden content. This is useful if you want to search for the titles of filter options in the fulltext search. For example if you have a filter option "Sweatshirt" and you want to find all products with this filter option, you can search for "Sweatshirt" in the fulltext search and all products with this filter option will be found.

Examples

Example "Car Website"

You have a website that's about cars. On some pages, your content is about tires, others are about "Accessories". You can now create a filter named "Accessories" and filter option named "Tires". Now when your customer uses the search function, she or he can narrow down the search to all pages marked with "Tires". That does not mean, that "Tires" must be on that page as a word, but it's marked as "relevant for tires" in the backend.

Example "University Courses"

As university you offer courses with different degrees. You could create a page for each course and add filter options for "bachelor" and "master" and let the users select which courses they want to see.

Using Filters

The next pages will show you how to setup and configure the filters:

Setup filters

Follow these steps to create filters and set up the faceted search:

Create a filter

Go to your search storage folder and use the list module to create a "filters" record. For each category you want to use in your faceted search in the frontend you will have to create one filter.
Create filter options

Add new filter options by adding them inside the filter record. For each option you want to display in the frontend you will have to add one filter option.
Tags

Tags are used internally to mark content as relevant for a certain filter option. You will have to choose a tag name for each filter.
Important
- You may freely choose a tag name, they're only used for internal purposes. But make sure the tag you choose for each filter option is unique. If you decide to give the same tag to different filter options, they will show the same search results. Although, this may be a desired behaviour in some cases.
- The tag has to be at least four characters long and must contain only alphanumeric characters. Up to version 3.3.1 it was possible to use dashes, colons, spaces and other non-alphanumeric characters as tags. This behaviour has changed and you may need to re-index when tags have changed or adjust your custom scripts if you set tags in your scripts.
- You must not use "syscat" as prefix for your tag names. This is reserverd for the automated generation of filter options, see Using system categories for filtering.
You have two possibilities to assign tags to pages:
1. Open the page properties and in the tab Search you find field Tags for faceted search
2. Use the function Set tag for all children of this page in the filter record itself.
If you use the option Set tag for all children of this page the tag will be set automatically to the subpages of the page you set while indexing that pages (you can select multiple pages).

With the exclude option you can prevent child pages from being tagged automatically.

If you do not want to set the tag for pages automatically, you can choose to set the tag on each page manually in the tab Search in the page properties.

The tags will be added to the index entry of that page at the time the indexer reads that page and writes its content to the index.

Note

You will first have to create at least one "filter" and one "filter option" to see any items in this list.

If you have a multilingual website, the tagging can be done only in the main language. But you can translate the filter options so that they will be visible in the frontend in the correct translation.

The filter options are coming from the whole system, no matter on what page you created them.

If you have more than one search plugin, you may want to restrict the filter options displayed here to a certain folder. You can do this by adding this to your Page TSconfig, where 1234 is the uid of your folder where the desired filter options are stored:
```
tx_kesearch.filterStorage = 1234
```
Copied!
Add filter to search plugin

Open your search plugin and select the filters you want to display in the tab "filter".

The filter will then be displayed in the frontend.

Hint

Note on indexing: The tags will be applied after the next indexing. So whenever you change the filters, re-index after that.

Tag based filters

There are three different types for filters which are based on tags (in opposite to the date range filter):

Selectbox

This options renders a dropdown selectbox.

List

This option renders link for each filter option. The list is put into a box which can be expanded. You can define if this list should be automatically expanded or not. You can define an additional CSS class in order to have a rendering that fits your website (smaller box, larger box).

Checkbox (multiselect, OR)

This mode has the same settings as the list mode. But there is an additional setting to mark all checkboxes as default. In this mode all filter options will be combined with logical "OR" in the search query.

Hidden filters

With this option you can add filter which are not changeable in the frontend (using the field preselected filter options). You can use this in order to reduce the result list to results matching a certain filter option. For example on a website with pages and news you can present a list of all indexed news.

Date range filter

The date range filter allows the user to enter a start date and an end date and finds results in between these two dates.

Add the date range filter

Create a filter

Create a new filter in your search storage page and set the type to "Date range". In opposite to the tag based filters there are no filter options to define.
Add filter to search plugin

Then add that filter to the list of filters which should be displayed in your search box plugin.

Where the date is fetched from

The date which is used for filtering is the same date which is used if the results are sorted by date. It is the field sortdate in the index table.

The page indexer fetches the date from the database field SYS_LASTCHANGED which is set by TYPO3 internally to the date when the page has been updated the last time. This can be overwritten by editors by setting the Last update field in the Metadata tab of the page properties.

The news indexer fetches the date from the "Date & Time" field of EXT:news.

Templating

Use a custom template for Partials/Filters/DateRange.html.

Using system categories for filtering

You can use the system categories the TYPO3 core provides for filtering.

ke_search automatically creates tags for assigned system categories, this applies only to the indexers for pages, files, news (EXT:news and EXT:tt_news) and (since version 4.6.0) tt_address as these records can have system categories assigned by default.

If you want to use system categories as filter options for other content, you will have to write a custom indexer or extend existing ones via a hook.

You can then select for which filter a category should be used as a filter option:

Create a "filter" record in your search data folder.
Select that filter in the tab "Search"

You can also choose to use sub-categories as filter options. You can select more than one filter if you want to use this category as a filter option for more than one filter.

Note

The filter options will be created via a backend hook whenever a category is edited (created, updated or deleted).
Up to version 3.1.6 ke_search only created the tags, you had to create the filter options yourself and use the auto-generated tag names.
For each assigned system category, two tags are created:
1. The first tag uses this naming schema syscat + UID of the system category (eg. syscat123). This tag is used in the auto-created filter options.
2. The second tag uses the title of the category. Non-alphanumeric characters will be removed.
If you use the tag derived from the category title, remember that this will change if the category title changes. It may be better to use the tag which relies on the uid of the category.

Custom Filter

You can provide your own code to render a filter based on the results and filter options ke_search has found.

Note

You can find the example below in the extension ke_search_hooks: https://extensions.typo3.org/extension/ke_search_hooks

These are the steps for adding custom filter:

In the ext_localconf.php file of your sitepackage extension add:

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['ke_search']['customFilterRenderer'][] =
    \Tpwd\KeSearchHooks\ExampleFilterRenderer::class;

Add the hook class

This is very basic example which replaces the "select" filter type with a custom type and uses a Fluid standalone view to render it. The class is placed in ke_search_hooks/Classes/ExampleFilterRenderer.php. You can freely choose the class name, the function name must be customFilterRenderer.

<?php

namespace Tpwd\KeSearchHooks;

use Tpwd\KeSearch\Lib\Pluginbase;
use TYPO3\CMS\Core\Utility\GeneralUtility;

class ExampleFilterRenderer
{
    public function customFilterRenderer(int $filterUid, array $options, Pluginbase $plugin, array &$filterData)
    {
        if ($filterData['rendertype'] == 'select') {
            /** @var \TYPO3\CMS\Fluid\View\StandaloneView $view */
            $view = GeneralUtility::makeInstance(\TYPO3\CMS\Fluid\View\StandaloneView::class);
            $view->setTemplateRootPaths([10 => 'EXT:ke_search_hooks/Resources/Private/Templates']);
            $view->setTemplate('Filters/CustomFilter');
            $view->assign('filter', $filterData);

            $filterData['rendertype'] = 'custom';
            $filterData['rawHtmlContent'] = $view->render();
        }
    }
}

In order to trigger the rendering of the custom filter the last two lines inside the loop are important:

$filterData['rendertype'] = 'custom';
$filterData['rawHtmlContent'] = $view->render();

When the "rendertype" is set to "custom", the given "rawHtmlContent" will be used instead of the rendering of ke_search itself.

Add the template

The template is placed in ke_search_hooks/Resources/Private/Templates/Filters/CustomFilter.html. Here we use radio buttons for the filter options.

<fieldset>
    <legend>Custom filter</legend>
    <div id="{filter.id}">
        <f:for each="{filter.options}" as="option">
            <input type="radio" name="{filter.name}" value="{option.value}" {f:if(condition: '{option.selected}', then: 'checked')} />
            {option.title}<f:if condition="{option.selected} == ''"><f:if condition="{filter.shownumberofresults} && {option.results}"> ({option.results})</f:if></f:if><br />
        </f:for>
    </div>
</fieldset>

Result

The result is a filter rendered as radio buttons.

Templating

The extension comes with some example CSS styles (which you can remove to use your own styles) and with fluid templates which control the frontend appearance.

Use your own CSS

ke_search comes with a default css file which is loaded automatically.

If you do not wish to use that file, you have the following possibilities.

Unset via TypoScript

If you do not wish to load that file, you can unset it via TypoScript:

plugin.tx_kesearch_pi1.cssFile >
plugin.tx_kesearch_pi2.cssFile >

Use plugin configuration

You can also make use of the field in the plugin configuration. This overwrites the default CSS file.

Use your own templates

In order to use your own fluid templates, please set the path to your templates in the TypoScript constants.

It's good practice to put the templates in a dedicated "site" package (an extension which holds all your templates, configuration and css files).

For example:

plugin.tx_kesearch.templateRootPath = EXT:mysite/Resources/Private/Templates/ke_search/
plugin.tx_kesearch.partialRootPath = EXT:mysite/Resources/Private/Partials/ke_search/
plugin.tx_kesearch.layoutRootPath = EXT:mysite/Resources/Private/Layouts/ke_search/

You can use the Constant Editor to set the paths to your templates, partials and layouts.

In order to use your own templates, please copy the default templates to your own package.

typo3conf/ext/ke_search/Resources/Private

It is not necessary to copy all templates, partials and layouts to adjust the templates, you can just copy the files you need to change. You will then have less hassle when templates change with a new version. All other files will fall back to the default.

Searchbox on every page

There are multiple ways to integrate the searchbox on every page.

By creating a content element and including this on every page
As plain HTML
Directly using TypoScript

Note

Since version 4.2.0 there's a plugin "Cachable searchbox". It is recommended to use this over "Searchbox" because this plugin won't disable caching of the entire page where it's used. Only drawback is that the plugin options won't get updated by the search currently performed, e. g. all the filter options will be shown even if they do not appear in the result list (the normal searchbox will only show the options which are available in the result list).

Best practice is to include the "Cachable searchbox" on every page and then on the result page use standard "Searchbox" plugin.

Including searchbox as a content element

Create a folder where you want to store the searchbox content element.
Insert a "Cachable searchbox" plugin and configure it as you wish.
Add that content element to every page with TypoScript, e. g.:

# Include searchbox as a content element
lib.searchbox = RECORDS
lib.searchbox {
   tables = tt_content
   source = 7
}

Where 7 is the UID of your searchbox content element. You can then include the searchbox in your main page template, e. g.:

page = PAGE
page.5 < lib.searchbox
page.10 < styles.content.get

On the search result page you should then remove the searchbox:

page.5 >

Include searchbox with plain HTML

# Include searchbox as plain HTML
lib.searchbox_html = TEXT
lib.searchbox_html.value (
   <form method="get" id="form_kesearch_searchfield" name="form_kesearch_searchfield" action="/search/">
     <input type="text" id="ke_search_searchfield_sword" name="tx_kesearch_pi1[sword]" placeholder="Your search phrase" />
     <input type="submit" id="ke_search_searchfield_submit" alt="Find" />
   </form>
)

# Default PAGE object:
page = PAGE
page.5 < lib.searchbox_html
page.10 < styles.content.get

The action "/search/" ist the slug of the page you created with your result list plugin.

Include searchbox with TypoScript

This is only possible without displaying filters as they are configured in a FlexForm. If you need filters, it's recommended to include the searchbox as content element as shown above.

# Include searchbox as a plugin
lib.searchbox_plugin = COA
lib.searchbox_plugin {
   10 < plugin.tx_kesearch_pi3

   # result page
   10.resultPage = 123

   # CSS file
   10.cssFile = EXT:ke_search/Resources/Public/Css/ke_search_pi1.css

   # Content element (search box plugin) from which additional
   # configuration should be loaded (UID of content element).
   # Important: If you have two search boxes on your result page
   # (eg. in the top and in the left area), you should set this value!
   # 10.loadFlexformsFromOtherCE = 123456
}

The number 123 in this case is a placeholder for the page ID you created with your result list plugin.

You can then include the searchbox in your main page template, e. g.:

page = PAGE
page.5 < lib.searchbox_plugin
page.10 < styles.content.get

On the search result page you should then remove the searchbox:

page.5 >

Images

ke_search renders images (icons and thumbnails) in the list view for the following cases.

Icons for the type of result (page, news, file, ...)
Preview images for pages
Preview images for news (using the first image of the news record)
Preview images for files (thumbnails will be created automatically for PDF and image files)

You can enable / disable them in the plugin configuration.

Type icons

You can change the icons which are used in the list view.

Example configuration (Template Setup):

plugin.tx_kesearch_pi2.resultListTypeIcon.page.file = EXT:mysite/Resources/Public/Images/example-icon.png

page stands for the record type and corresponds to the indexer type. For file formats like xls, doc etc. you can use file_xls, file_doc etc.

Page preview images

If enabled in the plugin configuration, the image set in the page properties Search > Search result image will be shown in the result list. If no image is set there, it falls back to Resources > Media.

Changing the size of images

To change the size of the images, you will have to adjust the corresponding fluid partial. Please have a look at the partial ResultRow.html in the section "typeIconOrPreviewImage".

Images in custom indexers

If you have implemented a custom indexer you can display images which are attached to the original record.

The image needs to be attached as FAL record to the original record.

The configuration setting shown below has to be added to your ext_localconf.php.

INDEXER_KEY is the key of your custom indexer (column type in the table tx_kesearch_index).
table refers to the column tablenames in the table sys_file_reference
field refers to the column fieldnames in the table sys_file_reference
TABLE_NAME is the table of the original record
IMAGE_FIELD_NAME is the column of the original record where the image is attached

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['ke_search']['fileReferenceTypes'][INDEXER_KEY]['table'] = TABLE_NAME;
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['ke_search']['fileReferenceTypes'][INDEXER_KEY]['field'] = IMAGE_FIELD_NAME;

This example shows the configuration for the fe_users table if your indexer configuration key is also fe_users:

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['ke_search']['fileReferenceTypes']['fe_users']['table'] = 'fe_users';
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['ke_search']['fileReferenceTypes']['fe_users']['field'] = 'image';

Template Selector

It is possible to provide selectable template layouts to the editors, eg. a "list layout" and a "tile layout", or different searchbox layouts.

Usage

Register your new template layout in the Page TSconfig. It will then appear in the plugin and the editor will be able to select it.

TCEFORM {
   tt_content {
      pi_flexform.ke_search_pi1.view.templateLayout.addItems {
         20 = Custom search box 1
      }
   }
}

Add the new layout inside a condition which checks for the setting conf.templateLayout.

<f:layout name="General"/>
<f:section name="content">
   <f:if condition="{conf.templateLayout}">
      <f:if condition="{conf.templateLayout} == 10">
         <f:render section="defaultLayout" arguments="{_all}"/>
      </f:if>
      <f:if condition="{conf.templateLayout} == 20">
         <f:render section="customLayout" arguments="{_all}"/>
      </f:if>
   </f:if>
</f:section>

<f:section name="defaultLayout">
   ... default template code
</f:section>

<f:section name="customLayout">
   ... custom template code
</f:section>

Custom values in the search result list

By default the indexed content will be shown in the search result list (either the abstract or a part of the indexed content).

The partial for a single result row is stored at ke_search/Resources/Private/Partials/ResultRow.html.

If you want to adapt your search result list to show other content than the indexed content, e.g. the content of individual database fields, you have the following possibilities.

Format date for different locales

The date of the search result will printed if the setting Show date is activated in the plugin settings. You will get the date as a unix timestamp and can use the <f:format.date> viewhelper to use different locales.

Example:

<f:format.date format="%d. %B %Y">{resultrow.date_timestamp}</f:format.date>

Accessing the original database row

The UID and the PID and also the original database row are available in the result list by using the following variables

{resultrow.orig_uid}
{resultrow.orig_pid}
{resultrow.orig_row.uid}
{resultrow.orig_row.title}
[...]

The orig_row is only available for pages, news and tt_address records or if the key for a custom indexer is exactly the same as the corresponding table name (e.g. create a custom indexer for frontend users and use the key fe_users). But you can also register your own table names. See ke_search/Classes/Domain/Repository/GenericRepository.php.

Add a hook

You can add a hook additionalResultMarker in order to add more variables to the fluid template, see ke:search/Classes/Lib/Pluginbase.php.

Write a view helper

You could also write a view helper. Since you have the type and the UID of the original record available, you could pass this to the view helper and you are then free to create whatever content you want.

Debug available values

By adding

<f:debug>{resultrow}</f:debug>

to the Fluid template, you can see all the available values for each result.

Link view helper

A link view helper is provided to generate links to search result pages including a search word and filters.

Parameters

page (integer): Target page
piVars (array): pivars (sword, filter[...], page, sortByField, sortByDir, resetFilters)
resetFilters (array): Filters to reset
keepPiVars (boolean): Should the piVars be kept?
uriOnly (boolean): Returns only the uri.
section (string): Anchor for links

Additionally the standard HTML universal attributes can be used (like class, dir, id, lang, style, title, accesskey, tabindex, onclick).

Setting filters

In order to use filters with the link view helper you will need to know the filter UID, the tag of the filter option and (if you use a checkbox filter) the UID of the filter option. Please see the example below.

Example

<html xmlns:f="http://typo3.org/ns/TYPO3/CMS/Fluid/ViewHelpers"
      xmlns:kesearch="http://typo3.org/ns/Tpwd/KeSearch/ViewHelpers"
      data-namespace-typo3-fluid="true">

<ul>
   <li><kesearch:link page="85" piVars="{sword: 'elephants'}" class="my-link-class">Search for elephants</kesearch:link></li>
   <li><kesearch:link page="85" piVars="{filter_13: 'page'}" class="my-link-class">Show all pages</kesearch:link></li>
   <li><kesearch:link page="85" piVars="{sword: 'elephants', filter_13: 'page'}" class="my-link-class">Search for elephants in pages</kesearch:link></li>
   <li><kesearch:link page="85" piVars="{sword: 'elephants', filter_13: 'page', filter_3_278: 'syscat92'}" class="my-link-class">Search for elephants in pages and use a checkbox filter</kesearch:link></li>
</ul>

</html>

Render filters in result list

Since version 3.9.0 it is possible to show filters also in the result list plugin which gives you more flexibility in placing them in relation to the result list.

In order to use this feature you need to enable the filters in the result list with the TypoScript setup setting plugin.tx_kesearch_pi2.includeFilters = 1.

This is useful if you want to show e.g. the filters in the right-hand side and only if they are present.

HowTo

Enable the filters in the result list with this snippet in the TypoScript setup:
```
plugin.tx_kesearch_pi2.includeFilters = 1
```
Copied!

Create a Resources/Private/Partials/FiltersForm.html:

This is a modification of the Resources/Private/Partials/Filters.html which looks like:

<f:for each="{filters}" as="filter">
   <f:switch expression="{filter.rendertype}">
      <f:case value="select"><f:render partial="Filters/Select" arguments="{conf: conf, filter: filter}" /></f:case>
      <f:case value="checkbox"><f:render partial="Filters/Checkbox" arguments="{conf: conf, filter: filter}" /></f:case>
   </f:switch>
</f:for>

Add a Resources/Private/Partials/FiltersResults.html which contains:

<f:for each="{filters}" as="filter">
   <f:switch expression="{filter.rendertype}">
      <f:case value="list"><f:render partial="Filters/List" arguments="{conf: conf, filter: filter}" /></f:case>
      <f:case value="custom"><f:format.raw>{filter.rawHtmlContent}</f:format.raw></f:case>
   </f:switch>
</f:for>

In Resources/Private/Templates/ResultList.html include:

<f:if condition="{filters}">
   <div class="filters filtersResults">
      <f:render partial="FiltersResults" arguments="{conf: conf, numberofresults: numberofresults, resultrows: resultrows, filters: filters}" />
   </div>
</f:if>

And in Resources/Private/Templates/SearchForm.html include:

<f:if condition="{filters}">
   <div class="filters filtersForm">
      <f:render partial="FiltersForm" arguments="{conf: conf, numberofresults: numberofresults, resultrows: resultrows, filters: filters}" />
   </div>
</f:if>

Configuration

The extension can be configured via the plugin settings (FlexForm), TypoScript and the extension configuration.

Additional word characters

By default MySQL treats certain characters as word delimiters, e.g. dot (".") and hyphen ("-"). That means words which contain one of these characters will be treated as two words and it is not possible to search for such a word.

But in some cases it would be helpful to be able to search for such words, e.g. serial numbers which contain a hyphen, e.g. "AB-123".

Since version 5.1.0 it is possible to make this words searchable. Go to the extension settings and add the desired characters in "Additional word characters". You can add multiple characters there.

After that you will have to start the indexer.

Words containing the configured characters can then be searched.

Search result with additional word characters

Allow only ke_search records on a page

Most likely you will create a sysfolder in which all the ke_search related data (indexer configurations, filters and the index) will be stored.

You can reduce the allowed records in the "New record" wizard to ke_search records by including the static Page TSconfig file "Restrict pages to ke_search records". This will reduce the items shown in the "New record" wizard to indexer configurations and filters.

Available records are reduced to ke_search indexer configurations and filters

Avoid 404 error from cacheHash validation

In TYPO3 10.4.35/11.5.23/12.2 an option enforcevalidation has been introduced to enforce the validation of the "cHash" parameter. This is recommended to be enabled and is enabled by default for new installations, see https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Configuration/Typo3ConfVars/FE.html#enforcevalidation

For ke_search it is not possible to calculate a cHash because ke_search uses a form using the GET method to send data and the options are modified by the user.

Therefore the default ke_search parameters are excluded in the ext_localconf.php using the ['FE']['cacheHash']['excludedParameters'] option.

If you have configured ke_search to use a different parameter for submitting the search word you will have to exclude the search word parameters manually:

If you have enabled the enforcevalidation option and
have changed the parameter for the search word using plugin.tx_kesearch_pi1.searchWordParameter

Example

// to be used in ext_localconf.php
$GLOBALS['TYPO3_CONF_VARS']['FE']['cacheHash']['excludedParameters'] = array_merge(
    $GLOBALS['TYPO3_CONF_VARS']['FE']['cacheHash']['excludedParameters'],
    [
        'q', // e.g. if "plugin.tx_kesearch_pi1.searchWordParameter = q"
    ]
);

Hint

If you don't add this setting you will get the following error: 404 Page Not Found The page did not exist or was inaccessible. Reason: Request parameters could not be validated (&cHash empty)

Highlighted Word

To overwrite the default HTML (<span class="hit"></span>) one may use the following TypoScript:

plugin.tx_kesearch_pi2.highlightedWord_stdWrap.wrap = <mark>|</mark>

TypoScript and FlexForm settings

Each property in FlexForm overwrites the property defined by TypoScript.

Each property has stdWrap properties.

With the following TypoScript, you can define the result page:

plugin.tx_kesearch_pi1.resultPage = 9

or you can define the result page with help of an URL param if you want:

plugin.tx_kesearch_pi1.resultPage.data = GP:tx_kesearch_pi1|resultPage

Notes on TypoScript and extension configuration

In Admin Tools > Settings > Extension Configuration you can define basic options like the minimal length of searchwords.

You can overwrite this configuration in your page TypoScript setup:

ke_search.extconf.override.searchWordLength = 3

ke_search_premium.extconf.override.enableSphinxSearch = 0

Override record storage page (Startingpoint)

It is possible to override the record storage page (or "Startingpoint") defined in the plugin using TypoScript. This is useful if you want to serve different search results depending on TypoScript conditions or if you include the plugin via TypoScript.

For example you could serve different search results to logged in users by setting the override record storage page to a different page id.

Setup TypoScript:

plugin.tx_kesearch_pi1.overrideStartingPoint = 123
plugin.tx_kesearch_pi1.overrideStartingPointRecursive = 1
plugin.tx_kesearch_pi2.overrideStartingPoint = 123
plugin.tx_kesearch_pi2.overrideStartingPointRecursive = 1

Reduce filters shown in the backend

If you have defined filter options in more than one page tree and you want to display only the filter options placed in a certain folder in the backend, you can set the following

Page TSconfig:

tx_kesearch.filterStorage = 123

Routing (Speaking URLs)

Speaking URLs (for TYPO3 9 and greater) can be achieved by adding a route enhancer configuration to the site configuration (see example below).

Speaking URLs can be configured for the search word, for filters, for the sorting and for the pagination.

There are two mappers provided to map tags to slugs (KeSearchTagToSlugMapper) and to use the search word as part of the route (KeSearchUrlEncodeMapper).

Example URL

https://example.org/search-page/score/desc/0/1/news-cat-1/search+word

Notes

Adjust the values for the parameter sortByField as it fits your needs (you may have different fields to sort by in your website).
For filters of type select and textlink you need one rule per filter. The rule is defined by using "filter" + "_" + the UID of the filter.
For filters of type checkbox (multi-select filters) you need one rule per filter option. The rules are defined by using filter + "_" + the UID of the filter + "_" + the UID of the filter option. Unfortunately this may render your URL very long if you have many filter options. Maybe there will be a better solution in the future.
Search words will be url encoded (eg. "schön" will become "sch%25C3%25B6n").
The StaticMappableAspectInterface can also be used for filters. In this case not the slug is used for URL generation but the tag. This may be useful in cases where you use the same tag for different filter options or if you have huge amounts of tags and want to improve performance (the KeSearchTagToSlugMapper accesses the database once for each routing parameter on every request).
For filters of type "select" or "list" you will need to set one character default value. That will be ignored in the filtering, but that is necessary to differentiate if the value is coming from the routing configuration or if the user wants to reset the filter (in that case an empty value is given). See also https://github.com/tpwd/ke_search/issues/126

Examples

These are examples for the site configuration file (config.yaml).

You need to adjust the filter UIDs (like in filter_13 where 13 is the UID of the filter) and the filter option UIDs (like in filter_3_267 where 267 is the UID of the filter option).

Simple example

This is a simple example with one filter and the search word mapped to a speaking URL.

If only a filter is given, this will give URLs like

https://www.example.org/search-page/filter-option

If additionally a searchword is given, this will result in

https://www.example.org/search-page/filter-option/score/desc/0/1/search-word

routeEnhancers:
   KeSearch:
      type: Plugin
      routePath: '{filter_13}/{sortByField}/{sortByDir}/{resetFilters}/{page}/{sword}'
      namespace: 'tx_kesearch_pi1'
      defaults:
         sortByField: 'score'
         sortByDir: 'desc'
         resetFilters: '0'
         page: '1'
         sword: ''
         filter_13: '-'
      requirements:
         sortByField: '(score|title|customranking|sortdate)?'
         sortByDir: '(asc|desc)?'
         resetFilters: '[0-9]?'
         page: '\d+'
         filter_13: '[0-9a-zA-Z-]*'
      aspects:
         sortByField:
            type: StaticValueMapper
            map:
               score: 'score'
               customranking: 'customranking'
               title: 'title'
         sortByDir:
            type: StaticValueMapper
            map:
               asc: 'asc'
               desc: 'desc'
         resetFilters:
            type: StaticRangeMapper
            start: '0'
            end: '1'
         page:
            type: StaticRangeMapper
            start: '1'
            end: '99'
         filter_13:
            type: KeSearchTagToSlugMapper
         sword:
            type: KeSearchUrlEncodeMapper

Full example

This is an example for a site configuration which adds multiple filters to the routing configuration. Filter no. 3 is a "checkbox" filter, therefore each filter option has to be a configured individually.

routeEnhancers:
   KeSearch:
      type: Plugin
      routePath: '{sortByField}/{sortByDir}/{resetFilters}/{page}/{filter_14}/{filter_13}/{filter_3_267}/{filter_3_273}/{filter_3_278}/{filter_3_283}/{sword}'
      namespace: 'tx_kesearch_pi1'
      defaults:
         sortByField: 'score'
         sortByDir: 'desc'
         resetFilters: '0'
         page: '1'
         filter_13: '-'
         filter_14: '-'
         filter_3_267: ''
         filter_3_273: ''
         filter_3_278: ''
         filter_3_283: ''
         sword: ''
      requirements:
         sortByField: '(score|title|customranking)?'
         sortByDir: '(asc|desc)?'
         resetFilters: '[0-9]?'
         page: '\d+'
         filter_13: '[0-9a-zA-Z-]*'
         filter_14: '[0-9a-zA-Z-]*'
         filter_3_267: '[0-9a-zA-Z-]*'
         filter_3_273: '[0-9a-zA-Z-]*'
         filter_3_278: '[0-9a-zA-Z-]*'
         filter_3_283: '[0-9a-zA-Z-]*'
      aspects:
         sortByField:
            type: StaticValueMapper
            map:
               score: 'score'
               customranking: 'customranking'
               title: 'title'
         sortByDir:
            type: StaticValueMapper
            map:
               asc: 'asc'
               desc: 'desc'
         resetFilters:
            type: StaticRangeMapper
            start: '0'
            end: '1'
         page:
            type: StaticRangeMapper
            start: '1'
            end: '99'
         filter_13:
            type: KeSearchTagToSlugMapper
         filter_14:
            type: KeSearchTagToSlugMapper
         filter_3_267:
            type: KeSearchTagToSlugMapper
         filter_3_273:
            type: KeSearchTagToSlugMapper
         filter_3_278:
            type: KeSearchTagToSlugMapper
         filter_3_283:
            type: KeSearchTagToSlugMapper
         sword:
            type: KeSearchUrlEncodeMapper

Upgrading

If you are upgrading from ke_search 3.3.1 or below and you are using your own templates, you will have to do a few adjustments to the templates as shown below.

Resources/Private/Templates/SearchForm.html

Add the kesearch namespace to the beginning of the file

<html xmlns:f="http://typo3.org/ns/TYPO3/CMS/Fluid/ViewHelpers"
      xmlns:kesearch="http://typo3.org/ns/Tpwd/KeSearch/ViewHelpers"
      data-namespace-typo3-fluid="true">

Add the snippet to rewrite the url to the beginning of the form

<f:comment> // Replace the URL with the speaking URL </f:comment>
<f:format.raw><script type="text/javascript">history.replaceState(null,'','</f:format.raw><kesearch:link keepPiVars="1" uriOnly="1" /><f:format.raw>');</script></f:format.raw>

Add conditions to the hidden fields

<f:if condition="{page}">
   <input id="kesearchpagenumber" type="hidden" name="tx_kesearch_pi1[page]" value="{page}" />
</f:if>
<input id="resetFilters" type="hidden" name="tx_kesearch_pi1[resetFilters]" value="0" />
<f:if condition="{sortByField}">
   <input id="sortByField" type="hidden" name="tx_kesearch_pi1[sortByField]" value="{sortByField}" />
</f:if>
<f:if condition="{sortByDir}">
   <input id="sortByDir" type="hidden" name="tx_kesearch_pi1[sortByDir]" value="{sortByDir}" />
</f:if>

Resources/Private/Partials/Filters/Checkbox.html

Change the "name" attribute of the options

<input type="checkbox" name="{option.key}" id="{option.id}" value="{option.tag}" {f:if(condition: '{option.selected}', then: ' checked="checked"')} {f:if(condition: '{option.disabled}', then: 'disabled = "disabled"')} />

Search word length

By default ke_search only finds words with a minimum length of four characters. This corresponds to the MySQL setting ft_min_word_len which is set to 4 by default.

Hint

As standard ke_search uses MyISAM as storage engine for the index table.

The value can be adjusted by following these steps:

Change ft_min_word_len to the desired value in your MySQL configuration

This can be done, e.g. in file my.cnf. In the example below it will set to 3:
```
[mysqld]
ft_min_word_len = 3
```
Copied!
A better way would be on Debian-derived systems to use a file like /etc/mysql/conf.d/fulltext.cnf to separate the default values in my.cnf with your custom settings.

After the adjustment the MySQL server has to be restarted.
Set Basic > Change searchword length in the extension configuration to the same value

Change searchword length in the extension configuration
Re-index your content

Either by just running the indexer again (full indexing is necessary) or by executing the following SQL command:
```
REPAIR TABLE tx_kesearch_index QUICK;
```
Copied!

Tip

Find more information about fine-tuning the full-text search in the MySQL documentation.

Change the search word parameter

By default the URL parameter for the searchword is tx_kesearch_pi1[sword] (following the TYPO3 "piVar" standard).

You may change this parameter to something else, e.g. query which allows the search words to be tracked by GA4 with the TypoScript configuration option searchWordParameter.

Example (Setup TypoScript):

plugin.tx_kesearch_pi1.searchWordParameter = query

Sorting

You may define the sorting method in the plugin configuration. Available options are relevance, title and date. More options may be added through third party extensions (see below Adding your own sorting options).

There are two sorting method options, one if a searchword was given and one if only filters have been used without a searchword. The reason for that is without a search word, you don't have a relevance ranking.

Default sorting is relevance descending if a searchword has been given and date descending if no search word has been given.

You may also activate the "frontend sorting" feature. This allows the visitor to decide for a sorting method.

You may then choose the fields you want to allow sorting for. By default these are relevancy, date and title.

Adding your own sorting options

If you want other sorting options than relevance, date or title, you will have to

Extend the table tx_kesearch_index by the fields you want to use for sorting (for example mysortfield) (ext_tables.sql, TCA configuration). Note: Don't use an underscore _ in the field name and use a string type (TEXT, VARCHAR) as field type.
Register your sorting fields by hook registerAdditionalFields, so that they are written to the database.
Write your own indexer or extend an existing one that fills your new field during the indexing process.

Note

If you add an "additional field" to the index every indexer must set this field. So make sure you use the provided hooks for every indexer you use. Additional fields cannot be integers, they must be strings. See https://github.com/tpwd/ke_search/issues/248

You can find an example in the extension ke_search_hooks: https://extensions.typo3.org/extension/ke_search_hooks

// in ext_localconf.php:

// Register hook to register additional fields in the index table
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['ke_search']['registerAdditionalFields'][] =
   \MyVendor\KeSearchHooks\AdditionalIndexerFields::class;

<?php
namespace MyVendor\KeSearchHooks;

/**
 * Class AdditionalIndexerFields
 * @package MyVendor\KeSearchHooks
 */
class AdditionalIndexerFields {
   public function registerAdditionalFields(&$additionalFields) {
      $additionalFields[] = 'mysorting';
   }
}

Your new database field will automatically appear in the backend selection of sorting fields.

For the frontend sorting links you will have to add a label for your new sorting field.

The key of the field is the same as the field name, prepended with oderlink_, so in this case it would be orderlink_mysorting.

You can add the label in your extension's locallang file for locallang_searchbox.xlf as described in Multilangual support.

Storage Engine

Why not use InnoDB for the index table?

The storage engine for the index table (tx_kesearch_index) is MyISAM while all other tables use the default InnoDB storage engine. Unfortunately this prevents using ke_search in Galera Clusters.

The reasons are explained below.

If the behaviour explained below does not affect how you use ke_search, you may switch to InnoDB for tx_kesearch_index.

Search for @-character ist not supported

You cannot search for the "at"-Character in InnoDB tables:

InnoDB full-text search does not support the use of the @ symbol in boolean full-text searches. The @ symbol is reserved for use by the @distance proximity search operator.

As a workaround you could add the @ character to the list of additional word characters as explained in Additional word characters.

Phrase search is not supported

Phrase search using double quotes ("my search word") is not supported in InnoDB.

Multilangual support

ke_search has multilingual support in a way that

if one searches in a specific language, the results will only be shown for that language.
filters can be translated and be shown in the respective language.

Indexing content in different languages

All available languages will be detected automatically and will be indexed.

Translating search result pages

On the search result page, insert the ke_search plugins in the translated page you just created. You can use the function "copy default content elements". You can leave the configuration as it has been copied from your default language.

Translating filters

In order to use the multilingual feature for filters you'll have to

Create page translations: Create alternative page languages for the storage folder where the index and filters are stored and for your search result page. You can do that with help of the page module by using the function "Create a new translation of this page".
Translate filters and filter options: Now you can translate the filters and filteroptions to the new language.

Use your own labels

You can overwrite labels used in ke_search by registering locallangXMLOverride files.

You can find the default language files in vendor/tpwd/ke_search/Resources/Private/Language and the language files for other languages in var/labels.

Copy the language files you want to modify (you dont' have to copy all the files) to your site package into the folder Resources/Private/Language/Overrides.

File structure for language file overrides

Register the files in the ext_localconf.php of your site package (assuming it's key is mysite), here an example for the file locallang_searchbox.xlf (you will have to do it for each file you want to overwrite):

// Custom translations
$GLOBALS['TYPO3_CONF_VARS']['SYS']['locallangXMLOverride']
['EXT:ke_search/Resources/Private/Language/locallang_searchbox.xlf'][]
    = 'EXT:mysite/Resources/Private/Language/Overrides/locallang_searchbox.xlf';

// Override a German ("de") translation
$GLOBALS['TYPO3_CONF_VARS']['SYS']['locallangXMLOverride']['de']
['EXT:ke_search/Resources/Private/Language/locallang_searchbox.xlf'][]
    = 'EXT:mysite/Resources/Private/Language/Overrides/de.locallang_searchbox.xlf';

Search behaviour

Users can fine-tweak the results by using search options. The result list ranking is based on an algorithm on which you find more information below.

Search options

The following parameters can be used in the fulltext search:

Search for phrases

A search for phrases can be executed by using the quote signs (""). The result will then contain exactly the searched phrase.

"french cooking"

Conjunctions

Note on conjuctions

If you have ke_search_premium installed and want to use the conjuctions, please note:

To use the "+" and "-" conjunctions the setting "Enable Sphinx-based in word search / partial word search" (Sphinx.enableInWordSearch) in the extension setting must turned off.

The result must contain the word

"french cooking" +eggs

If you want to to find only results which include both search words, you need to add the "+" to both search words, eg.

+"french cooking" +eggs

The result must not contain the word

"french cooking" -eggs

Partial word search

"french cooking" pepp*

finds eg. results with these words:

"french cooking" pepper
"french cooking" peppermint

Search for umlauts

If you search for words including umlauts, it doesn't matter if you use the umlaut character or the character in your search word (ä --> a, ö --> o etc.). Example: Searching for "Küche" or "Kuche" gives the same results.

The reason for that lies in MySQL itself and it's treatment of collations:

More than one search word

If you type in more than one searchword, all the words will be linked with "OR".

The results containing all searched words get the highest ranking and will be placed on top of the result list.

If you place a "+" in front of a word, the result must contain the word.(Conjunction).
If you place a "-" in front of a word, the result must not contain the word.(Disjunction).

Example:

“+Auto +cheap -expensive“

If you activate the enableExplicitAnd-option in the extension manager, all words will be conjuncted and the "+"-parameter becomes needles.

Note: If you are using the premium version of ke_search and you want to activate the searchengine Sphinx, all search words will automatically be conjuncted for it is the default behaviour of Sphinx.

Minimum length

If a word is shorter than 4 characters it will not be searched (Example: "come to" is the searched phrase and only "come" will be searched). This behaviour only shows if the short word stands at the beginning or the end of the searched phrase. If the short word stands between to longer words like "come to our company", this phrase will be searched exactly.

The minimum length can be changed, see Configuration.

Partial word search

The partial word search is enabled by default.

ke_search will find partial words if they are in the beginning of the words in the index.

Search for "Apple" will find:

Apple
Appletree

But will not find:

Bigapple

The partial word search can be deactivated in the extension setting (Extension-Manager), option enablePartSearch. Only full words which match the input will then be found.

You can activate the partial word search for single words by adding a "*" to the searched words. (Example see above).

In-Word-Partial-Search

Using the standard version of ke_search it is not possible to find partial words within other words, they have to be placed at the beginning of a word.

Example:

Searching for "back" will not find "paperback".
Searching for "paper" will find "paperback".

If you use the premium version of ke_search together with Sphinx you can enable partial in-word search so that searching for "back" will also find "paperback".

Ranking / Sorting

The ranking of search results can be defined in the plugin settings of the "searchbox" plugin in the tab "Sorting". By default, search results will be sorted by "relevance" if a searchword is given and by "date" if no searchword is given.

Ranking by relevance ("score")

The ranking of search results by relevance (in case "relevance" is selected in the plugin settings or selected by the user) is based on the MySQL fulltext search. You can find more information about the ranking algorithm of MySQL here:

https://dev.mysql.com/doc/internals/en/full-text-search.html

Searchwords in titles

If a user is searching for a word and that word appears in the title of index records, those records will be ranked higher. Most likely this will push those index records to the very top of the result list.

Note

This will only work, if the search word appears exactly in the title. If a user searches for "Australia", the results with "Australian shepherd" in the title won't get a higher weight. (see also Relevance calculation for partial words below)

Adjusting weight of the searchwords in titles

You can adjust how strong the search word in the title should be weighted. Go to the extension settings and increase the number for multiplyValueToTitle.

Note

Even with this value set to 1 (the lowest possible value), results having the search word in the title will be on top of the result list. Thus in most cases this setting will not have any effect.

Relevance calculation for partial words

Please note that MySQL will not calculate a relevance for partial words. The relevance can only be calculated if at least one search word is found as exact string in the index content

search for "testcontent" will deliver a relevance value if this word exists in index content
search for "testco" will NOT deliver a relevance value if "testcontent" exists in index content

Custom ranking

If you want to have more power about the ranking of search results or want to push certain results to the top of the list, consider the premium version of ke_search which adds the features "custom ranking" and "boost keywords".

More information

Stopwords

When using the extension with MySQL, some database servers may use the default MySQL MyISAM stopword list (see https://dev.mysql.com/doc/refman/8.0/en/fulltext-stopwords.html). This may lead to unexpected search results (e.g. search for german brief may return zero results), since brief is a default stopword.

It is therefore recommended to either disable the feature if possible by setting

ft_stopword_file = ""

Statistics

All the search words that are submitted by frontend users are stored in statistic tables. This function is activated by default but can be deactivated in the plugin configuration.

You can see the statistics in the backend module by selecting Searchword statistics or by using the dashboard widget.

The dashboard widget shows the search phrases used in the last seven days for the whole system (ignoring the folders in which the data is stored).

The backend module function shows a simple statistic of the submitted searchwords of the last 30 days. The statistic shows the cumulative values for single searchwords. Maybe this will be extended in a later version.

Differences between folders and other pages

If you call the statistics function for a folder you will get the cumulative values for all statistic data that is stored there. If you call it for a page of another type you will get the cumulative values for searchwords that were entered on this explicit page.

Technical background

Search phrases go to the table tx_kesearch_stat_search. Single search words are stored in the table tx_kesearch_stat_words.

The statistic data is stored in the folder that is set as the first storage point in your plugin configuration. Make sure that you set the FlexForm configuration option for activating the statistics function in the correct plugin if it does not work as expected. E.g. if you have several searchbox plugins that point to one central search result page, the value must be set on this result page.

Garbage collection

Since version 5.5.1 the statistic tables are registered for garbage collection. This means that the garbage collection scheduler task will delete old entries from the tables. The default value for the garbage collection is 180 days. You can change this value in the scheduler task configuration.

This is useful if you use the "Autocomplete" feature of ke_search_premium because the autocomplete function uses the statistic tables to propose search words. If you have a lot of old data in the tables, the autocomplete function will propose old search words that are not relevant anymore or the performance of the autocomplete function will decrease.

To activate garbage collection, please add two scheduler tasks of type Table garbage collection and select the tables tx_kesearch_stat_search and tx_kesearch_stat_words.

More options

Statistics using Google Analytics

If you use Google Analytics on your website, you can use it to generate a report for search queries:

Open the admin view
Activate site search

Enable Site search tracking in Google Analytics and enter the search query tx_kesearch_pi1[sword]. Go to Behaviour > Site search to see the report.
Search queries will be reported under Behaviour

Statistics using Matomo

If you use Matomo on your website, you can use it to generate a report for search queries:

Activate site search

In Matomo go to Administration > Measurables > Manage and select the according website. Select the option Site Search tracking enabled of the Site Search field:

You can disable the option Use default Site Search parameters (as these are not used) and enter the value tx_kesearch_pi1[sword] into the field Query parameter.
Display site search metrics

Go to Behaviour > Site Search.

Tip

It is also possible to add more information to Matomo like keywords with no results or the query time. Take a look into the blog post Display search metrics from TYPO3 extension ke_search in Matomo how to achieve that.

Hooks and Events

ke_search includes a lot of hooks you can use to include your own code and customize the behaviour of the extension.

Hooks

modifyPagesIndexEntry: Use this hook to modify the page data just before it will be saved into database.
modifyPageContentFields: Use this hook to modify the page content fields. See chapter “Indexing custom content fields” for further information.
modifyExtNewsIndexEntry: Use this hook to modify the news data just before it will be saved into database.
modifyExtTtNewsIndexEntry: Use this hook to modify the tt_news data just before it will be saved into database.
modifyAddressIndexEntry: Use this hook to modify the tt_address data just before it will be saved into database.
modifyFileIndexEntry: Use this hook to make custom modifications of the indexed data, e.g. the tags.
modifyFileIndexEntryFromContentIndexer: Use this hook to make custom modifications of the indexed data, e.g. the tags.
modifyFilterOptions: Use this hook to modify your filter options for type “select”, e.g. for adding special options, labels, css classes or to preselect an option.
modifyFilterOptionsArray: Use this hook to modify your filter options, independent from filter type, e.g. for adding special options, css classes or to preselect an option.
modifyFieldValuesBeforeStoring: Use this hook to manipulate the field values before they go to the database.
modifyContentFromContentElement: Use this hook for modifying a content element's content. See chapter “Indexing custom content fields” for further information.
modifyContentIndexEntry: Use this hook for custom modifications of the indexed data, e. g. the tags.
contentElementShouldBeIndexed: Use this hook to add a custom check if a specific content element should be indexed.
contentElementsHeaderShouldBeIndexed: Use this hook to add a custom check if the header of a specific content element should be indexed. Introduced in version 6.5.0. https://github.com/tpwd/ke_search/pull/297
initials: Change any variable while initializing the plugin.
modifyFlexFormData: Access and modify all returned values of ke_search FlexForm.
customFilterRenderer: You can write your own filter rendering function using this hook. You will have to add your custom filter type to TCA options array. See chapter “Custom filter rendering” for further information.
registerIndexerConfiguration: Use this hook for registering your custom indexer configuration in TCA. See chapter “Write your own custom indexer!” for further information.
customIndexer: Use this hook to register a custom indexer. See chapter “Write your own custom indexer!” for further information.
cleanup: Use this hook for cleanup.
registerAdditionalFields: This hook is important if you have extended the indexer table with your own columns.
getOrdering: Hook for third party extensions to modify the sorting.
getLimit: Hook for third party pagebrowsers or for modification $this->pObj->piVars['page'] parameter.
modifyResultList: Hook for adding new markers to the result list
fileReferenceTypes: Hook for adding third party file previews. See chapter “Images in custom indexers” for further information.

Events

MatchColumnsEvent: Allows to change the columns for which the "MATCH ... AGAINST" SQL clause should be created.
ModifyFieldValuesBeforeStoringEvent: Use this event to manipulate the field values before they go to the database.

Logging

Logging uses the TYPO3 Logging API.

The "minimum log level" is configurable in extension manager configuration (default: error).

Log messages matching the configured minimum log level are written into a separate logfile (e.g. var/log/typo3_kesearch_3765640ce4.log) with content like this:

Thu, 20 Dec 2018 14:12:20 +0100 [NOTICE] request="d40b2d51fe977" component="Tpwd.KeSearch.Indexer.IndexerRunner":
============================
= Indexing process started =
============================
Thu, 20 Dec 2018 14:12:21 +0100 [ERROR] request="d40b2d51fe977" component="Tpwd.KeSearch.Indexer.IndexerRunner": The path to catdoctools is not correctly set in the extension configuration. You can get the path with "which catdoc".
Thu, 20 Dec 2018 14:12:21 +0100 [ERROR] request="d40b2d51fe977" component="Tpwd.KeSearch.Indexer.IndexerRunner": The path for xls2csv is not correctly set in extConf. You can get the path with "which xls2csv".
Thu, 20 Dec 2018 14:12:21 +0100 [ERROR] request="d40b2d51fe977" component="Tpwd.KeSearch.Indexer.IndexerRunner": The path to catppttools is not correctly set in extension configuration. You can get the path with "which catppt".
Thu, 20 Dec 2018 14:12:21 +0100 [WARNING] request="d40b2d51fe977" component="Tpwd.KeSearch.Indexer.IndexerRunner": Could not index file /var/www/html/web/fileadmin/test/test-invalid.xlsx.
Thu, 20 Dec 2018 14:12:21 +0100 [WARNING] request="d40b2d51fe977" component="Tpwd.KeSearch.Indexer.IndexerRunner": Could not index file /var/www/html/web/fileadmin/test/test-invalid.docx.
Thu, 20 Dec 2018 14:12:21 +0100 [WARNING] request="d40b2d51fe977" component="Tpwd.KeSearch.Indexer.IndexerRunner": Could not index file /var/www/html/web/fileadmin/test/test-invalid.pptx.
Thu, 20 Dec 2018 14:12:21 +0100 [WARNING] request="d40b2d51fe977" component="Tpwd.KeSearch.Indexer.IndexerRunner": Could not index file /var/www/html/web/fileadmin/test/pdf-example-password.original.pdf.

Indexing errors and main indexing database operations are catched.

The "could not index file..." messages were removed from the indexing report (which is shown in the backend module and written to the sys_log) and replaced by a message pointing to the logfile.

Command line tools

Certain ke_search functions can be accessed via the command line.

Run vendor/bin/typo3 ke_search to see a list of available commands.

Start the indexer

Starts the indexer. All indexer configurations will be processed. Since version 5.4.0 you'll see a visual progression of the indexing process. Add this command in the TYPO3 scheduler to run the indexer automatically (via "Execute console commands").

vendor/bin/typo3 ke_search:indexing

Parameters:

-m, --indexingMode[=INDEXINGMODE]  Indexing mode, either "full" (default) or "incremental". [default: "full"]

Clear the index

Clears all records from the index (truncates the table).

vendor/bin/typo3 ke_search:clearindex

Remove the indexer lock

Removes the lock which prevents the indexer from running multiple times at the same time.

vendor/bin/typo3 ke_search:removelock

Show the status of the indexer

(since version 5.4.0)

Shows the status of the indexer and if the indexer is currently running also the progress. If "mode" is set to "short", it will only show "running" or "idle".

vendor/bin/typo3 ke_search:indexerstatus

Parameters

-m, --mode[=MODE]     Mode, either "full" (default) or "short". [default: "full"]

Changelog

The changelog can be found here:

https://github.com/tpwd/ke_search/blob/master/ChangeLog

Premium version

The package ke_search_premium is a paid add-on to ke_search. It can be installed additionally to ke_search and adds the following features:

Auto-complete
Boost Keywords (Push certain results to the top)
Custom Ranking (Change ranking of results)
Did you mean … ?
Distance search and map display (using Google Maps)
Faster search on very big data sets using Sphinx
Headless (output search results as JSON)
In-Word-Search
Remote-Indexer (adding one ore more external TYPO3 websites to the index)
Synonyms

More information can be found here:

https://www.kesearch.de/

The documentation can be found here:

http://dev.kesearch.de/documentation/ke_search_premium/

Faceted Search

Introduction

System requirements

Source code

Examples

Indexer configuration

Start indexing in the backend

Basic result list

Live examples

Quick start

Download and installation

Include TypoScript

Site Sets

TypoScript sets (the traditional way)

Create pages

Configure Plugins

Create the indexer configuration

Start Indexer

Indexing

Create indexer configurations

Configure the indexer configuration

Full indexing and incremental indexing

Starting the indexing process manually

Starting the indexing process automatically

Available indexers

Pages

Configuration

Content element types

File indexing

Container and Gridelements

Abstract

Advanced options

Index content from additional tables (eg. mask, bootstrap_package, content_blocks)

Options

Examples

Bootstrap Package

Mask

Mask with multiple additional tables

Mask with sub-elements in additional tables

More Mask examples

Content Blocks

Content Elements

News (EXT:news)

Configuration

Alternative single view page from category

Filtering with system categories

File indexing

News (EXT:tt_news)

Configuration

Alternative single view page from category

File indexing

Automated tagging

Addresses (EXT:tt_address)

Files

System requirements

Directory based file indexer

Configuration of the file indexer

Content based file indexer

Language handling

Custom Indexer

Extending existing indexers

Adding hidden content

Indexing custom content fields

Example

Backend

Backend module

Scheduler task

Dashboard widgets

Add a widget

Status widget

Index overview widget

Trending search phrases

Filters / Faceted Search

Examples

Example "Car Website"

Example "University Courses"

Using Filters

Setup filters

Tag based filters

Selectbox