Apache Solr for TYPO3

Extension key

solr

Package name

apache-solr-for-typo3/solr

Version

main

Language

en

Author

Ingo Renner, Markus Friedrich, Rafael Kähm, Timo Hund & Contributors

License

This document is published under the Open Publication License.

Rendered

Thu, 10 Jul 2025 11:39:15 +0000


A TYPO3 extension that integrates the Apache Solr Enterprise Search Server into the TYPO3 CMS.

This extension serves as a base module that covers the most frequently used functionalities.

Additional features can be obtained through the following free add-ons:

  1. Apache Tika for TYPO3
  2. Apache Nutch plugins for TYPO3 CMS

and many more by searching for "solr" in the TYPO3 Extension Repository (TER).


Table of Contents:

Introduction

What does it do?

Apache Solr for TYPO3 is the search engine you were looking for with special features such as Faceted Search or Synonym Support and an incredibly fast response times of results within milliseconds.

When development started, the primary goal was to create a replacement for Indexed Search. With the initial public release at T3CON09 in Frankfurt, Germany that goal was reached and even passed by adding features which Indexed Search does not support.

With EXT:solr it is simple to build a search for a TYPO3 website, that allows you to index any kind of TYPO3 records with TCA configuration and pages. The results can be rendered with flexible fluid templates, to render the results as you need them.

Feature List

  • Faceted Search
  • Spellchecking / Did you mean
  • Multi Language Support
  • Search word highlighting
  • Field Boosting for fine tuning the importance of certain index fields
  • Frontend User Group Access Restrictions Support
  • Stop word Support
  • Synonym Support
  • Auto complete / Auto suggest
  • Language Analysis / Support for inflected word forms
  • Content Elevation / Paid Search Results / Editorial Content
  • Sorting of Results
  • Content indexing through a near instant backend Index Queue
  • Auto correction (search for the first suggestion)
  • and more...

Development and Partnership

The extension is developed in an open source way and the source code is completely available on github. Releases to the TYPO3 TER are done in regular time frames (Usually every 3 months).

To make the development possible you can join a partner ship with dkd. By joining the partner program you have the following benefits:

  • You support the further development on EXT:solr
  • You get access to addon's that are not public available
    • Indexing of files from FAL (TYPO3 File Abstraction Layer)
  • You can be mentioned as a sponsor on http://www.typo3-solr.com
  • You can included support based on your subscription

By the financial invest of our partners it is possible to keep this extension uptodate and integrate great new features. If you like this extension please think about to become a partner as well!

|

If you are interested to become a partner visit http://www.typo3-solr.com or call dkd +49 (0)69 - 247 52 18-0.

Solr

First you need to install Solr itself. There are several ways to do so:

Using Hosted-solr.com

If you want to start simple and just create a Solr core with a click. You can use hosted-solr.com. For a small fee you get your own Solr core in seconds, configured to be used with EXT:solr.

Docker

You can use our official Docker image to start and maintain Solr server with a small effort.

To pull the TYPO3 Solr image from Docker hub, simply type the following in command line:

docker pull typo3solr/ext-solr:<EXT:Solr_Version_main_version eg 13.0 or exact 13.0.1>
Copied!

Persistent Data

Our Docker image is based on official Apache Solr image.

This volume will be mounted to persist the index and other resources from Apache Solr server. Following paths inside the exported volume are relevant for backups.

Path Contents
data/data/<language> the index data of corresponding core
data/configsets/ext_solr_<See_configset_on_version_matrix>/conf/_schema_analysis_(stopwords|synonyms)_<language>.json the managed stop words and synonyms of corresponding core

Start container with anonymous volume

To run the container with anonymous volume, simply type the following in command line:

docker run --name=typo3s-solr-server -d -p 8983:8983 typo3solr/ext-solr
Copied!

This will create a Docker anonymous volume and store the data inside of it. To find out the path of used anonymous volume, simply type the following in command line:

docker inspect -f '{{ .Mounts }}' typo3s-solr-server
Copied!

Start container with volume on hosts path

There are few steps required to be able to run the container with volume on hosts path.

Following commands will create the named volume "typo3s-solr-server-data" on hosts path and start the container with this volume.

mkdir .solrdata
docker volume create --name typo3s-solr-server-data --opt type=none --opt device=$PWD/.solrdata --opt o=bind
docker run --name=typo3s-solr-server --mount source=typo3s-solr-server-data,target=/var/solr -d -p 8983:8983 typo3solr/ext-solr
Copied!

Check if Solr is up and running

To check whether Solr is up and running head over to:

http://<ip>:8983/solr/#/core_en/query.

You should see the web interface of Solr to run queries:

Important: The image ships a default cores for all languages. The data of the cores is stored on an exported volume. When you want to update the container, you can just start a new container using the data volume of the old container. But at the same time this has the limitation, that you should only use this image with the default cores! If you want to create custom cores with a different configuration please read the section "Advanced Docker Usage"

Please note: The steps above show how to build the image from the Dockerfile. You can also download and use our compiled images from Docker Hub:

https://hub.docker.com/r/typo3solr/ext-solr/

Advanced Docker Usage

Our image has the intension to create running cores out of the box. This implies, that the schema is inside the container. The intension in our integration was to stay as close as possible to the official Apache Solr Docker images. Sometimes it might make sense that you use the official image directly instead of our image. An example could be when you want to have the solrconfig, schema and data outside of the container.

The following example shows how you can run our configuration with the official Apache Solr Docker container by mounting the configuration and data from a volume (When using Docker on macOS make sure you've added the volume folder to "Preferences -> File Sharing").

mkdir -p ~/mysolr
cp -r Resources/Private/Solr/* ~/mysolr/.
mkdir ~/mysolr/data
sudo chown -R 8983:8983 ~/mysolr
docker run -d -p 8983:8983 -v ~/mysolr:/var/solr/data solr:8.5
Copied!

Advanced settings and tweaks

For more settings and tweak possibilities for our Docker image, please refer to the Appendix - Docker Tweaks.

Other Setup

Beside the Docker there are various possibilities to setup solr. All of these possibilities are not officially supported, but the simplify the setup i want to mention them shortly here and summarize the needed steps.

Known Installers

All of these installers can be used to setup a plain, reboot save Solr server:

  • Use the installer shipped with Solr itself bin/install_solr_service.sh:

Allows to install Solr on many distributions including init scripts.

  • Use chef / ansible / whatever dev ops tool:

Allows you to setup a Solr server with your DevOps tool.

e.g. https://galaxy.ansible.com/geerlingguy/solr/ (ansible) or https://supermarket.chef.io/cookbooks/solr (chef)

Deployment of EXT:solr configuration into Apache Solr

Since EXT:solr 6.0.0 the configuration and all JAR files can be found in EXT:solr/Resources/Private/Solr folder. The goal of this approach is to make the deployment much easier.

All you need to do is, you need to copy the configSet directory into your prepared Solr installation and replace the solr.xml file. In the installer we do it like this:

cp -r ${EXTENSION_ROOTPATH}/Resources/Private/Solr/* ${SOLR_INSTALL_DIR}/server/solr
Copied!

After this, you can decide which cores you want to boot and delete not relevant cores from ${SOLR_INSTALL_DIR}/server/solr/cores/ directory.

If you want to create other cores, you can create them by adding a <core_name>/core.properties file like provided by EXT:solr. This is the recommended workflow, so you can deploy the Solr server in an Infrastructure as Code (IaC) manner.

Alternatively Create a core with the rest api (not recommended):

See: "V2 API" on https://solr.apache.org/guide/solr/latest/configuration-guide/coreadmin-api.html#coreadmin-create

curl -X POST http://localhost:8983/api/cores -H 'Content-Type: application/json' -d '
  {
    "create": {
      "name": "<core_name>",
      "configSet": "ext_solr_<XX_Y_Z>",
      "schema": "german/schema.xml",
      "instanceDir": "cores/<core_name>"
      "dataDir": "<absolute-path-to-SOLR-HOME-of_server(standard: /var/solr/data/)>/data/<core_name>"
    }
  }
'
Copied!

After installing the Solr server and deploying all schemata, the TYPO3 reports module helps you to verify if your setup fits to the requirements of EXT:solr

You now have a fully working, pre configured Solr running to start with

No you can continue with installing the extension Install EXT:solr.

Install EXT:solr

Install from TER using the TYPO3 Extension Manager

You can simply install stable versions of EXT:solr using the Extension Manager from the TYPO3 backend.

  1. Go to the Extension Manager, select Get Extensions and search for "solr".
  2. Install the extension.
  3. The Extension Manager will also install EXT:scheduler if not installed already for running the indexing tasks
  4. While developing we recommend installing devlog for easier error detection, too.

Install from git

Alternatively, you can also get the latest development version from GitHub:

$ git clone git@github.com:TYPO3-Solr/ext-solr.git solr
Copied!

Install with composer

Install this TYPO3 Extension solr via TYPO3 Extension Manager as usual, or via composer by running:

composer require apache-solr-for-typo3/solr
Copied!

Head over to the Extension Manager module and activate the Extension.

That's all you have to do, now head over to Configure Extension.

Configure Extension

After Install EXT:solr you need to configure the extension. Only a few steps from below are necessary for Index the first time.

Extension Configuration

Underlying settings can be found in the extension configuration, though the default settings are appropriate for most sites you should check and adapt this settings if necessary:

Static TypoScript

The extension already comes with basic TypoScript configuration that will work for small pages out of the box. For now create or edit an existing TypoScript Template record in your page tree and add the provided static TypoScript:

Search Markers

EXT:solr is indexing everything on a page between <!-- TYPO3SEARCH_begin --> and <!-- TYPO3SEARCH_end --> to ensure this is the case, check the output of you website and add the markers to your template.

If the markers are missing, you should add them to your template. To increase the quality of the search results the markes should only wrap the relevant content of a page and exclude e.g. menus, because they are same on each page. The markers can be used multiple times, but make sure each "TYPO3SEARCH_begin" is properly closed with "TYPO3SEARCH_end".

The most simple configuration for my page was:

page.10 {
    stdWrap.dataWrap = <!--TYPO3SEARCH_begin-->|<!--TYPO3SEARCH_end-->
}
Copied!

Enable indexing

Indexing will only work if you allow content to be indexed by stating so in your TypoScript setup configuration:

config {
    index_enable = 1
}
Copied!

Site Handling

In TYPO3 9.5 the new backend module called "Sites" was introduced. This Module can be used to setup and configure the Apache Solr connections.

Configure Solr Connections

As first step the authority part of URI to Apache Solr machine must be configured.

The Corename(aka path to core) for each available language must be assigned respectively.

Index the first time

After everything is setup, you need to index the contents of TYPO3 to enable searching in Solr. To do so open the Apache Solr module and navigate to the Index Queue. Select the contents to index and Queue Selected Content for Indexing.

Now the records are added to the index queue and ready to index. The indexing can be triggered from the backend module, or can be triggered by the TYPO3 scheduler.

Switch to the Scheduler module. If the module is not available, make sure to enable the extension first. It comes bundled with TYPO3 CMS but is not enabled by default.

Create a new scheduler task to run the indexing:

After the task was created, run it manually. The page will indicate a reload but won't reload after the task was run. Therefore you can reload the module to see the progress bar indicating the current progress of indexing:

The duration depends on things like the number of records to index and the number of languages configured in your system. Also whether caching is enabled and warmed up.

The extension will now index all records in the queue and send them to Solrs index.

Once you have some records inside the index, you can Display search and results.

Display search and results

After Solr has some documents inside his index, you can insert the plugin to provide a search with results from Solr. To do so create a new content record of type Search on a page:

Select Search: Form, Result, Additional Components if not already selected inside the content element:

Open the page and search for *, you should see all currently indexed records from Solr:

That's it. You now have a working TYPO3 Installation with Solr integration. You are able to queue items for indexing, index them and provide an interface for visitors to search the indexed records.

You can now adjust the fluid templates to your needs.

Backend

The chapter before gives you a short introduction about how to setup a Solr server, the extension and index a few documents into Solr. In this chapter we want to go deeper and learn how to write more complex indexing configurations and see what other possibilities the backend of EXT:solr provides.

ConnectionManager

TYPO3 Site mode

You can configure all Solr connection properties along with your TYPO3 site in the site module.

The configuration in the UI is limited to the following settings to keep the setup as simple as possible:

Global:

  • Scheme
  • Host
  • Port
  • Path to cores

Languagelevel:

  • Solr core name

If you have a more complex setup or need to configure username and password please configure Solr by editing the yaml file.

The fallback mechanism allows you to overwrite settings on the language level.

The fallback mechanism work like that:

Each setting has the following structure solr_{$setting}_{$scope}"`. The scope can be read or write. Every setting can be overwritten for the scope write, if nothing is configured it will fallback to the read setting. Every setting can be defined on the language level, if it is not configured on the language level it fallsback to the global setting.

Example:

base: 'http://solr-ddev-site.ddev.site/'
baseVariants: {  }
errorHandling: {  }
languages:
  -
    title: English
    enabled: true
    base: /
    typo3Language: default
    locale: en_US.UTF-8
    iso-639-1: en
    navigationTitle: ''
    hreflang: en-US
    direction: ''
    flag: global
    solr_host_read: solr-host-1
    solr_core_read: core_en
    languageId: '0'
  -
    title: German
    enabled: true
    base: /de/
    typo3Language: de
    locale: de_DE.UTF-8
    iso-639-1: de
    navigationTitle: ''
    hreflang: de-DE
    direction: ''
    flag: global
    solr_host_read: solr-host-2
    solr_core_read: core_de
    languageId: '1'
rootPageId: 3
routes: {  }
solr_enabled_read: true
solr_path_read: /
solr_port_read: 8983
solr_scheme_read: http
solr_use_write_connection: false
Copied!

The example above shows that you are able to define the setting solr_host_read on the language level. Since this is a more advanced configuration and the user interface should be kept simple, this can only be configured in the yaml.

IndexQueue Configuration

As you already learned you can index pages very easy with EXT:solr and setup a search for pages in seconds. Beside pages, there might be other records in your TYPO3 CMS that you want to have available in your search results.

Indexing custom records

As a core feature EXT:solr allows you to write custom TypoScript configuration to index records from any extension just with configuration. To see how this is working, we open the content of the TypoScript example "Search - Index Queue Configuration for news" that can be found in "Configuration/TypoScript/Examples/IndexQueueNews":

plugin.tx_solr.index.queue {

    news = 1
    news {
        type = tx_news_domain_model_news

        fields {
            abstract = teaser

            author = author
            authorEmail_stringS = author_email

            title = title

            content = SOLR_CONTENT
            content {
                cObject = COA
                cObject {
                    10 = TEXT
                    10 {
                        field = bodytext
                        noTrimWrap = || |
                    }
                }
            }

            category_stringM = SOLR_RELATION
            category_stringM {
                localField = categories
                multiValue = 1
            }

            keywords = SOLR_MULTIVALUE
            keywords {
                field = keywords
            }

            tags_stringM = SOLR_RELATION
            tags_stringM {
                localField = tags
                multiValue = 1
            }

            url = TEXT
            url {
                typolink.parameter =  {$plugin.tx_news.settings.detailPid}
                typolink.additionalParams = &tx_news_pi1[controller]=News&
                    tx_news_pi1[action]=detail&tx_news_pi1[news]={field:uid}
                typolink.additionalParams.insertData = 1
                typolink.useCacheHash = 1
                typolink.returnLast = url
            }
        }

        attachments {
            fields = related_files
        }
    }

}

plugin.tx_solr.logging.indexing.queue.news = 1
Copied!

|

By reading the example above you might recognize the following facts:

  • The indexing configuration is done in the TypoScript path 'plugin. tx_solr. index. queue. [configName]' and there can be multiple queue configurations.
  • The database table/type is configured in the property 'plugin.tx_solr.index.queue.[configName].type'. This allows you to have multiple index queue configurations for the same database table. This can be helpful when you have multiple queue configurations for news (e.g. if you have a press & corporate news section on your website).
  • The Solr fields are configured in 'plugin.tx_solr.index.queue.[configName].fields'. This allows you to flexibly fill any Solr field. The combination of dynamic fields (Appendix - Dynamic Fields) and the queue configuration allows you to write any kind of data into Solr without adapting the Solr schema.
  • There are custom TypoScript objects from EXT:solr that are used in the index queue configuration

When the index queue configuration of your custom record is ready, you can check the index queue in the backend module and add the news items to the queue.

Sysfolders outside the siteroot

It is a good practice not to nest the siteroots and do the configuration on the root page. Changes on records that are done in the TYPO3 backend are detected and the Solr document will be re-added to the index queue when something was changed.

By default only records are monitored for a site that are in the tree of the site. If you want to index records and detect changes on records in a different siteroot, the index queue configuration needs to contains "additionalPageIds" (e.g.: 'plugin.tx_solr.index.queue.<queueName>.additionalPageIds = 45,48').

Since the monitoring of changes in these records is expensive from performance perspective, you need to enable this feature in the extension configuration:

Enable tracking of records outside siteroot

Backend Modules

The backend modules are in the "Apache Solr" section available and can be unlocked for BE-users or/and groups. The modules help you to do maintenance tasks and get an overview on the system status:

During the next paragraphs we will go over the modules and explain, what could be done with them.

Info Module

Info Module shows you important infos about TYPO3 CMS and Solr state.

It contains different info tabs described below:

Connections

It lists all the for the site configured connections and their status.

Statistics

The Search Statistics module allows you to see Top Search Phrases with and without results. In addition it possible to see a complete listing with hits etc. ranked by Top search keywords.

Index Fields

The Index Fields module allows you to see, how many documents you have in which Solr core and which fields those documents have.

Core Optimization

Core optimization Module is responsive for managing the behaviour of cores. By modifying of following things, you can also change the ranking and/or the results.

Stop Words

With the stopwords module you can define a list of words that should be excluded from the search.

Common usecases are:

  • Very often occurring words like "the", "and" ... are excluded are filtered out because they are more or less "noize words".
  • You can add words that you want to avoid from indexing.

Synonyms

With the synonyms module you can allow to find documents by words that do not occur in the document but have the same meaning:

  • E.g. smartphone, cellphone, mobile, mobilephone

Note: The word that you want replace with a synonym needs to appear on both sides when you want to find it with the term itself later

Example

smartphone => smartphone, cellphone, mobile, mobilephone will match "smartphone, cellphone, mobile, mobilephone", when smartphone is missing on the right side, you will not find the document for smartphone anymore!

Index Queue

The Index Queue module is the most important module. It allows you to do the following things:

  • Select item types and add them for indexing to the indexing queue.
  • See the fill state of the indexing queue.
  • Check the indexing queue for errors when the indexing of an items failed.
  • Start an instant indexing run, directly from the module.
  • Clear the indexing queue and re-queue items.

Index Administration (earlier Index Maintenance)

The Index Administration module allows you, to do the following administrative operations on your Solr index:

  • Reload the Solr configuration.
  • Empty your Solr index. This removes all documents from the index of the current selected site.
  • Clear the indexing queue.

Index Inspector

Beside the own backend module, EXT:solr provides a feature called "Search Index Inspector". This Tool allows you to select a page or sysfolder and check what data is stored for this entity in the Solr index.

The "Search Index Inspector" can be opened with the TYPO3 Info Module ("Web > Info > Search Index Inspector") it shows the stored data in Solr from the page or sysfolder that is selected in the pagetree:

The EXT:solr Search Index Inspector

Page properties

EXT:solrs indexing behaviour depends on page properties. Depending on the state of page properties the pages and or sub pages may be indexed or not.

Behaviour

The Behaviour tab of the page properties

The Behaviour tab of the page properties

Include in Search (no_search)
By default, every allowed page is indexed. Use this flag to exclude the current page from being indexed.
Include sub entries in Search (no_search_sub_entries)

Use this flag, to be able to exclude sub pages from index recursively. By unsetting this flag, the sub pages/entries of desired page will be excluded from index.

Scheduler

When you want to index content from TYPO3 into Solr automatically EXT:solr ships scheduler tasks, that can be called at a configured time or directly from the backend.

Index Queue Worker

Changes that are done in the backend by an editor are written into a queue. This queue is processed asynchronously with a scheduler task and each item in the queue is indexed into solr.

The "Index Queue Worker" task has the following custom properties:

  • Site: Here you select the Solr site, that you want to index with this task instance.
  • Number of documents to Index: Here you can configure how many documents you want to index in one run. Depending on the performance of your system and the expected update time of the search you can choose a realistic number here.
  • Forced webroot: The scheduler task can be executed in the cli context, because no webserver is used there, TYPO3 is unable to detect your webroot. As assumption we use PATH_site as default here. When you need to configure something else, you can do it with this option. You can use the marker ###PATH_site### and ###PATH_typo3### to define relative pathes here, to be independent from the concrete instance.

The EXT:solr Index Queue Worker - Scheduler Task

Force Re-Indexing of a site

This task allows you to force the re-indexing of a site & indexing configuration at a planned time.

The Force Re-Indexing of a site task has the following custom properties:

  • Site: Here you select the Solr site, that you want to index with this task instance.
  • Index Queue configurations to re-index: Here you can limit the set of indexing configurations that should be Re-Indexed.

The EXT:solr Force Re-Indexing of a site - Scheduler Task

Optimizing cores of a site

This task allows you to optimize the indexes of given cores for a site at a planned time.

The Optimize index of a site task has the following custom properties:

  • Site : Here you select the Solr site, that you want to index with this task instance.
  • Cores to optimize index for : Here you can select the cores that should be optimized by the task.

The EXT:solr Optimize index of a site - Scheduler Task

Plugins

EXT:solr provides the following plugin instances that can be configured in the backend:

  • Results plugin: "Search: Form, Result, Additional Components"
  • Form plugin: "Search: Form only"
  • Frequent Searches plugin: "Search: Frequent Searches"

Results Plugin

The results plugin is the most important plugin of the extension. It is responsible to render a search form and the results.

Flexform Configuration

All configuration can be done with TypoScript and the settings from EXT:solr are used. For some settings it makes sence to overwrite them with the flexform in the plugin settings.

The following settings can be overwritten by instance with the flexform:

"Target Page":

Target page that should be used when a search is submitted. This can be usefull when you want to show the results on another page.

When nothing is configured the current page will be used.

Overwritten TypoScript Path plugin.tx_solr.search.targetPage
Type: Integer

"Initialize search with empty query":

If enabled, the results plugin issues a "get everything" query during initialization. This is useful, if you want to create a page that shows all available facets although no search has been issued by the user yet.

Note: Enabling this option alone will not show results of the get everything query. To also show the results of the query, see option Show results of initial empty query below.

Overwritten TypoScript Path plugin.tx_solr.search.initializeWithEmptyQuery
Type: Boolean

"Show results of initial empty query":

Requires "Initialize search with empty query" (above) to be enabled to have any effect. If enabled together with "Initialize search with empty query" the results of the initial "get everything" query are shown. This way, in combination with a filter you can easily list a predefined set of results.

Overwritten TypoScript Path plugin.tx_solr.search.showResultsOfInitialEmptyQuery
Type: Boolean

"Initialize with query":

This configuration can be used to configure an initial query string that is triggered when the plugin is rendered.

Overwritten TypoScript Path plugin.tx_solr.search.initializeWithQuery
Type: String

"Show results of initial query":

This option is used to configure if the results of an initial query should be shown.

Overwritten TypoScript Path plugin.tx_solr.search.showResultsOfInitialQuery
Type: Boolean

"Filters":

This flexform element allows you to define custom filters by selecting a Solr field and a value:

Overwritten TypoScript Path plugin.tx_solr.search.query.filter.
Type: Array

"Sorting":

When you want to sort initially by a field value and not by relevance this can be configured here.

Overwritten TypoScript Path plugin.tx_solr.search.query.sortBy
Type: String
Example: title desc

"Boost Function":

A boost function can be useful to influence the relevance calculation and boost some documents to appear more at the beginning of the result list. Technically the parameter will be mapped to the "bf" parameter in the Solr query.

Use cases for example could be:

"Give never documents a higher priority":

This could be done with a recip function:

recip(ms(NOW,created),3.16e-11,1,1)
Copied!

"Give documents with a certain field value a higher priority":

This could be done with:

termfreq(type,'tx_solr_file')
Copied!
Overwritten TypoScript Path plugin.tx_solr.search.query.boostFunction
Type: String
Example: recip(ms(NOW,created),3.16e-11,1,1)

See also:

https://cwiki.apache.org/confluence/display/solr/The+DisMax+Query+Parser#TheDisMaxQueryParser-Thebf%28BoostFunctions%29Parameter https://cwiki.apache.org/confluence/display/solr/Function+Queries

"Boost Query":

The boostQuery is a query that can be used for boosting. Technically it is mapped to the "bq" parameter of the Solr query. Compared to boost a function a boost query provides less use cases.

An example could be to boost documents based on a certain field value:

type:tx_solr_file

Overwritten TypoScript Path plugin.tx_solr.search.query.boostQuery
Type: String
Example: type:tx_solr_file

See also:

https://cwiki.apache.org/confluence/display/solr/The+DisMax+Query+Parser#TheDisMaxQueryParser-Thebq%28BoostQuery%29Parameter

Concepts

Since EXT:solr 7.0.0 the old templating of EXT:solr was droppend and rendering with fluid was added.

Along with this change some concepts have changed:

  • Until EXT:solr 7.0.0 EXT:solr css and javascript was loaded by EXT:solr automatically. In most cases you want to use custom css or you have custom javascript and the integrator want to decide which css or javascript to use. Therefore EXT:solr does not load it by default anymore and the integrator can load it with TypoScript. EXT:solr provides a lot of example TypoScript templates that load the default css or load the javascript that is needed to use a specific feature. Maybe take the time to explore the TypoScript templates that are shipped with the extension to see how they are implemented.
  • The were some old typosript settings that manipulate the data before it was passed to the view. With fluid this can also be done with core viewhelpers or custom viewhelpers. Therefore the following settings have been removed:
    • plugin.tx_solr.search.results.fieldRenderingInstructions
    • plugin.tx_solr.search.results.fieldProcessingInstructions (The fieldProcessingInstructions still exist at index time since there it is still needed)
  • Beside that there were some template related settings in the TypoScript, that can be solved just with fluid:
    • plugin.tx_solr.search.faceting.facetLinkATagParams or plugin.tx_solr.search.faceting.[facetName].facetLinkATagParams When you need something like this, you can just change the partials or render a facet with a custom partial (partialName = MyPartial) and add the properties there.
    • plugin.tx_solr.search.faceting.removeFacetLinkText This can be done just be rendering the text in the partial, that you need.
  • The setting faceting.facets.[facetName].selectingSelectedFacetOptionRemovesFilter has been removed, since it is possible to build this functionality just with Fluid ViewHelpers. The file "EXT:solr/Resources/Private/Templates/Partials/Facets/OptionsToggle.html" shows how f:if together with `option.selected` can be used to have this behaviour.

Fluid Template Structure

First we start with a short overview of the template structure. This is just to get an rought overview. The templates will be explained in detail in the template where they belong to:

  • Layouts: Layouts that are used in the search and the faceting.
  • Partials:
    • Facets: Partials that are use to render the specific facet types.
    • Result: Partials that are used during the result rendering (e.g. to render the result document, sorting or perPage selector)
    • Search: Partials that are used for the search also when no search was executed.
  • Templates:
    • Search: All templates that are used to render the actions in the SearchController
    • ViewHelper: All templates that are use in the widgets (FrequentSearched, LastSearches)

All backend related files are in a "Backend" folder. Everything else is frontend related.

Result List

The most important part of a search are the results. The rendering of the results is done in the "Results.html" template (Located in Templates/Search/Results.html)

The following part of the default template iterates over the results and renders every document with the Document.html partial (Partials/Frontend/Result/Document.html)

<ol start="{pagination.displayRangeStart}" class="results-list">
	<f:for each="{resultSet.searchResults}" as="document">
		<f:render partial="Result/Document" section="Document"
		 arguments="{resultSet:resultSet, document:document}" />
	</f:for>
</ol>
Copied!

This structure allows you to use e.g. the fluid if ViewHelper to render a result with a different partial, based on a field value. But as you see in the template above, by default the partial "Result/Document" is used.

The "document" partial is getting the document object. In our case this is an instance of "ApacheSolrForTypo3SolrfluidDomainSearchResultSetSearchResult" the api of this object allows to get the Solr field content with "Document->getFieldName()" that can be used as "document.fieldName" in fluid.

Facets

The goal of a good search is, that the user will find what he is looking for as fast as possible. To support this goal you can give information from the results to the user to "drill down" or "filter" the results up to a point where he exactly finds what he was looking for. This concept is called "faceting".

Imagine a user in an online shoe shop is searching for the term "shoe", wouldn't it be useful to allow the user to filter by "gender", "color" and "brand" to find exactly the model where he is looking for?

In the following paragraphs we will get an overview about the different facet types that can be created on a Solr field just by adding a few lines of configuration.

Facet Types

A Solr field can contain different type of data, where different facets make sence. The simplest facet is an option "facet". The "options facet" just contains a list of values and the user can choose one or many of them. A more complex type could be a "range facet" on a price field. A facet like this needs to allow to filter on a range of a minimum and a maximum value.

The "type" of a facet can be controlled with the "type" property. When nothing is configured there, the facet will be threated as option facet.

plugin.tx_solr.search.faceting.facets.[faceName].type = [typeName]
Copied!

Valid types could be: options | queryGroup | hierarchy | dateRange | numericRange

In the following paragraphs we will introduce the available facet types in EXT:solr and show how to configure them.

Option

The simplest and most often used facet type is the options facet. It renders the items that could be filtered as a simple list.

To setup an simple options facet you can use the following TypoScript snipped:

plugin.tx_solr.search {
    faceting = 1
    faceting {
        facets {
            contentType {
                label = Content Type
                field = type
            }
        }
    }
}
Copied!

By using this configuration you create an options facet on the Solr field "type" with the name "contentType". This field represents the record type, that was indexed into solr. Shown in the frontend it will look like this:

Options Facet

Summary:

Type options
DefaultPartial Partials\Facets\Options.html
Domain Classes Domain\Search\ResultSet\Facets\OptionBased\Options\*

Grouping by option prefix:

When you have an option facet with very much options you might want to group the options by an prefix of an option. This can be used e.g. to group the options alphabetically.

The following example shows how options can be grouped by prefix (from EXT:solr/Configuration/TypoScript/Examples/Facets/OptionsPrefixGrouped/setup.typoscript):

<s:facet.options.group.prefix.labelPrefixes options="{facet.options}" length="1" sortBy="alpha">
    <f:for each="{prefixes}" as="prefix">
        <li>
            {prefix}
            <ul>
                <s:facet.options.group.prefix.labelFilter options="{facet.options}" prefix="{prefix}">
                    <f:for each="{filteredOptions}" as="option">
                        <li class="facet-option" data-facet-item-value="{option.value}">
                            + <a class="facet solr-ajaxified" href="{s:uri.facet.addFacetItem(facet: facet, facetItem: option)}">{option.label}</a>
                            <span class="facet-result-count">({option.documentCount})</span>
                        </li>
                    </f:for>
                </s:facet.options.group.prefix.labelFilter>
            </ul>
        </li>
    </f:for>
</s:facet.options.group.prefix.labelPrefixes>
Copied!

Query Group

The query group facet renders an option list, compareable to the options facet, but the single options are not created from plain Solr field values. They are created from dynamic queries.

A typical usecase could be, when you want to offer the possiblity to filter on the creation date and want to offer options like "yesterday", "last year" or "more then five years".

With the following example you can configure a query facet:

plugin.tx_solr.search {
    faceting = 1
    faceting {
        facets {
             age {
                label = Age
                field = created
                type = queryGroup
                queryGroup {
                    week.query = [NOW/DAY-7DAYS TO *]
                    old.query = [* TO NOW/DAY-7DAYS]
                }
            }
        }
    }
}
Copied!

The example above will generate an options facet with the output "week" (for items from the last week) and "old" (for items older then one week).

The output in the frontend will look like this:

Solr queryGroup facet

An more complex example is shipped with this extension and can be enabled by including the template "Search - (Example) Fluid queryGroup facet on the field created", this example makes also use of renderingInstructions to render nice labels for the facet.

Summary:

Type queryGroup
DefaultPartial Partials\Facets\Options.html
Domain Classes Domain\Search\ResultSet\Facets\OptionBased\QueryGroup\*

Hierarchical

With the hierarchical facets you can render a tree view in the frontend. A common usecase is to render a category tree where a document belongs to.

With the following example you render a very simple rootline tree in TYPO3:

plugin.tx_solr.search {
    faceting = 1
    faceting {
        facets {
            pageHierarchy {
                field = rootline
                label = Rootline
                type = hierarchy
            }
         }
    }
}
Copied!

The example above just shows a simple example tree that is just rendering the uid's of the rootline as a tree:

Hierachy facet

A more complex example, that is rendering the pagetree with titles is shipped in the extension. You can use it by including the example TypoScript "Search - (Example) Fluid hierarchy facet on the rootline field":

Hierachy rootline facet

Summary:

Type hierarchy
DefaultPartial Partials\Facets\Hierarchy.html
Domain Classes Domain\Search\ResultSet\Facets\OptionBased\Hierarchy\*

Technical Solr background:

Technically the hierarchical facet for Solr is the same as a flat options facet. The support of hierarchies is implemented, by writing and reading the facet options by a convention:

[depth]-Level1Label/Level2Label/
Copied!

When you follow this convention by writing date into a Solr field you can render it as hierarchical facet. As example you can check indexing configuration in EXT:solr (EXT:solr/Configuration/TypoScript/Solr/setup.typoscript)

plugin.tx_solr {
    index {
        fieldProcessingInstructions {
            rootline = pageUidToHierarchy
        }
    }
}
Copied!

In this case the "fieldProcessingInstruction" "pageUidToHierarchy" is used to create the rootline for Solr in the conventional way.

Custom field processors can be registered with

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['fieldProcessor']['yourFieldProcessor'] = ACustomFieldProcessor::class;
Copied!

Date Range

When you want to provide a range filter on a date field in EXT:solr, you can use the type "dateRange".

The default partial generates a markup with all needed values in data attributes. Together with the provided jQuery UI implementation you can create an out-of-the-box date range facet.

With the following TypoScript you create a date range facet:

plugin.tx_solr.search {
    faceting = 1
    faceting.facets {
        creationDateRange {
            label = Created Between
            field = created
            type = dateRange
        }
    }
}
Copied!

In the extension we ship the TypoScript example "Search - (Example) dateRange facet with jQuery UI datepicker on created field" that shows how to configure a dateRange facet and load all required javascript files.

When you include this template a date range facet will be shown in the frontend that we look like this:

EXT:solr dateRange facet

As described before for the date range facet markup and javascript code is required, looking at the example template "Search - (Example) dateRange facet with jQuery UI datepicker on created field" in "Configuration/TypoScript/Examples/DateRange" you see that for the jQueryUi implementation the following files are included:

page.includeJSFooterlibs {
    solr-jquery = EXT:solr/Resources/Public/JavaScript/JQuery/jquery.min.js
    solr-ui = EXT:solr/Resources/Public/JavaScript/JQuery/jquery-ui.min.js
    solr-daterange = EXT:solr/Resources/Public/JavaScript/facet_daterange.js
}

page.includeCSS {
    solr-ui = EXT:solr/Resources/Public/Css/JQueryUi/jquery-ui.custom.css
}
Copied!

Numeric Range

Beside dates ranges are also usefull for numeric values. A typical usecase could be a price slider for a products page. With the user interface you should be able to filter the documents for a certain price range.

In the default partial, we also ship a partial with data attributes here to support any custom implementation. By default we will use the current implementation from EXT:solr based on jQueryUi.

The following example configures a numericRange facet for the field "pid":

plugin.tx_solr.search {
    faceting = 1
    faceting.facets {
        pidRangeRange {
            field = pid
            label = Pid Range
            type = numericRange
            numericRange {
                start = 0
                end = 100
                gap = 1
            }
        }
    }
}
Copied!

The numeric range facet requires beside the template also a javascript library to render the slider. The example TypoScript template "Search - (Example) Fluid numericRange facet with jQuery UI slider on pid field" can be used to see the range slider with jQuery UI for the Solr field pid by example.

When you configure a facet on the pid field like this, the frontend will output the following facet:

Numeric range facet

Beside the implementation with jQueryUi you are free to implement a range slider with any other javascript framework.

Rendering with fluid

Rendering facets with fluid is very flexible, because you can use existing ViewHelpers and implement your own logic in ViewHelpers to support your custom rendering logic.

In the default template the main faceting area on the left side, is done in the following file:

Resources/Private/Partials/Frontend/Result/Facets.html
Copied!

This template is used to render only the area for a few facets. The following part is the relevant part where we itterate over the facets:

<s:facet.area.group groupName="main" facets="{resultSet.facets.available}">
    <div class="facet-area-main">
        <div class="solr-facets-available secondaryContentSection">
            <div class="csc-header">
                <h3 class="csc-firstHeader">Narrow Search</h3>
            </div>
            <ul class="facets">
                <f:for each="{areaFacets}" as="facet">
                    <li class="facet facet-type facet-type-{facet.type}">
                        <f:render partial="Facets/{facet.partialName}"
                        arguments="{resultSet:resultSet, facet:facet}"/>
                    </li>
                </f:for>
            </ul>
        </div>
    </div>
</s:facet.area.group>
Copied!

Looking at the code above we see to important details that are important for solr.

Facet Grouping (Areas)

The first important part if the facet.area.group ViewHelper. By default all facets in the group main will be rendered. This value is the default value.

When you now want to render the facet at another place you can change the group with the following TypoScript configuration:

plugin.tx_solr.search {
    faceting = 1
    faceting.facets {
        contentType {
            field = type
            label = Content Type
            groupName = bottom
        }
    }
}
Copied!

Now the facet belongs to another group and will not be rendered in the "main" area anymore.

Default Partials

Another important fact is that Facet->getPartialName() is used to render the detail partial. The default implementation of a facet will return the default partial, that is able to render this facet.

If you need another rendering for one facet you can overwrite the used partial within the configuration:

plugin.tx_solr.search {
    faceting = 1
    faceting.facets {
        contentType {
            field = type
            label = Content Type
            partialName = mySpecialFacet
        }
    }
}
Copied!

Combining all of these concepts together with the flexibility of fluid you are able to render facets in a very flexible way.

Autosuggest

A user of the search typically want to find the results a fast as possible. To support the user and avoid to much typing solr can create a drop down list of common suggested search terms right after the search input box.

This feature can be easily configured with the following TypoScript setting:

plugin.tx_solr {
    suggest = 1
    suggest {
        numberOfSuggestions = 10
        suggestField = spell
    }
}
Copied!

Beside the server related part EXT:solr ships the jQuery autocomplete implementation to show the suggest results.

The suggest implementation uses the devbridge/jQuery-Autocomplete library(https://github.com/devbridge/jQuery-Autocomplete) to render the autocomplete.

If you want to configure an the autosuggest by example, you can include the TypoScript example template "(Example) Suggest/autocomplete with jquery".

When everything is configured the frontend will show you a drop down of suggestions when you are typing in the search field:

Autocomplete with jQuery

Beside search term suggestions the autocomplete an also show you the top search results for the user input.

This feature can be enabled with:

plugin.tx_solr.suggest.showTopResults = 1
Copied!

If you want to change the number of proposed top results you can also configure them:

plugin.tx_solr.suggest.numberOfTopResults = 5
Copied!

Sorting

When no sorting is selected the search will order the results by "relevance". This relevance is calculated by many factors and has the goal to deliver the best result for the query on the first position. That's what you expect from a search :)

For some usecases you want to change the sorting of the results by a certain field. In an onlineshop a user might want to order the results by the price to find the cheapest product that is matching his query.

A simple sorting can be configured with the following TypoScript snipped:

plugin.tx_solr.search.sorting >
plugin.tx_solr.search {
	sorting = 1
	sorting {
		defaultOrder = asc

		options {
			relevance {
				field = relevance
				label = Relevance
			}

			title {
				field = sortTitle
				label = Title
			}
		}
	}
}
Copied!

With the configuration above the possibility to sort by title is introduced. At the same time the sort by relevance link can be used to reset the sorting to sort by the natural Solr relevance.

Search with sorting

Templating

The rendering of the sorting is done on "Resources/Private/Partials/Results/Sorting.html" this partial is using the configuration and the view helpers to generate sorting links with the same behaviour as in ext:solr. For sure you can modifiy this template and use the ViewHelpers in the way how you want to implement your custom sorting.

Results per Page

EXT:solr allows you to configure how many result per page will be shown and at the same time the user can also change this value to an allowed value.

The following configuration can be used to configured the results per page:

plugin.tx_solr {
    search {
        results {
            resultsPerPage = 6
            resultsPerPageSwitchOptions = 12, 18, 24
        }
    }
}
Copied!

When you apply the configuration above, the frontend will show 6 search results by default and show the options 12, 18 and 24 to the user to change the amount of visible results

Results per page

Templating

The rendering of the "perPage selector" is done on "Resources/Private/Partials/Frontend/Results/PerPage.html". This partial is build in a way that the behaviour of the perPage selector is the same as in EXT:solr. If you want to do your custom rendering for example with links instead of a for, you can customize the rendering there.

Ajaxified Results

To improve the use experience and the performance it is possible to load most of the sub requests with ajax.

To activate the "ajaxification" you need to include the TypoScript template "Search - Ajaxify the searchresults with jQuery"

Include EXT:solr ajaxify template

How it works?

For all links with the css class "solr-ajaxified" the javascript search controller triggers the request against the same search page but with the type "7383" which is just rendering the search request. The response is replacing everything in the container "div.tx_solr" with the content of the response.

Limitations

Since the example only renders the plugin on a specific page type and not the same plugin instance as on the page, settings from the flexform are not taken into account by now.

Customize

Use custom Fluid Templates

After these steps solrfluid is usable and using the default Templates, Layouts and Partials. If you want to overwrite them, you can change the TypoScript configuration:

plugin.tx_solr {
    view {
        layoutRootPaths.10 = EXT:yourpath/Layouts/
        partialRootPaths.10 = EXT:yourpath/Partials/
        templateRootPaths.10 = EXT:yourpath/Templates/
    }
}
Copied!

Now you can copy the default partials from the extension to you project path and adapt them to your needs.

Languages

We recommend to create one Solr core per language. The shipped Solr example configuration provides a setup for the following languages:

  • Arabic (core_ar)
  • Armenian (core_hy)
  • Basque (core_eu)
  • Brazilian portuguese (core_ptbr)
  • Bulgarian (core_bg)
  • Burmese (core_my)
  • Catalan (core_ca)
  • Chinese (core_zh)
  • Czech (core_cs)
  • Danish (core_da)
  • Dutch (core_nl)
  • English (core_en)
  • Finnish (core_fi)
  • French (core_fr)
  • Galician (core_gl)
  • German (core_de)
  • Greek (core_el)
  • Hindi (core_hi)
  • Hungarian (core_hu)
  • Indonesian (core_id)
  • Irish (core_ie)
  • Italian (core_it)
  • Japanese (core_ja)
  • Khmer (core_km)
  • Korean (core_ko)
  • Lao (core_lo)
  • Latvia (core_lv)
  • Norwegian (core_no)
  • Persian (core_fa)
  • Polish (core_pl)
  • Portuguese (core_pt)
  • Romanian (core_ro)
  • Russian (core_ru)
  • Serbian (core_rs)
  • Spanish (core_es)
  • Swedish (core_sv)
  • Thai (core_th)
  • Turkish (core_tr)
  • Ukrainian (core_uk)

The configuration of the connection between Solr cores and sites is done in the site configuration.

Configure Routing

Currently only one route enhancer exists. It's purpose is to mask facets inside the query string or as part of the path segment.

The basement for routing is the enhancer SolrFacetMaskAndCombineEnhancer. Open your site configuration with an editor of your choice.

Locate the section of route enhancer routeEnhancers. If section available, add routeEnhancers to your configuration.

Use enhancer SolrFacetMaskAndCombineEnhancer as type of your route, and limit it to pages where the route should apply.

The extension key have to be set to tx_solr.

Solr specific configuration will placed inside of key solr.

The following example shows a the basement of the enhancer configuration.

routeEnhancers:
  products:
    type: SolrFacetMaskAndCombineEnhancer
    limitToPages:
      - 42
    extensionKey: tx_solr
    solr:
Copied!

If you use something else as tx_solr for the plugin namespace (see tx_solr.view) you need additional configuration.

Change the extensionKey from tx_solr to your plugin namespace.

routeEnhancers:
  products:
    type: SolrFacetMaskAndCombineEnhancer
    limitToPages:
      - 42
    extensionKey: search
Copied!

The next step is to configure a path segment or to mask the facets inside of the query.

Facets in path segments

This chapter describe how you can use a facet as part of your url path.

Imagine you have an online shop and use Solr to display the products. It would be convenient for the customer if can call an url path like /products/candy.

The first step is to create a site /products and place the Solr plugin as content.

Configure the path segment

In our example we are using facet productType to filter the search result for a specific product type.

The first step is to define a route path. Because we have already a side called /products only need to define a path contain the facet.

The name itself needs no connection to the facet, so you can choose something different.

routeEnhancers:
  products:
    routePath: '/{productType}'
Copied!

The next step is to map the variable to the facet itself.

Map the variable to the facet

The mapping from variable to facet differs from the standard as you know from TYPO3 or symfony.

A facet inside of the query is a composition of facet and value.

Example:

/products/?tx_solr[filter][0]=taste:matcha&tx_solr[filter][1]=productType:candy&tx_solr[filter][2]=color:yellow&tx_solr[filter][3]=color:green&tx_solr[filter][4]=taste:sweet

This example shows another issues:

  • there is not specific order of facets
  • a facet can contains multiple values

Define a query path would not work in this case.

The solution is to define a part of a query path and attach the facet you want to place inside of the url.

This is done by separating the query path and the facet by a dash.

routeEnhancers:
  products:
    # Note: All arguments inside of namespace tx_solr. See -> extensionKey
    # Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'
    _arguments:
      productType: filter-productType
    requirements:
      productType: '.*'
Copied!

The route enhancer now knows to place values of facet productType as path segment.

As result the URL now changed into

/products/candy/?tx_solr[filter][0]=taste:matcha&tx_solr[filter][1]=color:yellow&tx_solr[filter][2]=color:green&tx_solr[filter][3]=taste:sweet

Handling of multiple values

As the customer can combine as many values of a facet to filter the result, the path segment can contain multiple value.

The route enhancer will collect all values of the configured facet and combine them to a single string.

The values will sort alphanumeric and concat with a colon.

You can change the value separator by configuring a multi value separator:

routeEnhancers:
  products:
    solr:
      multiValueSeparator: ';'
Copied!

Additional you can replace specific characters inside of each value using a map to replace the characters:

routeEnhancers:
  products:
    solr:
      replaceCharacters:
        ' ': '-'
Copied!

Now lets have a look at the effect.

Before the configuration we had this URL:

/products/?tx_solr[filter][0]=taste:matcha&tx_solr[filter][1]=productType:dark+chocolate&tx_solr[filter][2]=taste:sweet&tx_solr[filter][3]=productType:candy

After applying the configuration, the URL changed into:

/products/candy;dark-chocolate?tx_solr[filter][0]=taste:matcha&tx_solr[filter][1]=taste:sweet

Full example

This example shows the all configuration done above

routeEnhancers:
  products:
    type: SolrFacetMaskAndCombineEnhancer
    limitToPages:
      - 42
    extensionKey: tx_solr
    routePath: '/{productType}'
    # Note: All arguments inside of namespace tx_solr. See -> extensionKey
    # Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'
    _arguments:
      productType: filter-productType
    # Important: Configure requirement for fields! If you wand to allow empty values, set .*
    requirements:
      productType: '.*'
    solr:
      multiValueSeparator: ';'
      replaceCharacters:
        ' ': '-'
Copied!

Simplify facets inside of the query

Sometimes you or your customer want to share links to a search result. The link itself could really hard to read, a specially if it contains a lot of filters.

There are two options that you can apply on the query string:

  1. Concat multiple values of facet into a single string
  2. Mask the filter inside of the query

All query related configurations will be placed below node query below node solr:

routeEnhancers:
  products:
    solr:
      query:
Copied!

Concat multiple values into a string

To activate the concatenation of multiple values add the concat entry and set the value to true.

By default the values will concat with a colon. If you need a different separator, add the entry valueSeparator and set a value that fits your need.

routeEnhancers:
  products:
    solr:
      query:
        concat: true
        # valueSeparator: ','
Copied!

Before Solr concatenates the values, they will be sorted in alphanumeric matter.

Imaging following filter:

tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha

By enabling the concat of multiple values, it would change into:

tx_solr[filter][0]=taste:matcha,sour,sweet

Mask filter inside of the query

Mask filter inside of the URL allows you to make the URL more readable and to shrink the length.

The configuration contains two parts:

  1. enable the mask
  2. map from the facet name to parameter (see faceting.facets.[facetName] - single facet configuration)

Add the entry mask with the value true to the query configuration.

Add the node map and configure the mapping.

The value of the map defines the facet key, the value the parameter that will be used inside of the URL.

routeEnhancers:
  products:
    solr:
      query:
        mask: true
        map:
          taste: taste
Copied!

Imaging following filter:

tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha

By enabling the concat of multiple values, it would change into:

taste=matcha,sour,sweet

Full example

This example shows all configurations together, which were done above in step by step:

routeEnhancers:
  products:
    solr:
      query:
        # To reduce the amount of parameters you can force Solr to concat the values.
        # For example you have following filter:
        #   tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha
        #
        # Concat will:
        # 1. collect all filters of the same type
        # 2. will sort all filter values alpha numeric
        # 3. join the values together
        #
        # As a result the query will modified into:
        #   tx_solr[filter][0]=taste:matcha,sour,sweet
        #
        # Note: If you active the mask option, the concat feature turn on automatically
        #
        concat: true
        # valueSeparator: ','

        # You can tell Solr to mask query facets. This feature require the map below
        #
        # For example you have following filter:
        #   tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha
        # Mask will:
        # 1. implode all values into a single string and sort it -> green,red,yellow
        # 2. replace tx_solr[filter][]=color: with color=
        #
        # As a result the query will modified into:
        # taste=matcha,sour,sweet
        #
        mask: true

        # In order to simplify a filter argument, you have to define a corresponding map value
        # There is no automatically reduction of filter parameters at the moment available.
        # The key is the name of your facet, the value what use instead.
        #
        # Important:
        # There are some restrictions for the values. The use of TYPO3 core parameters is prohibited.
        # This contains at the moment following strings: no_cache, cHash, id, MP, type
        map:
          taste: taste
Copied!

Example

routeEnhancers:
  Products:
    type: SolrFacetMaskAndCombineEnhancer
    limitToPages:
      - 10
    extensionKey: tx_solr
    solr:
      multiValueSeparator: '+'
      replaceCharacters:
        ' ': '_'
      query:
        # To reduce the amount of parameters you can force Solr to concat the values.
        # For example you have following filter:
        #   tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha
        #
        # Concat will:
        # 1. collect all filters of the same type
        # 2. will sort all filter values alpha numeric
        # 3. join the values together
        #
        # As a result the query will modified into:
        #   tx_solr[filter][0]=taste:matcha,sour,sweet
        #
        # Note: If you active the mask option, the concat feature turn on automatically
        #
        concat: true
        # valueSeparator: ','

        # You can tell Solr to mask query facets. This feature require the map below
        #
        # For example you have following filter:
        #   tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha
        # Mask will:
        # 1. implode all values into a single string and sort it -> green,red,yellow
        # 2. replace tx_solr[filter][]=color: with color=
        #
        # As a result the query will modified into:
        # taste=matcha,sour,sweet
        #
        mask: true

        # In order to simplify a filter argument, you have to define a corresponding map value
        # There is no automatically reduction of filter parameters at the moment available.
        # The key is the name of your facet, the value what use instead.
        #
        # Important:
        # There are some restrictions for the values. The use of TYPO3 core parameters is prohibited.
        # This contains at the moment following strings: no_cache, cHash, id, MP, type
        map:
          color: color
          taste: taste
          product: product
    routePath: '/{type}'
    # Note: All arguments inside of namespace tx_solr. See -> extensionKey
    # Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'
    _arguments:
      type: filter-type
    requirements:
      type: '.*'
Copied!

Logging

When you need to debug something in EXT:solr the extension allows you to write several information to the TYPO3 logging framework.

This can be helpful to analyze e.g.:

  • What is happening during the indexing?
  • What is going when a search in the frontend is triggered?

Configure Logging

There are several options that can be configured with TypoScript:

EXT:solr logging settings

You can find a detailed description of all options in the "tx_solr.logging" section in the reference.

Note: Make sure that you log on production only what you need, because log data can increase very quickly.

In the next steps we want to see as example how we can use the debug output to see which query was triggered in solr.

The first step is to debug the output for "rawGet":

plugin.tx_solr.logging.query.rawGet = 1
Copied!

Show Logging Output

As written in the introduction, the logging framework functionally is used to write the log entries.

For a quick debugging without any extension there is the possibility to show the debug messages directly:

  • For the frontend as formatted output.
  • For the backend as TYPO3 console output.

Note: You should use this on a development system only and make sure that the devIPmask does not allow other users to see the debug output.

To enable this features you need to:

  • Make sure that your ip address is configured to match the devIPMask ($GLOBALS['TYPO3_CONF_VARS']['SYS']['devIPmask'])
  • Enable the debug output in the EXT:solr TypoScript:
plugin.tx_solr.logging.debugOutput = 1
Copied!

You can now see the written log in the frontend:

EXT:solr frontend debug output

Configuration Structure

The configuration can be found in the folder "Resources/Private/Solr".

This folder contains:

  • The folder "configsets": This folder a set of configuration files that can be deployed into a Solr server, as a template.

It contains the "solrconfig.xml" file, the "schema.xml" files for all languages and the accessfilter libary that belongs to this version as a jar file. This configSet needs to be in place on a Solr server to create cores that are compatible to the EXT:solr extension.

  • The folder "cores": This folder ships an example "core.properties" file for all languages that are compatible with EXT:solr.

A "core.properties" file references a "configSet" that should be used. The path to the schema that is bound to a core is configured as "schema" relative to the root folder of the "configSet".

By example a "core.properties" file looks like this:

configSet=ext_solr_6_0_0
schema=german/schema.xml
name=core_de
dataDir=../../data/german
Copied!
  • The solr.xml file: This file configures Solr as required for the used Apache Solr version.

The EXT:solr configSet

As mentioned before the configSet is one single package, that contains all to customize a plain Apache Solr Server, to an Apache Solr Server, ready for EXT:solr & TYPO3.

The configset of the current release is located in "Resources/Private/Solr/configsets/ext_solr_<release>".

Inside the configSet you find the following folders:

  • conf: This folder contains the Solr configuration and all schemata files. There is one directory for each language which ships the schema.xml file for this language.

The schema.xml file contains the language specific adaptions and includes all general schema fields and types with an XInclude statement.

<!-- xinclude fields -->
<xi:include href="../general_schema_types.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>

<!--  xinclude fields-->
<xi:include href="../general_schema_fields.xml" xmlns:xi="http://www.w3.org/2001/XInclude"/>
Copied!
  • typo3lib: This folder ships the compiled access filter jar file, that should be used with this EXT:solr version.

The solrconfig (conf/solrconfig.xml) is configured to load all jar files from typo3lib.

Setup steps

With the extension we ship an installer for development and a Docker images that can be used to install Solr.

When you want to install Solr on your system in another way the following steps are required.

  • Install the Solr server
  • Copy the configsets into the configset folder (by default $SOLR_HOME/server/solr/configsets)
  • Make sure that the solr.xml file ($SOLR_HOME/server/solr/solr.xml) is in place and fits to your Solr version
  • Create an init script that starts Solr on boottime.
  • Secure your Solr port from outside.
  • Make sure that Solr is running with an own user.
  • Backup your data folders

Hint: Apache Solr ships an install script in newer version that might cover your requirements for production ($SOLR_HOME/bin/install_solr_service.sh). We don't use it in EXT:solr because there are currently problems when using it with Ubuntu Xenial (16.04)

Index Maintenance

Solr offers a lot of request handlers to do maintenance tasks.

Committing pending documents

curl http://host:port/solr-path/update -H "Content-Type: text/xml"
    --data-binary '<commit />'
Copied!

Clearing the index

curl http://host:port/solr-path/update -H "Content-Type: text/xml"
    --data-binary '<delete><query>*:*</query></delete>'

curl http://host:port/solr-path/update -H "Content-Type: text/xml"
    --data-binary '<commit />'
Copied!

Optimizing the index

This is no longer needed. The old scheduler task have also been removed.

Searching the index from the command line

Parameters:

q

what to search for. Format: fieldName:fieldValue

qt

defines the query type, for the command line we recommend "standard", the extension itself uses "dismax"

fl

comma separated list of fields to return

rows

number of rows to return

start

offset from where to return results

curl 'http://host:port/path-to-solr/select?q=hello&qt=standard&fl=title,content'
Copied!

Getting information / statistics about the index

curl 'http://host:port/path-to-solr/admin/luke'
Copied!

Create cores with the core admin api

The CoreAdmin API (https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API) allows you, manipulate the cores in your Solr server.

Since we support configSets a core could be generated with the following http call:

curl 'http://host:port/path-to-solr/admin/cores?action=CREATE&name=core_de&configSet=ext_solr_8_0_0&schema=german/schema.xml&dataDir=../../data/german'
Copied!

SolrConfig Parameters

There are several parameters in the solrconfig.xml that can be used to tune your Solr server. Our solrconfig.xml is designed to ship a reasonable configuration for the most standard use cases.

For use cases with very large indexes or high performance requirements it makes sence to tune those parameters

indexConfig.useCompoundFile

This value is "true" by default in our configuration. By setting this value to true Solr only writes one file for indexes instead of many. This is a little bit slower but more robust to prevent errors with "Too many open files".

Configuration Reference

tx_solr

This section defines general configuration options.

enabled

Type

Boolean

TS Path

plugin.tx_solr.enabled

Default

1

Options

0, 1

Since

1.2

A switch to completely turn on / off EXT:solr. Comes in handy with multi site installations where you want to enable EXT:solr only for certain sites, but still have the extension's configuration at a single place and include that for each site. Just set enabled = 0 for each site's root TS template or use conditions where you do not want EXT:solr.

enableDebugMode

Type

Boolean

TS Path

plugin.tx_solr.enableDebugMode

Default
Options

0, 1

Since

1.0

See

http://wiki.apache.org/solr/CommonQueryParameters#debugQuery

If enabled, the debugQuery query parameter is added to the Solr queries. Solr will then return additional information explaining the the query, scoring, timing, and other information.

tx_solr.general

This section defines general settings.

dateFormat.date

Type

String

TS Path

plugin.tx_solr.general.dateFormat.date

Default

d.m.Y H:i

Since

1.0

See

http://www.php.net/manual/de/function.strftime.php

Defines the format that is used for dates throughout the extension like in view helpers for example. The format uses the strftime() php function syntax, please consult the php documentation for available options.

tx_solr.view

All view related settings, these settings might also be relevant for Fluid.

pluginNamespace

Type

String

TS Path

plugin.tx_solr.view.pluginNamespace

Since

7.0

Default

tx_solr

Plugin namespace. Can be used to change the plugin namespace and can be changed by instance in the flexform.

templateFiles

By convention the templates is loaded from EXT:solr/Resources/Private/Templates/Frontend/Search/(ActionName).html. If you want to define a different entry template, you can do this here to overwrite the conventional default template. If you want to use FLUID fallbacks you can just configure the template name, otherwise you could also use a full reference EXT:/.../.

The templates that you configure in availableTemplate can be used in the flexform by the editor to select a template for the concrete plugin instance.

templateFiles.results

Type

String

TS Path

plugin.tx_solr.view.templateFiles.results

Since

7.0 (Replaces previous setting plugin.tx_solr.templateFiles.result)

Default

Results

By convention the "Results" template from you configured FLUID template path will be used As alternative you can configure a different template name here (e.g. MyResults or a full path to an entry template here).

templateFiles.results.availableTemplates

Type

Array

TS Path

plugin.tx_solr.view.templateFiles.results.availableTemplates

Since

7.0

Default

none

Allows to configure templates that are available in the flexform to switch.

Example:

plugin.tx_solr.view.templateFiles.results.availableTemplates {
    default {
        label = Default Searchresults Template
        file = Results
    }
    finder {
        label = Productfinder Template
        file = ProductFinder
    }
}
Copied!

templateFiles.form

Type

String

TS Path

plugin.tx_solr.view.templateFiles.form

Since

7.0 (Replaces previous setting plugin.tx_solr.templateFiles.form)

Default

Form

By convention the "Form" template from you configured FLUID template path will be used . As alternative you can configure a different template name here (e.g. MyForm or a full path to an entry template here).

templateFiles.form.availableTemplates

Type

Array

TS Path

plugin.tx_solr.view.templateFiles.form.availableTemplates

Since

7.0

Default

none

Allows to configure templates that are available in the flexform to switch.

Example:

plugin.tx_solr.view.templateFiles.form.availableTemplates {
    default {
        label = Default Searchform Template
        file = Form
    }
    specialform {
        label = Extended Search Form
        file = BetterForm
    }
}
Copied!

templateFiles.frequentSearched

Type

String

TS Path

plugin.tx_solr.view.templateFiles.frequentSearched

Since

7.0 (Replaces previous setting plugin.tx_solr.templateFiles.frequentSearched)

Default

FrequentlySearched

By convention the "FrequentlySearched" template from you configured FLUID template path will be used . As alternative you can configure a different template name here (e.g. FrequentlySearched or a full path to an entry template here).

tx_solr.index

This part contains all configuration that is required to setup your indexing configuration. You can use EXT:solr to easily index pages or any kind of records of your TYPO3 CMS.

Allows to prevent frontend indexing of pages when a backend editor is logged in and browsing the website.

additionalFields (deprecated)

Type

String, cObject (since 1.1)

TS Path

plugin.tx_solr.index.additionalFields

Since

1.0

Deprecated

2.0

A mapping of Solr field names to additional string values to be indexed with page documents. Use dynamic fields to index additional data, this way you don't have to modify the schema.xml

Example:

plugin.tx_solr.index.additionalFields {
  myFirstAdditionalField_stringS = some string

  mySecondAdditionalField_stringS = TEXT
  mySecondAdditionalField_stringS {
    value = some other value that can be constructed using any TypoScript cObject
    case = upper
    // more processing here as needed
  }
}
Copied!

Since version 1.1 you can use cObjects to generate the value for the field. The only thing to observe is that you generate strings. Other values may work, but haven't been tested yet.

Deprecated since 2.0, please use the Index Queue indexing configurations instead as it allows you to define more precisely for which types of documents you want which fields to be indexed.

fieldProcessingInstructions

Type

cObject

TS Path

plugin.tx_solr.index.fieldProcessingInstructions

Since

1.2 2.0

Options

timestampToIsoDate, uppercase, pathToHierarchy (2.5-dkd), pageUidToHierarchy (2.5-dkd)

Assigns processing instructions to Solr fields during indexing (Syntax: Solr index field = processing instruction name). Before documents are sent to the Solr server they are processed by the field processor service. Currently you can make a filed's value all uppercase, convert a UNIX timestamp to an ISO date, or transform a path into a hierarchy for hierarchical facets (2.0 only). Currently you can use only one processing instruction at a time.

Example:

fieldProcessingInstructions {
    changed = timestampToIsoDate
    created = timestampToIsoDate
    endtime = timestampToIsoDate
}
Copied!

Since version 11.5 it is possible to extend / add own processing instructions.

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['fieldProcessor']['yourFieldProcessor'] = ACustomFieldProcessor::class;
Copied!

Custom processors have to implement interface ApacheSolrForTypo3\Solr\FieldProcessor\FieldProcessor.

queue

The Index Queue is a powerful feature introduced with version 2.0. It allows you to easily index any table in your TYPO3 installation by defining a mapping of SolrFieldName = DatabaseTableFieldNameOrContentObject. The table must be configured / described in TCA, though. To index other, external data sources you might want to check out Solr's Data Import Handler (DIH).

The Index Queue comes preconfigured to index pages (enabled by default) and an example configuration for tt_news (provided as a separate TypoScript template).

Type

Array

TS Path

plugin.tx_solr.index.queue

Since

2.0

Default

pages

Defines a set of table indexing configurations. By convention the name of the indexing configuration also represents the table name. You can name the indexing configuration differently though by explicitly defining the table as a parameter within the indexing configuration. That's useful when indexing records from one table with different configuration - different single view pages / URLs for example.

Example:

// enables indexing of tt_news records
plugin.tx_solr.index.queue.news = 1
plugin.tx_solr.index.queue.news.fields {
    abstract = short
    author = author
    description = short
    title = title

    // the special SOLR_CONTENT content object cleans HTML and RTE fields
    content = SOLR_CONTENT
    content {
        field = bodytext
    }

    // the special SOLR_RELATION content object resolves relations
    category_stringM = SOLR_RELATION
    category_stringM {
        localField = category
        multiValue = 1
    }

    // the special SOLR_MULTIVALUE content object allows to index multivalue fields
    keywords = SOLR_MULTIVALUE
    keywords {
        field = keywords
    }

    // build the URL through typolink, make sure to use returnLast = url
    url = TEXT
    url {
        typolink.parameter = {$plugin.tt_news.singlePid}
        typolink.additionalParams = &tx_ttnews[tt_news]={field:uid}
        typolink.additionalParams.insertData = 1
        typolink.returnLast = url
        typolink.useCacheHash = 1
    }

    sortAuthor_stringS = author
    sortTitle_stringS  = title
}
Copied!

queue.[indexConfig]

Type

Boolean, Array

TS Path

plugin.tx_solr.index.queue.[indexConfig]

Since

2.0

Default

pages

An indexing configuration defines several parameters about how to index records of a table. By default the name of the indexing configuration is also the name of the table to index.

By setting plugin.tx_solr.index.queue.[indexConfig] = 1 or 0 you can en- / disable an indexing configuration.

Note: you could add L={field:__solr_index_language} in the additionalParams of the typolink to link to the correct language version (this was removed from the example above to simplify the example)

queue.[indexConfig].additionalWhereClause

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].additionalWhereClause

Since

2.0

A WHERE clause that is used when initializing the Index Queue, limiting what goes into the Queue. Use this to limit records by page ID or the like.

// only index standard and mount pages, enabled for search
plugin.tx_solr.index.queue.pages.additionalWhereClause = doktype IN(1, 7)
Copied!

queue.[indexConfig].initialPagesAdditionalWhereClause

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].initialPagesAdditionalWhereClause

Since

6.1

A WHERE clause that is used when initializing the Index Queue, limiting pages that goes into the Queue. This filter is applied prior to the plugin.tx_solr.index.queue.[indexConfig].additionalWhereClause filter and hence provides an even stronger filter mechanism - since it can be used to filter away page ID's that shouldn't be processed at all.

// Filter away pages that are "spacer" and have no_search, hidden and nav_hide set to zero
plugin.tx_solr.index.queue.pages.initialPagesAdditionalWhereClause = doktype <> 199 AND no_search = 0 AND hidden = 0 AND nav_hide = 0
Copied!

queue.[indexConfig].additionalPageIds

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].additionalPageIds

Since

2.0

Defines additional pages to take into account when indexing records for example. Especially useful for indexing DAM records or if you have your news outside your site root in a shared folder to use for multiple sites.

Additional page IDs can be provided as comma-separated list.

queue.[indexConfig].type

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].type

Since

11.5.1

Defines the type to index, which is usally the database table. Sometimes you may want to index records from a table with different configurations, f.e., to generate different single view URLs for tt_news records depending on their category or storage page ID. In these cases you can use a distinct name for the configuration and define the table explicitly.

plugin.tx_solr.index.queue.generalNews {
  type = tt_news
  fields.url = URL for the general news
  // more field configurations here ...
}

// extends the general news configuration
plugin.tx_solr.index.queue.pressNews < plugin.tx_solr.index.queue.generalNews
plugin.tx_solr.index.queue.pressNews {
  fields.url = overwriting URL for the press announcements
  // may overwrite or unset more settings from the general configuration
}

// completely different configuration
plugin.tx_solr.index.queue.productNews {
  type = tt_news
  fields.url = URL for the product news
}
Copied!

queue.[indexConfig].indexQueue

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].indexQueue

Since

11.6

Default
 

Class name of custom index queue implementation, falls back to the default index queue (ApacheSolrForTypo3SolrIndexQueueQueue).

queue.[indexConfig].initialization

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].initialization

Since

2.0

When initializing the Index Queue through the search backend module the queue tries to determine what records need to be indexed. Usually the default initializer will be enough for this purpose, but this option allows to define a class that will be used to initialize and add records to the Index Queue in special ways.

The extension uses this option for initializing the pages and more specifically to resolve Mount Page trees so they can be indexed too, although only being virtual pages.

queue.[indexConfig].indexer

Type

String, Array

TS Path

plugin.tx_solr.index.queue.[indexConfig].indexer

Since

2.0

When configuring tables to index a default indexer is used that comes with the extensions. The default indexer resolves the Solr field to database table field mapping as configured. However, in some cases you may reach the limits of TypoScript, when this happens you can configure a specialized indexer using this setting.

The indexer class is loaded using TYPO3's auto loading mechanism, so make sure your class is registered properly. The indexer must extend tx_solr_indexqueue_Indexer.

Example, pages use a specialized indexer:

plugin.tx_solr.index.queue.pages {
    indexer = tx_solr_indexqueue_PageIndexer
    indexer {
        // add options for the indexer here
    }
}
Copied!

Within the indexer configuration you can also define options for the specialized indexer. These are then available within the indexer class in $this->options.

Example, the TypoScript settings are available in PHP:

TypoScript:

plugin.tx_solr.index.queue.myIndexingConfiguration {
    indexer = tx_myextension_indexqueue_MyIndexer
    indexer {
        someOption = x
        someOtherOption = y
    }
}
Copied!

PHP:

namespace MyVendor\Namespace;

use ApacheSolrForTypo3\Solr\IndexQueue\Indexer;

class MyIndexer extends Indexer {
  public function index(tx_solr_indexqueue_Item $item) {
    if ($this->options['someOption']) {
      // ...
    }
  }
}
Copied!

queue.[indexConfig].indexingPriority

Type

Integer

TS Path

plugin.tx_solr.index.queue.[indexConfig].indexingPriority

Since

2.2

Default

Allows to define the order in which Index Queue items of different kinds are indexed. Items with higher priority are indexed first.

queue.[indexConfig].fields

Type

Array

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields

Since

2.0

Mapping of Solr field names on the left side to database table field names or content objects on the right side. You must at least provide the title, content, and url fields. TYPO3 system fields like uid, pid, crdate, tstamp and so on are added automatically by the indexer depending on the TCA information of a table.

You can also use a virtual field called __solr_content, which holds the content of the current page.

Example:

plugin.tx_solr.index.queue.[indexConfig].fields {
  content = bodytext
  title = title
  url = TEXT
  url {
    typolink.parameter = {$plugin.tx_extensionkey.singlePid}
    typolink.additionalParams = &tx_extenionkey[record]={field:uid}
    typolink.additionalParams.insertData = 1
    typolink.returnLast = url
  }
}
Copied!

queue.[indexConfig].attachments.fields

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].attachments.fields

Since

2.5-dkd

Comma-separated list of fields that hold files. Using this setting allows to tell the file indexer in which fields to look for files to index from records.

Example:

plugin.tx_solr.index.queue.tt_news.attachments.fields = news_files
Copied!

queue.[indexConfig].recursiveUpdateFields

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].recursiveUpdateFields

Since

6.1

Default

Empty

Allows to define a list of additional fields from the pages table that will trigger a recursive update.

plugin.tx_solr.index.queue.pages.recursiveUpdateFields = title
Copied!

The example above will trigger a recursive update of pages if the title is changed.

Please note that the following columns should NOT be configured as recursive update fields: "hidden" and "extendToSubpages". These fields are handled by EXT:solr already internally and thus they will have not effect.

queue.pages.excludeContentByClass

Type

String

TS Path

plugin.tx_solr.index.queue.pages.excludeContentByClass

Since

4.0

Can be used for page indexing to exclude a certain css class to be indexed.

Example:

plugin.tx_solr.index.queue.pages.excludeContentByClass = removeme
Copied!

The example above will remove the content of items in the page that have the css class "removeme".

queue.pages.allowedPageTypes

Type

List of Integers

TS Path

plugin.tx_solr.index.queue.pages.allowedPageTypes

Since

3.0

Default

1,7

Allows to set the pages types allowed to be indexed.

Even if you have multiple queues for pages, e.g. via different additionalWhereClause's, you have to set this value to allow further doktype's. Restrict the pages to be indexed by each queue via additionalWhereClause.

queue.pages.indexer.authorization.username

Type

String

TS Path

plugin.tx_solr.index.queue.pages.indexer.authorization.username

Since

2.0

Specifies the username to use when indexing pages protected by htaccess.

queue.pages.indexer.authorization.password

Type

String

TS Path

plugin.tx_solr.index.queue.pages.indexer.authorization.password

Since

2.0

Specifies the password to use when indexing pages protected by htaccess.

queue.pages.indexer.frontendDataHelper.scheme

Type

String

TS Path

plugin.tx_solr.index.queue.pages.indexer.frontendDataHelper.scheme

Since

2.0

Specifies the scheme to use when indexing pages.

queue.pages.indexer.frontendDataHelper.host

Type

String

TS Path

plugin.tx_solr.index.queue.pages.indexer.frontendDataHelper.host

Since

2.0

Specifies the host to use when indexing pages.

queue.pages.indexer.frontendDataHelper.path

Type

String

TS Path

plugin.tx_solr.index.queue.pages.indexer.frontendDataHelper.path

Since

2.0

Specifies the path to use when indexing pages.

Indexing Helpers

To make life even easier the Index Queue provides some indexing helpers. These helpers are content objects that perform cleanup tasks or content transformations.

SOLR_CONTENT

Since

2.0

Cleans a database field in a way so that it can be used to fill a Solr document's content field. It removes HTML markup, Javascript and invalid utf-8 chracters.

The helper supports stdWrap on its configuration root.

Example:

content = SOLR_CONTENT
content {
    field = bodytext
}
Copied!

Parameters:

value

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].value

Since

2.0

Defines the content to clean up. In this case the value would be hard-coded.

SOLR_MULTIVALUE

Since

2.0

Turns comma separated strings into an array to be used in a multi value field of an Solr document.

The helper supports stdWrap on its configuration root.

Example:

keywords = SOLR_MULTIVALUE
keywords {
    field = tags
    separator = ,
    removeEmptyValues = 1
}
Copied!

Parameters:

value

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].value

Since

2.0

Defines the content to clean up. In this case the value would be hard-coded.

separator

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].separator

Since

2.0

Default

,

The separator by which to split the content.

removeEmptyValues

Type

Boolean

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].removeEmptyValues

Since

2.0

Options

0,1

Default

1

The helper will clean the resulting array from empty values by default. If, for some reason, you want to keep empty values just set this to 0.

removeDuplicateValues

Type

Boolean

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].removeDuplicateValues

Since

2.9

Options

0,1

Default

Cleans the result from duplicate values.

SOLR_RELATION

Since

2.0

Resolves relations between tables.

Example:

category_stringM = SOLR_RELATION
category_stringM {
    localField = category
    multiValue = 1
}
Copied!

Parameters:

localField

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].localField

Since

2.0

Required

yes

The current record's field name to use to resolve the relation to the foreign table.

foreignLabelField

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].foreignLabelField

Since

2.0

Usually the label field to retrieve from the related records is determined automatically using TCA, using this option the desired field can be specified explicitly. To specify the label field for recursive relations, the field names can be separated by a dot, e.g. for a category hierarchy to get the name of the parent category one could use "parent.name" (since version:2.9).

foreignLabel

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].foreignLabel

Since

12.1

Defines how to build the label for indexing, stdWrap is applied. Can be used to overrule foreignLabelField. Referencing to "field" e.g. inside dataWrap will resolve to resolved record.

Example:

authorsNames_stringM = SOLR_RELATION
authorsNames_stringM {
    localField = author
    foreignLabel.dataWrap = {field : first_name} {field : last_name}
    multiValue = 1
}
Copied!

multiValue

Type

Boolean

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].multiValue

Since

2.0

Options

0,1

Default

Whether to return related records suitable for a multi value field. If this is disabled the related values will be concatenated using the following singleValueGlue.

singleValueGlue

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].singleValueGlue

Since

2.0

Default

", "

When not using multiValue, the related records need to be concatenated using a glue string, by default this is ", " (comma followed by space). Using this option a custom glue can be specified. The custom value must be wrapped by pipe (|) characters to be able to have leading or trailing spaces.

relationTableSortingField

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].relationTableSortingField

Since

2.2

Field in an mm relation table to sort by, usually "sorting".

enableRecursiveValueResolution

Type

Boolean

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].enableRecursiveValueResolution

Since

2.9

Options

0,1

Default

1

If the specified remote table's label field is a relation to another table, the value will be resolve by following the relation recursively.

removeEmptyValues

Type

Boolean

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].removeEmptyValues

Since

2.9

Options

0,1

Default

1

Removes empty values when resolving relations.

removeDuplicateValues

Type

Boolean

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].removeDuplicateValues

Since

2.9

Options

0,1

Default

Removes duplicate values

additionalWhereClause

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].additionalWhereClause

Since

5.0

Where clause that could be used to limit the related items to a subset that matches this where clause

Example:

category_stringM = SOLR_RELATION
category_stringM {
    localField = tags
    multiValue = 1
    additionalWhereClause = pid=2
}
Copied!

SOLR_CLASSIFICATION

Since

8.0

Allows to classify documents based on a configured pattern

Example:

topic_stringM = SOLR_CLASSIFICATION
topic_stringM {
    field = __solr_content
    classes {
        programming {
            matchPatterns = php, java, javascript, go
            class = programming
        }
        cms {
            matchPatterns = TYPO3, joomla
            class = cms
        }
        database {
            matchPatterns = mysql, MariaDB, postgreSQL
            class = database
        }
    }
}
Copied!

The `matchPatterns` can be used to configure pattern that can occure in the content to add that class. In addition `unmatchPatterns` can be configured to define patterns that should not occure in the content.

Patterns are regular expressions. You configure everything that is possible with regular expressions.

Example:s

The pattern `\ssmart[a-z]*\s` will match everything, that starts with a space followed by smart ending with any lowercase letter and ending by space. This would match e.g. smartphone, smarthome and every other word that starts with `smart`.

Note:

  • The configuration `patterns` is deprecated with 10.0.0 and will be removed in EXT:solr 11. Please use `matchPatterns` and `unmatchPatterns now.

field

Type

String

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].field

Since

8.0

Name of the database field, that should be used to as content to classify. The special field __solr_content can be used during indexing to classify the content of the page or file or any other record that fills the content field before.

classes

Type

Array

TS Path

plugin.tx_solr.index.queue.[indexConfig].fields.[fieldName].field

Since

8.0

Array of classification configurations. Each configuration needs to have the property "patterns", that is a list of patters that need to match and "class", that is the mapped class that will be indexed then.

Note:

The output field needs to be a multivalue field since an indexed item can have multiple classes.

enableCommits

Type

Boolean

TS Path

plugin.tx_solr.index.enableCommits

Since

6.1

Default

true

This setting controls whether ext-solr will implicitly cause Solr commits as part of its operation.

If this settings is set to false, you need to ensure that something else will periodically call commits. The Solr daemons AutoCommit feature would be a natural choice.

This feature is mainly useful, when you have many installations in the same Solr core.

Note: Calling some APIs may still cause commits, but these can always be explicitly disabled.

tx_solr.suggest

This feature allows you to show a suggest layer that suggest terms that start with the letters that have been typed into the search field

numberOfSuggestions

Type

Integer

TS Path

plugin.tx_solr.suggest.numberOfSuggestions

Since

1.1

Default

10

Sets the number of suggestions returned and displayed in the layer attached to the search field.

suggestField

Type

String

TS Path

plugin.tx_solr.suggest.suggestField

Since

1.1

Default

spell

Sets the Solr index field used to get suggestions from. A general advice is to use a field without stemming on it. For practical reasons this is currently the spell checker field.

Note: With EXT:solr 11.1.0 ASCII folding and language depending normalization filters were introduced, but due to the special behaviour of the auto suggestions ascii-terms were not treated correctly. So with 11.1.3 the untouched tokens are also kept, as this might lead to duplicate suggestions, a new field for exact suggestions is introduced, if you want to avoid duplicates and use stricter suggestions, just configure spellExact as suggest field.

forceHttps

Type

Boolean

TS Path

plugin.tx_solr.suggest.forceHttps

Since

1.1

Default
Options

0,1

If enabled, HTTPS will be used for querying suggestions. Otherwise HTTP will be used.

treatMultipleTermsAsSingleTerm

Type

Boolean

TS Path

plugin.tx_solr.suggest.treatMultipleTermsAsSingleTerm

Since

1.4 / 1.7-dkd

Default
Options

0,1

When a user types multiple words into your search field they usually are split up into full keywords used in the query's q parameter and the last part being the partial keyword in the facet.prefix parameter. Enabling this setting moves everything into the facet.prefix parameter. This is usually only useful when using a string field as suggest / auto-complete source field.

Example - Here "Hello Solr" are the full keywords and the user started typing "World" so that "Wo" is used as the partial keyword:

"Hello Solr Wo" -> q=Hello Solr, facet.prefix=Wo (default) "Hello Solr Wo" -> q=<empty>, facet.prefix=Hello Solr Wo (treatMultipleTermsAsSingleTerm)

showTopResults

Type

Boolean

TS Path

plugin.tx_solr.suggest.showTopResults

Since

8.0

Default

1

When this setting is enabled, the top results are shown in the suggest layer. The top results are build from the first search match, or when the first search delivers no hit, the results from the first suggestion are shown.

numberOfTopResults

Type

Integer

TS Path

plugin.tx_solr.suggest.numberOfTopResults

Since

8.0

Default

5

Defines the number of top results that will be shown.

additionalTopResultsFields

Type

String

TS Path

plugin.tx_solr.suggest.additionalTopResultsFields

Since

9.0

Comma-separated list of fields that should be added to the top results response json.

tx_solr.statistics

This section allows you to configure the logging for statistics.

Note: The statistics are logged into a mysql table. This might not make sense for high frequently used searches. In this case you should think about to connect a dedicated tracking tool.

statistics

Type

Boolean

TS Path

plugin.tx_solr.statistics

Since

2.0

Default

Set plugin.tx_solr.statistics = 1 to log statistics.

statistics.anonymizeIP

Type

Integer

TS Path

plugin.tx_solr.statistics.anonymizeIP

Since

2.0

Default

Allowed values are 0 (masking disabled), 1 (mask host), 2 (mask host and subnet).

statistics.addDebugData

Type

Boolean

TS Path

plugin.tx_solr.statistics.addDebugData

Since

6.1

Default

Adds debug data to the columns time_total, time_preparation and time_processing in the table tx_solr_statistics from the result of the search query.

Note: Enabling addDebugData can have performance impact since debugMode is appended to queries.

statistics.topHits.days

Type

Integer

TS Path

plugin.tx_solr.statistics.topHits.days

Since

12.0

Default

30

Number of days to read out the search top hits.

statistics.topHits.limit

Type

Integer

TS Path

plugin.tx_solr.statistics.topHits.limit

Since

12.0

Default

5

Number of records to read out the search top hits.

statistics.noHits.days

Type

Integer

TS Path

plugin.tx_solr.statistics.noHits.days

Since

12.0

Default

30

Number of days to read out non-search hits.

statistics.noHits.limit

Type

Integer

TS Path

plugin.tx_solr.statistics.noHits.limit

Since

12.0

Default

5

Number of records to read out non-search hits.

statistics.queries.days

Type

Integer

TS Path

plugin.tx_solr.statistics.queries.days

Since

12.0

Default

30

Number of days to read out search queries.

statistics.queries.limit

Type

Integer

TS Path

plugin.tx_solr.statistics.queries.limit

Since

12.0

Default

100

Number of records to read out search queries.

tx_solr.logging

This section defines logging options. All loggings will be available in the TYPO3 logging framework.

debugOutput

Type

Boolean

TS Path

plugin.tx_solr.logging.debugOutput

Default
Options

0,1

Since

6.1

If enabled the written log entries will be printed out as debug message in the frontend or to the TYPO3 debug console in the backend. This setting replaces the previous setting plugin.tx_solr.logging.debugDevLogOutput which was needed, when the devLog was used.

exceptions

Type

Boolean

TS Path

plugin.tx_solr.logging.exceptions

Default

1

Options

0,1

Since

1.0

If enabled, thrown exceptions are logged.

indexing

Type

Boolean

TS Path

plugin.tx_solr.logging.indexing

Default

1

Options

0,1

Since

1.0

If enabled, logs when pages / documents are indexed.

indexing.indexQueueInitialization

Type

Boolean

TS Path

plugin.tx_solr.logging.indexing.indexQueueInitialization

Default

1

Options

0,1

Since

2.0

If enabled, logs the query used to initialize the indexing queue.

indexing.indexQueuePageIndexerGetData

Type

Boolean

TS Path

plugin.tx_solr.logging.indexing.indexQueuePageIndexerGetData

Default

1

Options

0,1

Since

2.0

If enabled, the requested data will be logged. Request data includes item, url, parameters, headers, data, decodedData and report.

indexing.pageIndexed

Type

Boolean

TS Path

plugin.tx_solr.logging.indexing.pageIndexed

Default
Options

0,1

Since

2.0

If enabled, it will be logged if a page was indexed successful or not. Additionally, an error is logged if indexing is disabled (via config.index_enable)

query.filters

Type

Boolean

TS Path

plugin.tx_solr.logging.query.filters

Default

1

Options

0,1

Since

1.0

If enabled, filters will be logged when they get added to the Solr query.

query.searchWords

Type

Boolean

TS Path

plugin.tx_solr.logging.query.searchWords

Default

1

Options

0,1

Since

1.0

If enabled, received search queries will be logged.

query.queryString

Type

Boolean

TS Path

plugin.tx_solr.logging.query.queryString

Default

1

Options

0,1

Since

1.0

If enabled, query string parameters and the respective response will be logged.

query.rawPost

Type

Boolean

TS Path

plugin.tx_solr.logging.query.rawPost

Default

1

Options

0,1

Since

1.0

If enabled, POST requests against the Solr server will be logged.

query.rawGet

Type

Boolean

TS Path

plugin.tx_solr.logging.query.rawGet

Default

1

Options

0,1

Since

1.0

If enabled, GET requests against the Solr server will be logged.

Extension Configuration

The following settings can be defined in the "Settings":>"Extension Configuration":>"solr"

Basic

pluginNamespaces

Type

String

Since

11.1

Default

tx_solr

A list of white listed plugin namespaces (Required by cacheHash/excludedParameters and plugin flex form).

includeGlobalQParameterInCacheHash

Type

Boolean

Since

11.1

Default

Include/Exclude global q parameter in/from cacheHash.

useConfigurationFromClosestTemplate

Type

Boolean

Since

6.1

Default

When this setting is active the closest page with a TypoScript template will be used to fetch the configuration. This improves the performance but limits also the possibilities. E.g. conditions can not be used that are related to a certain page.

Monitoring

useConfigurationTrackRecordsOutsideSiteroot

Type

Boolean

Since

6.1

Default

1

A common scenario is to have a site and a storage folder for records parallel to it on the same level (f.e.) If you don't want this behaviour - it should be set to false.

useConfigurationMonitorTables

Type

String

Since

6.1

Default
 

Monitor tables - explicitly monitor these (still requires TypoScript configuration)

monitoringType

Type

Int

Since

11.2

Default

Defines how data updates should be monitored

By default (=0) recognized updates will be processed directly and the Solr index queue will be directly updated, also the Solr index where appropriate. As in huge instances the monitoring can slow down the TYPO3 backend, two more monitoring options are available:

  • 1: Delayed: Record update events will be queue and processed later, the scheduler task "Event Queue Worker" is required for processing.
  • 2: No monitoring: Monitoring is completely disabled, please note that you have to take care of Solr index updates yourself.

Route Enhancer

enableRouteEnhancer

Type

Boolean

Since

12.0

Default

To use the EXT:solr possibility to create speaking URLs for Solr facets, activate this option.

As this feature requires additional configuration and costly processing, it's disabled by default.

Advanced

allowSelfSignedCertificates

Type

Boolean

Since

6.1

Default

Can be used to allow self signed certificates - when using the SSL protocol.

Database

Database indexes

Some of the SQL statements performed on the pages table in TYPO3 perform extensive operations while copying page-trees. These operations can be speed up by by adding 2 indexes to the standard table pages.

The indexes are: content_from_pid_deleted (content_from_pid, deleted), doktype_no_search_deleted (doktype, no_search, deleted)

It is not required that these indexes are created by in above scenarios considerable performance gains can be achieved.

Development

There are many ways to extend and hook into EXT:solr to customize EXT:solr for your needs.

Indexing

This section describes the possibilities to extend page indexing in EXT:solr with custom code via PSR-14 events.

Page Indexing

There are several points to extend the Page Indexer class and register own classes that are used during the indexing.

BeforePageDocumentIsProcessedForIndexingEvent

Registered Event Listeners can be used to add additional documents to Solr when a page gets indexed.

Registration of an event listener in your extension's Services.yaml:

MyVendor\MyPackage\EventListeners\MyEventListener:
  tags:
    - name: event.listener
      identifier: 'my-package/modify-documents'
Copied!

The corresponding event listener class:

use ApacheSolrForTypo3\Solr\Event\Indexing\BeforePageDocumentIsProcessedForIndexingEvent;
use ApacheSolrForTypo3\Solr\System\Solr\Document\Document;

class MyEventListener {

    public function __invoke(BeforePageDocumentIsProcessedForIndexingEvent $event): void
    {
        $additionalDocument = new Document();
        $event->addDocuments([$additionalDocument]);
    }
}
Copied!

For other records than pages, the PSR-14 Event \ApacheSolrForTypo3\Solr\Event\Indexing\BeforeDocumentIsProcessedForIndexingEvent can be used.

AfterPageDocumentIsCreatedForIndexingEvent

Registered event listeners can be used to replace/substitute a Solr document of a page.

Registration of an event listener in your extension's Services.yaml:

MyVendor\MyPackage\EventListeners\MyEventListener:
  tags:
    - name: event.listener
      identifier: 'my-package/modify-page'
Copied!

The corresponding event listener class:

use ApacheSolrForTypo3\Solr\Event\Indexing\AfterPageDocumentIsCreatedForIndexingEvent;

class MyEventListener {

    public function __invoke(AfterPageDocumentIsCreatedForIndexingEvent $event): void
    {
        $event->setDocument($myCustomDocument);
    }
}
Copied!

BeforeDocumentsAreIndexedEvent

Registered Event Listeners can be used to process Solr documents (pages and records) before they are added to index.

Registration of an event listener in your extension's Services.yaml:

MyVendor\MyPackage\EventListeners\MyEventListener:
  tags:
    - name: event.listener
      identifier: 'my-package/modify-documents'
Copied!

The corresponding event listener class:

use ApacheSolrForTypo3\Solr\Event\Indexing\BeforeDocumentsAreIndexedEvent;

class MyEventListener {

    public function __invoke(BeforeDocumentsAreIndexedEvent $event): void
    {
        foreach ($event->getDocuments() as $document) {
           $document->addField('my_custom_field', 'my_custom_value');
        }
    }
}
Copied!

Independent indexer

If external data should be indexed or the RecordIndexer is not required, it is possible to fill the index with an extension as well. The class can then be called e.g. by a CLI call.

<?php

namespace Vendor\ExtensionName\Import;

use ApacheSolrForTypo3\Solr\ConnectionManager;
use ApacheSolrForTypo3\Solr\Domain\Site\SiteRepository;
use ApacheSolrForTypo3\Solr\System\Solr\Document\Document;
use ApacheSolrForTypo3\Solr\System\Solr\SolrConnection;
use TYPO3\CMS\Core\Utility\GeneralUtility;

class Indexer
{

    protected ConnectionManager $connectionManager;

    public function __construct()
    {
        $this->connectionManager = GeneralUtility::makeInstance(ConnectionManager::class);
    }

    /**
     * Send data to Solr index
     *
     * @param array $rows Data to be indexed, e.g. multiple DB rows
     * @param int $pageId root page
     * @param int $language language id
     */
    public function index(array $rows, int $pageId = 1, int $language = 0)
    {
        $documents = [];

        foreach ($rows as $row) {
            $documents[] = $this->createDocument($row, $pageId);
        }

        $connection = $this->connectionManager->getConnectionByPageId($pageId, $language);
        $connection->getWriteService()->addDocuments($documents);
    }


    /**
     * Remove all from index
     *
     * @throws \ApacheSolrForTypo3\Solr\NoSolrConnectionFoundException
     */
    public function clearIndex() {
        $connections = $this->connectionManager->getAllConnections();
        foreach ($connections as $connectionLanguage => $connection) {
            /** @var SolrConnection */
            $connection->getWriteService()->deleteByType('custom_type');
        }
    }

    /**
     * Create a Solr document which then is sent to Solr
     *
     * @param array $row
     * @param int $pageId
     * @return Document
     */
    protected function createDocument(array $row, int $pageId): Document
    {
        $document = $this->getBaseDocument($row, $pageId);

        $solrFieldMapping = [
            'title' => 'title',
            'summary' => 'abstract',
            'information' => 'content',
            'keywords' => 'keywords',
            'area' => 'area_stringS',
            'category' => 'category_stringS'
        ];

        foreach ($row as $key => $value) {
            if (isset($solrFieldMapping[$key])) {
                $document->setField($solrFieldMapping[$key], $value);
            }
        }

        // url generation
        $document->setField('url', 'todo'); // custom implementation required

        return $document;
    }

    /**
     * Creates a Solr document with the basic / core fields set already.
     *
     * @param array $itemRecord The record to use to build the base document
     * @param int $rootPageId root page id
     * @return Document A basic Solr document
     */
    protected function getBaseDocument(array $itemRecord, int $rootPageId): Document
    {
        $siteRepository = GeneralUtility::makeInstance(SiteRepository::class);
        $site = $siteRepository->getSiteByRootPageId($rootPageId);

        /** @var Document $document */
        $document = GeneralUtility::makeInstance(Document::class);

        // required fields
        $document->setField('id', 'custom_type_' . $itemRecord['uid']);
        $document->setField('variantId', 'custom_type' . $itemRecord['uid']);
        $document->setField('type', 'custom_type');
        $document->setField('appKey', 'EXT:solr');
        $document->setField('access', ['r:0']);

        // site, siteHash
        $document->setField('site', $site->getDomain());
        $document->setField('siteHash', $site->getSiteHash());

        // uid, pid
        $document->setField('uid', $itemRecord['uid']);
        $document->setField('pid', 1);

        return $document;
    }
}
Copied!

Developing Backend Modules

Backend Components

EXT:solr provides UI components for backend modules. Some components hold (GUI)state and some not, but all components calling actions(changing the extension and/or GUI state!), and then redirecting to the actions within the component was used(referrer also) or to the defined by callers action uri, if that is required by UX.

Below are all available components listed and their responsibility.

CoreSelector

Renders menu in backends doc header with available Solr cores for selected Site and changes the Solr core by clicking on option in drop down menu.

  • Provides following methods, which must be called inside the initializeView(...) method in your controller to render this component in Backend:

    • generateCoreSelectorMenuUsingSiteSelector()

      • Use this method together with SiteSelectorMenu component.
    • generateCoreSelectorMenuUsingPageTree()

      • Use this method if you are using original page tree from CMS.
  • Provides following Actions for changing state, must be added to actions list of your controller:

    • switchCore
  • Provides following fully initialized properties in utilizing action controller:

    • $selectedSolrCoreConnection from type \ApacheSolrForTypo3\Solr\SolrService

If you need the possibility to switch the core, you can extends the AbstractModuleController (in ApacheSolrForTypo3SolrControllerBackendSearch).

FAQ

Why should I add some action name to my action controller?

To allow calling this action within your controller, component can use own controller for changing state only if that is hardcoded with some module and allowed by ACL for all(or almost all) be users/groups, but this is a bag approach. Therefore allow changing something, only if that is needed.

What do I need to do for using Backend Components?

By extending ApacheSolrForTypo3SolrControllerBackendSearchAbstractModuleController your module has the pagetree (to select the side) and the core selector, to select the needed Solr core.

Best Practice

Optimize Page Indexing

Indexing of pages is usually done via this extensions PageIndexer which determines a pages content by crawling its URL and extracting everything between the two magic markers <!--TYPO3SEARCH_begin--> and <!--TYPO3SEARCH_end-->. When fetching the URL the header X-Tx-Solr-Iq is added to the request, which is (among other things) disabling all caches.

To reduce crawling runtimes you might wanna disable the rather expensive generation of navigation structures such as main and footer menus that you've defined for your page object in TypoScript.

You can do so by creating a TypoScript condition that unsets all sorts of non-content elements. Like this for example:

[request && traverse(request.getHeaders(), 'x-tx-solr-iq/0')]
page.10.dataProcessing >
page.10.variables >
page.10.variables {
    content < styles.content.get
    content.select.where = {#colPos}=0
    contentMarginal < styles.content.get
    contentMarginal.select.where = {#colPos}=1
}
[global]
Copied!

You should notice a significant difference right away:

bin/typo3 scheduler:run --task=<tast-id-for-site> --force
Copied!

or if EXT:solrconsole is installed on your system:

bin/typo3 solr:queue:index -n -a 1 -s <site>
Copied!

If not: go to the TYPO3 backend and check the page object definition in the TypoScript object browser with that condition being activated to see what it looks like now.

Development Environment

To simplify the development for TYPO3 and Solr related components we provide a development environment based on Docker and the DDEV of the TYPO3 core.

You can find the development box in the following git repository:

https://github.com/TYPO3-Solr/solr-ddev-site

When you start the box, you will find a pre-configured environment for the support TYPO3 version and the Solr installation that is needed for the installed EXT:solr version.

Testing and Continues Integration

The goal during the development of EXT:solr is, to test most of the components with unit and integration tests.

Unit Tests

For the single classes we've added unit tests whenever we though it is usefull. The unit tests should run very quickly, to git a quick response.

Integration Tests

As in EXT:solr the integration tests are more complex and test the integration of the different components. Since a database server and a Solr server is required this is needed to run the integration test suite. During the bootstrap of the test environment, we use the TYPO3 core functionality for database tests and we install a local Solr server with out install script.

To simplify the local usage of the unit and integration tests, we ship a few bash script that support you to get everything started.

Bootstrapping the Test Environment

When you want to start the testrunner in your shell you need to bootstrap it once:

source ./Build/Test/bootstrap.sh --local
Copied!

The bootstrapper will prompt for some values:

When the boostrapper was finished successful the following was done:
  • Environment variables for the TYPO3 testing framework have been exported
  • Test database was created
  • Test Solr instance was created

Afterwards you can run the ci suite in your shell

Running the ci Suite

When the test environment was boostrapped correctly you can start the test runner:

./Build/Test/cibuild.sh
Copied!

When everything is configured correctly all tests should run through and you should get a green bar:

Code Structure

The components of EXT:solrfluid have been developed with the domain driven design (DDD) approach (https://de.wikipedia.org/wiki/Domain-driven_Design) for our extension we tried to separate the code by the following layers:

  • Domain: Everything that is related to the "search" domain should be implemented here.
  • System: Everything that is related to the "system" (e.g. TYPO3 specific) should be implemented here.

Domain Layer & Domain Model

The classes of the domain layer are located in "Classes/Domain" and should contain everything that is related to the "search domain".

ResultSet

The "SearchResultSet" is the main entity that you get passed to the view. It can be used to access all search related objects on your result page.

UML diagram of the SearchResutSet

The SearchResultSet can be used e.g. to get facets and spelling suggestions. A focus for the first release was a new domain model for facets, that can be rendered with fluid or any other template engine.

Facets

The following UML diagram shows the implemented facets in EXT:solrfluid. Every facet has one or more facet items attached. For the OptionsFacet the FacetItem is an Option, for the NumericRangeFacet a NumericRange.

Rendering of a facet:

Based on the "type" TypoScript configuration the "FacetRegistry" chooses the responsible facet package class that is used to create the object structure from the Solr repsonse. Each facet type is shipped with a default fluid partial, that is able to render such a facet.

A facet package consists of the following parts:

  • The facet parser: Responsible to parse the Apache Solr response into a domain object structure
  • The url decoder: Responsible to decode the url fragment of the facet in EXT:solr
  • The query builder: Responsible to build the needed query fragment for Apache Solr to apply the facet

The TypoScript configuration "partialName" can be used to force the rendering with another fluid partial.

For advanced use cases you can use the "FacetRegistry" to register your own facet type or overwrite the facet behaviour for a certain facet type.

UML diagram of the *Facet classes

As you see in the diagram above Solr ships a clean object structure of the facets, that you can render in your custom templates as you need them.

Events

In version 11 we started with the implementation of events, using the EventDispatcher (PSR-14 Events).

Step by step the events will replace older hooks and signals. In the following you will find a description of the available events

Monitoring

Observing and processing of data updates is controlled by the following events:

  • ContentElementDeletedEvent, fired if a content element is deleted
  • PageMovedEvent, fired if a page is moved
  • RecordDeletedEvent, fired if a record is deleted
  • RecordGarbageCheckEvent, fired if a record garbage check is triggered
  • RecordMovedEvent, fired if a record is moved
  • VersionSwappedEvent, fired if a version is swapped

All events implement the DataUpdateEventInterface and at least provide information about the elements uid, table and modified fields.

Data update processing

ProcessingFinishedEvent

The ProcessingFinishedEvent indicates that the processing of a data update event is finished, if you want to implement an own monitoring you can use the ProcessingFinishedEvent to start the processing. The event provides the originally fired data update events listed above, so that you have all the required information about the update to process.

If you're using the event you're indepent of the monitoring setting, as this event if fired in the immediate and delayed monitoring mode as soon as an event is processed.

DelayedProcessingQueuingFinishedEvent / DelayedProcessingFinishedEvent

If you're using the delayed processing (see "monitoringType"), you can use one of the following events:

DelayedProcessingQueuingFinishedEvent This event is fired as soon as the update event is queued in the event queue (tx_solr_eventqueue_item), you can e.g. use this event to register own events or to implement processing that has to executed immediately even in delayed monitoring mode.

DelayedProcessingFinishedEvent The Scheduler task "Event Queue Worker" is required to process the data updates in delayed monitoring mode and will fire the DelayedProcessingFinishedEvent as soon as an event has been processed. If you require to perform actions only during delayed processing, the event can be used.

SearchUriBuilder

The SearchUriBuilder is responsible to build uris, that are used in the searchContext. Since the route enhancer is introduced you can use the following event to influence the build uris:

  • BeforeVariableInCachedUrlAreReplacedEvent
  • BeforeCachedVariablesAreProcessedEvent
  • AfterUriIsProcessedEvent

Facets

Currently EXT:solr provides following events for Facets-Component modification:

  • AfterFacetIsParsedEvent

ViewHelpers

Beside the controllers, the domain objects and the templates we ship a few useful view helpers. To avoid a strong coupling between the extension and fluid as template engine we tried to keep all ViewHelpers as "slim" as possible. Whenever it was possible we moved the logic into custom service classes and just use them in the ViewHelper.

Since everything belongs to the "SearchResultSet" and we wanted to avoid the need ob passing this object around from "template to template" and "partial to partial" we decided to provide an own "ControllerContext" that referenced the "SearchResultSet". With this approach, it is possible to access the "SearchResultSet" in every ViewHelper.

With the current release we ship the following concrete ViewHelpers:

|

Path Description
s:debug.documentScoreAnalyzer Used to render the score analysis.
s:debug.query Shows the Solr query debug information.
s:document.highlightResult Performs the highlighting on a document.
s:document.relevance Shows the relevance information for a document.
s:facet.area.group Filters the facets in the rendering scope to one group.
s:facet.options.group.prefix.labelPrefixes Provides an array of available label prefixes that can be used to filter with s:facet.options.group.prefix.labelFilter.
s:facet.options.group.prefix.labelFilter Filters the options of a facet by a given prefix.
s:uri.facet.addFacetItem Add's a facet item to the current url.
s:uri.facet.removeAllFacets Removes all facet items from the current url.
s:uri.facet.removeFacet Removes all options from one facet.
s:uri.facet.removeFacetItem Removes a single facet item from the url.
s:uri.facet.setFacetItem Sets one single item for a facet (and removes other setted)
s:uri.paginate.resultPage Creates a link to a result page of the current search.
s:uri.search.currentSearch Creates a link to the current search (with facets, sorting...)
s:uri.search.startNewSearch Creates a link for a new search by a term.
s:uri.sorting.removeSorting Creates a link to the current search and removes the sorting.
s:uri.sorting.setSorting Creates a link to the current search and sets a new sorting.
s:pageBrowserRange Provides the range data for the pagination.
s:searchForm Renders the searchForm.
s:translate Custom translate ViewHelper (uses translations from ext:solr)

FAQ - Frequently Asked Questions

What does the term "Core" mean?

This term relates to Apache Solr indexes and means a single distinct part of an index. It is possible to use multiple cores on one single Apache Solr instance. Good examples are using a different Apache Solr core for each language or of course a separate core for each website. For more information please refer to the Apache Solr documentation for "Core" explanation.

|

Where can I report a bug?

Please make sure that this bug is not reported already, use also the search function of our issue tracker. Our issue tracker is on GitHub.

|

Where can I report a security issue?

If you have found a security issue in our extension, please do not post about it in a public channel. Please send an email to the TYPO3 security team with detailed description of found vulnerability. For more details about security issue handling see https://typo3.org/teams/security/contact-us/

|

Is there some chat/irc channel for EXT:solr available?

Join us on the official Slack for TYPO3 and get answers related to EXT:solr in the #ext-solr channel immediately!

|

Which plugins(TYPO3 Frontend) are available?

  • Search: Form only
  • Search: Form, Result, Additional Components
  • Search: Frequent Searches

Just insert one of this plugins on corresponding page to fade in the search form and/or supply the front end with a search results.

|

When i open the search page i see the message 'Search is currently not available. ', whats wrong?

Did you configure your Solr connection as required?

  • Please read "Configure Extension" and check if you have configured everything
  • Did you configure Solr server and port and does the scheme and path match?
  • Did you click "Initialize connection" after configuring the Solr server?
  • Can you access the Solr server with wget or curl from the command line?
  • Is the system report of EXT:solr green?

|

In which cases do I want to trigger indexing manually?

  • after changing any configuration file.
  • after modifying synonyms, stop words, protected words in TYPO3 Backend -> Search

Moreover by changing core/index configuration you need to reload the core to make the changes become active. To reload configuration you can either restart the whole Solr server or simply reload a specific core.

|

I want to index files with EXT:solr. How can i do that?

We provide an addon called EXT:solrfal, that allows you to index files from FAL into Solr. This addon is currently available for partner only.

|

The indexer does not generate URLs to restricted pages. How can I force the generation of URLs?

If you have a detail page for e.g. news records that is restricted, please use the typolink attribute linkAccessRestrictedPages :

|

default = TEXT
default {
  // ... other typolink settings
  typolink.linkAccessRestrictedPages = 1
}
Copied!

|

How can i use Fluid templates with EXT:solr < v7.0.0?

For the Fluid rendering in EXT:Solr >= 5.0 <= 6.1 we provide the addon EXT:solrfluid, that allows you to render your search results with Fluid. Since EXT:Solr 7.0 Fluid is the default templating engine.

|

Which versions of EXT:solr / EXT:solrfal and EXT:solrfluid work together?

Please check the Appendix - Version Matrix, the you can find the proposed version combinations.

|

Pages are not indexed. I did everything by the book.

You forgot to set config.index_enable = 1 in your TypoScript setup: Enable indexing

|

My indexed documents are empty, i can not find the content of a page?

Did you configure the search markers ( "<!-- TYPO3SEARCH_begin -->" and "<!-- TYPO3SEARCH_end -->") on your page? Check the paragraph Search Markers and make sure your page renders them.

|

When i change a record, no update is detected. What's wrong?

Are your records inside of your site root? EXT:solr record monitor processes records that belong to your site, which means they need to be below your site root. If you want to index records that are outside your sideroot, you need to configure the page id's of the sysfolder as additionalPageIds:

|

plugin.tx_solr.index.queue.[yourQueueName].additionalPageIds = 4711,4712
Copied!

|

There are two data-types for text stringS and textS. When should i choose which datatype?

String data types like stringS store the raw string. No processing, like stemming, splitting etc. is applied. The processing is useful when you want to search in the field and support more then exact matches. When you just want to display the content you should choose a stringS type, when you want to search in the field you should choose textS.

|

I am adding content to a dynamic field but when i search for the content i can not find the document. What's wrong?

Beside the indexing part you need to configure the query part. Make sure that all relevant fields are configured as query fields:

|

plugin.tx_solr.search.query.queryFields := addToList(test_textS^1.0)
Copied!

I don't find the expected document on the first position. What can i do?

:) That's a good question. In the end, Solr is a search and the sorting depends on the score, not as in a database on one or two simple criterion.

In the end Solr provides a lot of settings that influence the score calculation and you need to tune the results to you needs. The following settings are helpful to tune your results.

Check your data

The quality of you data is important. Maybe a document is on the first position because, the search term is really relevant for it? Maybe it is an option to change the content?

Adjust the query field boost factors

For each query field there is a boost value after the ^ sign. To increase the factor of a single field for the whole query, you can increase the number in the query fields.

Example:

|

plugin.tx_solr.search.query.queryFields = title^20.0, title^15.0
Copied!

Use boostFunctions or boostQueries

For use cases like "news are always more important then pages" or "Newer documents should be at the beginning" you can use boostFunctions (query.boostFunction) or boostQueries (query.boostQuery)

The search term only exists as a synonym

You can use the backend module synonyms (Synonyms) to maintain synonyms and configure Solr to retrieve documents by a term that is not naturally inside the document.

Ask DKD support

Beside that, there are more options to tune. The DKD support can help you, to analyze and tune your search results. Call +49 (0)69 - 247 52 18-0.

Non ASCII characters like german umlauts do not work when i search, how do I fix that?

To allow search with umlauts Tomcat needs to be configured to use UTF-8 encoded urls. Go to apache-tomcat/conf/server.xml and change the URIEncoding parameter:

|

<Connector port="8080" protocol="HTTP/1.1"
    connectionTimeout="20000" redirectPort="8443"
    URIEncoding="UTF-8" />
Copied!

How can I change Solr's schema and add custom fields?

Please do not change the shipped Solr schema. There are a lot of dynamic fields (Appendix - Dynamic Fields) that can be used to index any kind of datatype.

I am using varnish before my site. How can i index pages properly?

SOLR Indexer might have some issues, when the page to index is behind a Varnish Proxy. We have collected two ways of solving this issue

Bypassing varnish:

Bypass when X-Tx-Solr-Iq is present

The Solr indexer request send the header X-Tx-Solr-Iq.

To have bypass the Varnish caching, put this into your sub vcl_recv part of the configuration

|

if (req.http.X-Tx-Solr-Iq) {
    return(pipe);
}

Copied!

Using Cache-Control:

Put this into your sub vcl_fetch part of the configuration

|

if (req.http.Cache-Control ~ "no-cache") {
    set beresp.ttl = 0s;
    # Make sure ESI includes are processed!
    esi;
    set beresp.http.X-Cacheable = "NO:force-reload";
    # Make sure that We remove all cache headers, so the Browser does not cache it for us!
    remove beresp.http.Cache-Control;
    remove beresp.http.Expires;
    remove beresp.http.Last-Modified;
    remove beresp.http.ETag;
    remove beresp.http.Pragma;

      return (deliver);
}

Copied!

I want to build the Dockerfile_full image on my mac with a local volume, how can i do that?

The following example shows how to build the Dockerfile image and start a container with a mapped local volume (only for the data). This was tested with "Docker for Mac" (not Docker Toolbox). Before executing the example, make sure, that you have added " /solrdata" as allowed volume in the Docker configuration.

# build the image
docker build -t typo3-solr -f Docker/SolrServer/Dockerfile .

# create volume directory locally
mkdir -p ~/solrdata

# add Solr group to volume directory
sudo chown :8983 ~/solrdata

# run Docker container from image with volume
docker run -d -p 127.0.0.1:8282:8983 -v ~/solrdata:/var/solr/data/data typo3-solr

Copied!

Can i index a https (SSL) site?

Yes. You need a ssl certificate (can be self signed) and change the following setting:

plugin.tx_solr.index.queue.pages.indexer.frontendDataHelper.scheme = https
Copied!

|

I want to index a value into a multiValue field from a user function. How can i do that?

You can do that, by using SOLR_MULTIVALUE

plugin.tx_solr.index.queue.indexConfigName {
    fields {
      somevalue_stringM = SOLR_MULTIVALUE
      somevalue_stringM {
           stdWrap.cObject = USER
           stdWrap.cObject.userFunc = Vendor\Ext\Classname->getValues
           separator=,
      }
    }
}
Copied!

|

How can i use a configuration from AdditionalConfiguration.php when i deploy my application on several instances?

The configuration of the connection is done with TypoScript. When you want to use a configuration from TYPO3_CONF_VARS or from the system environment, you can apply an stdWrap on the configuration that reads from these configurations.

The following example shows how a host can be configured in the AdditionalConfiguration.php and used in your TypoScript to connect to solr:

The following line is added to AdditionalConfiguration.php

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['host'] = 'mysolrserver.de';
Copied!

|

To use this configuration for the host, you can use a TEXT element in the configuration and use override.data to use the value from the AdditionalConfiguration.php

plugin.tx_solr.solr {
   host = TEXT
   host {
     value = localhost
     override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|host
   }
}
Copied!

|

I want to index extension records, what do i need to do?

EXT:solr provides a flexible indexing for TYPO3 pages and records. You can add a custom indexing configuration for your own records with a valid TCA configuration.

You can read more about this in the section IndexQueue Configuration.

The following things are important:

  • The extension ships several examples in the Folder "Configuration/TypoScript/Examples", read them and try to undestand them.
  • EXT:solr can not know the business logic of an extension to generate a link to a detail view. You need to use typolink to build an url that points to a valid, existing detail page.
  • When you index records, e.g. news it these records are indexed in Solr and point to a news details page. That's the reason why it makes sence to exclude the news detail page from the normal page indexing. Otherwise the indexing of this page will produce an error message, because only a url with a valid news uid produces a valid output.

|

Are in EXT:solr some cli commands available?

Yes, currently(v. 6.1) only one for initializing Solr connections. But check for new ones with bin/typo3 list command.

|

I want to overwrite the type field, why is this not possible?

The type field is a system field that EXT:solr uses to keep the system in sync. Overwritting this field might result in inconsistency. However, if you need something like a custom type you can also write the information to a dynamic Solr field and use that one as a type.

The following example shows, how to fill the field "mytype_stringS" and build a facet on this field:

plugin.tx_solr {
    index{
        queue{
            news = 1
            news {
                type = tt_news

                fields {
                    mytype_stringS = TEXT
                    mytype_stringS.value = news

                }
            }
        }
    }
    search.faceting.facets.mytype_stringS {
         label = Type
         field = mytype_stringS
    }
}
Copied!

|

I want to implement a toggle functionality for facet options as previously possible with selectingSelectedFacetOptionRemovesFilter. How can i do that?

This is completely possible with Fluid core ViewHelpers and the domain model. The following steps are required.

Register a custom partial to render the facet:

plugin.tx_solr.search.faceting.facets.<facetName>.partialName = OptionsToggle
Copied!

This is the content of the OptionsToggle Partial (Feel free to adapt it to your needs):

<h5 class="facet-label">{facet.label}</h5>
<ul class="facet-option-list facet-type-options fluidfacet" data-facet-name="{facet.name}" data-facet-label="{facet.label}">
    <f:for each="{facet.options}" as="option" iteration="iteration">
        <li class="facet-option{f:if(condition:'{iteration.index} > 9', then:' tx-solr-facet-hidden')}" data-facet-item-value="{option.value}">
            <f:if condition="{option.selected}">
                <f:then><a class="facet solr-ajaxified" href="{s:uri.facet.removeFacetItem(facet: facet, facetItem: option)}">{option.label}</a></f:then>
                <f:else><a class="facet solr-ajaxified" href="{s:uri.facet.addFacetItem(facet: facet, facetItem: option)}">{option.label}</a></f:else>
            </f:if>
            <span class="facet-result-count">({option.documentCount})</span>
        </li>
    </f:for>
    <f:if condition="{facet.options -> f:count()} > 10">
        <li>
            <a href="#" class="tx-solr-facet-show-all" data-label-more="{s:translate(key:'faceting_showMore', extensionName:'solr')}"
                data-label-less="{s:translate(key:'faceting_showFewer', extensionName:'solr')}">
                <s:translate key="faceting_showMore" extensionName="solr">Show more</s:translate>
            </a>
        </li>
    </f:if>
</ul>
Copied!

I want to store HTML in solr, how can i retrieve that?

In general it is not recommend to allow html in the Solr field. Especially when you index content that can be changed by the user.

However, if you want to allow html in a Solr field, you need to add the field as trusted field and the content will not be escaped during the retrieval from solr.

The following example shows how to avoid html in the content field:

plugin.tx_solr.search.trustedFields = url, content

Copied!

Note: When you allow html in the content please make sure that the usage of crop ViewHelpers or a limit of the field length does not break your markup.

I want to use two instances of the search plugin on the same page, how can i do that?

If you want to use two search plugins on the same page you can add two instances and assign a different "Plugin Namespace" in the flexform. If you want to avoid, that both plugins react on the global "q" parameter, you can disable this also in the flexform. Each instance is using the querystring from <pluginNamespace>[q] then.

How can i configure switchable templates for the results plugin?

The following example shows, how you can configure a custom switchable entry template for the Results plugin:

plugin.tx_solr {
    view {
        templateRootPaths.100 = EXT:your_config_extension/Resources/Private/Templates/
        partialRootPaths.100 = EXT:your_config_extension/Resources/Private/Partials/
        layoutRootPaths.100 = EXT:your_config_extension/Resources/Private/Layouts/
        templateFiles {
            results = Results
            results.availableTemplates {
                default {
                    label = Default Searchresults Template
                    file = Results
                }
                products {
                    label = Products Template
                    file = ProductResults
                }
            }
        }
    }
}

Copied!

I want to use EXT:solr with a deployment and pass connection settings from outside e.g. by the environment, how can i do that?

When you deploy a system automatically and you use EXT:solr there are some things that might be complicated:

  • You want to use a different Solr endpoint for each environment

To avoid that, you can set or generate these settings in the TYPO3 AdditionalConfigruation.php file and use them in your system.

To configure a used domain you cat set:

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['sites'][###rootPageId###]['domains'] = ['mydomain.com'];
Copied!

You can also define the data for your Solr endpoints there and use them in the TypoScript:

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['sites'][###rootPageId###]['solrhost'] = 'solr1.local';
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['sites'][###rootPageId###]['solrport'] = 8083;
Copied!

And use them in your TypoScript configuration:

plugin.tx_solr {
    solr {
        host = TEXT
        host {
            value = {$plugin.tx_solr.solr.host}
            override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|sites|###rootPageId###|solrhost
        }
        port = TEXT
        port {
            value = {$plugin.tx_solr.solr.port}
            override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|sites|###rootPageId###|solrport
        }
    }
}

Copied!

I want to use faceting.facets.[facetName].singleOptionMode why was it removed?

This setting belongs to the rendering and not to the facet itself. You can implement the same behaviour just with the given ViewHelpers.

The behaviour is the same, when you just call the ViewHelper s:uri.facet.setFacetItem instead of s:uri.facet.addFacetItem, which semantically just overwrites the current value.

We've added an example partial "OptionsSinglemode" that shows this behaviour. The example TypoScript template "Search - (Example) Options with singlemode (only one option at a time)" shows how to use this partial in combination with the setting "keepAllOptionsOnSelection".

I want to build a tab facet where all options remain, even with an option count of 0. How can i do that?

This can be done with the combination of several settings:

plugin.tx_solr.search.faceting {
    minimumCount = 0
    keepAllFacetsOnSelection = 1
    facets {
        typeTab {
            field = type
            keepAllOptionsOnSelection = 1
        }
    }
}
Copied!

The example above changes the minimumCount to 0, the default value i 1. Setting it to zero allows to have options without any results. The setting "keepAllFacetsOnSelection" let all facets remain and with keepAllOptionsOnSelection the options in the type facet remain.

How can i add a search-box on every page?

In most projects you want to add a searchbox on every content page. To support this, the default EXT:solr TypoScript template provides the TypoScript template path "plugin.tx_solr_PiSearch_Search" that contains a configured TypoScript code to render the searchbox. When you want to add that to your project in the most cases you would need to refer to a search result page. The following example shows how you can build a TypoScript lib object that configures the target page for this plugin instance:

lib.searchbox < plugin.tx_solr_PiSearch_Search
lib.searchbox.search.targetPage = 4711
Copied!

Afterwards you could render the TypoScript path "lib.searchbox" with several ways in TYPO3, e.g. with a FLUID ViewHelper:

<f:cObject typoscriptObjectPath="lib.searchbox" />
Copied!

By adding the snippet to a generic tempate you could render the searchbox on every page.

How can I index protected pages (htaccess protection)?

Protected pages can be accessed by passing the htpasswd username and password to the indexing queue. You can set the credentials by the following configuration:

plugin.tx_solr.index.queue.pages.indexer.authorization.username = your_username
plugin.tx_solr.index.queue.pages.indexer.authorization.password = your_password

Copied!

As credentials are stored as plain text, go for sure that your web server does not serve your TypoScript files publicly (protect the directory or by file endings). If you don't want to store plain text passwords, you can configure your web server to allow access from a specific domain (see below).

If you have multiple domains to index, the webserver requires the credentials for each domain accessed by the Solr indexer. The extension passes the credentials only once, so you will run into errors on a multi domain environment. Solution: Instead of passing the credentials as shown above, configure your webserver directory protection to allow access from the Solr IP:

AuthType Basic
AuthUserFile /path/to/.htpasswd
<RequireAny>
        Require ip XXX.XX.XX.XX (the IP of the Solr server)
        Require valid-user
</RequireAny>
Copied!

Be aware, that this will allow all accesses by given IP.

How can I use different host / port configurations in Solr v10 (e.g. for local environments)?

While you could use TypoScript conditions to change the configuration for different project evironments in the past, you can now use environment variables in the config.yaml like shown below.

In your sites config.yaml:

solr_host_read: '%env(SOLR_HOST)%'
solr_port_read: '%env(SOLR_PORT)%'

In your .env file:

SOLR_HOST=127.0.0.1
SOLR_PORT=8983
Copied!

Refer to TYPO3 documentation: https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/SiteHandling/UsingEnvVars.html#using-environment-variables-in-site-configuration

How can i register a custom statistic writer processor?

If the TypoScript option plugin.tx_solr.statistics = 1 is activated, a StatisticsWriterProcessor is used by EXT:solr, which writes the queries into the database.

If there are more requirements for the statistics or if you want to write the data into another system, you can register your own StatisticWriterProcessor in the ext_localconf.php of the extension:

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch']['statistics'] = MyVendor/Namespace/Statistics/CustomStatisticsWriterProcessor::class;
Copied!

The PHP class must have a certain structure, which is specified by the interface.

namespace MyVendor\Namespace;

use ApacheSolrForTypo3\Solr\Domain\Search\ResultSet\SearchResultSet;
use ApacheSolrForTypo3\Solr\Domain\Search\ResultSet\SearchResultSetProcessor;

class CustomStatisticsWriterProcessor implements SearchResultSetProcessor
{
    /**
     * @param SearchResultSet $resultSet
     * @return SearchResultSet
     */
    public function process(SearchResultSet $resultSet) {

        // your logic here

        return $resultSet;
    }
}
Copied!

I want to use the page content for a dynamic field , how can i do that?

You can use a virtual field called __solr_content, which holds the content of the current page.

Example:

plugin.tx_solr.index.queue.pages.fields {
    content_textEdgeNgramS = SOLR_CONTENT
    content_textEdgeNgramS {
        field = __solr_content
    }
}
Copied!

Solr claims to be not configured in backend context, although I did it well. What can be the reason?

Solr cannot read the configuration from access restricted pages, e. g. "only visible for logged in users". Although access to your root page is not restricted, this can occur if the root page is a shortcut to a access restricted page.

Does this make sense? Yes. If user is not logged in one can redirect to a login page with the help of an error handler. And after login one can re-redirect to a restricted page again. Please omit such scenarios.

The reason for that:

With TYPO3 10 and solr 11.0.1 it was possible for solr to read it's configuration from shortcut pages directly. With TYPO3 11 and solr 11.5 these shortcuts are followed. And if the access to the destination of the shortcut is restricted, solr cannot read the configuration because solr is not logged in.

The solution:

If the root page is a shortcut, the destination MAY NOT be access restricted! In addition a login page itself cannot get hidden after login. This would lead to errors and no redirections will take place. This can e. g. be achieved in the following way.

  • The root page is a shortcut to the 1st subpage.
  • The 1st subpage is a shortcut to an unprotected login page, and will be hidden after successfull login.
  • The 2nd subpage is an pagetree with the access restricted content.
  • And last but not least there is the login page (e. g. 3rd subpage) in an unprotected area, which will never get hidden. So ext-solr can read its configuration here.

This way already logged in users will be redirected directly to the restricted page / sub-tree, because the 1st subpage is hidden and so the restricted page is the first sub-page now.

The solr container in DDEV setup keeps quitting?

The issue was experienced on a OsX M2 ARM64 setup, but might be relevant for others.

We have experienced that using Colimas as docker provider in DDEV, caused the solr container to quit upon first HTTP request. This could happen when clicking the "Index Queue" backend module.

The container log did not contain any output relevant to the container quitting.

Changing to Docker Desktop as provider keep the container alive.

Relevant DDEV link: https://ddev.readthedocs.io/en/latest/users/providers/

Can mount points be indexed?

Mount points are supported in general and the mounted pages will be indexed like standard pages.

But there is a point to consider: Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs. The behavior is intentionally designed this way in TYPO3 core, the background is that it is not possible to specify the languages of the mounted page tree without Site Configuration.

Releases 13.0

Release 13.0.3

This is a maintenance release for TYPO3 13.4 LTS.

  • [TASK] disable dependabot on release-11.6.x branch by Rafael Kähm (b9771d029)
  • [BUGFIX] Fix mount point garbage collection by Markus Friedrich (a48f23369)
  • [TASK] Drop workaround for cObj IMAGE by Markus Friedrich (990046a10)
  • [BUGFIX] Avoid PHP warning if related record was not found by Albrecht Köhnlein (8ce468861)
  • [BUGFIX] Get current pageId by Julian Hofmann (868d60a72)

Release 13.0.2

This is a maintenance release for TYPO3 13.4 LTS.

List of all changes:

Release 13.0.1

List of all changes:

  • [TASK] Update version matrix 0ed896675
  • [BUGFIX] Fix coding standards issues 1f3a281ad
  • [BUGFIX] Fix monitoring of mounted pages ad2548006
  • [BUGFIX] Add check if generator is valid before traversing it 1e3fe3f70
  • !!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814 88918f61d
  • [FIX] docker image tests do not fail if core can not start 2b7b95602
  • [DOCS] Actually mention the values of monitoringType 508477f64
  • [BUGFIX] set PageInformation contentFromPid 1c29157cc
  • [BUGFIX] site has no attribute language 972950af5
  • [BUGFIX] Fix notice exception in ScoreCalculationService a7b819cbd
  • [FEATURE] Add an event for modifying the domain used for a site 2b848a77e
  • [FIX] Docker tests suite does not contain all logs a89de2f97
  • !!![FIX] Docker execution order issue for as-sudo tweaks 77c3c0f44
  • [BUGFIX] Docker tweaks as-sudo do not preserve the Docker image ENV eb8546858
  • [TASK] Use relative path to typo3lib in Apache Solr config dbbf4c5b4
  • [DOCS] Improve Solr core creation via API and other deployment parts 761894713

!!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814

Updates EXT:solr to Apache Solr 9.8.0.

Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default, which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality. All Apache Solr libs, modules or plugins must be configured within the main server configuration files. See: https://issues.apache.org/jira/browse/SOLR-16781

Impact:

Docker

By using our official Docker image from https://hub.docker.com/r/typo3solr/ext-solr, you want to pull the image v. 13.0.1+ and restart the container with that image, which will run a migrations-script automatically to secure the configuration in used volume automatically. No other steps are required.

Alternatively you can wipe the volume and start the container with v. 13.0.1+ image, but that method will wipe the index as well.

See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh

Other server setups

You have 2 possibilities to fix that issue in your Apache Solr Server:

(PREFERRED) Migrate the EXT:solr's Apache Solr configuration

Refer to https://github.com/TYPO3-Solr/ext-solr/pull/4290/files .

Following 3 files are relevant:

  • Changes in <Apache-Solr data dir>/configsets/ext_solr_13_0_0/conf/solrconfig.xml
  • Changes in <Apache-Solr data dir>/solr.xml
  • Movement from <Apache-Solr data dir>/configsets/ext_solr_13_0_0/typo3lib/solr-typo3-plugin-6.0.0.jar

    • to <Apache-Solr data dir>/typo3lib/solr-typo3-plugin-6.0.0.jar

Steps:

  1. Remove all occurrences of <lib dir=".* from <Apache-Solr data dir>/configsets/ext_solr_13_0_0/conf/solrconfig.xml file.
  2. Replace in <Apache-Solr data dir>/solr.xml file the snipped

    <str name="modules">scripting</str>
    Copied!

    by

    <str name="modules">scripting,analytics,analysis-extras,langid,clustering,extraction,${solr.modules:}</str>
    <str name="allowPaths">${solr.allowPaths:}</str>
    <str name="allowUrls">${solr.allowUrls:}</str>
    
    <!-- TYPO3 Plugins -->
    <str name="sharedLib">typo3lib/</str>
    Copied!
  3. Move the directory from <Apache-Solr data dir>/configsets/ext_solr_13_0_0/typo3lib

    • to <Apache-Solr data dir>/typo3lib

!!![FIX] Docker execution order issue for as-sudo tweaks

This change renames the file

  • from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
  • to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh

and moves the folder

  • from /docker-entrypoint-initdb.d/as-sudo/
  • to /docker-entrypoint-initdb.d-as-sudo/

to fix the execution order issue when setting the correct file permissions when starting the docker container, leading to a Operation not permitted errors.

More details see:

Impact:

This change requires adjustments in your Docker setup, only if you modified:

  • files in folder /docker-entrypoint-initdb.d/as-sudo/
  • file /docker-entrypoint-initdb.d/as-sudo-tweaks.sh.
Make sure to use:
  • /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh instead of

    • /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
  • /docker-entrypoint-initdb.d-as-sudo instead of

    • /docker-entrypoint-initdb.d/as-sudo/

Release 13.0.0

This is a new major release for TYPO3 13.4 LTS.

New in this release

!!! Upgrade to Apache Solr 9.7.0

This release requires Apache Solr v 9.7.0+.

Adjust mount point indexing

Mount point indexing and corresponding tests have been adjusted for TYPO3 13. Mount points are supported in general and the mounted pages will be indexed like standard pages.

But there is a point to consider: Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs. The behavior is intentionally designed this way in TYPO3 core, the background is that it is not possible to specify the languages of the mounted page tree without Site Configuration.

All Changes

  • [TASK] Prepare main branch for TYPO3 13 by Rafael Kähm (eaec73806)
  • [TASK] Set Apache Solr configsets to ext_solr_13_0_0 2024.05.13 by Rafael Kähm (460f919be)
  • [BUGFIX] Fix TYPO3 coding standards by Rafael Kähm (80cfe91dc)
  • [TASK] Simple blocker:: come through typo3 extension:setup command by Rafael Kähm (12de6ef21)
  • [BUGFIX] Set solr configSet to ext_solr_13_0_0 by Thomas Löffler (c3c317ffe)
  • [TASK] Update navigationComponent for page tree in v13 by Thomas Löffler (64673fd0f)
  • [TASK] Adapt Unit Tests for TYPO3 v13 by Benni Mack (c0baedbaa)
  • !!![TASK] Change default to not track last searches by Christoph Lehmann (e1f607a88)
  • [TASK] Beautify backend modules for v13 by Thomas Löffler (e51bd8286)
  • [TASK] Adapt Unit Tests for TYPO3 v13 by Benni Mack (1c3c35105)
  • [TASK] Make TSFE resolving work again by Benni Mack (6e2b3f3b4)
  • [TASK] Clean up usages of TSFE mocking by Benni Mack (87630a289)
  • [BUGFIX] Fix remaining integration tests by Benni Mack (f859f0c5b)
  • [TASK] fix PhpStan errors for TYPO3 13 by Rafael Kähm (89d9f0d27)
  • [TASK] Disable progress % within Github Actions by Rafael Kähm (396396979)
  • [TASK] Run tests daily by Rafael Kähm (a81626723)
  • [FIX] tests for TYPO3 13 @ 2024.07.02 by Rafael Kähm (20b35ea21)
  • [FIX] tests for TYPO3 13 @ 2024.07.09 by Rafael Kähm (c02a3ebbc)
  • [FIX] IntegrationSearchTest for TYPO3 13 by Rafael Kähm (3415e9871)
  • [FIX] require PHP 8.2 for TER version by Rafael Kähm (3d1092b84)
  • [FIX] Dependabot not working by Rafael Kähm (dfcdd98bb)
  • [TASK] Don't store build solrci-image longer as 1 days by Rafael Kähm (ed561a654)
  • [FIX] GitHub scheduled Actions for daily tests by Rafael Kähm (556b5d3df)
  • [FIX] deprecations for Fluid viewHelpers and stack by Rafael Kähm (216319eed)
  • [FIX] IntegrationExtbasePersistenceEventListenerTest errors by Rafael Kähm (97156bf19)
  • [FIX] Restore BE Modules functionality for TYPO3 13 by Rafael Kähm (cdd979018)
  • [TASK] migrate to typo3fluid/fluid v4 as required by TYPO3 13 by Rafael Kähm (064ce710d)
  • [TASK] Remove deprecated queue.[indexConfig].table TypoScript setting by Rafael Kähm (1a426a6dc)
  • [FIX] Translation handling by delegating requered context objects/values by Rafael Kähm (c3d9db33b)
  • [FIX] follow-up for removed queue.[indexConfig].table TypoScript setting by Rafael Kähm (7fee9368e)
  • [FIX] wrong Schema version in status checks by Rafael Kähm (590b34e8d)
  • [TASK] skip tests for mount-pages temporary #4160 by Rafael Kähm (32906dccf)
  • [TASK] skip tests for acces restrictions stack temporary #4161 by Rafael Kähm (f8eeaad03)
  • [BUGFIX] PhpStan Variable $parameters in empty() always exists and is not falsy by Rafael Kähm (2a8596519)
  • [FIX] Tests for TYPO3 dev-main @2024.09.23 by Rafael Kähm (ff7e038f7)
  • [BUGFIX] Failed to resolve module specifier '@apache-solr-for-typo3/solr//FormModal.js' by Rafael Kähm (3c86a707f)
  • [BUGFIX] @typo3/backend/tree/page-tree-element does not work in BE-Modules by Rafael Kähm (111f68404)
  • [FIX] access restrictions stack for TYPO3 13 by Rafael Kähm (dc7162b25)
  • [FIX] #[Group('frontend')] attribute has comment in SearchControllerTest by Rafael Kähm (0514886b4)
  • [TASK] Adjust configuration check and fallbacks in MultiValue CO by Markus Friedrich (ea883ce33)
  • [TASK] Adapt simulated environment for TYPO3 13 by Markus Friedrich (fb9fdd8c8)
  • Update TxSolrSearch.rst by Florian Seirer (f8d330082)
  • [TASK] Update dependencies by Rafael Kähm (01e5387c0)
  • [TASK] fix CS issues for newest typo3/coding-standards by Rafael Kähm (8c1e28850)
  • !!![TASK] Upgrade to Apache Solr 9.7.0 by Markus Friedrich (323b1f0c2)
  • [RELEASE] 13.0.0-alpha-1 by Markus Friedrich (3bd453d09)
  • [FIX] allow tags/releases from main branch by Rafael Kähm (26e38f8b7)
  • [TASK] migrate plugin subtype "list_type" by Rafael Kähm (0c0f2b953)
  • [TASK] Upgrade typo3/testing-framework to dev-main 2024.10.15 by Rafael Kähm (a4596d49e)
  • [TASK] Use TYPO3 13.4+ and 13.4.x-dev after TYPO3 13 LTS release by Rafael Kähm (0fd63e172)
  • [TASK] Remove JSONP callback in suggest by Benni Mack (094b4e5b2)
  • [FEATURE] Introduce method to unset the query string (#4136) by Ayke Halder (b0ddab00e)
  • Update ExtensionSettings.rst by Jon Echeveste González (d79c92c9d)
  • [FEATURE] Make Node->depth actually initialized and usable by snk-spo (b530a2f03)
  • [TASK] Update version matrix by Markus Friedrich (b6bfad8f1)
  • [TASK] 13.0.x-dev Update solarium/solarium requirement by dependabot[bot] (64e978646)
  • [TASK] improve exception handling by Rafael Kähm (8f1597b4d)
  • [FIX] Garbage collector does not get configuration by Rafael Kähm (f73de9da2)
  • [FIX] CS in Configuration/Backend/Modules.php by Rafael Kähm (08f717129)
  • [FIX] deprecations in Dockerfile by Rafael Kähm (af1e8cdcd)
  • [BUGFIX] Ensure index document is deleted by Markus Friedrich (10c0fde3c)
  • [DOCs] for release 12.0.4 by Rafael Kähm (7b61833ad)
  • [DOCs] Update EXT:solr 12.0.x line in version matrix by Rafael Kähm (ac1ff3663)
  • [FIX] phpstan: Method UrlHelper::withQueryParameter() has parameter $value with no type specified by Rafael Kähm (588564f27)
  • [TASK] Remove Scrutinizer integrations on release-12.0.x by Rafael Kähm (c2558c1d3)
  • [FIX] Re-added template variables for SearchFormViewHelper by thomashohn (f7ad16ae4)
  • [DOCs] for release 12.0.5 by Rafael Kähm (ec97b6fd1)
  • [TASK] Remove Implicitly nullable parameter declarations deprecated by Thomas Hohn (207a0e5fa)
  • Update composer requirement by Thomas Hohn (43f3baa94)
  • [TASK] CS change to multiline parameters with comma on last by Rafael Kähm (9aa403a65)
  • [TASK] Clean and improve ConnectionManagerTest by Markus Friedrich (edf482457)
  • [TASK] Adjust mount point indexing by Markus Friedrich (bf446c032)
  • [BUGFIX] Fix record monitoring if site is missing by Markus Friedrich (0dfd4b454)
  • [TASK] Evaluate all entries in Services.yaml regarding to shared setting by Rafael Kähm (f8083a616)
  • [TASK] Add int cast for sys_language_uid by Guido Schmechel (de7d7efa7)
  • [TASK] Add int cast for sys_language_uid by Guido Schmechel (5d659dd3a)
  • [DOCS] Switch documentation rendering to PHP-based rendering by Rafael Kähm (4f7b9a73e)
  • [DOCS] workaround for version matrix by Rafael Kähm (bc5bf0b6f)
  • [FEATURE] Add timeframe filter to statistics module by Bastien Lutz (0fc8d7cbd)
  • [BUGFIX] Respect foreignLabel in related items from mm table by Till Hörner (f5271b049)
  • [BUGFIX] Make getHasChildNodeSelected recursive by Tobias Wojtylak (a128c3018)
  • [BUGFIX] Add StartTimeRestriction to ConfigurationAwareRecordService by Amir Arends (27f36af68)
  • [FEATURE] Use PHP generator to prevent processing of all available site by Stefan Frömken (7fec14dc4)
  • [FIX] Indexing fails with SOLR_* cObj in TypoScript by Rafael Kähm (bcb252197)
  • [FIX] missing TypoScript configuration on RecordMonitor stack by Rafael Kähm (31199d2a1)

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Albrecht Köhnlein
  • Amir Arends
  • Ayke Halder
  • Bastien Lutz
  • Benni Mack
  • Bernd Wilke
  • Christoph Lehmann
  • @chrrynobaka
  • @derMatze82
  • Florian Seirer
  • Guido Schmechel
  • Hendrik vom Lehn
  • Jon Echeveste González
  • Lars Tode
  • Markus Friedrich
  • Rafael Kähm
  • @snk-spo
  • Stefan Frömken
  • Thomas Hohn
  • Thomas Löffler
  • Till Hörner
  • Tobias Wojtylak
  • Torben Hansen
  • Wolfgang Wagner

Also a big thank you to our partners who have already concluded one of our development participation packages such as Apache Solr EB for TYPO3 13 LTS (Feature):

  • +Pluswerk AG
  • .hausformat
  • 711media websolutions GmbH
  • Amt der Oö Landesregierung
  • Autorité des marchés financiers
  • Berlin-Brandenburgische Akademie der Wissenschaften
  • Bestellung EB13 SOS Software GmbH für Telekom
  • CS2 AG
  • F7 Media GmbH
  • Fachhochschule Erfurt
  • Getdesigned GmbH
  • Groupe Toumoro inc
  • KONVERTO AG
  • Kassenärztliche Vereinigung Rheinland-Pfalz
  • Kreis Euskirchen
  • LOUIS INTERNET GmbH
  • Leuchtfeuer Digital Marketing GmbH
  • LfdA - Labor für digitale Angelegenheiten GmbH
  • MOSAIQ GmbH
  • Marketing Factory Digital GmbH
  • ProPotsdam GmbH
  • SITE'NGO
  • Snowflake Productions GmbH
  • Stämpfli AG
  • THE BRETTINGHAMS GmbH
  • b13 GmbH
  • clickstorm GmbH
  • cron IT GmbH
  • graphodata GmbH
  • i-kiu motion
  • in2code GmbH
  • internezzo ag
  • jweiland.net e.K.
  • mehrwert intermediale kommunikation GmbH
  • network.publishing Möller-Westbunk GmbH
  • plan2net GmbH
  • queo GmbH
  • visol digitale Dienstleistungen GmbH
  • werkraum Digitalmanufaktur GmbH

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us by becoming an EB partner:

https://shop.dkd.de/Produkte/Apache-Solr-fuer-TYPO3/

or call:

+49 (0)69 - 2475218 0

Releases 12.0

Release 12.0.7

This is a maintenance release for TYPO3 12.4.

Release 12.0.6

List of all changes

  • [BUGFIX] GroupViewHelperTest 9e2b6bf9f
  • [TASK] Remove Scrutinizer integrations on release-12.0.x ac09602b4
  • [FIX] CS issue in Tests\Unit\System\ContentObject\ContentObjectServiceTest b5be2e1c5
  • [BUGFIX] Convert getPageOverlay hook into PSR-14 Event fe7c9eb08
  • [BUGFIX] PhpStan Variable $parameters in empty() always exists and is not falsy 1b3218b7c
  • [TASK] Apache Solr 9.7 compatibility 27ab5cd7f
  • [TASK] 12.0.x-dev Update solarium/solarium requirement 01440e68d
  • [TASK] Remove Implicitly nullable parameter declarations deprecated 825bb9bdb
  • Fix file a185cb863
  • [TASK] CS change to multiline parameters with comma on last b5b22eb3b
  • [TASK] Evaluate all entries in Services.yaml regarding to shared setting a3da67ac2
  • [FEATURE] Add timeframe filter to statistics module 9e8d69772
  • [BUGFIX] Respect foreignLabel in related items from mm table fc67246a5
  • [BUGFIX] Make getHasChildNodeSelected recursive 5d3ab6b0b
  • [BUGFIX] Add StartTimeRestriction to ConfigurationAwareRecordService 7369adf95
  • [FEATURE] Use PHP generator to prevent processing of all available site fe71a4f48
  • [BUGFIX] Fix monitoring of mounted pages f4924d144
  • [DOCS] Switch documentation rendering to PHP-based rendering 82773617f
  • [BUGFIX] Add check if generator is valid before traversing it 1c45b5318
  • [FIX] docker image tests do not fail if core can not start f52a50ec2
  • !!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814 71ac3ee1a
  • [DOCS] Actually mention the values of monitoringType 7f72a9fab
  • !!![TASK] Remove JSONP callback in suggest fca9069ed
  • [BUGFIX] Fix notice exception in ScoreCalculationService 52800162b
  • [FIX] Docker tests suite does not contain all logs 9ce209824
  • !!![FIX] Docker execution order issue for as-sudo tweaks 6ac30c124
  • [BUGFIX] Docker tweaks as-sudo do not preserve the Docker image ENV c88254506
  • [TASK] Use relative path to typo3lib in Apache Solr config ad32c9905
  • [DOCS] Improve Solr core creation via API and other deployment parts 387826fca

!!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814

Updates EXT:solr to Apache Solr 9.8.0.

Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default, which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality. All Apache Solr libs, modules or plugins must be configured within the main server configuration files. See: https://issues.apache.org/jira/browse/SOLR-16781

Impact:

Docker

By using our official Docker image from https://hub.docker.com/r/typo3solr/ext-solr, you want to pull the image v. 12.0.6+ and restart the container with that image, which will run a migrations-script automatically to secure the configuration in used volume automatically. No other steps are required.

Alternatively you can wipe the volume and start the container with v. 12.0.6+ image, but that method will wipe the index as well.

See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh

Other server setups

You have 2 possibilities to fix that issue in your Apache Solr Server:

(PREFERRED) Migrate the EXT:solr's Apache Solr configuration

Refer to https://github.com/TYPO3-Solr/ext-solr/pull/4290/files .

Following 3 files are relevant:

  • Changes in <Apache-Solr data dir>/configsets/ext_solr_12_0_0/conf/solrconfig.xml
  • Changes in <Apache-Solr data dir>/solr.xml
  • Movement from <Apache-Solr data dir>/configsets/ext_solr_12_0_0/typo3lib/solr-typo3-plugin-6.0.0.jar

    • to <Apache-Solr data dir>/typo3lib/solr-typo3-plugin-6.0.0.jar

Steps:

  1. Remove all occurrences of <lib dir=".* from <Apache-Solr data dir>/configsets/ext_solr_12_0_0/conf/solrconfig.xml file.
  2. Replace in <Apache-Solr data dir>/solr.xml file the snipped

    <str name="modules">scripting</str>
    Copied!

    by

    <str name="modules">scripting,analytics,analysis-extras,langid,clustering,extraction,${solr.modules:}</str>
    <str name="allowPaths">${solr.allowPaths:}</str>
    <str name="allowUrls">${solr.allowUrls:}</str>
    
    <!-- TYPO3 Plugins -->
    <str name="sharedLib">typo3lib/</str>
    Copied!
  3. Move the directory from <Apache-Solr data dir>/configsets/ext_solr_12_0_0/typo3lib

    • to <Apache-Solr data dir>/typo3lib

!!![TASK] Remove JSONP callback in suggest

by @bmack and @dkd-kaehm in #4267

In own implementation of autosuggest JS parts with usage of JSONP, the action must be migrated to non-JSONP calls.

!!![FIX] Docker execution order issue for as-sudo tweaks

This change renames the file

  • from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
  • to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh

and moves the folder

  • from /docker-entrypoint-initdb.d/as-sudo/
  • to /docker-entrypoint-initdb.d-as-sudo/

to fix the execution order issue when setting the correct file permissions when starting the docker container, leading to a Operation not permitted errors.

More details see:

Impact:

This change requires adjustments in your Docker setup, only if you modified:

  • files in folder /docker-entrypoint-initdb.d/as-sudo/
  • file /docker-entrypoint-initdb.d/as-sudo-tweaks.sh.
Make sure to use:
  • /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh instead of

    • /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
  • /docker-entrypoint-initdb.d-as-sudo instead of

    • /docker-entrypoint-initdb.d/as-sudo/

Release 12.0.5

This is a maintenance release for TYPO3 12.4 LTS, which mainly fixes 2 regressions introduced in version 12.0.4:

  • [TASK] Re-added template variables by @thomashohn in #4118
  • [BUGFIX] Fix index queue clearance by @sal-lochbaum & @dkd-friedrich in #4120
  • [TASK] Update to solarium/solarium 6.3.5 by @dkd-friedrich in #4120

Release 12.0.4

This is a maintenance release for TYPO3 12.4 LTS, containing:

  • [TASK] Use request object to retrieve query params instead of _GET by @sfroemkenjw in #4045
  • [TASK] Use Attributes for PHPUnit tests by @bmack in #4048
  • [TASK] Update PHP-Stan to at least 1.11.* by @sfroemkenjw in #4055
  • [TASK] Apply and repair rector refactorings by @sfroemkenjw in #4049
  • [TASK] Migrate requireJS to ES6. Solr BE Modal JS by @sfroemkenjw in #4057
  • [TASK] Apache Solr 9.6 compatibility by @dkd-friedrich in #4056
  • [TASK] Use new template module API by @sfroemkenjw in #4054
  • [FEATURE] Add contentObjectData to searchController by @spoonerWeb in #4059
  • [BUGFIX] Add empty array as fallback if null by @spoonerWeb in #4061
  • [BUGFIX] Add empty array defaults in SearchFormViewHelper by @hnadler in #4042
  • [TASK] Integrate content of Module layout into WithPageTree by @sfroemkenjw in #4066
  • [TASK] Repair statistics chart because of CSP in Solr Info module by @sfroemkenjw in #4068
  • [FEATURE:BP:12] Be able to disable tracking of last searches by @dkd-kaehm in #4064
  • [TASK] Add access plugin tests by @dkd-friedrich in #4069
  • [TASK] Update authors by @sfroemkenjw in #4071
  • [TASK] Remove content stream usage by @dkd-friedrich in #4073
  • [BUGFIX] Fix synonym and stop word upload by @dkd-friedrich in #4074
  • [TASK] Call getLabelFromItemListMerged with the current row data by @3l73 in #4081
  • [BUGFIX] numeric facet range slider sends lot of requests to server by @hvomlehn-sds in #4084
  • [BUGFIX] Typecast $userGroup to integer by @derhansen in #4079
  • [TASK] Remove getIsSiteManagedSite as all site are managed now by @sfroemkenjw in #4070
  • [BUGFIX] [4026] treat non-overlayed mount points as valid by @derMatze82 in #4029
  • [TASK] New Crowdin updates by @dkd-kaehm in #4094
  • [BUGFIX] Fix range string calculation in DateRange facet by @derhansen in #4090
  • [BUGFIX:12] scheduler task "Optimize index of a site" is not functional by @dkd-kaehm in #4104
  • [TASK] Apache Solr 9.6.1 compatibility by @dkd-kaehm in #4106
  • [BUGFIX] deprecations in Dockerfile by @dkd-kaehm in #4110
  • [BUGFIX] Ensure index document is deleted by @dkd-friedrich in #4113

Release 12.0.3

This is a maintenance release for TYPO3 12.4 LTS, containing:

  • [BUGFIX] PHPStan 1.11+ by @dkd-kaehm in #4030
  • [BUGFIX] Allow fe_group modification when indexing subpage of restricted page by @deschilter in #3982
  • [TASK:P:main] Full fixed french translation by @megamisan in #3996
  • [TASK] Update all GitHub-Actions on main branch by @dkd-kaehm in #4028
  • [TASK] Remove deprecations for CI-Build by @dkd-kaehm in #4028
  • [BUGFIX] Debug output if response body is read once stream by @Kanti in #4009
  • [BUGFIX] Use tx-solr-suggest-focus for autofocus on form-submit by @felixranesberger in #4005
  • [FEATURE] Support stdWrap to resolve relation label field by @DanielSiepmann in #4021
  • [BUGFIX] Remove superfluous quotation marks in Partials/Result/FacetsActive.html by @julianhofmann in #4021
  • [TASK] Remove switchableControllerActions from TypoScript by @derhansen in #4015
  • [Internal] Collection of fixes recognized by SDS task by @dkd-kaehm in #3974
  • [BUGFIX] Prevent exception in queue module when no site can be determined by @pschriner in #3988
  • [FIX] Actions errors 2024.04.05 by @dkd-kaehm in #3994
  • [BUGFIX] column changed out of range by @ferfrost in #3980
  • [TASK] Update testing framework by @dkd-friedrich in #3992
  • [BUGFIX] Solves PHP Warning: Undefined array key "label". by @serens #3998

Release 12.0.2

This is a maintenance release for TYPO3 12.4 LTS, containing:

  • [BUGFIX] false as field value is not allowed by @dkd-kaehm in #3901
  • [FEATURE] send an event for suggest queries by @dmitryd in #3903
  • [TASK] Remove unused field 'cookie' in tx_solr_statistics by @derhansen in #3893
  • [BUGFIX] Unit tests missing mock of EventDispatcher for AfterSuggestQueryHasBeenPreparedEvent by @dkd-kaehm in #3910
  • [BUGFIX] Resolve custom partialName in hierarchy facet by @tillhoerner in #3908
  • [BUGFIX] Fix #3896: adjust paths in jquery-ui.custom.css by @dmitryd in #3906
  • [BUGFIX] Broken IndexQueueModule.css asset path in backend by @DanielSiepmann in #3898
  • [FEATURE] Improve BeforeSearchResultIsShownEvent by @georgringer in #3915
  • [BUGFIX] 'AfterFacetIsParsedEvent' was never dispatched by @dkd-kaehm in #3921
  • [BUGFIX] update acces to backend user modules by @mhirdes in #3924
  • [DOC] Add infos about adding own field processors by @kitzberger in #3930
  • [BUGFIX] Ensure method return value of root-page-UID is an integer by @dkd-kaehm in #3929
  • [BUGFIX] Fix #3916: exception in CLI mode when using suggest TS example by @dmitryd in #3917
  • [BUGFIX] Prevent exception in AccessComponent by @dkd-friedrich in #3945
  • [TASK] Describe issues with Colima as Docker provider by @sorenmalling in #3950
  • [TASK] Provide encryptionKey in unit tests by @dkd-friedrich in #3959
  • [DOCS] Make values of faceting.urlParameterStyle more clear by @linawolf in #3951
  • [BUGFIX] Handle if some tags are upper-case and strict-comparison by @thomashohn in #3941
  • [FEATURE] Power up for magic filter __pageSections by @kitzberger in #3937
  • [TASK] Add content stream check by @dkd-friedrich in #3967
  • [TASK] Support several Apache Solr versions by @dkd-friedrich in #3956
  • [RELEASE] 12.0.2

Release 12.0.1

This is a maintenance release for TYPO3 12.4 LTS, containing:

Note: This release requires the database schema update, due of database schema change from pull-request #3881

Release 12.0.0

We are happy to release EXT:solr 12.0.0. The focus of this release has been on TYPO3 12 LTS compatibility.

Please note that we require at least TYPO3 12.4.3, as this version contains some change concerning to Fluid.

New in this release

Support of TYPO3 12 LTS

With EXT:solr 12.0 we provide the support of TYPO3 12 LTS.

!!! Upgrade to Apache Solr 9.3.0

This release requires Apache Solr v 9.3.0+.

Following variables must be set in solr.in.sh file or in Solr system props:

  • SOLR_ENABLE_REMOTE_STREAMING=true
  • SOLR_ENABLE_STREAM_BODY=true

For more information see:

Reworked Search Query Component System

The Search Component system, which is used to enrich the search query (e.g. by faceting, boosting, debug analysis), has been completely reworked by utilizing the PSR-14 event system.

At the same time the Search Query Modifiers have been merged into the Query Component systems.

All built-in components are now reworked and utilize the \ApacheSolrForTypo3\Solr\Event\Search\AfterSearchQueryHasBeenPreparedEvent PSR-14 event.

The interface \ApacheSolrForTypo3\Solr\Domain\Search\SearchRequestAware has been removed.

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifySearchQuery'] and the interfaces \ApacheSolrForTypo3\Solr\Query\Modifier as well as \ApacheSolrForTypo3\Solr\Search\QueryAware and \ApacheSolrForTypo3\Solr\Search\SearchAware have been removed. The modifiers have been merged into Components.

Registration does not happen in ext_localconf.php anymore via \ApacheSolrForTypo3\Solr\Search\SearchComponentManager which has been removed, but now happens in Configuration/Services.yaml as documented in TYPO3 Core's PSR-14 Registration API.

Related hooks around this system have been moved to PSR-14 events as well:

  • $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['beforeSearch'] has been replaced by \ApacheSolrForTypo3\Solr\Event\Search\AfterInitialSearchResultSetHasBeenCreatedEvent
  • $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] has been replaced by \ApacheSolrForTypo3\Solr\Event\Search\AfterSearchHasBeenExecutedEvent

SignalSlots replaced by PSR-14 events

The previously available Extbase Signals have been removed from EXT:solr in favor of PSR-14 Events.

  • The signal ApacheSolrForTypo3\Solr\Domain\Index\IndexService::beforeIndexItems has been replaced by \ApacheSolrForTypo3\Solr\Event\Indexing\BeforeItemsAreIndexedEvent
  • The signal ApacheSolrForTypo3\Solr\Domain\Index\IndexService::beforeIndexItem has been replaced by \ApacheSolrForTypo3\Solr\Event\Indexing\BeforeItemIsIndexedEvent
  • The signal ApacheSolrForTypo3\Solr\Domain\Index\IndexService::afterIndexItem has been replaced by \ApacheSolrForTypo3\Solr\Event\Indexing\AfterItemHasBeenIndexedEvent
  • The signal ApacheSolrForTypo3\Solr\Domain\Index\IndexService::afterIndexItems has been replaced by \ApacheSolrForTypo3\Solr\Event\Indexing\AfterItemsHaveBeenIndexedEvent
  • The signal ApacheSolrForTypo3\Solr\Domain\Search\ResultSet\Facets\OptionBased\Options\OptionFacetParser::optionsParsed has been replaced by \ApacheSolrForTypo3\Solr\Event\Parser\AfterFacetIsParsedEvent
  • The signal ApacheSolrForTypo3\Solr\Controller\SearchController::resultsAction has been replaced by \ApacheSolrForTypo3\Solr\Event\Search\BeforeSearchResultIsShownEvent
  • The signal ApacheSolrForTypo3\Solr\Controller\SearchController::formAction has been replaced by \ApacheSolrForTypo3\Solr\Event\Search\BeforeSearchFormIsShownEvent
  • The signal ApacheSolrForTypo3\Solr\Controller\SearchController::frequentlySearchedAction has been replaced by \ApacheSolrForTypo3\Solr\Event\Search\AfterFrequentlySearchHasBeenExecutedEvent
  • The signal ApacheSolrForTypo3\Solr\Controller\SearchController::beforeSearch has been removed (see the new PSR-14 events below)

Hooks replaced by PSR-14 events

The previously available hooks and their respective interfaces have been removed from EXT:solr.

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['Indexer']['indexPageAddDocuments'] and its interface \ApacheSolrForTypo3\Solr\AdditionalPageIndexer are now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\Indexing\BeforePageDocumentIsProcessedForIndexingEvent.

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifyVariantId'] and its interface \ApacheSolrForTypo3\Solr\Variants\IdModifier are now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\Variants\AfterVariantIdWasBuiltEvent.

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['IndexQueueIndexer']['preAddModifyDocuments'] and its interface \ApacheSolrForTypo3\Solr\PageIndexerDocumentsModifier are now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\Indexing\BeforeDocumentIsProcessedForIndexingEvent.

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['IndexQueueIndexer']['indexItemAddDocuments'] and its interface \ApacheSolrForTypo3\Solr\AdditionalIndexQueueItemIndexer are now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\Indexing\BeforeDocumentsAreIndexedEvent.

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['Indexer']['indexPageSubstitutePageDocument'] and its interface \ApacheSolrForTypo3\Solr\SubstitutePageIndexer are now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\Indexing\AfterPageDocumentIsCreatedForIndexingEvent.

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['postProcessIndexQueueInitialization'] and its interface \ApacheSolrForTypo3\Solr\IndexQueue\InitializationPostProcessor are now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\IndexQueue\AfterIndexQueueHasBeenInitializedEvent

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['postProcessFetchRecordsForIndexQueueItem'] is now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\IndexQueue\AfterRecordsForIndexQueueItemsHaveBeenRetrievedEvent

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['IndexQueuePageIndexer']['dataUrlModifier'] and the according interface \ApacheSolrForTypo3\Solr\IndexQueue\PageIndexerDataUrlModifier is now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\Indexing\AfterFrontendPageUriForIndexingHasBeenGeneratedEvent

The hook $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['postProcessIndexQueueUpdateItem'] is now superseded by the PSR-14 event \ApacheSolrForTypo3\Solr\Event\Indexing\AfterIndexQueueItemHasBeenMarkedForReindexingEvent

PSR-14 events renamed

Previous PSR-14 events have been renamed to be consistent with other PSR-14 Events in EXT:solr.

  • \ApacheSolrForTypo3\Solr\Event\Routing\PostProcessUriEvent is now named \ApacheSolrForTypo3\Solr\Event\Routing\AfterUriIsProcessedEvent
  • \ApacheSolrForTypo3\Solr\Event\Routing\BeforeProcessCachedVariablesEvent is now named \ApacheSolrForTypo3\Solr\Event\Routing\BeforeCachedVariablesAreProcessedEvent
  • \ApacheSolrForTypo3\Solr\Event\Routing\BeforeReplaceVariableInCachedUrlEvent is now named \ApacheSolrForTypo3\Solr\Event\Routing\BeforeVariableInCachedUrlAreReplacedEvent

!!! Shortcut pages not indexed anymore

Currently there is no important reason to index the shortcut pages, because the target pages are indexed as expected and the shortcuts are 307-redirected to their targets. So contents can be found in search results as expected.

!!! Deprecated Node class removed

Former EXT:solr versions used an own node implementation for Solr endpoints, this implementation (ApacheSolrForTypo3SolrSystemSolrNode) is now removed in favor of the Endpoint implementation of Solarium.

If you've used this class or the SolrConnection directly, you have to adapt your PHP code:

  • use SolariumCoreClientEndpoint instead of ApacheSolrForTypo3SolrSystemSolrNode
  • call ApacheSolrForTypo3SolrSystemSolrSolrConnection->getEndpoint() instead of ApacheSolrForTypo3SolrSystemSolrSolrConnectiongetNode(), method will return Solarium Endpoint
  • Node could be converted to string to get the core base URI, getCoreBaseUri() can be used instead.

Note: With dropping the Node implementation we also dropped the backwards compatibility that allows to define the Solr path segment "/solr" within "solr_path_read" or "solr_path_write". Be sure your configuration doesn't contain this path segment!

!!! Changed visibility of ApacheSolrForTypo3SolrIndexQueueFrontendHelperPageIndexer methods

For testing purposes some methods of the PageIndexer were defined as public, these methods are now protected. The tests are adapted accordingly, so that there is no need to declare the methods as public. If you have used one of this methods, you have to adapt your code. Affected methods:

  • setupConfiguration
  • index
  • indexPage

!!! Solr route enhancer disabled by default

EXT:solr offers the possibility to create speaking URLs for Solr facets, but as this feature requires additional configuration and costly processing this feature is now disabled by default.

If you've already used the route enhancer you must set option "enableRouteEnhancer":

$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['enableRouteEnhancer']

Frontend Helper Changes

The FrontendHelper logic revolving around PageIndexer has been reduced to a minimum by only having two methods available:

  • ApacheSolrForTypo3\Solr\IndexQueue\FrontendHelper\FrontendHelper::activate() - used to register hooks and PSR-14 event listeners
  • ApacheSolrForTypo3\Solr\IndexQueue\FrontendHelper\FrontendHelper::deactivate(PageIndexerResponse $response) - used to populate data into the PageIndexerResponse object

The actual PageIndexerRequest object is now available as a property of TYPO3's Request object as attribute named "solr.pageIndexingInstructions".

!!!Complex query in FlexForm filter value

It is now possible to use complex query in FlexForm filter value. If the value contains space and no special characters, the value is always automatically escaped.

The old behaviour is still working, so if a string value contains space(s) and no special characters of the solr query parser, the string is always wrapped with double quotes. But if the string contains special characters no wrapping happen special characters are: + - && || ! ( ) { } [ ] ^ " ~ * ? : \

There is some cases where this change can break, for example if the filter value is something like toto AND tata or music (rock) or my "flow" is. Here the wrapping and the escaping of the inner double quote have to be manually updated like this "toto AND tata" or "music (rock)" and "my \"flow\" is".

All Changes

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Achim Fritz
  • Albrecht Köhnlein
  • Alexander Nitsche
  • Andreas Kießling
  • André Buchmann
  • Bastien Lutz
  • Benni Mack
  • Benoit Chenu
  • Christoph Lehmann
  • @chrrynobaka
  • Daniel Siepmann
  • @derMatze82
  • Dmitry Dulepov
  • Elias Häußler
  • Eric Chavaillaz
  • Ernesto Baschny
  • Fabio Norbutat
  • Felix Ranesberger
  • ferfrost
  • Florian Rival
  • Georg Ringer
  • Harald Witt
  • Hendrik vom Lehn
  • @hnadler
  • Henrik Elsner
  • Ingo Fabbri
  • Jennifer Geiß
  • Julian Hofmann
  • Kai Lochbaum
  • Lars Tode
  • Lukas Niestroj
  • Marc Hirdes
  • Mario Lubenka
  • Markus Friedrich
  • Matthias Vogel
  • @n3amil / Cypelt
  • Oliver Bartsch
  • Patrick Schriner
  • Philipp Kitzberger
  • Pierrick Caillon
  • Rafael Kähm
  • René Maas
  • Roman Schilter
  • Sascha Nowak
  • Sascha Schieferdecker
  • Sebastian Schreiber
  • Silvia Bigler
  • Søren Malling
  • Stefan Frömken
  • Steve Lenz
  • Stämpfli Kommunikation
  • Sven Erens
  • Sven Teuber
  • Thomas Löffler
  • Till Hörner
  • Tim Dreier
  • Tobias Hövelborn
  • Tobias Schmidt
  • Torben Hansen
  • @twojtylak
  • Wolfgang Wagner | wow! solution

Also a big thank you to our partners who have already concluded one of our new development participation packages such as Apache Solr EB for TYPO3 12 LTS (Feature or Maintenance):

  • +Pluswerk AG
  • .hausformat
  • 3m5. Media GmbH
  • 4eyes GmbH
  • 711media websolutions GmbH
  • ACO Ahlmann SE & Co. KG
  • AVM Computersysteme Vertriebs GmbH
  • AmedickSommer Neue Medien GmbH
  • Ampack AG
  • Amt der Oö Landesregierung
  • Autorité des Marchés Financiers (Québec)
  • Bandesinstitut für Schule und Medien Berlin-Brandenburg
  • Beeeh IT
  • Bytebetrieb GmbH & Co. KG
  • CARL von CHIARI GmbH
  • CDG 59
  • CPS GmbH
  • CS2 AG
  • Columbus Interactive GmbH
  • Connecta AG
  • DGB Rechtsschutz GmbH
  • DMK E-BUSINESS GmbH
  • DP-Medsystems AG
  • DSCHOY GmbH
  • Davitec GmbH
  • Deutsches Literaturarchiv Marbach
  • Die Medialen GmbH
  • Digitale Offensive GmbH
  • EB-12LTS-FEATURE
  • Eidg. Forschungsanstalt WSL
  • F7 Media GmbH
  • FTI Touristik GmbH
  • Fachagentur Nachwachsende Rohstoffe fnr.de
  • Forte Digital Germany GmbH
  • GPM Deutsche Gesellschaft für Projektmanagement e. V.
  • Gernot Leitgab
  • Getdesigned GmbH
  • Groupe Toumoro inc
  • HEAD acoustics GmbH
  • HSPV NRW
  • Hochschule Koblenz Standort Remagen
  • INOTEC Sicherheitstechnik GmbH
  • IW Medien GmbH
  • Internezzo
  • Intersim AG
  • KONVERTO AG
  • Kassenzahnärztliche Vereinigung Bayerns (KZVB)
  • Kassenärztliche Vereinigung Rheinland-Pfalz
  • Kreis Euskirchen
  • Kwintessens B.V.
  • L.N. Schaffrath DigitalMedien GmbH
  • LOUIS INTERNET GmbH
  • La Financière agricole du Québec
  • Land Tirol
  • Landeskriminalamt Thüringen
  • Leuchtfeuer Digital Marketing GmbH
  • Lingner Consulting New Media GmbH
  • MEDIENHAUS der Evangelischen Kirche in Hessen und Nassau GmbH
  • Macaw Germany Cologne GmbH
  • Marketing Factory Consulting GmbH
  • NEW.EGO GmbH
  • OST Ostschweizer Fachhochschule
  • ProPotsdam GmbH
  • Provitex GmbH Provitex GmbH
  • Québec.ca gouv.qc.ca
  • Randstad Digital
  • Rechnungshof Österreich
  • Red Dot GmbH & Co. KG
  • SIWA Online GmbH
  • SUNZINET GmbH
  • Sandstein Neue Medien GmbH
  • Schoene neue kinder GmbH
  • Serviceplan Suisse AG
  • Snowflake Productions GmbH
  • Stadtverwaltung Villingen-Schwenningen
  • Statistik Österreich
  • Studio 9 GmbH
  • Stämpfli AG
  • Systime/Gyldendal A/S
  • Südwestfalen IT
  • THE BRETTINGHAMS GmbH
  • Talleux & Zöllner GbR
  • Typoheads GmbH
  • UEBERBIT GmbH
  • Universität Regensburg
  • VisionConnect.de
  • WACON Internet GmbH
  • WIND Internet BV
  • Webtech AG
  • Werbeagentur netzpepper
  • XIMA MEDIA GmbH
  • b13 GmbH
  • bgm websolutions GmbH & Co. KG
  • chiliSCHARF GmbH
  • clickstorm GmbH
  • cosmoblonde GmbH
  • cron IT GmbH
  • cyperfection GmbH
  • digit.ly
  • gedacht
  • graphodata GmbH
  • grips IT GmbH
  • helhum.io
  • in2code GmbH
  • jweiland.net
  • keeen GmbH
  • medien.de mde GmbH
  • mehrwert intermediale kommunikation GmbH
  • mellowmessage GmbH
  • morbihan.fr
  • ochschule Furtwangen
  • pietzpluswild GmbH
  • plan2net GmbH
  • rms. relationship marketing solutions GmbH
  • rocket-media GmbH & Co KG
  • sgalinski Internet Services
  • studio ahoi Weitenauer Schwardt GbR
  • visuellverstehen GmbH
  • webconsulting business services gmbh
  • werkraum Digitalmanufaktur GmbH
  • wow! solution
  • zimmer7 GmbH

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us by becoming an EB partner:

https://shop.dkd.de/Produkte/Apache-Solr-fuer-TYPO3/

or call:

+49 (0)69 - 2475218 0

Releases 11.6

Release 11.6.3 ELTS

This is a non-public maintenance release for TYPO3 11.5 ELTS, containing:

New in this release

Apache Solr 9.8.1 support

EXT:solr 11.6.3 has been tested with Apache Solr 9.8.1, this version can be used without any update steps to consider.

Full list of changes

  • [TASK] Allow Apache Solr 9.8.1 by @dkd-friedrich
  • [BUGFIX:P:11.6] Add check if generator is valid before traversing it by @jacobsenj and @dkd-friedrich
  • [FEATURE:P:11.6] Use PHP generator to prevent processing of all available site @sfroemkenjw and @dkd-friedrich

Release 11.6.2 ELTS

This is a non-public security release for TYPO3 11.5 ELTS, containing:

!!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814

Updates EXT:solr to Apache Solr 9.8.0.

Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default, which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality. All Apache Solr libs, modules or plugins must be configured within the main server configuration files. See: https://issues.apache.org/jira/browse/SOLR-16781

Impact:

Docker

You can wipe the volume and start the container with v. 11.6.2+ image, but that method will wipe the index as well.

See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh

Other server setups

You have 2 possibilities to fix that issue in your Apache Solr Server:

(PREFERRED) Migrate the EXT:solr's Apache Solr configuration

Refer to https://github.com/TYPO3-Solr/ext-solr/pull/4290/files .

Following 3 files are relevant:

  • Changes in <Apache-Solr data dir>/configsets/ext_solr_11_6_0_elts/conf/solrconfig.xml
  • Changes in <Apache-Solr data dir>/solr.xml
  • Movement from <Apache-Solr data dir>/configsets/ext_solr_11_6_0_elts/typo3lib/solr-typo3-plugin-6.0.0.jar

    • to <Apache-Solr data dir>/typo3lib/solr-typo3-plugin-6.0.0.jar

Steps:

  1. Remove all occurrences of <lib dir=".* from <Apache-Solr data dir>/configsets/ext_solr_11_6_0_elts/conf/solrconfig.xml file.
  2. Replace in <Apache-Solr data dir>/solr.xml file the snipped

    <str name="modules">scripting</str>
    Copied!

    by

    <str name="modules">scripting,analytics,analysis-extras,langid,clustering,extraction,${solr.modules:}</str>
    <str name="allowPaths">${solr.allowPaths:}</str>
    <str name="allowUrls">${solr.allowUrls:}</str>
    
    <!-- TYPO3 Plugins -->
    <str name="sharedLib">typo3lib/</str>
    Copied!
  3. Move the directory from <Apache-Solr data dir>/configsets/ext_solr_11_6_0_elts/typo3lib

    • to <Apache-Solr data dir>/typo3lib

!!![FIX] Docker execution order issue for as-sudo tweaks

This change renames the file

  • from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
  • to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh

and moves the folder

  • from /docker-entrypoint-initdb.d/as-sudo/
  • to /docker-entrypoint-initdb.d-as-sudo/

to fix the execution order issue when setting the correct file permissions when starting the docker container, leading to a Operation not permitted errors.

More details see:

Impact:

This change requires adjustments in your Docker setup, only if you modified:

  • files in folder /docker-entrypoint-initdb.d/as-sudo/
  • file /docker-entrypoint-initdb.d/as-sudo-tweaks.sh.
Make sure to use:
  • /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh instead of

    • /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
  • /docker-entrypoint-initdb.d-as-sudo instead of

    • /docker-entrypoint-initdb.d/as-sudo/

Minor changes & bugfixes

  • [DOCS] Improve Solr core creation via API and other deployment parts by @dkd-kaehm & @dkd-friedrich in #41
  • [TASK] Use relative path to typo3lib in Apache Solr config by @dkd-kaehm & @dkd-friedrich in #41
  • [BUGFIX] Docker twaks as-sudo do not preserve the Docker image ENV by @dkd-kaehm & @dkd-friedrich in #41
  • [BUGFIX] Docker tests suite does not contain all logs by @dkd-kaehm & @dkd-friedrich in #41
  • [BUGFIX] docker image tests do not fail if core can not start by @dkd-kaehm & @dkd-friedrich in #41
  • [TASK] Replace "Publish to TER" to release by @dkd-kaehm in #38

Release 11.6.1 ELTS

This is a non-public maintenance release for TYPO3 11.5.

Full list of changes

  • [TASK] Integrate TYPO3 11.5 ELTS by @dkd-kaehm & @dkd-friedrich
  • [TASK] Prepare release-11.6.x ELTS by @dkd-friedrich
  • [BUGFIX] NPE in Index Queue module when no site is selected by @adamkoppede
  • [FEATURE] Add timeframe filter to statistics module by @BastiLu

Release 11.6.0

We are happy to release EXT:solr 11.6.0. The focus of this release has been on Apache Solr upgrade to v9.7.0.

Important: This version is installable with TYPO3 11 LTS on v11.5.14+ only and contains some breaking changes, see details below.

New in this release

!!! Upgrade to Apache Solr 9.7.0

This release requires Apache Solr v9.7.0+.

Along with the compatibility to Solr 9.7 the dependency to SOLR_ENABLE_STREAM_BODY is removed.

Full list of changes

  • !!![BUGFIX:BP:11.6] Queue check considers indexing configuration by @dkd-friedrich
  • !!![TASK:BP:11.6] Introduce queue and queue item interfaces by @dkd-friedrich
  • [TASK:BP:11.6] Consider queue initialization status by @dkd-friedrich
  • !!![TASK:BP:11.6] Introduce specific EXT:solr exceptions by @dkd-friedrich
  • [TASK] Prepare release-11.6.x for TYPO3 11.5 LTS by @dkd-kaehm
  • [TASK] sync the CI stuff from main branch into 11.6.x by @dkd-kaehm
  • [TASK] Activate PHPStan by @dkd-friedrich
  • [TASK] Use Apache Solr 9.2 for EXT:solr 11.6 by @dkd-friedrich
  • [FEATURE] Introduce TYPO3_SOLR_ENABLED_CORES docker env variable by Christoph Lehmann
  • [TASK] Make it possible to change solr unix GID:UID on docker image… by @dkd-kaehm
  • [TASK] revert unnecessary changes on Apache Solr 9.2 upgrade by @dkd-kaehm
  • !!![TASK] Upgrade to Apache Solr 9.3.0 by @dkd-friedrich
  • [BUGFIX] Fix result highlighting fragment size by @dkd-friedrich
  • [TASK] Update to Solr 9.5 by @dkd-friedrich
  • New Crowdin updates 2024.07.03 by @dkd-kaehm
  • [TASK] Upgrade to Apache Solr 9.7 by @dkd-friedrich

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Adam Koppe
  • @BastiLu
  • Christoph Lehmann
  • Markus Friedrich
  • Rafael Kähm

Also a big thank you to our partners who have already concluded one of our new development participation packages such as Apache Solr EB for TYPO3 11 LTS:

  • .hausformat GmbH
  • 3m5. Media GmbH
  • abteilung_digital GmbH
  • ACO Ahlmann SE & Co. KG
  • Agence E-magineurs
  • Agenda d.o.o.
  • AgenturWebfox GmbH
  • Amedick & Sommer Neue Medien GmbH
  • Ampack AG
  • Atol CD
  • Autorité des Marchés Financiers (Québec)
  • avenit AG
  • b13 GmbH
  • brandung GmbH Oliver Krause
  • Bundesanstalt Statistik Österreich
  • Bytebetrieb GmbH & Co. KG
  • Canton de Neuchâtel - SIEN
  • CARL von CHIARI GmbH
  • chiliSCHARF GmbH
  • clickstorm GmbH
  • co-operate Wegener & Rieke GmbH
  • Columbus Interactive GmbH
  • cosmoblonde GmbH
  • creativ clicks GmbH
  • cyperfection GmbH
  • digit.ly
  • DMK E-BUSINESS GmbH
  • Earlybird GmbH & Co KG
  • elancer-team GmbH
  • eulenblick Kommunikation und Werbung
  • Fa. .hausformat
  • Fa. Ausy
  • Fachagentur Nachwachsende Rohstoffe fnr.de
  • Fachhochschule Erfurt
  • Fourdegrees GbR
  • FTI Touristik GmbH
  • für novotegra GmbH (BayWa r.e. AG)
  • Getdesigned GmbH
  • GFE Media GmbH
  • graphodata GmbH
  • gressourcenmangel integral gmbh
  • grips IT GmbH
  • hiroki digital GmbH
  • Hirsch & Wölfl GmbH
  • Hochschule Furtwangen
  • Hochschule Koblenz - Standort Remagen
  • HSPV NRW
  • in2code
  • INOTEC Sicherheitstechnik GmbH
  • Institut national d'excellence en santé et en services sociaux inesss.qc.ca
  • internezzo ag
  • IW Medien GmbH
  • jweiland
  • Kassenärztliche Vereinigung Rheinland-Pfalz
  • Kreis Euskirchen
  • L.N. Schaffrath DigitalMedien GmbH
  • La Financière agricole du Québec
  • Landeskriminalamt Thüringen
  • Leuchtfeuer Digital Marketing GmbH
  • Lingner Consulting New Media GmbH
  • LOUIS INTERNET GmbH
  • Marketing Factory
  • media::essenz
  • medien.de mde GmbH
  • mehrwert intermediale kommunikation GmbH
  • Meridium Technologies
  • MOSAIQ GmbH
  • NEW.EGO GmbH
  • Overlap GmbH & Co KG
  • Patrick Gaumond Québec.ca gouv.qc.ca
  • peytz.dk
  • pietzpluswild GmbH
  • Pixelant / Resultify
  • Plan Software GmbH
  • Plan.Net France
  • ProPotsdam GmbH
  • Provitex GmbH
  • queo GmbH
  • rms. relationship marketing solutions GmbH
  • RR Anwendungsentwicklung
  • Sandstein Neue Medien GmbH
  • Schoene neue kinder GmbH
  • seam media group gmbh
  • Shop F7 Media GmbH
  • SITE'NGO
  • Snowflake Productions GmbH
  • SOS Software Service GmbH
  • Stadtverwaltung Villingen-Schwenningen
  • Stämpfli AG
  • systime.dk
  • Talleux & Zöllner GbR
  • tirol.gv.at
  • toumoro.com
  • Typoheads GmbH
  • UEBERBIT GmbH
  • unternehmen online GmbH Co. KG (UO)
  • Verband der Vereine Creditreform e.V.
  • VisionConnect.de
  • visol digitale Dienstleistungen GmbH
  • visuellverstehen GmbH
  • WACON Internet GmbH
  • webconsulting business services gmbh
  • Webtech AG
  • Werbeagentur netzpepper
  • werkraum Digitalmanufaktur GmbH
  • wow! solution
  • zimmer7 GmbH

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

Support us by becoming an EB partner:

https://shop.dkd.de/Produkte/Apache-Solr-fuer-TYPO3/

or call:

+49 (0)69 - 2475218 0

Releases 11.2

Release 11.2.5 ELTS

This is a non-public security release for TYPO3 10.4 ELTS, containing:

!!![SECURITY] Update to Apache Solr 9.8.0 : CVE-2025-24814

Updates EXT:solr to Apache Solr 9.8.0.

Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default, which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality. All Apache Solr libs, modules or plugins must be configured within the main server configuration files. See: https://issues.apache.org/jira/browse/SOLR-16781

Impact:

Docker

You can wipe the volume and start the container with v. 11.2.5+ image, but that method will wipe the index as well.

See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh

Other server setups

You have 2 possibilities to fix that issue in your Apache Solr Server:

(PREFERRED) Migrate the EXT:solr's Apache Solr configuration

Refer to https://github.com/TYPO3-Solr/ext-solr/pull/4290/files .

Following 3 files are relevant:

  • Changes in <Apache-Solr data dir>/configsets/ext_solr_11_2_0_elts/conf/solrconfig.xml
  • Changes in <Apache-Solr data dir>/solr.xml
  • Movement from <Apache-Solr data dir>/configsets/ext_solr_11_2_0_elts/typo3lib/solr-typo3-plugin-6.0.0.jar

    • to <Apache-Solr data dir>/typo3lib/solr-typo3-plugin-6.0.0.jar

Steps:

  1. Remove all occurrences of <lib dir=".* from <Apache-Solr data dir>/configsets/ext_solr_11_2_0_elts/conf/solrconfig.xml file.
  2. Replace in <Apache-Solr data dir>/solr.xml file the snipped

    <str name="modules">scripting</str>
    Copied!

    by

    <str name="modules">scripting,analytics,analysis-extras,langid,clustering,extraction,${solr.modules:}</str>
    <str name="allowPaths">${solr.allowPaths:}</str>
    <str name="allowUrls">${solr.allowUrls:}</str>
    
    <!-- TYPO3 Plugins -->
    <str name="sharedLib">typo3lib/</str>
    Copied!
  3. Move the directory from <Apache-Solr data dir>/configsets/ext_solr_11_2_0_elts/typo3lib

    • to <Apache-Solr data dir>/typo3lib

!!![FIX] Docker execution order issue for as-sudo tweaks

This change renames the file

  • from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
  • to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh

and moves the folder

  • from /docker-entrypoint-initdb.d/as-sudo/
  • to /docker-entrypoint-initdb.d-as-sudo/

to fix the execution order issue when setting the correct file permissions when starting the docker container, leading to a Operation not permitted errors.

More details see:

Impact:

This change requires adjustments in your Docker setup, only if you modified:

  • files in folder /docker-entrypoint-initdb.d/as-sudo/
  • file /docker-entrypoint-initdb.d/as-sudo-tweaks.sh.
Make sure to use:
  • /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh instead of

    • /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
  • /docker-entrypoint-initdb.d-as-sudo instead of

    • /docker-entrypoint-initdb.d/as-sudo/

Upgrade to Apache Solr 9.7.0

This release requires Apache Solr v9.7.0+.

Along with the compatibility to Solr 9.7 the dependency to SOLR_ENABLE_STREAM_BODY is removed.

Minor changes & bugfixes

  • [TASK] Update GitHub actions by @dkd-friedrich in #42
  • [DOCS] Improve Solr core creation via API and other deployment parts by @dkd-kaehm & @dkd-friedrich in #42
  • [TASK] Use relative path to typo3lib in Apache Solr config by @dkd-kaehm & @dkd-friedrich in #42
  • [BUGFIX] Docker twaks as-sudo do not preserve the Docker image ENV by @dkd-kaehm & @dkd-friedrich in #42
  • [BUGFIX] Docker tests suite does not contain all logs by @dkd-kaehm & @dkd-friedrich in #42
  • [BUGFIX] docker image tests do not fail if core can not start by @dkd-kaehm & @dkd-friedrich in #42

Release 11.2.4 ELTS

Support of Apache Solr 9.5.0

Compatibility with Apache Solr 9.5.0 is checked and EXT:solr now accepts the following Apache Solr versions:

  • 9.3.0
  • 9.4.0
  • 9.4.1
  • 9.5.0

Small improvements and bugfixes

  • [BUGFIX] Handle float values in options facet parser by @dkd-kaehm in #11
  • [BUGFIX:11.2] Exception with tx_solr_statistics after latest TYPO3 security update by @dkd-kaehm in #14
  • !!![TASK] Update to Apache Solr 9.2 for TYPO3 10.4 ELTS by @dkd-kaehm in #19
  • [TASK] Remove unused field cookie in tx_solr_statistics by @dkd-kaehm in #21
  • [BUGFIX] Fix autosuggest with non-ascii terms by @dkd-friedrich in #28
  • [TASK] Prepare release-11.2.x ELTS by @dkd-kaehm in #17
  • [BUGFIX] Fix branch alias by @dkd-friedrich in #30
  • [TASK] Allow custom ELTS repository by @dkd-friedrich in #30
  • [BUGFIX:P:11.2] prevent undefined array key warning if filter is empty by Achim Fritz in #32
  • [TASK] Allow Apache Solr 9.5 by @dkd-friedrich in #31

Release 11.2.3 - Last non ELTS release

This is a maintenance release for TYPO3 10.4 and the last non ELTS release, as TYPO3 10 LTS reaches the ELTS phase on April 30, 2023.

EXT:solr release-11.2.x will not be maintained in TYPO3-Solr/ext-solr repository any more. The maintenance and builds will be moved to a private repository and ELTS versions, EXT:solr 11.2.4+ for TYPO3 10 ELTS versions, can be obtained through the dkd EB program.

This release contains:

  • [BUGFIX:P:11.2] make CE search form in backend editable again by @dkd-kaehm in #3640
  • [DOC] Fix wrong type for boostQuery in the docs and example by @rr-it and @dkd-kaehm in a997a2f4

Release 11.2.2

This is a maintenance release for TYPO3 10.4.

EXT:solr release-11.2.x will not be maintained in TYPO3-Solr/ext-solr repository any more. The maintenance and builds will be moved to a private repository and ELTS versions, EXT:solr 11.2.3+ for TYPO3 10 ELTS versions, can be obtained through the dkd EB program.

This release contains:

  • [BUGFIX] Type-hinting for SiteUtility::getConnectionProperty() by @dkd-kaehm in #3395
  • [TASK:Security:P:11.2] Update jQuery and its plugin libs by @dkd-kaehm in #3430
  • [FEATURE] Add signal before search in resultsAction by @stat1x in #3391
  • [BUGFIX:BP:11.2] Don't use minimum-stability dev on TYPO3 stable in build/CI by @dkd-kaehm in #3466
  • [TASK] Docker version check on docker image build by @dkd-kaehm in #3524
  • [BUGFIX:P:11.2] Use ConfigurationManager to get typscript in plugin FlexForm by @dkd-kaehm in #3497
  • [TASK] Use PCOV for coverage instead of xDebug :: Upgrade to phpunit 8 by @dkd-kaehm in #3528
  • [BUGFIX:BP:11.2] Make API eID script compatible with TYPO3 v11.5 by @dkd-friedrich in #3550
  • [BUGFIX:BP:11.2] Use plugin namespace as label for flexforms by @dkd-friedrich in #3553
  • [BUGFIX:BP:11.2] Respect indexingPriority in QueueItemRepository by @dkd-friedrich in #3557
  • [BUGFIX:BP:11.2] use siteLanguage TypoScript object to get language id by @dkd-friedrich in #3555
  • [TASK:11.2] equalize CI/Github-Actions for release-11.0|2|5 and main by @dkd-kaehm in #3467
  • [BUGFIX:BP:11.2] Sanitize frequent and last searches output by @dkd-friedrich in #3592
  • [BUGFIX:BP:11.2] changed from always picking the 0 array value to pic… by @dkd-friedrich in #3594
  • [BUGFIX] Enforce visibility context in Tsfe by @saitho in #3050
  • [TASK:BP:11.2] Disable sql handler by @dkd-friedrich in #3604

Release 11.2.1

This is a maintenance release for TYPO3 10.4, containing:

  • [BUGFIX] Prepend wrong fe language on empty cache (#3375)
  • [TASK] Use TYPO3 IpAnonymizationUtility (#3262)
  • [BUGFIX:BP:11.2] Shortcircuit work in SolrRoutingMiddleware (#3202)
  • [TASK] Fix scrutinizer on release-11.2.x
  • [BUGFIX:BP:11.2] Do not handle page updates on new page with uid 0 (#3338)
  • [TASK] Remove not used strptime() adaption for windows.
  • [BUGFIX] Respect indexing configuration for new and updated subpages (#3276)
  • [BUGFIX] Ensure keywords string does not exceed database field length (#3321)
  • [TASK:BP:11.2] Adapt column arrangement within sites config (#3295)
  • [DOCS:BP:11.2] add missing doc for ..pageIndexed
  • [BUGFIX] Fix PSR-4 Namespaces and Paths (#3285)
  • [BUGFIX] Silence DebugWriter for PageIndexerRequest (#3030)
  • [BUGFIX] AbstractSolrTask::setRootPageId(): Argument #1 must be of type int, string given (#3267)
  • [BUGFIX:11.2] Update GarbageCollector.php (#3249)
  • [DOCS] Align with new TYPO3 documentation standards (#3242)
  • [DOCS] Align README.md with other extensions (#3218)
  • [BUGFIX] Missing dot in configuration in numberOfResultsPerGroup method (#3098
  • [TASK] Added info about using page content in fields
  • [TASK] Added info about the virtual field __solr_contents
  • [BUGFIX:BP:11.2] Fix write connection (#2916)

Release 11.2.0

We are happy to release EXT:solr 11.2.0. The focus of this release has been on supporting the latest Apache Solr version 8.11.1 and on optimizing the data update monitoring.

Apache Solr 8.11.1 support

With EXT:solr 11.2.0 we support Apache Solr 8.11.1, the latest release of Apache Solr.

To see what has changed in Apache Solr please read the release notes of Apache Solr: https://solr.apache.org/docs/8_11_1/changes/Changes.html

Improved data update monitoring and handling

To ensure the Solr index is up-to-date an extensive monitoring is done, in huge sites this may slow down the TYPO3 backend, as many records and pages have to be checked and updated. With EXT:solr 11.2 you can configure how EXT:solr will monitor and handle data updates, by default EXT:solr acts as in all former versions, but you can now configure a delayed update handling or even turn the monitoring off.

Small improvements and bugfixes

Beside the major changes we did several small improvements and bugfixes:

Contributors

Special thanks to ACO Ahlmann SE & Co. KG for sponsoring the improved data update handling, #3153!

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Georg Ringer
  • Lars Tode
  • Mario Lubenka
  • Markus Friedrich
  • Marc Bastian Heinrichs
  • Michael Wagner
  • Rafael Kähm

Also a big thank you to our partners who have already concluded one of our new development participation packages such as Apache Solr EB for TYPO3 11 LTS (Feature), Apache Solr EB for TYPO3 10 LTS (Maintenance) or Apache Solr EB for TYPO3 9 ELTS (Extended):

  • ACO Ahlmann SE & Co. KG
  • avenit AG
  • b13 GmbH
  • cyperfection GmbH
  • in2code GmbH
  • Leitgab Gernot
  • medien.de mde GmbH
  • TOUMORØ
  • WIND Internet

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us by becoming an EB partner:

https://shop.dkd.de/Produkte/Apache-Solr-fuer-TYPO3/

or call:

+49 (0)69 - 2475218 0

Releases 11.5

Release 11.5.5

This is a maintenance release for TYPO3 11.5 and the last release that supports Apache Solr 8.11. Next EXT:solr release for TYPO3 11.5 will be 11.6.0, it will contain support for Apache Solr 9 and some breaking improvements.

Note: This change requires the database schema update, due of database schema change from pull-request #3880

11.5.5 contains the following changes:

  • !!![BUGFIX:11.5] Exception with tx_solr_statistics after latest TYPO3 security update by @dkd-kaehm in #3880

Release 11.5.4

11.5.4 contains the following changes:

  • [TASK] Fix CI 2023.09.11 on release-11.5.x by @dkd-kaehm in #3777
  • [BUGFIX:BP:11.5] Fix EXT:solr route enhancer by @dkd-friedrich in #3743
  • [BUG] Fix detection of "draft records" in workspaces by @baschny in #3642
  • [BUGFIX] Do not index translations on default language in languages free mode by @dkd-kaehm in #3786
  • [BUGFIX:BP:11.5] Retry Uri Building after exception by @dkd-friedrich in #3789
  • [BUGFIX] Delete index documents without available site by @dkd-kaehm in #3778
  • [TASK:BP:11.5] Ensure recursive page update on page movement by @dkd-friedrich in #3771
  • [FEATURE:BP:11.5] Add index queue indices by @dkd-friedrich in #3791
  • [TASK:BP:11.5] Migrate top.fsMod by @dkd-friedrich in #3795
  • [BUGFIX:BP:11.5] Return value getPageItemChangedTime() must be of the type int by @dkd-kaehm in #3813
  • [TASK:BP:11.5] Remove duplicate withHeader() by @dkd-kaehm in #3817
  • [BUGFIX:BP:11.5] Do not list cores twice in Index Inspector by @dkd-kaehm in #3818
  • [BUGFIX] Fixes multiple sortings by @BastiLu in #3762
  • [BUGFIX:BP:11.5] Fix missing frontend.typoscript request attribute while indexing by @dkd-kaehm in #3822
  • [BUGFIX] Prevent indexing error on missing 'foreignLabelField' by @kitzberger in #3740
  • [BUGFIX:BP:11.5] Force score to float by @dkd-kaehm in #3824
  • [BUGFIX:BP:11.5] Fix possible notice by @dkd-kaehm in #3825
  • [DOC:BP:11.5] Add FAQ how to generate URLs to restricted pages by @dkd-kaehm in #3826
  • [BUGFIX:BP:11.5] Handle float values in options facet parser by @dkd-kaehm in #3827
  • [BUGFIX:BP:11.5] handle localizations with un-available tsfe more gracefully by @dkd-kaehm in #3832
  • [TASK] Update the version matrix by @dkd-friedrich in #3860

Release 11.5.3

This is a maintenance release for TYPO3 11.5, containing:

  • [BUGFIX] make CE search form in backend editable again by @rr-it in #3626
  • [DOC] Fix wrong type for boostQuery in the docs and example by @dkd-kaehm in #3e7ff72
  • [TASK] Fix unit tests for 2023.06.07 by @dkd-kaehm in #3695

Release 11.5.2

This is a maintenance release for TYPO3 11.5, containing:

  • [BUGFIX:BP:11.5] Fix error when indexing pages with field processing instruction categoryUidToHierarchy by @dkd-kaehm in #3462
  • [BUGFIX:BP:11.5] Custom doktype is deleted from Solr after saving with custom queue configuration by @dkd-friedrich in #3468
  • [BUGFIX:BP:11.5] Don't use minimum-stability dev on TYPO3 stable in build/CI by @dkd-kaehm in #3464
  • [BUGFIX:BP:11.5] Fix value resolution in SOLR_RELATION by @dkd-friedrich in #3484
  • [FEATURE:P:11.5] Add new option manualSortOrderDelimiter for facets by @dkd-kaehm in #3494
  • [BUGFIX:P:11.5] Casting after check by @dkd-kaehm in #3495
  • [TASK] Docker version check on docker image build by @dkd-kaehm in #3525
  • [BUGFIX:P:11.5] Use ConfigurationManager to get typscript in plugin FlexForm by @dkd-kaehm in #3496
  • [BUGFIX:P:11.5] Exception on search plugin if no Solr connection is configured by @dkd-kaehm in #3498
  • [BUGFIX:BP:11.5] Fix handling of non existing pages on deletions by @dkd-friedrich in #3520
  • [TASK:BP:11] Verify the record before accessing the pid by @3l73 in #3537
  • [TASK:BP:11.5] Handle Solr connection exception by @dkd-friedrich in #3542
  • [BUGFIX:BP:11.5] Fix Solr response handling during indexing by @dkd-friedrich in #3544
  • [BUGFIX:BP:11.5] Use plugin namespace as label for flexforms by @dkd-friedrich in #3552
  • [BUGFIX:BP:11.5] Respect indexingPriority in QueueItemRepository by @dkd-friedrich in #3556
  • [BUGFIX:BP:11.5] add empty string as fallback by @dkd-friedrich in #3559
  • [BUGFIX:BP:11.5] use siteLanguage TypoScript object to get language id by @dkd-friedrich in #3554
  • [BUGFIX:BP:11.5] Sanitize frequent and last searches output by @dkd-friedrich in #3591
  • [BUGFIX:BP:11.5] changed from always picking the 0 array value to pic… by @dkd-friedrich in #3593
  • [BUGFIX:BP:11.5] Avoid getSolrConfiguration() on null by @dkd-friedrich in #3599
  • [TASK:BP:11.5] Disable sql handler by @dkd-friedrich in #3603
  • [BUGFIX:BP:11.5] Avoid PHP 8 warning when page indexing fails by @dkd-friedrich in #3609
  • [BUGFIX:BP:11.5] Ensure record exists before asserting if draft by @dkd-friedrich in #3610
  • [BUGFIX:BP:11.5] Fix usage of null coalescing operator by @dkd-friedrich in #3611
  • [BUGFIX:BP:11.5] return empty string for renderStatic if there is no content … by @dkd-friedrich in #3612

Release 11.5.1

We are happy to publish EXT:solr 11.5.1 maintenance release

New in this release

  • [BUGFIX] Do not include removed strptime() by @dkd-kaehm in #3335
  • [BUGFIX:BP:11.5] Do not handle page updates on new page with uid 0 by @rr-it in #3344
  • [BUGFIX:BP:11.5] Shortcircuit work in SolrRoutingMiddleware by @christophlehmann in #3341
  • !!![TASK] Use preAddModifyDocuments hook for pages by @christophlehmann in #3076
  • [BUGFIX] Fix array key access in ext_getSetup (Backport 11.5) by @saitho in #3361
  • [TASK:BP:11.5] Indexing configuration icon fallback by @dkd-friedrich in #3371
  • [BUGFIX:BP:11.5] Do not index missing fields by @dkd-friedrich in #3372
  • [TASK:BP:11.5] Introduce index queue type setting by @dkd-friedrich in #3370
  • [TASK:BP:11.5] Do not index language with unconfigured core by @christophlehmann in #3373
  • [BUGFIX] Make API eID script compatible with TYPO3 v11.5 by @peterkraume in #3350
  • [BUGFIX] Type-hinting for SiteUtility::getConnectionProperty() by @dkd-kaehm in #3396
  • [TASK:BP:11.5] Introduce generic EXT:solr exception by @dkd-friedrich in #3422
  • [BUGFIX:BP:11.5] Fix frontend Solr connection initialization by @dkd-friedrich in #3425
  • [ACTIONS:2022.12.22] Use fixed typo3/coding-standards 0.6.x < 0.7.0 for TYPO3 11.5 by @dkd-kaehm in #3429
  • [TASK:Security] Update jQuery and its plugin libs by @dkd-kaehm in #3428
  • [BUGFIX:P:11.5] Proper check for config.index_enable by @georgringer in #3433
  • [BUGFIX:P:11.5] Typecast $timestamp to int in TimestampToUtcIsoDate by @derhansen in #3434
  • [BUGFIX:P:11.5] prevent undefined array key warning if filter is empty by @achimfritz in #3435
  • [FEATURE] Add signal before search in resultsAction by @stat1x in #3392
  • [BUGFIX] Fix php warning undefined array key no_search_sub_entries by @DrWh0286 in #3381

Please read the release notes: https://github.com/TYPO3-Solr/ext-solr/releases/tag/11.5.1

Release 11.5.0

We are happy to release EXT:solr 11.5.0. The focus of this release has been on TYPO3 11 LTS compatibility.

#standwithukraine #nowar

New in this release

Support of TYPO3 11 LTS

With EXT:solr 11.5 we provide the support of TYPO3 11 LTS.

Please note that we require at least TYPO3 11.5.14, as this version contains some change concerning the usage of local TypoScriptFrontendController objects that are solving some issues during indexing.

Bootstrap 5.1

The default templates provided by EXT:solr were adapted for Bootstrap 5.1.

The templates are also prepared to display some icons with Bootstrap Icons, but the usage is optional and the icons are no longer provided with EXT:solr as the former Glyphicons were.

Custom field processors

fieldProcessingInstructions can be used for processing values during indexing, e.g. timestampToIsoDate or uppercase. Now you can register and use your own field processors via: $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['fieldProcessor']['yourFieldProcessor'] = ACustomFieldProcessor::class;

Custom processors have to implement interface ApacheSolrForTypo3SolrFieldProcessorFieldProcessor.

N-Gram Filter for strings

Provides a new field type and dynamic fields for strings with enabled Edge-N-Gram filter.

Now the following fields can be used: - *_stringEdgeNgramS - *_stringEdgeNgramM

Improve and Fix TSFE Initialization

The initialization of TSFE within indexing and Backends modules contexts is refactored.

In this change the setting and usage of $GLOBALS['TSFE'] is removed and replaced by TYPO3s Core Context API. The "Context" is always cloned instead of using its singleton instance. The "Context", "Language", "TSFE" and "ServerRequest", which are required for TypoScript parsing in BE-modules and indexing contexts, are highly isolated/capsuled and not visible anymore for all things not belonging to EXT:solr internals.

Byside of isolation/encapsulation of TSFE, the language handling is restored to pure and default EXT:solr "fallback" mode approach. So all page records, which are involved in indexing(All page types and above all all with sys_template or records-to-index), must be translated. Otherwise the translation records will be indexed in default language.

Note: Since TYPO3 11 LTS does not allow to instantiate TSFE for sys folders and spacer,
the initialization of TSFE will be done for first and closest page(not spacer or folder) within the site rootline.

Get "free content mode" working

In previous releases of EXT:solr the language handling for "free-content-mode" inconsistent. The behavior of "free-content-mode" related records varied in RecordMonitor, Initializing and Indexing contexts, which was the source of troubles for mixing overlay records in wrong cores/languages.

This change brings the RecordMonitor, Initializing and Indexing contexts for "free-content-mode" related records into the same line, so the "free-content-mode" records are processed the same way.

Make pageRangeFirst and pageRangeLast accessible in fluid

With these two additional getters it is possible to access the variables in fluid templates. See: #3254

Add custom field processors

Custom field processors can be registered with

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['fieldProcessor']['yourFieldProcessor'] = ACustomFieldProcessor::class;
Copied!

And many more

Please see the list of changes below or the full changelog:.

The list of all changes:

  • [TASK] Prepare schemas for EXT:solr 11.5.x c0a3e6205
  • [TASK] Provide N-Gram Filter for strings 13b90a996
  • [TASK] composer branch aliases ebfee76bb
  • [BUGFIX] Recursive constants 8af25d03f
  • [BUGFIX:BP:11.5] Follow up to recursive constants a57960763
  • [TASK] Migrate TYPO3#88366 deprecated cache_ prefix on caches a8f111592
  • [BUGFIX] Filter within route enhancers b6d77ee52
  • [BUGFIX] Fix NON-Composer mod libs composer.json for composer v2 e9ec5c11c
  • [TASK] Setup Dependabot to watch "solarium/solarium" dfc99f4b0
  • [TASK] Setup Github Actions :: Basics ceb892408
  • [TASK] Bump to and test against TYPO3 ^11.5 e7eeb2b3d
  • [TASK] Bump solarium to 6.1.4, which supports PHP 8.0 e56c32436
  • [TASK] Bump nimut/testing-framework to v. 6.0, which supports PHP 8.0 e5353ab3c
  • [FIX] Fix GH actions on branches push event 85e413d39
  • [BUGFIX:P:11.5] Don't use jQuery.ajaxSetup() 6714590a8
  • [TASK] Restructure version matrix 9535750f4
  • [Bugfix:BP:11-5] routeenhancer with empty filters 578e0153b
  • [TASK:11.5] Replace mirrors for Apache Solr binaries on install-solr.sh 7f998d221
  • [TASK] Make TYPO3 11 LTS compatible : Backend Modules Templates 871c5b00f
  • [TASK] Make TYPO3 11 LTS compatible : rector run 7e104a499
  • Make TYPO3 11 LTS compatible : TSFE initialization : record indexing 66f512b12
  • [TASK] Make collapse work in BE 800384e48
  • [TASK] Style index fields tab in info module 8f9a0ce9d
  • [TASK] Fix loading Chart module 8fd1182ac
  • [TASK] Adapt namespaces f1f5521b9
  • !!! [TASK] Switch to hook contentPostProc-cached of TypoScriptFrontendController e1c8c3afc
  • [TASK] Apply rectors 0e6bf902e
  • [BUGFIX] Enforce visibility context in Tsfe d50947375
  • [TASK] Fix scrutinizer for EXT:Solr 11.5 43dcbd43f
  • [TASK-11.5C] Fix - "Unit" Tests : Remove usages of UriBuilder::setUseCacheHash() d71ec451c
  • [TASK-11.5C] Fix - "Unit" Tests 5e047c520
  • [TASK-11.5C] Fix - "Unit" Tests : PHP 8.0 6023d78d7
  • [TASK] Let PHP 8.0 Job allow to fail temporarily d36c22e3e
  • TBD!!! [TASK-11.5C] Fix - "Integration" Tests 4005e974b
  • !!![TASK] Improve and Fix TSFE Initialization a246cb8e3
  • [TASK] Refactor IntegrationTest base class : auto import root pages d14b82ec5
  • [TASK] Refactor Integration tests : SiteHashServiceTest 280271d04
  • [TASK] Refactor Integration tests : ResultSetReconstitutionProcessorTest 1317a2792
  • [TASK] Refactor Integration tests : IndexerTest f87a5f5d7
  • [TASK] Refactor Integration tests : IndexerTest additionalPageIds 723ccea67
  • [TASK] Refactor Integration tests : IndexerTest "hide default language" 1538c61dc
  • [TASK] Refactor Integration tests : IndexerTest "Relation (MM) translation overlays" 82bfe55d4
  • [TASK] Reactivate tests for indexing records without L parameter 66afd4f59
  • [TASK] Refactor Integration tests : Schrink fixtures 25cf5b911
  • [BUGFIX] Remove hidden translated record in index 1b7642115
  • [FEATURE] Get "free content mode" working 0986a24c9
  • [BUGFIX] TypoScript configuration for "Hide default language" sites ddcbc3bb6
  • [TASK] Refactor pagination bb42410af
  • [TASK] Fix indentation, add more documentation 56922bdb4
  • [TASK] Fix Index-Queue module: "Clear Index Queue" functionality 1307974e9
  • [TASK] Disable temporary testing against TYPO3 < v11.5.4 9faf73fb6
  • [TASK] Refactor LastSearches and FrequentlySearched widgets b3a9fef4c
  • [TASK] Allow to publish (-PRE)-(ALPHA|BETA|RC) releases to TYPO3 TER 5cb71c168
  • [FIX] Allow to edit pages outside of site root 6c8801154
  • [FIX] Allow to mark pages as site root 09009909b
  • [FIX] Don't auto select first configured Solr site if non configured exists 352998671
  • [FIX] Can't create SchedulerTask 05ae55ec7
  • [TASK] Add Czech translation a3805b287
  • [Bugfix] Prevent unwanted filter parameters from being generated 3e156981d
  • !!![TASK] Refactor Site stack 5120a68b7
  • !!![FIX] Index Queue initialization is not robust enought bc7133237
  • [FIX] TypoScript in Tsfe::initializeTsfe() parsed twice aafc18de3
  • [FIX] Integration tests on release-11.5.x 210a64a88
  • [TASK] Upgrade to Apache Solr 8.11.1 b3ab72de1
  • [BUGFIX] Catch Throwables instead Exceptions a2988d2ff
  • [FEATURE] Fix #3143: improve variant handling by sorting user groups e38785eb8
  • [BUGFIX] Fix #3145: exception in scheduler with php 8 75b1237e0
  • [BUGFIX] Fix #3141: TypeError in TranslateViewHelper bc12bfafd
  • [BUGFIX] Fix autosuggest with non-ascii terms 6687bcd4f
  • Allow to generate indexing error log from throwable 4abdba3f3
  • [FIX] Can't index pages which require a user session 2e35a8c05
  • [CLEANUP] Remove unused "Initialize Solr connections" code bc03310cc
  • [TASK] Make FE/Search tests working 14c45a210
  • [TASK] Remove IntegrationTest::importDumpFromFixture() method b7e4c6f59
  • [TASK] make scrutinizer ocular working on PHP 8+ e58050fb4
  • [FIX] Call to undefined method ResponseFactory::createJsonResponse() 6b65feccb
  • [TASK] Fix Scrutinizer issues d40bcd67a
  • [BUGFIX] Prevent some "undefined array key" warnings with php 8 5a4ef9038
  • [BUGFIX] TER releases missing composer dependencies be3eafc0d
  • [TASK] unite all intgeration tests in same suite a227fe7f9
  • [TASK] Test TYPO3 11+ with PHP 8.1 as well 4be1ccc5f
  • [TASK:11.5] Upgrade solarium/solarium to 6.6.2 efe7c5614
  • [WIP] PHP 8.1 compatibility 15c1221e5
  • [BUGFIX] Fix notice in TranslateViewHelper 3b91901e6
  • [TASK] Avoid different Solarium versions in non- and composer modes 4091c6261
  • [P:11.5:FEATURE] Improve data update handling 6561e3585
  • [TASK] Add proper annotations on GH actions job failures. f145285e2
  • [TASK] Migrate to PhpUnit 9+ Api and cleanup the obsolete method mocks cc8cc7885
  • [BUGFIX] Fix write connection 9a16a743d
  • [BUGFIX] core optimization module PHP 8.1 compatibility c81407540
  • [TASK] Remove not used strptime() adaption for windows. ad5c03932
  • [BUGFIX] Ensure BE_USER is kept when initializing TSFE c7c0ba8ad
  • [TASK:11.5] Minimal changes to Templates to make Bootstrap 5.1 working d5940d393
  • [TASK] Standardize *.php files header declaration 514717864
  • [TASK] Use and apply TYPO3 coding standards, rector and type hinting 61076e3ed
  • [BUGFIX] Skip rootline check in be for records stored at pid 0 6800394c0
  • [BUGFIX] Prevent "undefined array key" warnings with php 8 in page indexer d4afa18d1
  • FIX: Argument 1 passed to ApacheSolrForTypo3SolrTaskAbstractSolrTask::setRootPageId() must be of the type int, string given 2858e45aa
  • Added info about the virtual field __solr_contents 8002707ed
  • Added info about using page content in fields 85741400b
  • FIX: Argument 1 passed to ApacheSolrForTypo3SolrSystemUrlUrlHelper::setPort() must be of the type int, string given 9afe701ad
  • [TASK] Bootstrap 5.1 adaptions 0d6f62a30
  • [CLEANUP] Delete obsolete TypoScript example ConnectionFromConfVars cb5b5284a
  • [BUGFIX] Ensure proper items per page setting 84d70b1f1
  • [BUGFIX:11.5] Access restricted pages can not be indexed on TYPO3 11.5 74d316358
  • [BUGFUX] Fix #3221: exception in page browser 094e70fa6
  • [TASK] Follow-up changes by EXT:solrfal for TYPO3 11.5 41ac7ffd5
  • [TASK] Supress warnings of strftime fbf20c41d
  • [TASK] Remove unnecessary bootstrap_package 1582b646f
  • [TASK] Reenable skipped test of SearchControllerTest 1e0be7a51
  • [BUGFIX] Fix feuser initialisation in BE context 3ea33b4f8
  • [TASK] Improve error handling in index queue module cb0292d6f
  • [BUGFIX] Add type cast to TaskProviders ab070482e
  • [BUGFIX] Missing dot in configuration in numberOfResultsPerGroup method 59a49ba41
  • [DOCS] Align README.md with other extensions (#3218) 9b4a1153b
  • [DOCS] Align with new TYPO3 documentation standards (#3242) ec66f49e5
  • [TASK] Prevent type errors 061ef243a
  • [TASK] Allow SearchResultSetService instantiation via makeInstance b15f2444e
  • [TASK] Move ObjectManager to constructor in AbstractFacet 35405f349
  • [FEATURE] Make pageRangeFirst and pageRangeLast accessible in fluid 31ba843a1
  • [BUGFIX] Fix return type error for option facet 002661140
  • [BUGFIX] change detection of free mode records eb87e83ba
  • [BUGFIX] Avoid yoda-style conditions in PHP 48e52dbd0
  • [TASK] Sync with new TYPO3 coding standards b15838961
  • [TASK] Sync with EXT:solrfluidgrouping for TYPO3 11.5 1ef155471
  • Update GarbageCollector.php eab5887f1
  • [BUGFIX] AbstractSolrTask::setRootPageId(): Argument #1 () must be of type int, string given 506b540e4
  • Silence DebugWriter for PageIndexerRequest 56203dfa0
  • [BUGFIX] Undefined array key in ..DomainSiteSite:L130 8e1d5ed0e
  • [BUGFIX] Fix PSR-4 Namesppaces and Paths 49a797884
  • [BUGFIX] Ensure array value is set when accessing 3fa4ff496
  • [BUGFIX:11.5] Frequent Searches plugin does not work 49b32a195
  • [BUGFIX] Class properties must not be accessed before initialization 5a9556488
  • [BUGFIX] Respect indexing configuration for new and updated subpages 6196913be
  • [BUGFIX:BP:11.5] Empty suggest query triggers a PHP error f564a31b9
  • [TASK:BP:11.5] Adjust typo3/coding-standards settings c0b0e1a6f
  • [DOCS:BP:11.5] add missing doc for plugin.tx_solr.logging.indexing.pageIndexed e309f0f9f
  • [TASK:BP:11.5] Require TYPO3 11.5.14 b698f86e9
  • [TASK:BP:11.5] Adapt column arrangement within sites config bd628be99
  • [FEATURE:BP:11.5] Add custom field processors 173c7a5d4
  • [TASK:11.5] Fix TYPO3 coding standards issues after upgrade to v0.5.5 55830f209
  • Ensure keywords string does not exceed database field length 9f2c81768
  • [BUG] make sure that $currentPageNumber in resultsAction is always >= 1 (#3324) be8cc90b6
  • [FEATURE] add logging for failed http requests f9edd8bc4
  • [BUGFIX] fix infinite loop in Tsfe::getPidToUseForTsfeInitialization() 3a2b8d0e8

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Achim Fritz
  • Andreas Beutel
  • Andreas Kießling
  • ayacoo
  • Christoph Lehmann
  • Christopher Schnell
  • Daniel Koether
  • dev-rke
  • Dmitry Dulepov
  • dsone
  • FearFreddy
  • Georg Ringer
  • garfieldius
  • Guido Schmechel
  • Henrik Elsner
  • Jan Delius
  • Jens Jacobsen
  • Lars Tode
  • leslawp
  • Marc Bastian Heinrichs
  • Mario Lubenka
  • Marcus Balasch
  • Marcus Schwemer
  • Markus Friedrich
  • Markus Kobligk
  • Michael Kettel
  • Michael Wagner
  • Michiel Roos
  • Nicola Widmer
  • Pascal Hofmair
  • Peter, CyberForum e.V
  • Peter Kraume
  • Philipp Kitzberger
  • Rafael Kähm
  • René Maas
  • Rudy Gnodde
  • rr-it
  • Sascha Egerer
  • Sebastian Hofer
  • Sebastian Michaelsen
  • Soren Malling
  • stat1x
  • Stefan Frömken
  • Stefano Kowalke
  • twojtylak
  • Thomas Löffler
  • Tobias Kretschmann
  • Tobias Schmidt
  • Torben Hansen

Also a big thank you to our partners who have already concluded one of our new development participation packages such as Apache Solr EB for TYPO3 11 LTS (Feature), Apache Solr EB for TYPO3 10 LTS (Maintenance) or Apache Solr EB for TYPO3 9 ELTS (Extended):

  • .hausformat GmbH
  • ACO Ahlmann SE & Co. KG
  • AgenturWebfox GmbH
  • Amedick & Sommer Neue Medien GmbH
  • avenit AG
  • b13 GmbH
  • Bytebetrieb GmbH & Co. KG
  • Cobytes B.V.
  • Connetation Web Engineering GmbH
  • cosmoblonde GmbH
  • creativ clicks GmbH
  • cyperfection GmbH
  • DVT - Daten-Verarbeitung-Tirol GmbH
  • Earlybird GmbH & Co KG
  • elancer-team GmbH
  • eulenblick Kommunikation und Werbung
  • FONDA GmbH
  • GFE Media GmbH
  • graphodata GmbH
  • Hirsch & Wölfl GmbH
  • Hochschule Niederrhein
  • i-fabrik GmbH
  • in2code GmbH
  • internezzo ag
  • Intersim AG
  • IW Medien GmbH
  • Jochen Weiland
  • Kassenärztliche Vereinigung Rheinland-Pfalz
  • Kreis Euskirchen
  • Landeskriminalamt Thüringen
  • L.N. Schaffrath DigitalMedien GmbH
  • Leitgab Gernot
  • LOUIS INTERNET GmbH
  • Marketing Factory Consulting GmbH
  • medien.de mde GmbH
  • MEDIA::ESSENZ
  • mehrwert intermediale kommunikation GmbH
  • Neue Medien GmbH
  • NEW.EGO GmbH
  • novotegra GmbH
  • Overlap GmbH & Co KG
  • Pädagogische Hochschule Karlsruhe
  • peytz.dk
  • ProPotsdam GmbH
  • Proud Nerds
  • Provitex GmbH
  • queo GmbH
  • Québec.ca
  • rms. relationship marketing solutions GmbH
  • Sandstein Neue Medien GmbH
  • Schoene neue kinder GmbH
  • seam media group gmbh
  • SITE'NGO
  • Snowflake Productions GmbH
  • SOS Software Service GmbH
  • Stämpfli AG
  • Studio 9 GmbH
  • systime.dk
  • techniConcept Sàrl
  • TOUMORØ
  • Typoheads GmbH
  • UEBERBIT GmbH
  • visol digitale Dienstleistungen GmbH
  • WACON Internet GmbH
  • we.byte GmbH
  • wegewerk GmbH
  • werkraum Digitalmanufaktur GmbH
  • WIND Internet
  • zimmer7 GmbH

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us by becoming an EB partner:

https://shop.dkd.de/Produkte/Apache-Solr-fuer-TYPO3/

or call:

+49 (0)69 - 2475218 0

Releases 11.1

Release 11.1.0

We are happy to release EXT:solr 11.1.0. The focus of this release has been on URL and SEO optimizations.

New in this release

Route enhancers

Introduce the TYPO3s route enhancer functionality for facets. This feature allows to mask facets inside the query string or as part of the path segment.

See:

Associative keys for tx_solr[filter] facet URL parameters

Introduce a new style how the facet array represented inside of the url with a combination of key and value in order to be able to keep a specific order.

This feature allows to change the url parameters from index based to associative keys for facets. Using associative keys, the value of a facet will be replaced with 1. A value of 1 means, that the facet is active. A value of 0 means, the value is inactive.

Additionaly a new sort option for url parameters is available. The sort of parameters is mandatory for associative keys.

By default Solr behaves as before and will be changed in future releases.

See: https://github.com/TYPO3-Solr/ext-solr/pull/2705

ASCII and Scandinavian Folding Filter

To improve the search behaviour we introduce folding filters, e.g. allowing to skip accents in search terms. The following languages are now using the ASCII folding filter:

  • dutch
  • english
  • finish
  • french
  • german
  • hungarian
  • irish
  • italian
  • polish
  • portuguese
  • serbian (for fields that don't include the Serbian Normalization Filter)
  • spanish
  • turkish

For the Scandinavian languages, Norwegian, Swedish and Danish, a similiar approach is used, but we're using the more specialized Scandinavian Normalization and Scandinavian Folding Filters.

Folding process usally takes place at a late stage, so your configurations shouldn't be affected. But for the Scandinavian languages the Scandinavian Normalization Filter processes the terms earlier, so your protected words for the Snowball Porter Filter, e.g. danish/protwords.txt, might be affected, please be sure to use the right spelling (see https://solr.apache.org/guide/8_8/language-analysis.html#scandinavian-normalization-filter).

See: https://github.com/TYPO3-Solr/ext-solr/pull/2963

cHash configuration

EXT:solrs components like range facets can not be properly handled by cHash stack, because the amount of possible range-combinations is infinite, therefore they must be excluded from cHash calculation.

This change makes it possible to exclude all EXT:solr parameters from cache hash. To prevent misconfigurations, the new extension configuration setting "pluginNamespaces" was introduced, which is used in FlexForm and in TYPO3_CONF_VARS/FE/cacheHash/excludedParameters. This setting makes it impossible to chose invalid/unhandled EXT:solr plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

Please follow the following migration instructions

Plugin namespaces: Needed only if other as default (tx_solr) plugin namespace is used in instance. Add the used namespace[s] to $GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A list of white listed plugin namespaces"

Global q parameter: Needed only if global "q" parameter without plugin namespace is used and wants to be included in cache hash calculation. Set the setting $GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] = '1' or enable it via backend "Settings" -> "Extension Configuration" -> "solr" -> "Include/Exclude global q parameter in/from cacheHash"

See:

Scheduler task to optimize Solr cores

This task allows you to optimize the indexes of given cores for a site at a planned time.

See:

Apache Solr 8.9.0 support

With EXT:solr 11.1 we support Apache Solr 8.9.0, the latest release of Apache Solr.

To see what has changed in Apache Solr please read the release notes of Apache Solr: https://solr.apache.org/docs/8_9_0/changes/Changes.html

Map managed resources to core-name

Makes it possible to manage resources per core instead of previously used per schema approach. Now is it possible to maintain the stopwords and synonyms for each core/site separately, and avoid mixing the synonyms between sites using same core.

See: https://github.com/TYPO3-Solr/ext-solr/issues/2635 https://github.com/TYPO3-Solr/ext-solr/pull/2794 * https://github.com/TYPO3-Solr/ext-solr/commit/fde8a64be3de538339e1608fbe44a8160ab9f023

Update to Solarium 6

Solarium is upgraded from version 4 to version 6, so due to changes in Solarium various classes and data types had to be adapted.

There are two major changes you have to consider while upgrading:

  • TypoScript option plugin.tx_solr.solr.timeout is dropped, settings for HTTP client $GLOBALS['TYPO3_CONF_VARS']['HTTP'] are now taken into account
  • Solr path mustn't be prepended with "/solr/", refer to the "Getting Started > Configure Extension" section in our manual

Drop TYPO3 9 compatibility

To simplify the development we've dropped the compatibility for TYPO3 9 LTS. If you need to use TYPO3 9 please use the 11.0.x branch.

Small improvements and bugfixes

Beside the major changes we did several small improvements and bugfixes:

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Achim Fritz
  • Benni Mack
  • Christoph Lehmann
  • Daniel Koether
  • Daniel Siepmann
  • Dmitry Dulepov
  • @dev-rke
  • @dsone
  • @FearFreddy
  • @Figilano
  • @frommo
  • Georg Ringer
  • Jens Jacobsen
  • Lars Tode
  • @leslawp
  • Marc Bastian Heinrichs
  • Markus Friedrich
  • Markus Kobligk
  • Michiel Roos
  • Peter, CyberForum e.V
  • Philipp Kitzberger
  • Rafael Kähm
  • Ruud Silvrants
  • Sascha Egerer
  • Sebastian Hofer
  • Sebastian Michaelsen
  • Stefano Kowalke
  • Stephan Brun
  • Sybille Peters
  • Timo Hund
  • Tobias Kretschmann

Also a big thanks to our partners that have joined the EB2021 program:

  • +Pluswerk AG
  • 711media websolutions GmbH
  • Abt Sportsline GmbH
  • ACO Severin Ahlmann GmbH & Co. KG
  • AVM Computersysteme Vertriebs GmbH
  • cosmoblonde GmbH
  • creativ clicks GmbH
  • cron IT GmbH
  • CS2 AG
  • CW Media & Systems
  • Earlybird GmbH & Co KG
  • Earlybird GmbH & Co KG
  • FLOWSITE GmbH
  • form4 GmbH & Co. KG
  • Getdesigned GmbH
  • Granpasso Digital Strategy GmbH
  • Ikanos GmbH
  • internezzo ag
  • Intersim AG
  • Ion2s GmbH
  • Leitgab Gernot
  • mellowmessage GmbH
  • Moselwal Digitalagentur UG (haftungsbeschränkt)
  • network.publishing Möller-Westbunk GmbH
  • OST Ostschweizer Fachhochschule
  • Plan.Net Suisse AG
  • Provitex GmbH
  • punkt.de GmbH
  • queo GmbH
  • Rechnungshof
  • Schoene neue kinder GmbH
  • SIT GmbH
  • SIZ GmbH
  • Stämpfli AG
  • Triplesense Reply Frankfurt
  • TWT reality bytes GmbH
  • visol digitale Dienstleistungen GmbH
  • Web Commerce GmbH
  • webconsulting business services gmbh
  • webschuppen GmbH
  • Webstobe GmbH
  • Webtech AG
  • wow! solution
  • XIMA MEDIA GmbH

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us by becoming an EB partner:

https://shop.dkd.de/Produkte/Apache-Solr-fuer-TYPO3/

or call:

+49 (0)69 - 2475218 0

Releases 11.0

Release 11.0.9 ELTS

We are happy to release EXT:solr 11.0.9 maintenance release, this release is a non-public ELTS release.

Support of Apache Solr 9.5.0

Compatibility with Apache Solr 9.5.0 is checked and EXT:solr now accepts the following Apache Solr versions:

  • 9.2.0
  • 9.2.1
  • 9.3.0
  • 9.4.0
  • 9.4.1
  • 9.5.0

To avoid breaking changes EXT:solr 11.0.9 doesn't require Solr 9.5.0, but it's strongly recommended to use the latest supported version.

Note that due to CVE-2023-50290 you should at least check your configuration or update to at least Apache Solr 9.3.0.

Small improvements and bugfixes

  • [TASK:BP:11.0] Prepend wrong fe language on empty cache by @goldi42
  • [BUGFIX:BP:11.0] Fix connection initialization by @dkd-friedrich in #10
  • [BUGFIX] Handle float values in options facet parser by @dkd-kaehm in #12
  • [BUGFIX:11.0] Exception with tx_solr_statistics after latest TYPO3 security update by @dkd-kaehm in #13
  • [TASK] Remove unused field cookie in tx_solr_statistics by @dkd-kaehm in #22
  • [TASK] Allow Apache Solr 9.4 by @dkd-friedrich in #25
  • [BUGFIX] Fix result highlighting fragment size by @dkd-friedrich in #26
  • [TASK] Allow Apache Solr 9.5 by @dkd-friedrich in #27

Release 11.0.8 ELTS

We are happy to release EXT:solr 11.0.8 maintenance release, this release is a non-public ELTS release.

Support of Apache Solr 9.2.0

Apache Solr 8.5.1 is outdated, thus we now provide support of Apache Solr 9.2. Technically an update is not required, but recommended to avoid possible security issues in no longer supported Apache Solr versions. If you can't update your Apache Solr server right now, EXT:solr 11.0.8 should still work with 8.5, but you'll see warnings in the reports module.

Note: With Apache Solr 9 the following components are no longer available and you have to adapt the configuration if needed. No longer available components are:

1) Data Import Handler (DIH) DIH is an independent project now; it is no longer a part of Solr

2) VelocityResponseWriter VelocityResponseWriter is an independent project now; it is no longer a part of Solr. This encompasses all previously included /browse and wt=velocity examples.

Small improvements and bugfixes

  • [TASK:Security:P:ELTS_9.5] Update jQuery and its plugin libs by @dkd-kaehm in #3
  • [BUGFIX:BP:11.0] Respect indexingPriority in QueueItemRepository by @dkd-friedrich in #5
  • [TASK] Integrate in packagist.com : conflict with non ELTS EXTsolr by @dkd-kaehm in #4
  • [BUGFIX:BP:11.0] Sanitize frequent and last searches output by @dkd-friedrich in #6
  • [BUGFIX] Fix TypeError in StatisticsWriterProcessor by @dkd-friedrich in #6
  • [TASK] Remove unneeded GitHub actions by @dkd-friedrich in #6
  • [TASK] Remove unneeded TYPO3 version from matrix by @dkd-friedrich in #6
  • [BUGFIX] Try to solve conflicts issue on composer on CI by @dkd-kaehm in #7
  • [TASK] Prepare configsets for 11.0 ELTS by @dkd-friedrich in #8
  • [BUGFIX:BP:11.0] Fix expected variant results by @dkd-friedrich in #8
  • [TASK] Update version matrix by @dkd-friedrich in #8

Release 11.0.7 - Last non ELTS release

TYPO3 9 LTS reached the ELTS stage: free community support for TYPO3 9 LTS ended on 30 sept. 2021. We'll join the TYPO3s ELTS regiment and provide EXT:solr support for TYPO3 9 ELTS upwardly via our EB program. Therefore the EXT:solr release-11.0.x will not be maintained in TYPO3-Solr/ext-solr repository any more. The maintenance and builds will be moved to other place. The new EXT:solr 11.0.8+ for TYPO3 9 ELTS versions will be provided via dkds EB program.

Release 11.0.6

This is a bugfix-only release that contains only bugfixes

This is a bugfix-only release that contains:

  • [BUGFIX:BP:11-0] Respect TCA setting of 'tstamp' field (#3037)
  • [BUGFIX:BP:11.0] Update SolrNotAvailable.html (#3020)
  • [BUGFIX] Recursive constants (#3048)
  • [BUGFIX] Follow up to recursive constants (#3058)
  • [BUGFIX:BP:11.0] Don't use jQuery.ajaxSetup (#2503)
  • [TASK:11.0] Replace mirrors for Apache Solr binaries on install-solr.sh (#3094)

Release 11.0.5

This is a bugfix-only release that contains only bugfixes

  • [TASK] 2021.12.13 Rebuild Docker images due of(CVE-2021-44228)
  • [BUGFIX] Delete documents for valid connections only b99d7ad (#2940)
  • [TASK] Make Apache Solr v8.6+ compatible 12b9483 (#2938)
  • [TASK] Bump Chart.js to v2.9.4 4eacf89 (#2946)
  • [BUGFIX] use pages configuration as default #issue-2742 d05f77e (#2742)
  • [BUGFIX] ENV vars not handled correctly in site management module d83c7d1 (#2576)
  • [BUGFIX] Delete synonyms with URL special chars 5905fdb (#2336)
  • [BUGFIX] Fix typo in CoreOptimizationModule/Index.html 3163d25 (#2965)
  • [BUGFIX] Use correct html tags in templates 31e2d2c (#2970)
  • [BUGFIX] Add missing applicationType to faked request 19baedc (#2932)
  • [TASK] Add language cache to SiteUtility 6f7e4d1 (#2908)
  • [BUGFIX] writing errorHandling of site configuration 1ff6ca3 (#2913)
  • [TASK] Make language cache work with multi site setups b7a39c1 (#2986)
  • [BUGFIX] set base uri to face frontend request 2c34ae9 (#2914)
  • [BUGFIX] getRangeString(): check type before format() - call a99275a (#2942)
  • [BUGFIX] Fix type error in UrlHelper 17f1653 (#2756)

Release 11.0.4

This is a bugfix-only release that contains

  • [BUGFIX] Removes secondary parameter c6a9dcc (#2746)
  • [DOCS] Use *_PORT variable for setting the port fca6f68 (#2759)
  • [BUGFIX] Correct Content-Type header for suggest response e843b44 (#2783)
  • [TASK] Change configuration files to TYPO3 file extensions 6d513e6 (#2813)
  • [BUGFIX] garbage collector on translations 9adcc40 (#2797)
  • [BUGFIX] Quote field within score calculation 3969340 (#2824)
  • [BUGFIX] disabled Solr Sites e7bc3ab (#2795)
  • [BUGFIX] Enable unicode when fetching pages eb33376 (#2810)
  • [TASK] Disable cache time information for ajax request f54213f (#2834)
  • [TASK] Adjust composer TYPO3 version constrains for EXT:Solr 11.0.4+ 50df86a (#2844)
  • [FEATURE] Allow stdWrap on sorting label 5f2cee2 (#2339)
  • [BUGFIX] Fix handling of case sensitive variant ids (#2865)
  • [FEATURE] Store number of existing variants 9c88401 (#2870)
  • [BUGFIX] Function call with non existing variable 0a69d45 (#2842 / #2520)
  • [BUGFIX:BACKPORT:11] Fix missing variant field value 8e0c648 (#2878)
  • [BUGFIX] Exception on Cached state of TranslateViewHelper 1765751 (#2830)
  • [BUGFIX] Check if $recordUid is non-numeric before substitution a9cf555 (#2836)
  • [TASK] Remove usages of Prophet by all occurrences within TYPO3 API 3bbf25a, 45b1703, 4f2b37a (#2862)
  • [TASK] Remove TYPO3 long time ago deprecated cache class 79cafe9 (#2782)
  • [BUGFIX] Change filter for workspace 5408889 (#2847)
  • [BUGFIX] Use Iconfactory to retrieve record icons fa77962 (#2900)
  • [BUGFIX] Language overlay for records is not retrieved since solr Version 11.x (#2788)
  • [BUGFIX] Temporary free mode fix d5e936b
  • [BUGFIX] Content id in language aspect c84ce1b
  • [BUGFIX] Language aspect for indexer 9af09f3
  • [BUGFIX] remove escaping on suggestion prefix f70829e (#2917)
  • [FEATURE] Exclude sub entries of page/storage recursively 4151a25 (#2934)
  • [BUGFIX] Make relevance sorting option markable as active bc813c8 (#2922)

Release 11.0.3

This is a bugfix-only release that contains only a few bugfixes

  • [TASK] Use minor version of solr docker image (#2740)
  • [BUGFIX] Make sure HtmlContentExtractor::cleanContent() is UTF-8 safe (#2514)
  • [BUGFIX] Database exception in RecordMonitor for records no-"enable" columns (#2512)
  • [BUGFIX] Indexing of records fails with solr 10.x (#2521)
  • [BUGFIX] Hard codes plugin namespace (#2732)
  • [BUGFIX] Restricted pages are not being indexed in Typo3 10 (#2695)
  • [BUGFIX] Prevent duplicate urls for page 0 (#2718)
  • [BUGFIX] Fix assignment for page uid variable (#2664)
  • [BUGFIX] Use num_found in static db table (#2668)
  • [BUGFIX] Build core base path right, when path is slash only (#2692)
  • [BUGFIX] Fix missing renderType attribute in flexform for search plugin (#2669)
  • [BUGFIX] Add option to override 'port' in frontend indexing URL (#2618)
  • [BUGFIX] Reset uriBuilder before building a new uri (#2658)
  • [DOCS] Multiple improvements to the docs

Release 11.0.2

This is a bugfix-only release that contains only a few bugfixes

  • [TASK] Add warning in the docs that a fqdn is required for the sitehandling
  • [BUGFIX] Re-enable Integration Tests for TYPO3 v10
  • [BUGFIX] Fix unit tests with new controller context check
  • [BUGFIX] Fix tests and add groups for tests
  • [BUGFIX] Remove mocks in TYPO3 v10 Integration tests
  • [BUGFIX] Remove unneeded constant
  • [BUGFIX] Fix travis.yml to use correct stable versions
  • [BUGFIX] Ensure to hand in PSR-7 Request to TSFE->getPageAndRootlineWithDomain
  • [BUGFIX] Remove unneeded is_siteroot flag in nested storage folder
  • [BUGFIX] Always return array on non-mounted sites
  • [BUGFIX] Fix multiple rootpages in nested sites
  • [BUGFIX] Prevent SiteNotFoundException in reports module

Release 11.0.1

This is a bugfix-only release that contains only a few bugfixes

  • [BUGFIX] Fix documentation and Versionmatrix
  • [BUGFIX] Fix failing build on docker hub

Release 11.0.0

We are happy to release EXT:solr 11.0.0. The focus of this release was the support of TYPO3 10 LTS.

Important: This version is installable with TYPO3 9 and 10 LTS. For TYPO3 9 LTS at least version 9.5.16 is required. EXT:solr 11 requires the usage of the TYPO3 site handling for the configuration of solr.

The `legacyMode` that allows the usage of domain records and configuration of Solr cores in TypoScript was dropped with EXT:solr 11.

New in this release

With EXT:solr 11 we provide the support of TYPO3 10 LTS. If you update to EXT:solr 11, make sure, that you are using the TYPO3 site management to manage your Apache Solr endpoints. Thanks to: Achim Fritz & b13 for the support on that topic

Support of Apache Solr 8.5.1

With EXT:solr 11 we support Apache Solr 8.5.1, the latest release of Apache Solr.

To see what was changed in Apache Solr 8.5.x please read the release notes of Apache Solr

Small improvements and bugfixes

Beside the major changes we did several small improvements and bugfixes:

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • dev-rke
  • Florian Langer
  • Georg Ringer
  • Goddart Goth
  • Hannes Lau
  • Jens Jacobsen
  • Kai Lochbaum
  • Klaus Hörmann-Engl
  • Koen Wouters
  • Markus Friedrich
  • Markus Schwemer
  • Patrick Daxböck
  • Philipp Parzer
  • Rafael Kähm
  • Sascha Nowak
  • Thomas Löffler
  • Timo Hund

Also a big thanks to our partners that have joined the EB2020 program:

  • +Pluswerk AG
  • .hausformat GmbH
  • 3m5. Media GmbH
  • 4eyes GmbH
  • Agora Energiewende Smart Energy for Europe Platform (SEFEP) gGmbH
  • Amedick & Sommer Neue Medien GmbH
  • AUSY SA
  • b13 GmbH
  • BARDEHLE PAGENBERG Partnerschaft mbB
  • BIBUS AG Group
  • Bitmotion GmbH
  • brandung GmbH & Co. KG
  • cab services ag
  • clickstorm GmbH
  • comwrap GmbH
  • cron IT GmbH
  • CS2 AG
  • cyperfection GmbH
  • digit.ly GmbH
  • Digitale Offensive GmbH Internetagentur
  • E-Magineurs
  • Eidg. Forschungsanstalt WSL
  • FGTCLB GmbH
  • FTI Touristik GmbH
  • GAYA - Manufacture digitale
  • Hochschule für Polizei und öffentliche Verwaltung Nordrhein-Westfalen
  • hotbytes GmbH & Co. KG
  • IHK Neubrandenburg
  • in2code GmbH
  • Inotec Sicherheitstechnik GmbH
  • jweiland.net
  • Kassenzahnärztliche Vereinigung Bayerns (KZVB)
  • Kassenärztliche Vereinigung Rheinland-Pfalz
  • Landeskriminalamt Thüringen
  • LfdA – Labor für digitale Angelegenheiten GmbH
  • Macaw Germany Cologne GmbH
  • Marketing Factory Consulting GmbH
  • Masterflex SE
  • mehrwert intermediale kommunikation GmbH
  • mm Online Service
  • netlogix GmbH & Co. KG
  • Open New Media GmbH
  • plan.net - agence conseil en stratégies digitales
  • plan2net GmbH
  • PROFILE MEDIA GmbH
  • ressourcenmangel dresden GmbH
  • RKW Rationalisierungs- und Innovationszentrum der Deutschen Wirtschaft e. V.
  • ruhmesmeile GmbH
  • Sandstein Neue Medien GmbH
  • Stadt Wien - Wiener Wohnen Kundenservice GmbH
  • Stefan Galinski Internetdienstleistungen
  • TOUMORØ
  • Typoheads GmbH
  • unternehmen online GmbH & Co. KG
  • VisionConnect GmbH
  • werkraum Digitalmanufaktur GmbH
  • WIND Internet
  • zimmer7 GmbH

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us by becoming an EB partner:

http://www.typo3-solr.com/en/contact/

or call:

+49 (0)69 - 2475218 0

Releases 10.0

Release 10.0.0

We are happy to release EXT:solr 10.0.0. The focus of EXT:solr 10.0.0 was, to support the site handling that was introduced in TYPO3 9 LTS .

Important: This version is installable with TYPO3 9 LTS only.

Since the development budget is limited at one side and we have other project requests at the other side we could spend less time on the development of EXT:solr. If you want to support us please consider to sponsor us.

New in this release

TYPO3 9 LTS site handling support

The major new feature of EXT:solr 10 is the support of the TYPO3 site handling. You can now configure the Apache Solr connections along with your TYPO3 site. To get a typical Solr site running you now need only a few settings and can use the site handling user interface for that.

Apache Solr 8.2 support

EXT:solr 10 ships Apache Solr 8.2.

Note: The data volume of the Apache Solr Docker container was changed from `/opt/solr/server/solr/data` to `/var/solr/data/data` this might require changes in your infrastructure configuration.

Extensionscanner cleanup

Most (not all) of the extension scanner violations have been fixed. We hope to get the extension scanner violations cleaned in the next releases.

Drop TYPO3 8 compatibility

To simplify the development we've dropped the compatibility for TYPO3 8 LTS. If you need to use TYPO3 8 please use the 9.0.x branch.

Add default field for image and price

To allow external applications to index common information for product's we've added a field for price and image. Along with that we've changed the suggest to render the content of the "image" field instead of "previewImage_stringS", this might require changes in your index configuration.

Migration from EXT:solr 9 to EXT:solr 10

Important things on sites with activated Site Handling

By default EXT:solr 10 expects the configuration of Solr connections and cores in the site handling module, along with your TYPO3 site. The configuration of the Solr connections with your site are available immediatly, so now initialization of connections is required anymore.

The old fashioned setup (domain records and Solr connections in TypoScript) are now the legacy mode. If you want or need to use that still, you can enable the legacy mode, with your extension configuration by setting `allowLegacySiteMode = 1`.

The following things will become standard, and should be preferred and activated/configured as close as possible and in some cases immediately, otherwise the things wil not work or break the whole setup:

TypoScript

plugin.tx_solr.solr

This TypoScript configurations for Constants and for Setup are at least partially obsolete and are ignored on Site Handling activated sites. All Apache Solr connections must be stored in Site Handling "config.yaml" file for each language.

config.absRefPrefix

The "config.absRefPrefix" is obsolete and must be replaced with Site Handlings "base" or "baseVariants" settings.

Limitations of the site UI and yaml configuration

Important: The goal of the configuration with your TYPO3 site and the site module UI was to simplify the setup, especially for new users. Therefore, not all options are still possible as before, but the most of them are still possible by editing the yaml file.

There are the following known and wanted limitations:

  • It is not possible to configure a username and a password for the solr server with the UI. You still have the possibility to add that configuration in the yaml file, in that case make sure that this file is not readable from the web!. Another approach is to configure the usage from the environment variables. To configure a username the setting solr_username_read or solr_password_read can be used in the yaml file.
  • It is not possible to configure a different Solr hostname with the UI. If you need that you can still configured that in the yaml file, by using the fallback mechanism.

The fallback mechanism work like that:

Each setting has the following structure solr_{$setting}_{$scope}"`. The scope can be read or write. Every setting can be overwritten for the scope write, if nothing is configured it will fallback to the read setting. Every setting can be defined on the language level, if it is not configured on the language level it fallsback to the global setting.

Example:

base: 'http://solr-ddev-site.ddev.site/'
baseVariants: {  }
errorHandling: {  }
languages:
  -
    title: English
    enabled: true
    base: /
    typo3Language: default
    locale: en_US.UTF-8
    iso-639-1: en
    navigationTitle: ''
    hreflang: en-US
    direction: ''
    flag: global
    solr_host_read: solr_node_1
    solr_core_read: core_en
    languageId: '0'
  -
    title: German
    enabled: true
    base: /de/
    typo3Language: de
    locale: de_DE.UTF-8
    iso-639-1: de
    navigationTitle: ''
    hreflang: de-DE
    direction: ''
    flag: global
    solr_host_read: solr_node_2
    solr_core_read: core_de
    languageId: '1'
rootPageId: 3
routes: {  }
solr_enabled_read: true
solr_path_read: /solr/
solr_port_read: 8983
solr_scheme_read: http
solr_use_write_connection: false
Copied!

The example above shows that you are able to define the setting solr_host_read on the language level. Since this is a more advanced configuration and the user interface should be kept simple, this can only be configured in the yaml.

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Achim Fritz
  • Georg Ringer
  • Helmut Hummel
  • Marc Bastian Heinrichs
  • Marco Pfeiffer
  • Markus Kobligk
  • Netcoop
  • Nicole Cordes
  • Rafael Kähm
  • Rémy DANIEL
  • Sascha Egerer
  • Stefan Frömken
  • Stephan Jorek
  • Timo Hund
  • Yann Weyer
  • Gerald Aistleitner

Also a big thanks to our partners that have joined the EB2019 program:

  • 21TORR GmbH
  • 3m5, Media GmbH
  • Absolut Research GmbH
  • AgenturWebfox GmbH
  • Amedick & Sommer Neue Medien GmbH
  • arndtteunissen GmbH
  • Arrabiata Solutions GmbH
  • artif GmbH & Co. KG
  • Atol Conseils & Développements
  • b13 GmbH
  • bgm business websolutions GmbH & Co KG
  • Bitmotion GmbH
  • BIBUS AG Group
  • Bitmotion GmbH
  • Columbus Interactive GmbH
  • Consulting Piezunka und Schamoni - Information Technologies GmbH
  • cosmoblonde GmbH
  • CS2 AG
  • datamints GmbH
  • Diesel Technic AG
  • Die Medialen GmbH
  • Direction des Systèmes d’Information - Département du Morbihan
  • dörler engineering services
  • E-Magineurs
  • Fachhochschule für öffentliche Verwaltung NRW Zentralverwaltung
  • fixpunkt werbeagentur gmbh
  • Flowd GmbH
  • Frequentis Comsoft GmbH
  • GAYA - La Nouvelle Agence
  • Gernot Leitgab
  • Getdesigned GmbH
  • .hausformat GmbH
  • Haute école de travail social et de la santé - EESP
  • Hirsch & Wölfl GmbH
  • Hochschule Furtwangen
  • Hypo Tirol Bank AG
  • Intera Gesellschaft für Software-Entwicklung mbH
  • interactive tools GmbH - Agentur für digitale Medien
  • internezzo ag
  • iresults gmbh
  • ITK Rheinland
  • LOUIS INTERNET GmbH
  • Kassenärztliche Vereinigung Bayerns (KZVB)
  • KONVERTO AG
  • kraftwerk Agentur für neue Kommunikation GmbH
  • Landesinstitut für Schule und Medien Berlin-Brandenburg
  • Libéo
  • LINGNER CONSULTING NEW MEDIA GMBH
  • MaxServ B.V.
  • McLicense GmbH
  • MeinEinkauf AG
  • NEW.EGO GmbH
  • medien.de mde GmbH
  • mehrwert intermediale kommunikation GmbH
  • mellowmessage GmbH
  • mentronic . Digitale Kommunikation
  • MOSAIQ GmbH
  • pietzpluswild GmbH
  • plan2net GmbH
  • plan.net - agence conseil en stratégies digitales
  • Proud Nerds
  • +Pluswerk AG
  • punkt.de GmbH
  • Redkiwi
  • ressourcenmangel dresden GmbH
  • rrdata
  • RKW Rationalisierungs- und Innovationszentrum der Deutschen Wirtschaft e.V.
  • Site’nGo
  • SIWA Online GmbH
  • Stadt Wien - Wiener Wohnen Kundenservice GmbH
  • Stadtverwaltung Villingen-Schwenningen
  • Stefan Galinski Internetdienstleistungen
  • Studio Mitte Digital Media GmbH
  • TOUMORO
  • Ueberbit Gmbh
  • WACON Internet GmbH
  • webconsulting business services gmbh
  • webschuppen GmbH
  • Webstobe GmbH
  • webit! Gesellschaft für neue Medien mbH
  • wegewerk GmbH
  • werkraum Digitalmanufaktur GmbH
  • XIMA MEDIA GmbH

Special thanks to our premium EB 2019 partners:

  • jweiland.net
  • sitegeist media solutions GmbH

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us by becoming an EB partner:

http://www.typo3-solr.com/en/contact/

or call:

+49 (0)69 - 2475218 0

Releases 9.0

Release 9.0.0

We are happy to release EXT:solr 9.0.0. The focus of EXT:solr 9.0.0 was, to support the latest version of Apache Solr (7.6.0) and to drop the usage of the solrphpclient and use the solarium php API instead.

New in this release

Support of Apache Solr 7.6

EXT:solr 9.0.0 ships a ready to use docker container with Apache Solr 7.6. This makes new features of Apache Solr available to EXT:solr.

Replaced solrphpclient with solarium php api

For the communication between EXT:solr we've used the solrphpclient library. This library was not maintained anymore and had several custom modifications. Therefore we made the decision to move to the solarium php api.

This brings us the following advantages:

  • Use a common, robust, maintained library
  • Join the forces with other PHP projects to improve solarium and benefit from that

The migration to solarium required several changes in EXT:solr and all add-on's and we will provide compabtility releases for them as well.

With the move to solarium we donated some parts to the solarium API (e.g. the Solr core handling). This allows us to remove some redundant logic in EXT:solr in the future.

Thanks:

Pull requests and Links:

Outlook:

By now we use the Queries and Httpclient of solarium, but not the domain classes because this requires additional changes in EXT:solr and solarium.

In the future we want to get rid of redundant code and use the API where we can and it makes sence and support solarium with the features that we need for EXT:solr.

TYPO3 9 compatibility

The current release is installable and useable with TYPO3 9 LTS but not all features are supported.

Currently it is supported to:

  • Use EXT:solr with sites that do have a domain record or domain configuration from EXT:solr

The following parts require additional work and are not supported:

  • SiteHandling
  • Extensionscanner proofed

Since a backward compatibility to TYPO3 8.7 LTS make it harder to support the previous mentioned topics. We will drop the support for TYPO3 8 LTS in the next version and improve the support of those TYPO3 9 LTS features.

Nevertheless a lot of work was allready done for the basic support of TYPO3 9 LTS in the following pull requests:

Allow open query in DateRangeFacet

This patch allows to create data range facets with an open beginning or open end.

Support to differ between read and write connections

By now each site had one Solr connection for reading and writing. In most of the cases this good enough when you want to index and search in the same core.

Some setups require a more flexible approach:

  • E.g. when you want to clean a core and re-index that data your index is not complete for some time on the live site
  • When you want to do a primary/replica setup for performance reasons(e.g. by using a replica node on the web server) this was not possible by now

With a separation of read and write connections this is now possible. With these building blocks you could e.g.

  • Index into a shadow core (that is the write core) and swap read and write core when your re-index is done
  • Install a replica server on your frontend server and index into a dedicated primary node that act's as a Solr primary server

The new setup can be configured like that:

plugin.tx_solr.solr {
        read {
            scheme = https
            host   = 127.0.0.1
            port   = 8983
            path   = /solr/core_en/
        }
        write < .read
        write {
            port   = 8984
        }
}
Copied!

For compatibility reasons EXT:solr is falling back to `plugin.tx_solr.solr.*` when nothing is configured here:

Important: When you update from EXT:solr 8.1.0 you need to re-initialize your EXT:solr connections.

More flexible facet uri ViewHelpers with optional facet object

You could create now a facet item link (add, set, remove) somewhere else in the results view where no facet object is available.

Beside

{s:uri.facet.setFacetItem(facet: facet, facetItem: option)}
Copied!

you could create a set link now with this vh arguments:

{s:uri.facet.setFacetItem(facetName: 'type', facetItemValue: 'pages', resultSet: resultSet)}
Copied!

Thanks to Marc Bastian Heinrichs for creating a patch for that.

Allow to configure additionalExcludeTags for option facets

When you want to exclude facets from the counts of another facets, Apache Solr uses tags and excludeTags to realize that.

With the setting `additionalExcludeTags` you can add custom exclude tags for a facet and `addFieldAsTag` allows you, to force the creation of a tag for a certain facet.

Thanks to Marc Bastian Heinrichs for creating a patch for that and to in2code for paying for the finalization and documentation.

Bugfixes

Migration from EXT:solr 8.1.0 to EXT:solr 9.0.0

  • We ship Apache Solr 7.5.0, you need to install that Version with our configSet.
  • The argument "hasSearched" was removed from the searchAction and is no longer passed. You can now retrieve this information by calling "SearchResultSet::getHasSearch" or "{resultSet.hasSearched}" in the FLUID template. When you access this argument in your FLUID Template, you need to change that as well.
  • EXT:solr 9 differs between read and write connections now. As fallback the old configuration is still supported and used for reading and writing. Nevertheless you need to re-initialize the Solr connections that the data in the registry is rewritten. If you want to make use of the new configuration you can configure the connections like that:
plugin.tx_solr.solr {
    read {
        scheme = https
        host   = 127.0.0.1
        port   = 8983
        path   = /solr/core_en/
    }
    write < .read
    write {
        port   = 8984
    }
}
Copied!

Removed Code

The following code parts have been removed as announced in previous versions of EXT:solr:

  • SearchResultSetService::getHasSearched Please use SearchResultSet::getHasSearched now
  • SortingHelper::getSortFields
  • SortingHelper::getSortOptions
  • Queue::initialize
  • Queue::initializeIndexingConfigurations
  • Search::hasSearched
  • Search::getResultDocumentsRaw
  • Search::getResultDocumentsEscaped
  • Search::getFacetCounts
  • Search::getFacetFieldOptions
  • Search::getFacetQueryOptions
  • Search::getFacetRangeOptions
  • Search::getSpellcheckingSuggestions
  • Util::isLocalizedRecord

Outlook

In the next release we will drop the support of TYPO3 8 and focus on the integration into TYPO39. Depending on the funding we would like to support the integration into the TYPO3 site management and want to allow to configure you Solr site with the TYPO3 site management module.

With the move to the solarium php api, we take the first step of the integration. In the next releases we want to use more parts of the solarium API and also contribute to that API to share the improvements with other PHP projects.

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Aljoscha Weber
  • Benni Mack
  • Felix Nagel
  • Florian Wessels
  • Helmut Hummel
  • Jens Jacobsen
  • Marc Bastian Heinrichs
  • Patrick Gaumond
  • Rafael Kähm
  • Sasche Egerer
  • Thomas Hohn
  • Timo Hund

Also a big thanks to our partners that have joined the EB2019 program:

  • Amedick & Sommer Neue Medien GmbH
  • BIBUS AG Group
  • Bitmotion GmbH
  • CS2 AG
  • Gernot Leitgab
  • Getdesigned GmbH
  • Hirsch & Wölfl GmbH
  • ITK Rheinland
  • Kassenärztliche Vereinigung Bayerns (KZVB)
  • TOUMORO
  • Ueberbit Gmbh
  • XIMA MEDIA GmbH
  • b13 GmbH
  • bgm business websolutions GmbH & Co KG
  • datamints GmbH
  • medien.de mde GmbH
  • mehrwert intermediale kommunikation GmbH
  • mellowmessage GmbH
  • plan2net GmbH
  • punkt.de GmbH

Special thanks to our premium EB 2019 partners:

  • jweiland.net
  • sitegeist media solutions GmbH

In addition i want to thank Markus Kalkbrenner and the whole solarium team for the support.

Thanks to everyone who helped in creating this release!

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us in 2019 by becoming an EB partner:

http://www.typo3-solr.com/en/contact/

or call:

+49 (0)69 - 2475218 0

Releases 8.1

Release 8.1.0

We are happy to release EXT:solr 8.1.0. The focus of EXT:solr 8.1.0 was, to improve the API for the new EB addon's "solrconsole" and "solrdebugtools".

New in this release

Groups should be respected in suggest as well

When solrfluidgrouping is installed the groups are also respected for the top results in the suggest.

Display plugin name in page module

This patch extends the hook PageModuleSummary displaying information about the plugins flexform settings in the page module. Now the plugin name will be displayed and linked to the edit form, similar to the default behavior.

TypoScriptService moved from Extbase to Core

The TypoScriptService was moved from extbase to the core and we now use the service in the TYPO3 core.

Improvements for solrconsole

The implementation of solrconsole required several API changes for the implementation of the commands. These changes have been added with the following pull requests:

Use Apache Solr 6.6.3

We've updated shipped Apache Solr version to 6.6.3

Documentation improvements

Several issues have been fixed in the documentation to keep it up-to-date.

Add datetime fields for new configuration

For the example index configuration for EXT:news, the datetime fields have been added.

Anonymize ip addresses by default

Since several versions it is possible to anonymize the ip-address in the statistics. This is now enabled by default.

Remove setting search.spellchecking.wrap

This setting is not used anymore. It was replaced by a label in fluid in version 7.

Set mm.autoRelax to true by default

When terms get removed because they are stopwords this might have an unwanted impact when the mm condition is evaluated. Setting mm.autoRelax to true fixes this.

See also:

Dispatch signals in search controller actions

This patch add's signals to the SearchController that allow passing custom arguments to the search view's.

Optimizations on suggest

With the suggest there where several issues:

  • The type could not be passed before, now you can pass the type to the SearchFormViewHelper as "suggestPageType" argument.
  • The filters have been passed as global url arguments "filter" as json encoded array, what was hard to debug and inconsistent to the other parameters. We changed the parameter to the name "additionalFilters" and each filter is passed as an array item.

Impact: If you just use the default ViewHelpers and templates you need to change nothing. When you add custom filters to the suggest you now need to pass each filter as an array item of the argument "additionalFilters"

TYPO3 9 compatibility

With 8.1.0 we do not officially support TYPO3 9. You can install it on TYPO3 9.3.99 for development and we tried to fix the most important issues during the development but there are already a few known issues:

https://github.com/TYPO3-Solr/ext-solr/issues?q=is%3Aissue+is%3Aopen+label%3A9LTS

The following pull requests have already been merged in order to optimize the TYPO3 9 compatibility:

Bugfixes

Deprecated Code

The following methods have been marked as deprecated and will be removed in EXT:solr 9.0.0:

  • SearchResultSetService::getHasSearched() please use SearchResultSet::getHasSearched along with that the global template variable "hasSearched" will be removed with 9.0.0 as well.
  • Search::getHasSearched() please use SearchResultSet::getHasSearched instead along with that the global template variable "hasSearched" will be removed with 9.0.0 as well.
  • Util::isLocalizedRecord() please use TCAService::isLocalizedRecord instead
  • Queue::initialize() please use Queue::initializeBySiteAndIndexConfiguration instead
  • Queue::initializeIndexingConfigurations() please use Queue::initializeBySiteAndIndexConfiguration instead
  • SortingHelper::getSortFields() please use the SearchResultSet to get the parsed sorting
  • SortingHelper::getSortOptions() please use the SearchResultSet to get the parsed sorting

Outlook

In the next release, we want to focus on the move to solarium and the support of the latest Apache Solr version.

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors to this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Andreas Lappe
  • Benni Mack
  • Florian Schöppe
  • Frans Saris
  • Jens Jacobsen
  • Marc Bastian Heinrichs
  • Markus Friedrich
  • Michael Telgkamp
  • Olivier Dobberkau
  • Rafael Kähm
  • Rémy DANIEL
  • Thomas Löffler
  • Timo Hund
  • Thomas Hohn

Also a big thanks to our partners that have joined the EB2018 program:

  • 4eyes GmbH
  • Albervanderveen
  • Agentur Frontal AG
  • AlrweNWR Internet BV
  • Amedick & Sommer
  • AUSY SA
  • Bibus AG
  • Bitmotion GmbH
  • bgm Websolutions GmbH
  • bplusd interactive GmbH
  • Centre de gestion de la Fonction Publique Territoriale du Nord (Siège)
  • Citkomm services GmbH
  • Consulting Piezunka und Schamoni - Information Technologies GmbH
  • Cobytes GmbH
  • Cows Online GmbH
  • creativ clicks GmbH
  • DACHCOM.DIGITAL AG
  • Deutsches Literaturarchiv Marbach
  • food media Frank Wörner
  • Fachhochschule für öffentliche Verwaltung NRW
  • FTI Touristik GmbH
  • GAYA - La Nouvelle Agence
  • Hirsch & Wölfl GmbH
  • Hochschule Furtwangen
  • ijuice Agentur GmbH
  • Image Transfer GmbH
  • JUNGMUT Communications GmbH
  • Kreis Coesfeld
  • LINGNER CONSULTING NEW MEDIA GMBH
  • LOUIS INTERNET GmbH
  • L.N. Schaffrath DigitalMedien GmbH
  • MEDIA::ESSENZ
  • Mehr Demokratie e.V.
  • mehrwert intermediale kommunikation GmbH
  • Mercedes AMG GmbH
  • Petz & Co
  • pietzpluswild GmbH
  • pixelcreation GmbH
  • plan.net
  • Pluswerk AG
  • Pottkinder GmbH
  • PROVITEX GmbH
  • Publicis Pixelpark
  • punkt.de GmbH
  • PROFILE MEDIA GmbG
  • Q3i GmbH & Co. KG
  • ressourcenmangel an der panke GmbH
  • Roza Sancken
  • Site'nGo
  • SIWA Online GmbH
  • snowflake productions gmbh
  • Studio B12 GmbH
  • systime
  • SYZYGY Deutschland GmbH
  • Talleux & Zöllner GbR
  • TOUMORO
  • THE BRETTINGHAMS GmbH
  • TWT Interactive GmbH
  • T-Systems Multimedia Solutions GmbH
  • Typoheads GmbH
  • Q3i GmbH
  • Ueberbit GmbH
  • zdreicon GmbH
  • zimmer7 GmbH

Special thanks to our premium EB 2018 partners:

Thanks to everyone who helped in creating this release!

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us in 2018 by becoming an EB partner:

http://www.typo3-solr.com/en/contact/

or call:

+49 (0)69 - 2475218 0

Releases 8.0

Release 8.0.0

We are happy to release EXT:solr 8.0.0. The focus of EXT:solr 8.0.0 was, to improve the user experience in the frontend and backend.

New in this release

In the following paragraphs we want to summarize the new features that will be shipped with EXT:solr 8.0.0

New suggest

We've replaced the old jQuery UI based autosuggest with a new suggest (https://github.com/devbridge/jQuery-Autocomplete). The advanced suggest can not only show the suggestions, it can also show a configurable amount of top search results.

When the user clicks on the result, he can directly jump to the result page without opening the search results page.

Thanks:

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1638

JSON Faceting for options facets

Apache Solr offers a JSON API for faceting since several versions. Starting with the options facet we've added the support to use this JSON faceting API in EXT:solr.

The support of the JSON API, in general, allows us to build new features on top of that API, that was impossible before. With the first implementation we've added the following features:

By now an option was simply the value and the count, that reflects the number of documents that belong to that option. At EXT:solr 8.0.0 we've added a TypoScript option that is called "metrics", that allows us to collect and show several metrics from documents that belong to a facet option. Examples of metrics are e.g "sum of downloads", "average price",... These metrics will be available in Option model in the FLUID template and can also be used to sort the facet options.

The following example shows an configured options facet with a configured metric:

plugin.tx_solr.search.faceting.facets.type.metrics {
    newest = max(created)
    oldest = min(created)
}
Copied!

In the FLUID template you could use the following code in the facet partial to render those metrics:

<span>
   newest: {option.metrics.newest -> f:format.date(format: 'Y-m-d H:i:s')}
</span>
<span>
   oldest: {option.metrics.oldest -> f:format.date(format: 'Y-m-d H:i:s')}
</span>
Copied!

Thanks:

  • Thanks to Jens Jacobsen and UEBERBIT for sponsoring Jens work on that feature at our code sprint.

Since we'replaced the whole internal communication from EXT:solr to Apache Solr when options facets are used we are very happy to get your feedback and bug reports when you use the options facets with EXT:solr

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1764

Group facet options by prefix

When you have option facets with a lot of options, it would be nice to group those options by a prefix. An example is that you group all options by the starting letter to organize them in tabs:

With EXT:solr 8 we ship the following components that allow grouping your facet options to arrange them as you need them in your template:

  • LabelFilterViewHelper: Can be used to filter options based on a prefix of the label.
  • LabelPrefixesViewHelper: Can be used to access all available prefixes of the facet options.
  • TypoScript example template "(Example) Options grouped by prefix" that configures a grouped facet on the author field

Thanks: This feature was sponsored by https://www.linnearad.no/

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1717

Filterable options facet

In the previous section, the facets get grouped by prefix to organize a large number of options. Another way that you also often see on the web is to allow to filter the options with an additional input box above the facet.

The implementation of that feature is possible just with a partial and a few JavaScript components. To simplify the integration of that feature in a project we ship

  • Example FLUID partial that uses the filter for options
  • Example JavaScript that implements the filter functionality
  • Example TypoScript "Search - (Example) Options filterable by option value" that uses the partials and javascript for a facet

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1741

Default partials with bootstrap.css

The old templating was created with custom CSS that was shipped with the extension. Since we want to decrease the effort that is required to create a mobile search and many integrators use bootstrap.css we decided to ship bootstrap templates by default. If you want to use another framework or your own custom CSS you are still able to do that with custom templates.

Nevertheless, the mobile search in a TYPO3 introduction installation with bootstrap is much better than before and your effort to adopt it should be reduced.

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1738

Performance improvements

In EXT:solr 7.x and below a ping request was done before each search. In EXT:solr 8.0.0 we just catch a failed search and handle the unavailability. This saves up to 30% time because we just need one HTTP request to Apache Solr instead of 2.

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1660

Improved index inspector

In the previous versions, we've introduced own backend modules that can also be used by regular TYPO3 users to perform several tasks. With EXT:solr 8.0.0 the index inspector will be moved from the common info module to our info module:

Besides the move, we also added the functionality to ReQueue a single document from the index inspector when you have permissions on the index queue module.

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1763

Use TYPO3 Guzzle for page index requests

The indexing of pages is now done with the shipped Guzzle client in TYPO3.

Thanks: Thanks to Benni Mack from b13 who has implemented that feature http://www.b13.de/

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1837

SOLR_CLASSIFICATION cObject

When you index a lot of documents you might want to create facets based on patterns that occur in the content.

The cObject SOLR_CLASSIFICATION allows you to do a lightweight classification based on regex patterns that you configure in the index configuration.

The following example shows how SOLR_CLASSIFICATION can be used to map patterns on classes that are indexed into a Solr field that could be used for faceting:

plugin.tx_solr.index.queue.pages.businessarea_stringM = SOLR_CLASSIFICATION
plugin.tx_solr.index.queue.pages.businessarea_stringM {
   field = __solr_content
   classes {
      automotive {
         patterns = car,jeep,SUV
         class = automotive
      }
      pharma {
         patterns = pharma,doc,medicine
         class = pharma
      }
   }
}
Copied!

With the configuration above Solr documents get the value "automotive" assigned in the Solr field "businessarea_stringM" when the content contains the term "car", "jeep" or "SUV".

Thanks: Thanks to http://www.bibus.ch who sponsored the implementation of this feature.

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1723

Phrase support (phrase, bigram, trigram)

With plugin.tx_solr.search.query.(phrase/bigramPhrase/trigramPhrase).fields you can control what is passed to Solr with the ps,ps2 and ps3 value.

With these phrase fields, you can boost documents where phrases occur in close proximity. This can be very handy when you want to tune your search in terms of relevancy.

Related links:

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1735

Tie parameter support

With plugin.tx_solr.search.query.tieParameter you can now configure the tie value that is passed to Apache Solr.

This value allows you to configure the impact of low scoring fields to the overall score. 0.0 means, that only high score fields will matter, 0.99 means that all fields have the same impact

Related links:

Thanks: Thanks to Marcus Schwemer and in2code that sponsored and shared that feature.

Related pull request: https://github.com/TYPO3-Solr/ext-solr/pull/1690

Doctrine ready

TYPO3 8 introduced Doctrine DBAL for database queries and the old API will be removed in TYPO3 9. Since we've used a lot of repositories with custom SQL queries, we had to rewrite a lot of queries.

In EXT:solr we've used the chance to restructure the SQL related code and move them to repositories whenever this was possible.

With EXT:solr 8 every usage of the old database API is removed and we are prepared in that way to be ready for TYPO3 9.

Fluent API for Queries with the QueryBuilder

Many parts of the code of EXT:solr deal with queries for Apache Solr that's no surprise :). The corresponding parts in the code especially the Query class had grown over time and reached a huge complexity.

This has several drawbacks:

  • It is hard to integrate new features (e.g the tiebreaker or bigram features)
  • TYPO3 specific logic and common Apache Solr logic is mixed and makes it hard to switch to frameworks like e.g. Solarium
  • The Query class does multiple things: Build the query, initialize the query from the configuration,... This could be split into multiple components.

To get better in that regards our goal is to split the Query into:

  • Query: Aggregate that is responsible to build the Solr query string based on the options
  • QueryBuilder: Builder class that is responsible to build an initialized Query object e.g. based on TypoScript configuration and user input.

With the current state the QueryBuilder does the following to build a Query from the user input:

$query = $queryBuilder->newSearchQuery($rawQuery)
 ->useResultsPerPage($resultsPerPage)
 ->useReturnFieldsFromTypoScript()
 ->useQueryFieldsFromTypoScript()
 ->useInitialQueryFromTypoScript()
 ->useFiltersFromTypoScript()
 ->useFacetingFromTypoScript()
 ->useVariantsFromTypoScript()
 ->useGroupingFromTypoScript()
 ->useHighlightingFromTypoScript()
 ->usePhraseFieldsFromTypoScript()
 ->useBigramPhraseFieldsFromTypoScript()
 ->useTrigramPhraseFieldsFromTypoScript()
 ->getQuery();
Copied!

Finally, this allows us to:

  • Integrate new features faster
  • Allow devs to compose own queries that use or ignore several aspects of EXT:solr
  • Simplify the switch or integration of a generic Solr API that is independent of TYPO3 (e.g. Solarium)

On the way to TYPO3 9

With EXT:solr 8.0.0 we will not officially support TYPO3 9 since it is not an LTS release! Nevertheless, we want to stay close to the TYPO3 core and allow the usage in 9 already.

By now we mainly fix Doctrine and Composer related issues and support the dropped "pageslanguageoverlay" table.

So to sum up... EXT:solr 8.0.0 will mainly support TYPO3 8 LTS and we will support TYPO3 9.x a good as we can without losing the backward compatibility to TYPO3 8 LTS.

Bugfixes

Removed Code

Query Refactoring

In the long run we want to be able to use other PHP frameworks for Apache Solr e.g. solarium(http://www.solarium-project.org/). To make this possible, we need to split the pure Solr query related logic from the TYPO3Solr specific query logic (e.g. accessFilter,...). To get a step closer into this direction, we've extracted the logic that is required to build a TYPO3 specific Solr query into the QueryBuilder. The pure Solr related query logic remains in the Query class.

Impact:

  • Whenever you create or modify queries you should use the QueryBuilder class for that. In one of the next releases we will support to create solarium queries with this QueryBuilder.

Beside the query refactoring, that required to remove and change several methods, the following code has been removed:

Hooks:

  • $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifySearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
  • $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['processSearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.

Deprecated Code

The following methods have been marked as deprecated and will be removed in EXT:solr 9.0.0

  • ApacheSolrForTypo3SolrSearch::getResultDocumentsRaw - Use the SearchResultsSet::getSearchResults now
  • ApacheSolrForTypo3SolrSearch::getResultDocumentsEscaped - Use the SearchResultsSet::getSearchResults now
  • ApacheSolrForTypo3SolrSearch::getFacetCounts - Use the SearchResultSet::getFacets now
  • ApacheSolrForTypo3SolrSearch::getFacetFieldOptions - Use the SearchResultSet::getFacets now
  • ApacheSolrForTypo3SolrSearch::getFacetQueryOptions - Use the SearchResultSet::getFacets now
  • ApacheSolrForTypo3SolrSearch::getFacetRangeOptions - Use the SearchResultSet::getFacets now
  • ApacheSolrForTypo3SolrSearch::getSpellcheckingSuggestions - Use SearchResultSet::getSpellcheckingSuggestions
  • ApacheSolrForTypo3SolrQuery is deprecated, use ApacheSolrForTypo3SolrDomainSearchQueryQuery now
  • ApacheSolrForTypo3SolrSuggestQuery is deprecated, use ApacheSolrForTypo3SolrDomainSearchQuerySuggestQuery now

Outlook

In the next release we want to focus on the move to Solarium and the support of the lastest Apache Solr version.

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors for this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Andreas Lappe
  • Andri Steiner
  • Benni Mack
  • Daniel Diesenreither
  • Daniel Mann
  • Daniel Ruf
  • Georg Ringer
  • Hannes Lau
  • Jeffrey Nellissen
  • Jens Jacobsen
  • Marco Bresch
  • Marcus Schwemer
  • Markus Friedrich
  • Markus Kobligk
  • Markus Sommer
  • Nicole Cordes
  • Patrick Schriner
    1. Golmann
  • Rafael Kähm
  • Sascha Egerer
  • Simon Schmidt
  • Thomas Löffler
  • Timo Hund
  • Tomas Norre Mikkelsen

Also a big thanks to our partners that have joined the EB2018 program:

  • Albervanderveen
  • Amedick & Sommer
  • AUSY SA
  • bgm Websolutions GmbH
  • Citkomm services GmbH
  • Consulting Piezunka und Schamoni - Information Technologies GmbH
  • Cows Online GmbH
  • food media Frank Wörner
  • FTI Touristik GmbH
  • Hirsch & Wölfl GmbH
  • Hochschule Furtwangen
  • JUNGMUT Communications GmbH
  • Kreis Coesfeld
  • LOUIS INTERNET GmbH
  • L.N. Schaffrath DigitalMedien GmbH
  • Mercedes AMG GmbH
  • Petz & Co
  • Pluswerk AG
  • ressourcenmangel an der panke GmbH
  • Site'nGo
  • Studio B12 GmbH
  • systime
  • Talleux & Zöllner GbR
  • TOUMORO
  • TWT Interactive GmbH

Special thanks to our premium EB 2018 partners:

Thanks to everyone who helped in creating this release!

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us in 2017 by becoming an EB partner:

http://www.typo3-solr.com/en/contact/

or call:

+49 (0)69 - 2475218 0

Releases 7.0

Release 7.0.0

We are happy to release EXT:solr 7.0.0. This release brings several smaller and some bigger changes

New in this release

FLUID Templating

One and a half years ago we started to implement FLUID templating for EXT:solr. This project was initially started as the addon solrfluid. Solrfuid was only available for our partners.

With EXT:solr 7.0.0 the new templating is the default templating in EXT:solr. A lot of code was added and several old stuff was removed. Since some things are conceptional different in FLUID and you also have a lot of possibilities we also dropped some parts, that can be build with fluid itself or do not make sence to do them before rendering the result in the view.

Most of the things just work like before but in the following parts we made conceptional changes (for good reasons):

  • No css or javascript will be added to the page automatically with the page renderer! Because the integrator wants to have control on that and TYPO3 allows to add this with TypoScript we propose to add these things via TypoScript. EXT:solr offers a lot of example TypoScript templates e.g. to add the default css or to add the javascript for a range facet.

The following TypoScript settings have been removed because they can be implemented with FLUID:

plugin.tx_solr.search.faceting.facetLinkATagParams
Copied!

You can add them in your project partials. If you need it just for one facet, please overwrite the render partial with facet.partialName and render the attributes different there

plugin.tx_solr.search.faceting.[facetName].facetLinkATagParams
Copied!

You can add them in your project partials. If you need it just for one facet, please overwrite the render partial with facet.partialName and render the attributes different there

plugin.tx_solr.search.faceting.facets.[facetName].selectingSelectedFacetOptionRemovesFilter
Copied!

This can be implemented with FLUID logic. Please check the example "Search - (Example) Options with on/off toggle" that implements that (by using the partial Facets/OptionsToggle.html)

plugin.tx_solr.search.results.fieldRenderingInstructions
Copied!

Please use custom ViewHelpers or the cObject ViewHelper for that.

plugin.tx_solr.search.results.fieldProcessingInstructions
Copied!

Please use custom ViewHelpers or the cObject ViewHelper for that.

Backend Modules Restructured

In EXT:solr 7.0.0 the backend modules are structured into multiple backend modules. This makes the user experience in the TYPO3 backend more consistent and allows you, to give different permissions on each module.

When you login into the backend, you now have the following modules available:

  • Info: Gives information of your Solr system, index fields and search usage.
  • Core Optimization: This module can be used to maintain the synonyms and stopwords in the Apache Solr server.
  • Index Queue: Gives an overview on indexed records and can be used to requeue records for indexing.
  • Index Administration: This module can be used for administrator tasks on your Solr system (clear index, index queue or reload a core)
  • https://github.com/TYPO3-Solr/ext-solr/pull/1300

Add excludeValues for Facets

If you want to exclude an optionValue from the facets when they get retrieved, you can configure this now:

plugin.tx_solr.search.faceting.facets.colors_stringM.excludeValues = red
Copied!

The example below will exclude the option "red" from the results when it is in the response.

Allow to configure custom entry Template

In previous EXT:solr versions it was possible to set a custom entry templating using:

plugin.tx_solr.templateFiles.search = EXT:solr/Resources/Templates/PiSearch/search.htm
Copied!

This configuration could be overwritten with a text value in the flexform.

With the move to FLUID we improved this part and made it more editor friendly:

  • Since the view related settings are located in the <view> section we've move the template configuration there as well.
  • You can now set a Templatename only (e.g. MySearch) to benefit from FLUID fallbacks (while setting a full path is still supported.
  • You can configure availableTemplates that can be selected by the editor in the flexform.

The following example shows, how you can load your own partials and provide different entry templates for the editor:

plugin.tx_solr {
    view {
        templateRootPaths.100 = EXT:your_config_extension/Resources/Private/Templates/
        partialRootPaths.100 = EXT:your_config_extension/Resources/Private/Partials/
        layoutRootPaths.100 = EXT:your_config_extension/Resources/Private/Layouts/
        templateFiles {
            results = Results
            results.availableTemplates {
                default {
                    label = Default Searchresults Template
                    file = Results
                }
                products {
                    label = Products Template
                    file = ProductResults
                }
            }
        }
    }
}
Copied!

With the prevision configuration the editor can switch from "Default Searchresults Template" to "Products Template".

Refactoring of Query API

The Query class is one of the biggest classes in EXT:solr and grown over time. This class has now been splitted into several classes. Along with that a concept of "ParameterBuilder" has been introduced. A ParameterBuilder is responsible to build a parameter part of the query. E.g. the Grouping ParameterBuilder is responsible to build all parameters of the Solr query for the grouping.

Move FilterEncoder and FacetBuilder to Facet Package

In Solrfluid there was one folder for each facet, that contains the facet class and a parser that parsers the Solr response into the facet object. The opposite part(parse the url, build the Solr query) was previously done in EXT:solr, with a FilterEncoder that was registered in the FacetRendererFactory.

Now because solrfluid and Solr have been merged, this logic can also be streamlined. Every facet is now structured in a FacetPackage.

A FacetPackage describes:

  • Which parser should be used to parse the Solr response
  • Which url decoder should be used to parse the EXT:solr query data
  • Which query builder should be used to build the faceting query part

You can also implement custom facet types by registering an on FacetPackage with the FacetRegistry.

Migration:

When you have implemented an own FacetParser for solrfluid, you should add a FacetPackage, that references a UrlDecoder and QueryBuilder. If you have used a custom FacetParser without registring a custom facet type in EXT:solr (ApacheSolrForTypo3SolrFacetFacetRendererFactory::registerFacetType) you can just reference DefaultUrlDecorder and DefaultFacerQueryBuilder in your FacetPackage.

Custom plugin namespace - Multiple Instances

Before solrfluid was merged there were several parts in EXT:solr where the data was read using GeneralUtility::_GET. The drawback of this approach is that the structure of the urls is hard to change and it is not possible to have custom namespaces for each instance of a plugin.

With solrfuid a SearchRequest object was introduced. This object holds all data from the user request. Now this object is used, whenever data from the user action is read. This allows us to make the request namespace changeable. You can now add your custom plugin namespace to a search plugin instance.

Doctrine Migration

As an ongoing task, we started with the migration of database queries to doctrine. Since the database is used in many parts of the extension there are still many parts open. If you want to work on that, your help is very welcome.

Add --rootpageid to CLI command

If you want or need to limit the initialization of Solr connections to a special rootpage, you can now do this by adding the argument --rootpageid.

Respect Setting includeInAvailableFacets and includeInUsedFacets

This setting was not evaluated in EXT:solrfluid before and is now available also with FLUID rendering.

Respect requirements facet setting with fluid

This setting was not evaluated in EXT:solrfluid before and is now available also with the FLUID rendering.

Respect setting searchUsingSpellCheckerSuggestion with Fluid

This setting was not evaluated in EXT:solrfluid before and is now available also with the FLUID rendering.

Get rid of dependency to sys_domain record

By now EXT:solr had the dependency on an existing domain record. This can be a problem, when you domain is dynamic or you need to be able to generate it.

Now you can configure a domain by the rootPageId in the TYPO3_CONF_VARS, the domain record is still used, when nothing is configured here.

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['sites'][###rootPageId###]['domains'] = ['mydomain.com'];
Copied!

During the implementation of this the logic to retrieve the SiteHash and get the SolrConfiguration was moved to the SiteRepository, this requires an update of the scheduler instances because the scheduler saves a serialized task. Please run the shipped migration to update scheduler tasks created with 6.1.x.

Preparations for TYPO3 9

Several things that will be removed with 9 have been changed:

Bugfixes

Removed Code

The following code has been removed since it is not used anymore:

Classes:

  • ScriptViewHelper
  • StyleViewHelper
  • AbstractSolrBackendViewHelper
  • StringUtility

Methods:

  • Util::camelize
  • Util::camelCaseToLowerCaseUnderscored
  • Util::underscoredToUpperCamelCase
  • Util::pageExists

Deprecated Code

Methods:

  • Query::setQueryFieldsFromString use setQueryFields(QueryFields::fromString('foo')) with QueryFields instead, will be removed in 8.0
  • Query::getQueryFieldsAsString use getQueryFields()->toString() now if needed, will be removed in 8.0
  • Query::setQueryField use getQueryFields()->set() now, will be removed in 8.0
  • Query::escape Use EscapeService::escape now, when needed
  • Query::addReturnField use getReturnFields()->add() now, will be removed in 8.0
  • Query::removeReturnField use getReturnFields()->remove() now, will be removed in 8.0
  • Query::getFieldList use getReturnFields()->getValues() now, will be removed in 8.0
  • Query::setFieldList use setReturnFields() now, will be removed in 8.0
  • Query::escapeMarkers not needed anymore, use your own implementation when needed
  • Query::setNumberOfGroups use getGrouping()->setNumberOfGroups() instead, will be removed in 8.0
  • Query::getNumberOfGroups use getGrouping()->getNumberOfGroups() instead, will be removed in 8.0
  • Query::addGroupField use getGrouping()->addField() instead, will be removed in 8.0
  • Query::getGroupFields use getGrouping()->getFields() instead, will be removed in 8.0
  • Query::addGroupSorting use getGrouping()->addSorting() instead, will be removed in 8.0
  • Query::getGroupSortings use getGrouping()->getSortings() instead, will be removed in 8.0
  • Query::addGroupQuery use getGrouping()->addQuery() instead, will be removed in 8.0
  • Query::getGroupQueries use getGrouping()->getQueries() instead, will be removed in 8.0
  • Query::setNumberOfResultsPerGroup use getGrouping()->setResultsPerGroup() instead, will be removed in 8.0
  • Query::getNumberOfResultsPerGroup use getGrouping()->getResultsPerGroup() instead, will be removed in 8.0
  • Query::setFacetFields use getFaceting()->setFields() instead, will be removed in 8.0
  • Query::addFacetField use getFaceting()->addField() instead, will be removed in 8.0
  • Query::removeFilter use getFilters()->removeByFieldName() instead, will be removed in 8.0
  • Query::removeFilterByKey use getFilters()->removeByName() instead, will be removed in 8.0
  • Query::removeFilterByValue use getFilters()->removeByValue() instead, will be removed in 8.0
  • Query::addFilter use getFilters()->add() instead, will be removed in 8.0

Method Arguments:

  • Query::setGrouping now expects the first argument to be a Grouping object, compatibility for the old argument (bool) will be dropped in 8.0
  • Query::setHighlighting now expects the first argument to be a Highlighting object, compatibility for the old arguments (bool, int) will be dropped in 8.0
  • Query::setFaceting now expects the first argument to be a Faceting object, compatibility for the old arguments (bool) will be dropped in 8.0

Hooks:

  • $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifySearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
  • $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['processSearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.

Outlook

In the next release we want to focus on the user experience in the backend and in the frontend. As preparation we collected several tasks.

The goal of some of them (e.g. bootstrap templating, checkbox facets, filterable options partial) is to make more things possible out of the box and make the extension more user friendly:

https://github.com/TYPO3-Solr/ext-solr/issues?q=is%3Aissue+is%3Aopen+label%3AUX

If you have allready implemented one this (or something else), that you want to share or make available outofthebox feel free to contanct us!

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors for this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Andreas Lappe
  • Felix Eckhofer
  • Frans Saris
  • Georg Ringer
  • Helmut Hummel
  • Jonas Ulrich
  • Marco Bresch
  • Markus Friedrich
  • Michael Skrynski
  • Rafael Kähm
  • Rémy DANIEL
  • Sascha Egerer
  • Sebastian Hofer
  • Timo Hund

Also a big thanks to our partners that have joined the EB2017 program:

  • .hausformat
  • AGENTUR FRONTAG AG
  • amarantus - media design & conding Mario Drengner & Enrico Nemack GbR
  • Amedick & Sommer Neue Medien GmbH
  • Andrea Pausch
  • Animate Agentur für interaktive Medien GmbH
  • artig GmbH & Co. KG
  • b:dreizehn GmbH
  • BIBUS AG Group
  • Bitmotion GmbH
  • cab services ag
  • Causal Sarl
  • CHIARI GmbH
  • Citkomm services GmbH
  • clickstorm GmbH
  • Connecta AG
  • Creative360
  • cron IT GmbH
  • CYBERhouse Agentur für interaktive Kommukation GmbH
  • cyperfection GmbH
  • data-graphis GmbH
  • Deutsche Welthungerhilfe e.V.
  • Deutscher Ärzteverlag
  • Deutscher Volkshochschul-Verband
  • Die Medialen GmbH
  • die_schnittsteller gmbh
  • Dörfer engineering services
  • E-Magineurs
  • EYE Communications AG
  • Fachhochschule für öffentliche Verwaltung NRW Zentralverwaltung Gelsenkirchen
  • familie redlich AG
  • Fork Unstable Media GmbH
  • hauptsache.net GmbH
  • Havas Düsseldorf GmbH
  • Hirsch & Wölfl GmbH
  • Hochschule Furtwangen - IMZ Online Services
  • Hochschule Konstanz
  • Institut der deutschen Wirtschaft Köln Medien GmbH
  • iresults gmbh
  • ITK Rheinland
  • itl Institut für technische Literatur AG
  • jweiland.net
  • Kassenärztliche Vereinigung Rheinland-Pfalz
  • Kerstin Nägler Web & Social Media Beratung
  • Landesinstitut für Schule und Medien Berlin-Brandenburg
  • Leibniz Universität IT Services
  • Libéo
  • Lime Flavour GbR
  • LINGNER CONSULTING NEW MEDIA GMBH
  • LOUIS INTERNET
  • Maximilian Walter
  • MEDIA:ESSENZ
  • mehrwert intermediäre kommunikation GmbH
  • Mercedes-AMG GmbH
  • mlm media process management GmbH
  • n@work Internet Informationssystems GmbH
  • Netcreators
  • netz-haut GmbH
  • neuwerk interactive
  • Nintendo of Europe GmbH
  • Onedrop Solutions GmbH
  • Open New Media GmbH
  • Paints Multimedia GmbG
  • pixelcreation GmbH
  • plan2net
  • Pluswerk AG
  • polargold GmbH
  • punkt.de GmbH
  • Raiffeisen OnLine GmbH
  • ruhmesmeile GmbH
  • Rundfunk und Telekom Regulierung GmbH
  • Schweizer Alpen-Club SAC
  • sitegeist media solutions GmbH
  • Star Finanz-Software Entwicklung und Vertriebs GmbH
  • Stefan Galinski Internetdienstleistungen
  • Stratis - Toulon
  • Studio Mitte Digital Media GmbH
  • Studio 9 GmbH
  • Systime A/S
  • SYZYGY Deutschland GmbH
  • takomat Agentur GbR
  • THE BRETTINGHAMS GmbH
  • TOUMORO
  • Triplesense Reply GmbH
  • Typoheads GmbH
  • unternehmen online GmbH & Co. KG
  • Universität Bremen
  • VERDURE Medienteam GmbH
  • WACON Internet GmbH
  • webedit AG
  • Webstore GmbH
  • Webtech AG
  • wegewerk GmbH
  • Wohnungsbau- und Verwaltungsgesellschaft mbH Greifswald
  • XIMA MEDIA GmbH
  • zdreicom GmbH
  • zimmer7 GmbH

Thanks to everyone who helped in creating this release!

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us in 2017 by becoming an EB partner:

http://www.typo3-solr.com/en/contact/

or call:

+49 (0)69 - 2475218 0

Releases 6.1

Release 6.1.0

Close to the release of TYPO3 8 LTS we are happy to announce EXT:solr 6.1 that is compatible with TYPO3 8 LTS.

New in this release

In this release we've merged over 160 pull requests! With these pull requests several new features and bugfixes were added.

Compatible with TYPO3 8 LTS and PHP 7.1 ready

EXT:solr 6.1 is ready for TYPO3 8 LTS and php 7.1 while keeping the compatibility to TYPO3 7.6 LTS and PHP 5.5 and 5.6.

Backend Performance Improvements

During the development of version 6.1 there have been a lot of smaller and bigger performance improvements. The following changes might be interesting.

Allow to use the closest configuration in the page tree

When a record is saved in the backend by now the whole TypoScript configuration is evaluated for the page where the record is located. In many setups it is enough to just use the closest template in the rootline to parse the configuration. Since there are cases, where this method does not work (e.g. when you use conditions based on page ids) you need to switch this behaviour explicitly on, by enable "useConfigurationFromClosestTemplate".

Add caching when record monitor is used

To optimize the performance the RecordMonitor and GarbageCollector class have been splitted into several components. These components use the TYPO3 caching framework to cache result that do not need to be retrieved multiple times.

Add a global list of monitored tables

If a record is relevant can only be decided in the context of a Solr site because the TypoScript configuration defines the indexing configuration.

When you want to tweek the backend performance you can define a global list of monitoredTables. Other tables will then be ignored and also the parsing of the TypoScript configuration is not needed then.

New Index Inspector

Since extjs will be removed more and more from the TYPO3 core we decided to migrate the index inspector to fluid. You can use the index inspector as usual from "Web > Info > Search Index Inspector" to analyze which documents are in the solr server for a specific page.

Related Issues:

Allow to monitor records outside the siteroot

When the property "additionalPageIds" is configured for an indexing configuration this configuration is now taken into account in the record monitor to re-queue these elements when a change in the backend is done.

Since this is an expensive operation, you can disable this feature by configuring the extension setting (useConfigurationTrackRecordsOutsideSiteroot) in the extension manager.

Related Issues:

Use TYPO3 logging framework instead of devlog

By now the devLog from TYPO3 was used to log messages in the extension. Since several years there is a more flexible logging framework available in the TYPO3 core. In EXT:solr 6.1 we make use of this and use the TYPO3 logging framework to write the logs.

Related Issues:

Use ::class and shorten array syntax

The minimum php version (PHP 5.5) allows us to benefit from PHP language features that make the code more readable:

  • Short array syntax: We replaced the parts in the code where [] can be used instead of array()
  • ::class: By using ::class all places where classes get instanciated with GeneralUtility::makeInstance can be simplyfied

Related Issues:

Support partial matches with N-Gram and Edge N-Gram

For some search use cases it makes sence to support partial matches e.g. when you search for "boch" and want to mach a field with the content "bochum".

EXT:solr 6.1.0 shippes new dynamic field types that make "N-Gram" and "Edge N-Gram" processed fields available:

  • Edge N-Gram Singlevalue: \*_textEdgeNgramS Edge Ngram (hello => hello, hell..)
  • Edge N-Gram Multivalue: \*_textEdgeNgramM Edge Ngram (hello => hello, hell..)
  • N-Gram Singlevalue: \*_textNgramS Ngram (hello => he,ll,lo,hel,llo)
  • N-Gram Multivalue: \*_textNgramM Ngram (hello => he,ll,lo,hel,llo)

See also:

https://cuiborails.wordpress.com/2012/07/13/apache-solr-ngramedgengram/

Related Issues:

Allow to disable siteHash check by using allowedSites = *

The siteHash is unique for a Solr site in the TYPO3 system. When Solr does a query the allowedSites setting can be used to control the set of documents that are queried in a Solr core. This is useful when you want to split the content from multiple sites in a single Solr core. In some cases it is useful to disable this limitation.

E.g. when:

  • You have data in Solr that comes from another system
  • When you want to search across multiple sites

Before the extension "solr_disablesitehash" was required to turn this sitehash check off. With EXT:solr 6.1.0 we've changed the meaning of the allowedSites:

  • Before: * was the same as __all, which means all sites in the system
  • After: __all is still handled as __all sites in the system, but * now means every site (same as no check at all)

Migration: When you are using * for query.allowedSites change the setting to __all.

Related Issues:

Update jQuery version

In this release we have updated the shipped jQuery and jQueryUi components to the following versions:

  • jQuery: v3.1.1
  • jQuery UI: v1.12.1

Related Issues:

Fix scrutinizer issues

A lot of issues (formatting, small bugs, documentation,...) reported by scrutinizer have been resolved in this release.

Related Issues:

Add Solr access filter plugin 2.0.0

The Solr access filter plugin has been optimized to use a Solr post filter. By using a post filter the performance of this plugin is much better, because less documents need to be evaluated. In this release we ship this new version 2.0.0 of the access filter with the default configSet and in our docker container.

Related Issues:

Preparation for Doctrine migration

In the next release we will drop the support of TYPO3 7 LTS and only support 8 LTS. Since in the TYPO3 core the database access was migrated to Doctrine, we will use Doctrine as well.

To simplify this, we have centralized a lot of database queries in this release and also dropped duplicate queries to reduce to amount of work that needs to be done for this in the next releaese.

Related Issues:

Allow self signed ssl certificates

Not every project can use officially signed SSL certificates for all stages. Because of that we allowed for the page indexing to index a host, with a self signed certificate.

You can enable this feature by configuring the extension setting (allowSelfSignedCertificates) in the extension manager to true.

Related Issues:

Add cObject support for Solr settings

When you use EXT:solr in a deployment scenario (e.g. platform.sh) you maybe want to define the Solr endpoints by environment variables or from variables in TYPO3_CONF_VARS. Both approaches are supported by the TypoScript TEXT object. Therefore it makes sence for the Solr endpoint settings in plugin.tx_solr.solr to support the usage of cObjects there. This allows you to define connections like this:

Addition to AdditionalConfiguration.php:

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['host'] = 'mysolrserver.de';
Copied!

Usage in TypoScript:

plugin.tx_solr.solr {
    host = TEXT
    host {
        value = localhost
        override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|host
    }
}
Copied!

Related Issues:

Deprecated code

For the continues improvement of the codestructure and preparation for upcoming tasks, the following methods have been marked as deprecated and will be removed in EXT:solr 7.0 (next release):

  • GarbageCollector::cleanIndex
  • Query::addSortField
  • Query::getSortingFields
  • SolrService::getSchemaName
  • Util::timestampToIso
  • Util::isoToTimestamp
  • Util::timestampToUtcIso
  • Util::utcIsoToTimestamp
  • Util::getRootPageId
  • Util::isRootPage
  • Util::getSiteHashForDomain
  • Util::resolveSiteHashAllowedSites
  • Item::updateIndexedTime
  • Queue::getIndexingConfigurationByItem
  • Queue::getIndexingConfigurationsByItem
  • Queue::getItemsCountBySite
  • Queue::getRemainingItemsCountBySite
  • TypoScriptConfiguration::getIndexQueuePagesAllowedPageTypesArray
  • QueryTest::canAddSortField
  • Site::getLanguages
  • Site::getFirstAvailableSite
  • Site::getAvailableSites
  • Site::getAvailableSitesSelector

Please check your deprecation log and replace the usages in your code.

Breaking changes

In this release some breaking changes have been required. Please follow the notes below when you migrate from 6.0 to 6.1

Default variantId changed

The default variantId was table/uid before since this id is not unique across multiple TYPO3 system, a system hash was added before. The scheme of the new variantId is systemhash/table/uid and allows to use grouping when data from multiple TYPO3 systems get indexed and searched.

Migration:

Reindex the Solr data to get the new variantId.

Related Issues:

Semantic of allowedSites changed

Migration:

When you use plugin.tx_solr.search.query.allowedSites = * you should now use plugin.tx_solr.search.query.allowedSites = __all

Related Issues:

Setting debugDevlogOutput replaced by debugOutput

Because the devLog was replaced by the logging framework, the setting plugin.tx_solr.logging.debugDevlogOutput does not make sence anymore.

Migration:

When you want to see the log output in the backend or frontend please use plugin.tx_solr.logging.debugOutput now.

Related Issues:

"Only variables should be assigned by reference" in IndexQueueIndexer::preAddModifyDocuments()

Along we the removal of the reference GeneralUtility::getUserObj was replaced with GeneralUtility::makeInstance, because the usage with ":" is deprecated since TYPO3 8 and will be removed.

Migration:

When you reference custom indexer (in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['IndexQueueIndexer']['indexItemAddDocuments']) you should reference only an autoloadable classname.

The following steps are required:

  • Move your Indexer to an own extension with proper autoloading configuration
  • Reference the autoloadable classname

Related Issues:

Bugfixes

StatisticRepository::getTopKeyWordsWithOrWithoutHits $limit, $withoutHits have no default values

https://github.com/TYPO3-Solr/ext-solr/issues/1143

Wrong TS path in frontendDataHelper documentation

https://github.com/TYPO3-Solr/ext-solr/issues/964

Invalid include paths after scheduler run

https://github.com/TYPO3-Solr/ext-solr/issues/921

Backend Summary not working

https://github.com/TYPO3-Solr/ext-solr/issues/731

Queue initialization returns wrong (duplicate) results for second site root in multisite environment

https://github.com/TYPO3-Solr/ext-solr/issues/488

solrconfig.xml More Like This Handler configuration is broken

https://github.com/TYPO3-Solr/ext-solr/issues/765

Fix position of default stopwords

https://github.com/TYPO3-Solr/ext-solr/issues/578

Bug/unwanted feature in method getPages in Site.php

https://github.com/TYPO3-Solr/ext-solr/issues/652

Custom field is not indexed for custom page queue configuration

https://github.com/TYPO3-Solr/ext-solr/issues/842

GarbageCollector fails to check endtime correctly

https://github.com/TYPO3-Solr/ext-solr/issues/1212

"Only variables should be assigned by reference" in IndexQueueIndexer::preAddModifyDocuments()

https://github.com/TYPO3-Solr/ext-solr/issues/1183

Use urlencode for deletion of synonyms and stopwords

https://github.com/TYPO3-Solr/ext-solr/issues/1205 https://github.com/TYPO3-Solr/ext-solr/issues/1206

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors for this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Andreas Allacher
  • Andriy Oprysko
  • Arno Schoon
  • Frank Nägler
  • Frans Saris
  • Ingo Renner
  • Josef Glatz
  • Markus Kobligk
  • Rafael Kähm
  • Rasmus Larsen
  • Sascha Egerer
  • Thomas Hohn
  • Timo Hund
  • Tomas Norre Mikkelsen

Also a big thanks to our partners that have joined the EB2017 program:

  • Amedick & Sommer
  • amarantus - media design & coding Mario Drengner & Enrico Nemack GbR
  • Animate Agentur für interaktive Medien GmbH
  • artif GmbH & Co. KG
  • AVM Computersysteme Vertriebs GmbH
  • b:dreizehn GmbH
  • Bitmotion GmbH
  • cab services ag
  • Causal Sàrl
  • Creative360
  • cron IT GmbH
  • data-graphics GmbH
  • Deutscher Ärzteverlag GmbH
  • Deutscher Volkshochschul-Verband
  • Die Medialen GmbH
  • dörler engineering services
  • hauptsache.net
  • Havas Düsseldorf GmbH
  • itl AG
  • jweiland.net
  • KEMWEB GmbH & Co. KG
  • Leibniz Universität IT Services, Hannover
  • Lime Flavour GbR
  • LOUIS INTERNET
  • Maximillian Walter
  • Mercedes-AMG GmbH
  • mpm media process management GmbH
  • n@work Internet Informationssysteme GmbH
  • Netcreators BV (Netherlands)
  • NetSpring s.r.l.
  • netz-haut GmbH
  • polar gold
  • punkt.de
  • sitegeist media solutions GmbH
  • Star Finanz GmbH
  • Studio 9 GmbH
  • stratis
  • systime
  • takomat Agentur GbR
  • Triplesense Reply
  • Typoheads GmbH
  • UEBERBIT GmbH
  • WACON Internet GmbH
  • Universität Bremen
  • webconsulting business services gmbh
  • zdreicom AG
  • zimmer7 GmbH

Thanks to everyone who helped in creating this release!

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us in 2017 by becoming an EB partner:

http://www.typo3-solr.com/en/contact/

or call:

+49 (0)69 - 2475218 0

Releases 6.0

Release 6.0.0

Dia dhuit - Hello "Apache Solr for TYPO3" 6.0

We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 6.0.0. With this release we ship support for Apache Solr 6.3 and provide a Docker file, that can be used to spin up a Solr server configured and ready to run within a few steps. Besides that the release contains many other features and bugfixes.

New in this release

In this release we've merged over 120 pull requests! With these pull requests several new features and bugfixes were added.

Apache Solr 6.3 Support

The focus of this release was adding support for the latest version of Apache Solr and making setup easier. Configuration of Apache Solr is now provided via configSets. A configSet contains everything you need - solrconfig.xml, schema.xml & custom access filter JAR files - to configure and run a Solr server for use with TYPO3.

Related PRs:

Access Filter Support for multi-value Access Fields

The next release of EXT:solrfal will allow merging of duplicates into one document. The implementation of this feature requires that the permissions are stored in a multi-value field because different references to a file can have different permissions. Thus the access field is now a multi-value field and the access filter plugin is able to evaluate multiple access values.

Related PRs:

https://github.com/TYPO3-Solr/ext-solr/pull/675

Fluid Templates for System Status Reports

By using the Fluid standalone views we have adapted the status reports to use these views and create better output.

Related PRs:

https://github.com/TYPO3-Solr/ext-solr/pull/740

Support for New Languages (Irish, Serbian and Latvian)

With the new Apache Solr version we have added languages Irish, Serbian, and Latvian. You can now use them to index sites in these languages by creating a Solr core using the configuration provided by the extension.

Related PRs:

https://github.com/TYPO3-Solr/ext-solr/pull/779

### Use Extbase Command Controller to Update Connections

The old cli commands have been replaced with an Extbase command controller.

Please use the following commands to update Solr connections through cli:

php ./typo3/cli_dispatch.phpsh extbase solr:updateConnections
Copied!

Related PRs:

https://github.com/TYPO3-Solr/ext-solr/pull/694

Closer to 8 LTS

Our goal is to be able to support TYPO3 CMS v8 as soon as version 8 LTS will be released. Many patches from the community have been merged to improve the compatibility for version 8 already. Thanks to everyone who worked on that and provided contributions!

Related PRs:

Performance Improvements

Avoid schema retrieval:

Until now the Solr schema was fetched in the frontend even when it was not needed. Now we only fetch the schema in the backend when it's really needed (when updating stopswords or synonyms).

Reduce ping requests:

The amount of ping requests has been reduced to only do one ping when the plugin is rendered.

Improved configuration caching:

Configuration object, are now cached in an in memory cache. This gives an improvement when pages in the backend are copied.

Related PRs:

Allow Solr Server-Side Facet Option Limits

Until now you could only limit the facet options shown when they get rendered in the frontend. Sometimes however you want to reduce the available facet options at an earlier time when retrieving them from Solr.

The following setting now allows you to configure this limit:

plugin.tx_solr.search.faceting.facetLimit = 50
Copied!

Related PRs:

Improved Code Quality With Scrutinizer

To ensure changes don't reduce the quality of the code we have introduced Scrutinizer-CI to provide feedback on code quality. After adding Scrutinizer-CI we also started to resolve initial issues reported step by step and to increase the coverage of automated tests.

You can have a look at the results of the inspections and see the impact there:

https://scrutinizer-ci.com/g/TYPO3-Solr/ext-solr/statistics/

A couple pull requests have been merged to introduce Scrutinizer and to fix several issues:

Related PRs:

Our goal is to improve the code step by step and don't get worse.

Use of Compound Index Format

To avoid an error in Apache Solr with "too many open files", it is possible to use the compound index format. Apache Solr is then writting to only one index file instead of many. The downside however, is that the performance may be slightly reduced but should not be significant for most projects. We decided to change the default configuration here to use a more robust setting. If you have very high performance requirements you can still change this behavior by setting <useCompoundFile>false</useCompoundFile> in your solrconfig.xml.

Related PRs:

Added Schema Field for Exact Matches

We've added a new data type "textExact" to the Solr schema. Beside that copyFields have been added for the following fields:

  • titleExact
  • contentExact
  • tagsH1Exact
  • tagsH2H3Exact
  • tagsH4H5H6Exact
  • tagsInlineExact

Beside that this type is also available as dynamic fields with the following suffixes:

  • _textExactS
  • _textExactM

Related PRs:

Username and Password for Solr Connection

Username and password can now be configured for the Solr connection:

plugin.tx_solr.solr.username = username
plugin.tx_solr.solr.password = password
Copied!

Related PRs:

Timeout for Solr Connection

A timeout for the Solr connection can now be configured:

plugin.tx_solr.solr.timeout = 20
Copied!

Related PRs:

Rendering Instruction for DateFormatting

The following rendering instruction can be used, when you want to format a date as option facet and store it as date or timestamp.

plugin.tx_solr.search.faceting.facets.created {
   field = created
   label = Created
   sortBy = alpha
   reverseOrder = 1
   renderingInstruction = TEXT
   renderingInstruction {
      field = optionValue
      postUserFunc = ApacheSolrForTypo3\Solr\Domain\Search\ResultSet\Facets\RenderingInstructions\FormatDate->format
   }
}
Copied!

Related PRs:

Breaking Changes

Updating from version 5.1.1 or earlier there are some breaking changes that your need to keep in mind when updating.

Apache Solr 6 and Access Filter

Since version 6 Apache Solr comes with an embedded Jetty server which completely changes the setup compared to Apache Solr 4.10. It is now required to install a new instance of Apache Solr Server. This can be done with our install script for development or you can use our Docker image to start a new Solr server as a container. Both approaches are described in our documentation. The embedded Solr Jetty server also uses a new default port (8983) to serve connections.

Migration to Command Controllers for CLI

We are now using a command controller to update the Solr server connections. Please use the following command to update your connections:

php ./typo3/cli_dispatch.phpsh extbase solr:updateConnections
Copied!

Bugfixes

The following bugs have been fixed in this release.

Outlook

In the next release (6.1) our focus will be to prepare support for TYPO3 CMS version 8 LTS as well as possible. The next release (6.1) will also be the last release for TYPO3 CMS version 7.6 LTS.

Contributors

Like always this release would not have been possible without the help from our awesome community. Here are the contributors for this release.

(patches, comments, bug reports, reviews, ... in alphabetical order)

  • Anjey
  • Claus Due
  • Daniel Siepmann
  • Dominique Kreemers
  • Georg Ringer
  • Hendrik Putzek
  • Ingo Renner
  • Josef Glatz
  • Markus Friedrich
  • Markus Kasten
  • Michiel Roos
  • Olivier Dobberkau
  • Patrick Oberdorf
  • Peter Kraume
  • Philipp Gampe
  • Pierrick Caillon
  • Sascha Egerer
  • Thomas Hohn
  • Timo Hund
  • Tomas Norre Mikkelsen

Also a big thanks to our partners that have joined the EB2016 program:

  • Arrabiata Solutions GmbH & Co. KG
  • avonis
  • Bank CIC AG
  • Bitmotion GmbH
  • Citkomm services GmbH
  • cron IT
  • CS2 AG
  • Cosmoblonde GmbH
  • Daniz online markting
  • datenwerk innovationsagentur gmbh
  • Die Medialen GmbH
  • die_schnittsteller GmbH
  • E-magineurs
  • Fernando Hernáez Lopez
  • Future Connection AG
  • Gernot Leitgab
  • .hausformat
  • Hirsch & Wölfl GmbH
  • hs-digital GmbH
  • IHK Neubrandenburg
  • internezzo AG
  • jweiland.net
  • L.N. Schaffrath DigitalMedien GmbH
  • mehrwert intermediale kommunikation GmbH
  • netlogix GmbH & Co. KG
  • Pixel Ink
  • Pixelpark AG
  • pixolith GmbH & Co. KG
  • polargold GmbH
  • portrino GmbH
  • Q3i GmbH & Co. KG
  • raphael gmbh
  • RUAG Corporate Services AG
  • sitegeist media solutions GmbH
  • ST3 Elkartea
  • Star Finanz-Software Entwicklung und Vertriebs GmbH
  • Stefan Galinski Interndienstleistungen
  • Speedpartner GmbH
  • sunzinet AG
  • Systime A/S
  • SYZYGY Deutschland GmbH
  • tecsis GmbH
  • web-vision GmbH
  • websedit AG - Internetagentur
  • Webstobe GmbH
  • werkraum GmbH
  • WIND Internet
  • wow! solution
  • zdreicon AG

Thanks also to our partners who already singed up for a 2017 partnership (EB2017):

  • Amedick & Sommer Neue Medien GmbH
  • cron IT GmbH
  • b:dreizehn GmbH
  • Die Medialen GmbH
  • Leibniz Universität IT Services, Hannover
  • LOUIS INTERNET
  • polargold GmbH
  • Mercedes-AMG GmbH
  • Triplesense Reply GmbH
  • zdreicom AG

Thanks to everyone who helped in creating this release!

How to Get Involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports and feature requests on GitHub
  • Ask or help or answer questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Support us in 2017 by becoming an EB partner:

http://www.typo3-solr.com/en/contact/

or call:

+49 (0)69 - 2475218 0

Releases 5.1

Release 5.1.0

Apache Solr for TYPO3 version 5.1 released

We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 5.1.0. This release ships a few bugfixes and smaller features

New in this Release

The focus of this release was to provide the new features from the codesprint and bugfixes from the issue tracker. Beside that, we've added a variants feature that could be used together with the upcomming solrfal release 3.2 as one approach to avoid duplicated files in the search frontend.

Document Variants

This feature adds a new optional field variantId that is filled with type/uid and can be used for collapsing by default.

The following example shows the required TypoScript configuration that is needed:

plugin.tx_solr.search {
    variants = 0
    variants {
        expand = 1
        variantField = variantId
        limit = 10
    }
}
Copied!

The collapsing can be used to collapse on numeric and string fields, text fields are not supported for collapsing by Solr.

Details: https://github.com/TYPO3-Solr/ext-solr/pull/550

Improve Documentation

During the transition from forge to github, we've moved the rest of the documentation to our rst documentation.

Details: https://github.com/TYPO3-Solr/ext-solr/pull/547

Backend statistics

During the codesprint in Frankfurt a new statistics module was added for that backend that allows you to get some usefull insights of your search installation.

  • Which terms are the most frequent search terms?
  • Which terms are frequently searched but deliver no result?

UX Improvements

During the codesprint several UX improvements in the backend module have been done:

Replace content element wizard icon with svg icon:

  • Replace content element wizard icon with svg icon

Detail: https://github.com/TYPO3-Solr/ext-solr/pull/581

Make init connections icon use SVG:

  • Adds new SVG icon for init connection

Details: https://github.com/TYPO3-Solr/ext-solr/pull/566

Backend module layout fixes

  • Makes sure everything is aligned properly
  • Removes some whitespace, giving the module content more room

Details: https://github.com/TYPO3-Solr/ext-solr/pull/575

Bugfix: absRefPrefix auto in cli mode with webroot different from PATH_site

When you are using absRefPrefix auto and index in cli mode and you webroot differs from PATH_site you have the possibility to configure a webroot in the scheduler task to make absRefPrefix=auto work in this case

Details: https://github.com/TYPO3-Solr/ext-solr/pull/495

Bugfix: Keep selected site in scheduler task

When a site in the scheduler task was selected the selection could get lost.

Details: https://github.com/TYPO3-Solr/ext-solr/pull/557

Outlook

In the next release (6.0) we will focus on the integration of the latest Apache Solr Version (currently 6.1.0)

Contributors

Like always this release would not have been possible without the help from our awesome community. These are the contributors for this release.

(patches, comments, bug reports, review, ... in alphabetical order)

  • Daniel Siepmann
  • Ingo Renner
  • Steffen Ritter
  • Timo Hund
  • Thomas Hohn
  • Thomas Scholze

Also a big thanks to our partners that have joined the EB2016 program:

  • Bank CIC AG
  • Bitmotion GmbH
  • Citkomm services GmbH
  • CS2 AG
  • Cosmoblonde GmbH
  • Daniz online markting
  • datenwerk innovationsagentur gmbh
  • die_schnittsteller GmbH
  • E-magineurs
  • Fernando Hernáez Lopez
  • Future Connection AG
  • Gernot Leitgab
  • Hirsch & Wölfl GmbH
  • hs-digital GmbH
  • IHK Neubrandenburg
  • L.N. Schaffrath DigitalMedien GmbH
  • mehrwert intermediale kommunikation GmbH
  • netlogix GmbH & Co. KG
  • pixolith GmbH & Co. KG
  • Q3i GmbH & Co. KG
  • RUAG Corporate Services AG
  • ST3 Elkartea
  • Star Finanz-Software Entwicklung und Vertriebs GmbH
  • Stefan Galinski Interndienstleistungen
  • Systime A/S
  • SYZYGY Deutschland GmbH
  • web-vision GmbH
  • websedit AG - Internetagentur
  • Webstobe GmbH
  • WIND Internet

Thanks to everyone who helped in creating this release!

How to get involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports, and feature requests on GitHub
  • Ask or help answering questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Releases 5.0

Release 5.0.0

Apache Solr for TYPO3 version 5.0 released

We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 5.0.0. With this release we ship the requirements to use our new addon EXT:solrfluid, that allows you to render your search results with the fluid templating engine.

Beside the adaptions for solrfluid we mainly did bugfixes and cleanup in the code base. If you want to use solrfluid in your projects, you should signup a partnership with dkd (visit typo3-solr.com or call dkd +49 (0)69 - 247 52 18-0 for more information).

New in this Release

Preparing fluid support

  • Adding a SearchRequest object
  • Improving the SearchResultSetService

Dropping deprecated code

As announced in Version 4.0 the following methods have been removed:

  • Util::getTypoScriptObject please use TypoScriptConfiguration::getObjectByPath() instead.
  • Util::isValidTypoScriptPath please use TypoScriptConfiguration::isValidPath() instead.
  • Util::getTypoScriptValue please use TypoScriptConfiguration::getValueByPath() instead.
  • IndexQueueQueue::getTableToIndexByIndexingConfigurationName please use TypoScriptConfiguration::getIndexQueueTableNameOrFallbackToConfigurationName instead.
  • IndexQueueQueue::getTableIndexingConfigurations please use TypoScriptConfiguration::getEnabledIndexQueueConfigurationNames instead.
  • PluginPluginBase::search / PluginBase::getSearch / PluginBase::setSearch please use $pi->getSearchResultSetService()->getSearch() instead.
  • PluginResultsResults::getAdditionalFilters please use $pi->getSearchResultSetService()->getAdditionalFilters() instead.
  • PluginResultsResults::getNumberOfResultsPerPage use $pi->getSearchResultSetService()->getNumberOfResultsPerPage() instead.
  • PluginResultsResults::getAdditionalFilters please use $pi->getSearchResultSetService()->getAdditionalFilters() instead.
  • TypoScriptConfiguration::offsetGet / offsetExists / offsetSet please use TypoScriptConfiguration::getObjectByPath / isValidPath / getValueByPath instead. These functions have only been implemented for backwards compatibility in will be removed in 5.0

Dropping the class maps

Since EXT:solr 5.0 is only running on TYPO3 7.6 LTS and classmap files for backwards compatibility have been dropped. This also gives am small performance improvement for the current version.

Unify xliff files in one file per language

Because we want to use the labels in EXT:solr and EXT:solrfluid, and extbase requires to use one xliff file per language, we take the chance to streamline the language files into one file per language.

Performance improvement for TypoScriptConfiguration

Because TypoScriptConfiguration is used quite frequently, we did some performance improvements by optimizing the access to the underling array structure.

Add additionalWhereClause to SOLR_RELATION

You can now add an additionalWhereClause for SOLR_RELATION items.

Example:

plugin.tx_solr.index.queue {
    record = 1
    record {
        table = tx_extension_domain_model_record

        fields {
            title = title
            category_stringM = SOLR_RELATION
            category_stringM {
                localField = category
                multiValue = 1
                additionalWhereClause = pid=2
            }
        }
    }
}
Copied!

See: https://github.com/TYPO3-Solr/ext-solr/issues/426

Show devLog entries as debug message or in the TYPO3 console

You can now show the written log messages as output of the debug console in the backend or debug messages in the frontent, when you enable the following setting:

plugin.tx_solr.logging.debugDevlogOutput = 1
Copied!

RST Documentation

We've ported the TypoScript reference from the forge wiki into an rst format. At the same time we've enhanced the documentation with chapters about:

  • Getting Started
  • The Backend
  • Logging
  • FAQ section

If you want to contribute and bring the documentation one step further feel free to proof-read or provide additional parts as a pull request.

Deprecations

  • Typo3Environment: The usage of ApacheSolrForTypo3SolrTypo3Environment is deprecated and should be removed

Outlook

In the next release (6.0) we will focus on our codesprint (11. to 13. august in frankfurt). Beside that we will focus on issues in the issue tracker and general improvements of the codebase and documentation.

Contributors

Like always this release would not have been possible without the help from our awesome community. These are the contributors for this release.

(patches, comments, bug reports, review, ... in alphabetical order)

  • Daniel Siepmann
  • Frans Saris
  • Gerald Spreer
  • Hendrik Nadler
  • Ingo Renner
  • Marc Bastian Heinrichs
  • Markus Friedrich
  • Olivier Dobberkau
  • Patrick Oberdorf
  • Peter Kraume
  • Sascha Löffler
  • Timo Hund

Also a big thanks to our partners that have joined the EB2016 program:

  • Bank CIC AG
  • CS2 AG
  • Cosmoblone GmbH
  • Daniz online markting
  • datenwerk innovationsagentur gmbh
  • die_schnittsteller GmbH
  • E-magineurs
  • Fernando Hernáez Lopez
  • Future Connection AG
  • Hirsch & Wölfl GmbH
  • hs-digital GmbH
  • L.N. Schaffrath DigitalMedien GmbH
  • pixolith GmbH & Co. KG
  • Q3i GmbH & Co. KG
  • RUAG Corporate Services AG
  • ST3 Elkartea
  • Star Finanz-Software Entwicklung und Vertriebs GmbH
  • Stefan Galinski Interndienstleistungen
  • Systime A/S
  • websedit AG - Internetagentur
  • Webstobe GmbH
  • web-vision GmbH

Thanks to everyone who helped in creating this release!

How to get involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports, and feature requests on GitHub
  • Ask or help answering questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Releases 4.0

Release 4.0.0

Apache Solr for TYPO3 version 4.0 released

We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 4.0.0. With this release we now support TYPO3 CMS versions 7.6LTS together with Apache Solr version 4.10. At the same time we also started making the extension work with TYPO3 8.

New in this release

Support of TYPO3 7.6 LTS and TYPO3 8.0 (Dropping Support of 6.2 LTS)

To stay up-to-date with the TYPO3 core we decided to drop the support of 6.2 LTS and support 7.6 LTS only. We also made sure that EXT:solr works with TYPO3 8.0, this will be an ongoing effort until the LTS release of TYPO3 8. Until then you may experience occasional bugs as we try to keep up with the core's development.

Improve the Test Coverage for Fluid Integration and Refactoring

During the preparation for the upcoming fluid integration we used the chance to refactor the frontend part of the extension.

What we did here:

  • Move the logic from the Result/Search and FrequentSearches plugins to service classes in order to have the possibility to use them in different places (e.g. other controllers)
  • Introduce a SearchResultSet and SearchResultSetService to improve the object
    model of search results

TypoScriptConfiguration Object and Streamlining of Configuration usage

TypoScript configuration is used in many places and accessing it with an object has some advantages:

  1. Merging configuration with Flexforms and other sources is easier
  2. We have a single point of access for configuration
  3. We can move and remove isset()/is_array()/if..else logic to read and evaluate configuration into this object and make the code that is using it more readable
  4. Since the implementation is a cross cutting concern for the whole extension, it can be used to improve the coverage of integration tests for all parts
  5. By reading the annotations later it could be used to automatically generate
    a TypoScript reference

During the preparation of the current release we implemented this TypoScriptConfiguration object and replaced all parts in the scope of EXT:solr

Migration:

For backwards compatibility the TypoScriptConfiguration object can be used with array access operators. If you want to read the configuration you can use the speaking method name for a configuration path or use the methods getValueByPath() or getObjectByPath().

PHP 7.0 ready

During the development we made sure that the code of EXT:solr works with PHP 7.0.

General Changes

Beside the major changes that have been mentioned before, there are a few minor changes that might be interesting:

ExcludeContentByClass

You can now use a new TypoScript setting plugin.tx_solr.index.queue.pages.excludeContentByClass.

e.g.

plugin.tx_solr.index.queue.pages.excludeContentByClass = removeme
Copied!

Removal of q.op Parameter from solrconfig.xml

Since solrconfig.xml contained a default value for the mm (MinimumMatch) and q.op parameters and it does not make sense to have both at the same time for eDismax we removed the q.op parameter from the default configuration to avoid confusion.

Allow Indexing from Backend Module

To avoid switching from the backend module to scheduler and back, we added a button to do a simple indexing run with just 10 documents.

Resolving Overlays of SOLR_RELATION objects

The SOLR_RELATION object is now taking an overlay into account when relations are resolved. This allows you to resolve different relations from a translated entity for an original entity.

Auto-Correct in Default Template

The goal of the search is to keep the visitor on the site and provide the best results for them with the smallest possible interaction.

There was a feature in EXT:solr (searchUsingSpellCheckerSuggestion) that could be used to automatically trigger a search using the spellchecker's suggestions when a search with the original term did not retrieve any results. To simplify the integration we added the needed markup to the default templates so that you now just need to enable the feature and use it with the shipped default template.

How to use it?

plugin.tx_solr.search.spellchecking.searchUsingSpellCheckerSuggestion = 1
Copied!

Use the shipped default templates or add the following snippet to your template (between <!-- ###SOLR_SEARCH_NO_RESULTS### begin --> and <!-- ###SOLR_SEARCH_NO_RESULTS### end -->:

<div class="tx-solr-search-suggestion_results">
    <!-- ###IF:###NO_RESULTS.HAS_SUGGESTION_RESULTS###|==|1### begin -->
    ###NO_RESULTS.SUGGESTION_RESULTS###
    <!-- ###IF:###NO_RESULTS.HAS_SUGGESTION_RESULTS###|==|1### end -->
</div>
Copied!

Dynamic Field Type for Spellchecking

For spellchecking we had no dynamic field type before. Since 4.0.0 we now ship the dynamic field types *_textSpellS and *_textSpellM to allow you to easily define fields for spell checking.

Usage of Styles and Modals from TYPO3 7 LTS

To keep the styling of the backend up-to-date and use the new JavaScript modals we updated the backend module to use these new features.

Add Support of FastVectorHighlighter

Highlighting can be slow for large text documents - f.e. created when indexing pdf documents - and slow down search queries. To help prevent this issue we added support for Solr's [FastVectorHighlighter](https://cwiki.apache.org/confluence/display/solr/FastVector+Highlighter)

The FastVectorHighlighter will now be used by default if fragmentSize is set to at least 18 (this is required by the FastVectorHighlighter to work).

Use Caching Framework in Util::getConfigurationFromPageId

Configuration is retrieved and evaluated many times during indexing. Thus it made sense to use the TYPO3 caching framework to cache these expensive operations and improve their performance.

Use Xliff as Translation Format

Translation files have been migrated to the xliff format. The EXT:solr team would be happy if you contribute missing translations.

Resolve affected pages when "extendToSubpages" is set and "hidden" flag is changed

The Record Monitor is now able to evaluate the "hidden" flag on subpages when the extendToSubpages page property is set or unset.

Add signal after IndexQueueWorkerTask::indexItems

You can now register for a new signal "afterIndexItems".

Bugfixes

Beside the improvements mentioned before many smaller bugs have been fixed in this release as well.

Installation and Updating

This version of EXT:solr is based on the same Apache Solr Version (4.10) as the last version so an update for the Solr server itself is not needed.

However, a few modifications to the schema and Solr configuration where made. Please make sure to use the latest solrconfig.xml and schema versions.

You can always check TYPO3's system status report to see whether your setup meets the requirements and is up-to-date to be used with EXT:solr.

Deprecations

The following functions and methods have been marked as deprecated and will be removed in version 5.0

  • Util::getTypoScriptObject please use TypoScriptConfiguration::getObjectByPath() instead.
  • Util::isValidTypoScriptPath please use TypoScriptConfiguration::isValidPath() instead.
  • Util::getTypoScriptValue please use TypoScriptConfiguration::getValueByPath() instead.
  • IndexQueueQueue::getTableToIndexByIndexingConfigurationName please use TypoScriptConfiguration::getIndexQueueTableNameOrFallbackToConfigurationName instead.
  • IndexQueueQueue::getTableIndexingConfigurations please use TypoScriptConfiguration::getEnabledIndexQueueConfigurationNames instead.
  • PluginPluginBase::search / PluginBase::getSearch / PluginBase::setSearch please use $pi->getSearchResultSetService()->getSearch() instead.
  • PluginResultsResults::getAdditionalFilters please use $pi->getSearchResultSetService()->getAdditionalFilters() instead.
  • PluginResultsResults::getNumberOfResultsPerPage use $pi->getSearchResultSetService()->getNumberOfResultsPerPage() instead.
  • PluginResultsResults::getAdditionalFilters please use $pi->getSearchResultSetService()->getAdditionalFilters() instead.
  • TypoScriptConfiguration::offsetGet / offsetExists / offsetSet please use TypoScriptConfiguration::getObjectByPath / isValidPath / getValueByPath instead. These functions have only been implemented for backwards compatibility in will be removed in 5.0

Outlook

The next release (5.0) will focus on the needed refactorings to allow implementing fluid template rendering. At the same time we will ship the first version of our new extension "solrfluid" to our EAP partners.

Contributors

Like always this release would not have been possible without the help from our awesome community. These are the contributors for this release.

(patches, comments, bug reports, review, ... in alphabetical order)

  • Dmitry Dulepov
  • Florian Seirer
  • Frank Nägler
  • Frans Saris
  • Ingo Pfennigstorf
  • Ingo Renner
  • Marc Bastian Heinrichs
  • Markus Friedrich
  • Markus Kobligk
  • Mickael Vanclooster
  • Olivier Dobberkau
  • Sascha Egerer
  • Timo Schmidt

Thanks to everyone who helped in creating this release!

How to get involved

There are many ways to get involved with Apache Solr for TYPO3:

  • Submit bug reports, and feature requests on GitHub
  • Ask or help answering questions in our Slack channel
  • Provide patches through Pull Request or review and comment on existing Pull Requests
  • Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3

Releases 3.1

Release 3.1.0

Apache Solr for TYPO3 version 3.1 released

We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 3.1.0. With this release we now support TYPO3 CMS versions 6.2LTS and 7.6LTS together with Apache Solr version 4.10.

New in this Release

Support of TYPO3 7.6 LTS:

We added support for TYPO3 CMS 7.6LTS while still supporting version 6.2LTS to allow an easy upgrade. Support for TYPO3 version 4.5LTS has been dropped with this release. The next release will likely require at least TYPO3 7.6LTS.

On the code side we now use PSR-2 coding style like in TYPO3 7.6LTS. This way it is easy for contributors and developers to stick with a common coding style. For contributions PSR-2 style is also enforced through automatic checks now.

Overall we resolved over 40 issues and added almost 700 commits.

Apache Solr 4.10

The extension comes with an installer that installs Apache Tomcat 8.0.29 and Apache Solr 4.10.4. Please make sure to update to Apache Solr 4.10.4 and the new Solr schema and configuration provided by the extension to make sure the extension works as intended.

Manage Stopwords from the Backend

With EXT:solr 3.0 we introduced an all new backend module to manage the Index Queue, check index data, and manage synonyms. With this release we're adding a new sub module to allow editing stopwords.

Stopwords are usually used to discard common words when indexing. This can increase relevance of documents.

Introducing PHP Namespaces

Our complete codebase is now namespaced and uses root namespace "ApacheSolrForTypo3Solr" for all classes. We also added a class map to provide backwards compatibility. Nevertheless we suggest updating your extensions to use the namespaced class names if you use them.

Introducing Travis CI

For a long time we already had a small unit test suite. However, it was not kept up-to-date, tests weren't executed automatically and generally in a state that left room for improvement. Starting with this release we will focus on increasing test coverage and keeping them up-to-date.

The unit test suite is now executed automatically for each Pull Request through Travis CI (https://travis-ci.org/TYPO3-Solr/ext-solr). Currently we run tests against TYPO3 version 6.2 LTS, 7.6 LTS, and dev-main with the same PHP versions as TYPO3 CMS, PHP 5.3-5.6.

Additionally Travis CI also checks for PSR-2 compliance and will fail a PR in case the code does not fit that coding standard.

In case a Pull Request is rejected by Travis CI you can follow the link to the build and inspect its output to see what needs to be fixed.

General Changes

HTML escaping after retrieval from Solr

The data ist now escaped right after the retrieval from Solr. In rare cases when you need to store HTML in Solr documents you can use the following configuration to mark these fields as trusted fields and skip the escaping:

plugin.tx_solr.search.trustedFields = my_first_html_field,my_second_html_field
Copied!

Synonym handling (Lowercasing and Evaluation at Index- and Querytime)

Synonyms are now handled at query and index time. The advantage is, that when editing synonyms from the backend module those changes become effective immediately.

Removed page browser dependency

As we were preparing to be compatible with TYPO3 7.6LTS we found that there is no compatible version of EXT:pagebrowse. We used that extension to provide the page browser in results listings. To become compatible with TYPO3 7.6LTS we now have integrated the page browser into EXT:solr itself. So if you are not using EXT:pagebrowse for anything else you can safely remove the extension from your installation.

Installation and Updating

To install EXT:solr version 3.1.0 you will need either TYPO3 6.2LTS or TYPO3 7.6LTS. On the Solr side we require Apache Solr 4.10.4, which requires Java 7.

When updating an existing installation to EXT:solr 3.1.0 please make sure to update to Apache Solr 4.10.4 including the new schema and configuration files. The extension comes with a script that will install a working setup for you automatically. The script can be found in EXT:solr/Resources/Install/.

In rare cases you may have to re-initialize the Solr connections. You can do so from TYPO3's clear cache menu in the top toolbar. Because of the changes made to the schema you will need to re-index your site.

Make sure to check the system status report for any warnings or errors.

### TypoScript Paths changed

During the migration to namespace we also had to change the TypoScript paths:

tx_solr_pi_results => tx_solr_PiResults_Results tx_solr_pi_search => tx_solr_PiSearch_Search tx_solr_pi_frequentsearches => tx_solr_PiFrequentSearches_FrequentSearches

If you're building custom TypoScript content objects referencing Solr plugin configurations you should make to update these as well.

Outlook

With this release out the door we will focus on updating the add-on extensions like EXT:solrgrouping, EXT:tika, and EXT:solrfal for file indexing, these should be available soon, too.

The next release within the 3.x series will likely require at least TYPO3 7.6LTS and might as well arrive soon as there are no bigger changes planned yet besides dropping support for TYPO3 6.2LTS.

After that we will start working on the long-awaited move to Fluid as the template engine for version 4.0.

Contributors

Like always this release would not have been possible without the help from our awesome community. These are the contributors for this release.

(patches, comments, bug reports, review, ... in alphabetical order)

  • Alexander Stehlik
  • Andreas Allacher
  • Andreas Fernandez
  • Andreas Wolf
  • Daniel Siepmann
  • Dmitry Dulepov
  • Frank Nägler
  • Frans Saris
  • Gordon Brüggemann
  • Hans Höchtl
  • Heiko Hardt
  • Hendrik Nadler
  • Ingo Renner
  • Javn Wagner
  • Jens Jacobsen
  • Jigal van Hemert
  • Joschi Kuphal
  • Josef Florian Glatz
  • Kevin von Spiczak
  • Marc Bastian Heinrichs
  • Markus Friedrich
  • Markus Günther
  • Markus Sommer
  • Markus Kobligk
  • Maxime Lafontaine
  • Michael Knabe
  • Michiel Roos
    1. Tillmann
  • Olivier Dobberkau
  • Peter Kraume
  • Phuong Doan
  • Romain Canon
  • Sascha Affolter
  • Sascha Egerer
  • Sascha Nowak
  • Sebastian Enders
  • Soren Malling
  • Stefan Galinski
  • Stefan Neufeind
  • Steffen Müller
  • Steffen Ritter
  • Thomas Heilmann
  • Thomas Janke
  • Timo Schmidt
  • Witali Rott
  • Thomas Heilmann

Thanks to everyone who helped in creating this release!

Introducing Timo Schmidt

Last but not least we would like to introduce Timo Schmidt. Timo recently joined dkd to work on Apache Solr for TYPO3 in support and development. If you've been active in our Slack channel or followed our GitHub commits you may have been in contact with Timo already. We're looking forward to working with Timo.

How to get involved

There are many ways to get involved with Apache Solr for TYPO3:

Releases 3.0

Release 3.0.0

Apache Solr for TYPO3 version 3.0 released

We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 3.0.0. With this release we now support TYPO3 CMS versions 4.5 to 6.2, and Apache Solr version 4.8.

A couple weeks ago we released Apache Solr for TYPO3 version 3.0 to the TYPO3 Extension Repository. Even without this announcement we have seen quite a few downloads already and have also been able to help projects update to the current version even during the development phase.

New in this Release

TYPO3 CMS 6.2 LTS

We added support for TYPO3 CMS 6.2 LTS while still supporting version 4.5 LTS to allow an easy upgrade. Which means you can now use system categories to build hierarchical facets, and on the code side we moved a lot of code around to match the file and folder structure of extbase extensions. All New Search Backend Module

If you are running TYPO3 CMS version 6.0 or newer you will be pleased to find a completely new backend module to get an overview of your Solr installation, the Index Queue, and execute index maintenance tasks.

The new backend module is designed so that it can easily be extended with sub-modules. One such sub-module is the new synonyms module allowing you to manage synonyms for your site keywords. Further modules are planned and we also encourage others to add functionality using the new API.

Apache Solr 4.8

In the background we also updated Apache Solr to version 4.8 which brings new features and allows us to make things even easier for integrators.

New features resulting from the Solr update include highlighting elevated search results (editorial or paid content).

When indexing content before, you had to wait for Solr to commit newly added documents before seeing the changes show up in the search results. This is not necessary anymore as Solr 4.0 added near real time search and automatic soft commits, meaning that documents added to the Solr index now become searchable immediately.

General Changes

Overall we have resolved close to 80 issues adding up to almost 400 changes. Other than the changes listed above we also added support for limiting a search to certain branches of the page tree. Of course the usual bug fixes and performance improvements were added as well, too.

Updating

During the reorganization of the files and folders we also moved the TypoScript configuration files. Because of this the include path for these files will not match anymore. You can simply re-add the TypoScript in the Include Static section of your template record. We also added an upgrade script that does that for you automatically.

Two scheduler tasks have been removed; the commit scheduler task was meant for development purposes only anyway and is not needed anymore thanks to Solr's automatic soft commits. The optimize task has been removed since its name was a bit misleading and in certain situations it could do more harm than any good.

Outlook

The release of version 3.0 lays the foundation for further changes to come in keeping the extension up to date with developments in both TYPO3 CMS and Apache Solr, providing the best search experience available for TYPO3 CMS.

New add-ons like FAL file indexing and updated versions of existing add-on extensions will be provided through the Early Access Program.

There may be one or two more minor release in the 3.x branch to deliver features that did not make it in 3.0. After that we will be moving on to the development of 4.0 which will have a minimum requirement of TYPO3 CMS 6.2 LTS. The main goal for version 4 will be switching to Fluid for templating and adding namespaces.

Contributors

Like always this release would not have been possible without the help from our awesome community. These are the contributors for this release.

(patches, comments, bug reports, ... no particular order)

  • Sascha Egerer
  • John Foushee
  • Tolleiv Nietsch
  • Dmitry Dulepov
  • Stefan Galinski
  • Franz G. Jahn
  • Hauke Meyer
  • Jonas Götze
  • Joschi Kuphal
  • Kay Strobach
  • Stefan Sprenger
  • Marc Bastian Heinrichs
  • Lienhart Woitok
  • Kai Vogel
  • Jochen Rieger
  • Irene Höppner
  • Tomita Militaru
  • Jan Kiesewetter
  • Lars Peipmann
  • Markus Kobligk
  • Michael Knabe
  • Michiel Roos
  • Peter Kraume
  • Gerrit Venema
  • Michel Tremblay
  • Lucas Jenss
  • Steffen Ritter
  • Hans Höchtl
  • Stefan Neufeind
  • Soren Malling
  • Witali Rott
  • Bernhard Kraft
  • Phuong Doan
  • Stephan Schuler
  • Tim Werdin
  • Rik Willems
  • Rémy Daniel
  • Andreas Lappe
  • Sascha Nowak
  • Alexander Stehlik
  • Hans Höchtl
  • Cyrel Wolfangel
  • Georg Kuehnberger
  • Jan-Erik Revsbech
  • Gabe Blair
  • Bart Gijswijt
  • Arjen Hoekema

Thanks to everyone who helped in creating this release!

Sitemap

Appendix - Docker Tweaks

Beside of original Apache Solr Docker image, on which is our image based on we provide some tweaks to make our and your work simpler.

Disable unnecessary cores on container start

By defining env TYPO3_SOLR_ENABLED_CORES with a space separated list of languages/cores to enable, only those cores will be initialized on start-up. This allows to save memory usage of Apache Solr server instance.

Usage:

docker run -e 'TYPO3_SOLR_ENABLED_CORES=english german' -it typo3solr/ext-solr
Copied!

Appendix - Dynamic Fields

Dynamic fields allow you to add custom fields to Solr documents. That said, you never need to modify Solr's schema (which could cause problems or at least unnecessary additional work when updating the Solr extension). The following sections describe how to use dynamic fields with your Solr for TYPO3 installation. Usage of dynamic fields

You can use dynamic fields by following a special naming convention for document fields. E.g. to create a dynamic field that is a string the field name should end with _stringS. So if you want to create a field for storing a title you would name it title_stringS. We suggest you use lower camel case for the field name followed by an underscore followed by the dynamic field type "extension".

We've predefined the following dynamic fields:

Extension Type Multivalue Comment
*_stringS String No  
*_stringM String Yes  
*_stringCollatedS string_collated No  
*_stringCollatedM string_collated Yes  
*_binS binary No Stored but not indexed
*_binM binary Yes Stored but not indexed
*_boolS Boolean No  
*_boolM Boolean Yes  
*_intS Integer No  
*_intM Integer Yes  
*_tIntS Integer No Deprecated use _intS removed in EXT:solr 10
*_tIntM Integer Yes Deprecated use _intM removed in EXT:solr 10
*_longS Long No  
*_longM Long Yes  
*_tLongS Long No Deprecated use _longS removed in EXT:solr 10
*_tLongM Long Yes Deprecated use _longM removed in EXT:solr 10
*_floatS Float No  
*_floatM Float Yes  
*_tFloatS Float No Deprecated use _floatS removed in EXT:solr 10
*_tFloatM Float Yes Deprecated use _floatM removed in EXT:solr 10
*_doubleS Double No  
*_doubleM Double Yes  
*_tDoubleS Double No Deprecated use _doubleS removed in EXT:solr 10
*_tDoubleM Double Yes Deprecated use _doubleS removed in EXT:solr 10
*_tDouble4S Double No Deprecated use _double4S removed in EXT:solr 10
*_tDouble4M Double Yes Deprecated use _double4M removed in EXT:solr 10
*_dateS Date No  
*_dateM Date Yes  
*_tDateS Date No Deprecated use _dateS removed in EXT:solr 10
*_tDateM Date Yes Deprecated use _dateM removed in EXT:solr 10
*_random Random No  
*_textS Text No  
*_textM Text Yes  
*_textTS Text Tight No  
*_textTM Text Tight Yes  
*_textSortS Sortable Text No  
*_textSortM Sortable Text Yes  
*_textWstS Whitespace tokenized Text No  
*_textWstM Whitespace tokenized Text Yes  
*_textEdgeNgramS Edge Ngram (hello => hello, hell..) No  
*_textEdgeNgramM Edge Ngram (hello => hello, hell..) Yes  
*_textNgramS Ngram (hello => he,ll,lo,hel,llo) No  
*_textNgramM Ngram (hello => he,ll,lo,hel,llo) Yes  
*_textPath textPath No  
*_textExactS textExact No  
*_textExactM textExact Yes  
*_textSpellS textSpell No  
*_textSpellM textSpell Yes  
*_textSpellExactS textSpellExact No  
*_textSpellExactM textSpellExact Yes  
*_phoneticS Phonetic No  
*_phoneticM Phonetic Yes  
*_point point No  
*_location location No  
*_coordinate double    
*_locationRpt locationRpt No  
*_currency currency No  

Appendix - Version Matrix

Supported versions

List of EXT:solr versions and the matching versions of Apache Solr and TYPO3 that are supported:

TYPO3 EXT:solr (↻) EXT:tika (↻) EXT:solrfal ($) EXT:solrconsole ($) EXT:solrdebugtools ($) EXT:solrfluidgrouping ($↺) Apache Solr Configset
13.4 13.0 13.0 13.0 13.0 13.0 N/A (integrated in EXT:solr) 9.8.1¹ ext_solr_13_0_0
12.4 12.0 12.0 12.0 12.0 12.0 N/A (integrated in EXT:solr) 9.8.1¹ ext_solr_12_0_0
$ - Funding contribution extensions. See: https://www.typo3-solr.com/solr-for-typo3/open-source-version/
$↺ - Published funding contribution. Previously as ($), but merged in EXT:solr core
↻ - Open Source and financed by ($)
Ø - not yet available
ᾱ - non stable alpha release
β - non stable beta release
rc - release candidate available
¹ - recommended Apache Solr version, check version matrix in composer.json (composer info:solr-versions) for full list

Extended Long Term Support (ELTS)

Since January 2022, we have been following the TYPO3 release cycles and actively support the last two TYPO3 versions; in addition, we offer ELTS support for selected older versions. The following table illustrates the offers and available and upcoming versions:

TYPO3 ELTS EXT:solr ($) EXT:tika EXT:solrfal ($) EXT:solrconsole ($) EXT:solrdebugtools ($) Apache Solr Configset
11.5 11.6.3+ 11.0 11.0 11.0 11.0 9.8.1¹ ext_solr_11_6_0_elts
10.4 11.2.5+ 10.0 10.0 10.0 10.0 9.8.1¹ ext_solr_11_2_0_elts

Our Apache Solr for TYPO3 EB-partners newsletter will keep you updated!

$ - Funding contribution extensions as ELTS. See: ELTS on https://shop.dkd.de/produkte/apache-solr-fuer-typo3/
ELTS - In Extended program. See: https://typo3.com/services/extended-support-elts
Ø - not yet available
¹ - recommended Apache Solr version, check version matrix in composer.json (composer info:solr-versions) for full list

No longer supported versions

TYPO3 EXT:solr EXT:tika EXT:solrfal EXT:solrconsole EXT:solrdebugtools EXT:solrmlt Apache Solr Configset
11.5 11.6.0 11.0 11.0 11.0 11.0 N/A 9.7 ext_solr_11_6_0
11.5 11.5.0-7 11.0 11.0 11.0 11.0 N/A 8.11 ext_solr_11_5_0
10.4 11.2.0-3 10.0 10.0 10.0 10.0 10.0 8.11 ext_solr_11_2_0
10.4 11.2.0-3 10.0 10.0 10.0 10.0 10.0 8.11 ext_solr_11_2_0
10.4 11.1 10.0 10.0 10.0 10.0 10.0 8.9 ext_solr_11_1_0
9.5-10.4 11.0.0-9 6.0.0-3 8.0.0-1 4.0.0-1 1.1.2 3.1 8.5 ext_solr_11_0_0, ext_solr_11_0_0_elts

Obsolete versions

TYPO3 EXT:solr EXT:tika EXT:solrfal EXT:solrconsole EXT:solrfluidgrouping EXT:solrdebugtools EXT:solrfluid EXT:solrgrouping EXT:solrmlt Apache Solr Schema Solrconfig Accessplugin
9.5 10.0 5.0 7.0 4.0 2.0 1.1.2 N/A N/A 3.0 8.2.0 tx_solr-10-0-0--20191010 tx_solr-10-0-0--20191010 4.0
8.7 - 9.5 9.0 4.0 6.0 2.0 2.0 1.0.0-1.1.1 N/A N/A 3.0 7.6.0 tx_solr-9-0-0--20180727 tx_solr-9-0-0--20180727 3.0
8.7 8.1 3.1 5.1 1.0 1.1 1.0.0-1.1.1 N/A N/A 2.0 6.6.3 tx_solr-8-1-0--20180615 tx_solr-8-1-0--20180615 2.0
8.7 8.0 3.0 5.0 N/A 1.0 1.0.0-1.1.1 N/A N/A N/A 6.6.2 tx_solr-8-0-0--20171020 tx_solr-8-0-0--20171020 2.0
8.7 7.5 2.4 4.2 N/A N/A 1.0.0-1.1.1 N/A N/A N/A 6.6.2 tx_solr-7-5-0--20171023 tx_solr-7-5-0--20171023 2.0
8.7 7.0 2.4 4.2 N/A N/A 1.0.0-1.1.1 N/A N/A N/A 6.3 tx_solr-7-0-0--20170530 tx_solr-7-0-0--20170530 2.0
7.6 - 8.x 6.5 2.3 4.1 N/A N/A N/A 2.0 1.3 N/A 6.6.2 tx_solr-6-5-0--20171023 tx_solr-6-5-0--20171023 2.0
7.6 - 8.7 6.1 2.3 4.1 N/A N/A N/A 2.0 1.3 N/A 6.3 tx_solr-6-1-0--20170206 tx_solr-6-1-0--20161220 2.0
7.6 6.0 2.2 4.0 N/A N/A N/A 1.2 1.3 N/A 6.3 tx_solr-6-0-0--20161209 tx_solr-6-0-0--20161122 1.7
7.6 5.1 2.1 3.2 N/A N/A N/A 1.2 1.3 N/A 4.10 tx_solr-5-1-0--20160725 tx_solr-4-0-0--20160406 1.3
7.6 5.0 2.1 3.1 N/A N/A N/A 1.0 1.3 N/A 4.10 tx_solr-4-0-0--20160406 tx_solr-4-0-0--20160406 1.3
7.6 4.0 2.1 3.0 N/A N/A N/A N/A 1.2 N/A 4.10 tx_solr-4-0-0--20160406 tx_solr-4-0-0--20160406 1.3
6.2 - 7.6 3.1 2.0 2.1 N/A N/A N/A N/A 1.1 1.1 4.10 tx_solr-3-1-0--20150614 tx_solr-3-1-0--20151012 1.3