Apache Solr for TYPO3 is the search engine you were looking for with special
features such as Faceted Search or Synonym Support and an incredibly fast
response times of results within milliseconds.
When development started, the primary goal was to create a replacement for
Indexed Search. With the initial public release at T3CON09 in Frankfurt, Germany
that goal was reached and even passed by adding features which Indexed Search
does not support.
With EXT:solr it is simple to build a search for a TYPO3 website, that allows you to index any kind of TYPO3 records with TCA configuration and pages.
The results can be rendered with flexible fluid templates, to render the results as you need them.
Feature List
Faceted Search
Spellchecking / Did you mean
Multi Language Support
Search word highlighting
Field Boosting for fine tuning the importance of certain index fields
Frontend User Group Access Restrictions Support
Stop word Support
Synonym Support
Auto complete / Auto suggest
Language Analysis / Support for inflected word forms
Content indexing through a near instant backend Index Queue
Auto correction (search for the first suggestion)
and more...
Development and Partnership
The extension is developed in an open source way and the source code is completely available on github. Releases to the TYPO3 TER are done in regular time frames (Usually every 3 months).
To make the development possible you can join a partner ship with dkd. By joining the partner program you have the following benefits:
You support the further development on EXT:solr
You get access to addon's that are not public available
Indexing of files from FAL (TYPO3 File Abstraction Layer)
You can included support based on your subscription
By the financial invest of our partners it is possible to keep this extension uptodate and integrate great new features.
If you like this extension please think about to become a partner as well!
|
If you are interested to become a partner visit http://www.typo3-solr.com or call dkd +49 (0)69 - 247 52 18-0.
Getting Started
In this chapter we would like to give you a quick introduction into the very first steps with EXT:solr.
After reading this chapter you will know:
How to...
Start a Solr server
Install the extension and connect your TYPO3 system to your Solr server
Index a few pages and see the search results on your website
First you need to install Solr itself. There are several ways to do so:
Using Hosted-solr.com
If you want to start simple and just create a Solr core with a click. You can use hosted-solr.com. For a small fee you get your own Solr core in seconds, configured to be used with EXT:solr.
Docker
You can use our official Docker image to start and maintain Solr server with a small effort.
To pull the TYPO3 Solr image from Docker hub, simply type the following in command line:
docker pull typo3solr/ext-solr:<EXT:Solr_Version_main_version eg 13.0 or exact 13.0.1>
Our and official Apache Solr image exports a volume /var/solr for persistent data.
This volume will be mounted to persist the index and other resources from Apache Solr server.
Following paths inside the exported volume are relevant for backups.
the managed stop words and synonyms of corresponding core
Tip
To be save for other scenarios(e.g. SVC of modified Solr Schemas and managed resources), simply backup the whole "data/" folder.
Start container with anonymous volume
To run the container with anonymous volume, simply type the following in command line:
docker run --name=typo3s-solr-server -d -p 8983:8983 typo3solr/ext-solr
Copied!
This will create a Docker anonymous volume and store the data inside of it.
To find out the path of used anonymous volume, simply type the following in command line:
To check whether Solr is up and running head over to:
http://<ip>:8983/solr/#/core_en/query.
You should see the web interface of Solr to run queries:
Important: The image ships a default cores for all languages. The data of the cores is stored on an exported volume.
When you want to update the container, you can just start a new container using the data volume of the old container.
But at the same time this has the limitation, that you should only use this image with the default cores!
If you want to create custom cores with a different configuration please read the section "Advanced Docker Usage"
Please note: The steps above show how to build the image from the Dockerfile. You can also download and use our compiled images from Docker Hub:
Our image has the intension to create running cores out of the box. This implies, that the schema is inside the container.
The intension in our integration was to stay as close as possible to the official Apache Solr Docker images. Sometimes it might make
sense that you use the official image directly instead of our image.
An example could be when you want to have the solrconfig, schema and data outside of the container.
The following example shows how you can run our configuration with the official Apache Solr Docker container by mounting the configuration
and data from a volume (When using Docker on macOS make sure you've added the volume folder to "Preferences -> File Sharing").
For more settings and tweak possibilities for our Docker image, please refer to the Appendix - Docker Tweaks.
Other Setup
Beside the Docker there are various possibilities to setup solr. All of these possibilities are not
officially supported, but the simplify the setup i want to mention them shortly here and summarize the needed steps.
Known Installers
All of these installers can be used to setup a plain, reboot save Solr server:
Use the installer shipped with Solr itself bin/install_solr_service.sh:
Allows to install Solr on many distributions including init scripts.
Use chef / ansible / whatever dev ops tool:
Allows you to setup a Solr server with your DevOps tool.
Deployment of EXT:solr configuration into Apache Solr
Since EXT:solr 6.0.0 the configuration and all JAR files can be found in EXT:solr/Resources/Private/Solr folder.
The goal of this approach is to make the deployment much easier.
All you need to do is, you need to copy the configSet directory into your prepared Solr installation and replace the solr.xml file. In the installer we do it like this:
After this, you can decide which cores you want to boot and delete not relevant cores from ${SOLR_INSTALL_DIR}/server/solr/cores/ directory.
If you want to create other cores, you can create them by adding a <core_name>/core.properties file like provided by EXT:solr.
This is the recommended workflow, so you can deploy the Solr server in an Infrastructure as Code (IaC) manner.
Alternatively Create a core with the rest api (not recommended):
After installing the Solr server and deploying all schemata, the TYPO3 reports module helps you to verify if your setup fits to the requirements of EXT:solr
You now have a fully working, pre configured Solr running to start with
No you can continue with installing the extension Install EXT:solr.
Install EXT:solr
Install from TER using the TYPO3 Extension Manager
You can simply install stable versions of EXT:solr using the Extension Manager
from the TYPO3 backend.
Go to the Extension Manager, select Get Extensions and search for
"solr".
Install the extension.
The Extension Manager will also install EXT:scheduler if not installed already
for running the indexing tasks
While developing we recommend installing devlog for easier error detection, too.
Install from git
Alternatively, you can also get the latest development version from GitHub:
Underlying settings can be found in the extension configuration, though the default settings are appropriate for most sites you should check and adapt this settings if necessary:
Static TypoScript
The extension already comes with basic TypoScript configuration that will work for small pages out of the box. For now create or
edit an existing TypoScript Template record in your page tree and add the provided static TypoScript:
Search Markers
EXT:solr is indexing everything on a page between <!-- TYPO3SEARCH_begin --> and <!-- TYPO3SEARCH_end --> to ensure this is the case, check the output of you website and add the markers to your template.
If the markers are missing, you should add them to your template. To increase the quality of the search results the markes should only wrap the relevant content of a page and exclude e.g. menus, because they are same on each page.
The markers can be used multiple times, but make sure each "TYPO3SEARCH_begin" is properly closed with "TYPO3SEARCH_end".
EXT:Solr needs a proper domain/host as base for the site. If the base is set to "/" or a similar relative path, the Indexing will throw an error, so please make sure that "base" is a fully qualified domain like "https://my.project.tld/"
Configure Solr Connections
Important
The Site-Handling GUI provides the common setup scenario for most users only.
There is no possibility in the GUI to setup different things for each language in site configuration except Corename(core path).
Tip
All in sites config.yaml global defined thing for EXT:solr can be configured for each language differently by editing this file manually.
As first step the authority part of URI to Apache Solr machine must be configured.
The Corename(aka path to core) for each available language must be assigned respectively.
Index the first time
After everything is setup, you need to index the contents of TYPO3 to enable searching in Solr.
To do so open the Apache Solr module and navigate to the Index Queue. Select the contents to index
and Queue Selected Content for Indexing.
Now the records are added to the index queue and ready to index. The indexing can be triggered from the backend module, or
can be triggered by the TYPO3 scheduler.
Switch to the Scheduler module. If the module is not available, make sure to enable the extension
first. It comes bundled with TYPO3 CMS but is not enabled by default.
Create a new scheduler task to run the indexing:
After the task was created, run it manually. The page will indicate a reload but won't reload after
the task was run. Therefore you can reload the module to see the progress bar indicating the current
progress of indexing:
The duration depends on things like the number of records to index and the number of languages
configured in your system. Also whether caching is enabled and warmed up.
The extension will now index all records in the queue and send them to Solrs index.
After Solr has some documents inside his index, you can insert the plugin to provide a search with
results from Solr. To do so create a new content record of type Search on a page:
Select Search: Form, Result, Additional Components if not already selected inside the content
element:
Open the page and search for *, you should see all currently indexed records from Solr:
That's it. You now have a working TYPO3 Installation with Solr integration. You are able to queue
items for indexing, index them and provide an interface for visitors to search the indexed records.
You can now adjust the fluid templates to your needs.
Backend
The chapter before gives you a short introduction about how to setup a Solr server, the extension and index a few documents into Solr.
In this chapter we want to go deeper and learn how to write more complex indexing configurations and see what other possibilities the backend of EXT:solr provides.
You can configure all Solr connection properties along with your TYPO3 site in the site module.
The configuration in the UI is limited to the following settings to keep the setup as simple as possible:
Global:
Scheme
Host
Port
Path to cores
Languagelevel:
Solr core name
If you have a more complex setup or need to configure username and password please configure Solr by editing the yaml file.
The fallback mechanism allows you to overwrite settings on the language level.
The fallback mechanism work like that:
Each setting has the following structure solr_{$setting}_{$scope}"`. The scope can be read or write. Every setting can be overwritten for the scope write, if nothing is configured it will fallback to the read setting. Every setting can be defined on the language level, if it is not configured on the language level it fallsback to the global setting.
The example above shows that you are able to define the setting
solr_host_read on the language level. Since this is a more advanced
configuration and the user interface should be kept simple, this can only be
configured in the yaml.
IndexQueue Configuration
As you already learned you can index pages very easy with EXT:solr and setup a search for pages in seconds. Beside pages, there might be other records in your TYPO3 CMS that you want to have available in your search results.
Indexing custom records
As a core feature EXT:solr allows you to write custom TypoScript configuration to index records from any extension just with configuration. To see how this is working, we open the content of the TypoScript example "Search - Index Queue Configuration for news" that can be found in "Configuration/TypoScript/Examples/IndexQueueNews":
By reading the example above you might recognize the following facts:
The indexing configuration is done in the TypoScript path 'plugin. tx_solr. index. queue. [configName]' and there can be multiple queue configurations.
The database table/type is configured in the property 'plugin.tx_solr.index.queue.[configName].type'. This allows you to have multiple index queue configurations for the same database table. This can be helpful when you have multiple queue configurations for news (e.g. if you have a press & corporate news section on your website).
The Solr fields are configured in 'plugin.tx_solr.index.queue.[configName].fields'. This allows you to flexibly fill any Solr field. The combination of dynamic fields (Appendix - Dynamic Fields) and the queue configuration allows you to write any kind of data into Solr without adapting the Solr schema.
There are custom TypoScript objects from EXT:solr that are used in the index queue configuration
When the index queue configuration of your custom record is ready, you can check the index queue in the backend module and add the news items to the queue.
Custom records - links and detail page
In the example above typolink is used to build a link to the detail view of the news. This is required, because EXT:solr can not know the business logic of the news extension to build a detail link.
The TypoScript constant "plugin.tx_news.settings.detailPid" is used to configure the target pageId of the news single view. This has two important impacts:
The constant (plugin.tx_news.settings.detailPid) need to point to a valid news single page.
The page with the news single view, should be configured with "Include in Search => Disable" because indexing this page with the normal page indexing without a news id will produce an error page.
Include in Search - Disable
Sysfolders outside the siteroot
It is a good practice not to nest the siteroots and do the configuration on the root page.
Changes on records that are done in the TYPO3 backend are detected and the Solr document will be re-added to the index queue when something was changed.
By default only records are monitored for a site that are in the tree of the site. If you want to index records and detect changes on records in a different siteroot, the
index queue configuration needs to contains "additionalPageIds" (e.g.: 'plugin.tx_solr.index.queue.<queueName>.additionalPageIds = 45,48').
Since the monitoring of changes in these records is expensive from performance perspective, you need to enable this feature in the extension configuration:
Enable tracking of records outside siteroot
Backend Modules
The backend modules are in the "Apache Solr" section available and can be unlocked for BE-users or/and groups. The modules help you to do maintenance tasks and get an overview on the system status:
During the next paragraphs we will go over the modules and explain, what could be done with them.
Info Module
Info Module shows you important infos about TYPO3 CMS and Solr state.
It contains different info tabs described below:
Connections
It lists all the for the site configured connections and their status.
Statistics
The Search Statistics module allows you to see Top Search Phrases with and without results. In addition it possible to see a
complete listing with hits etc. ranked by Top search keywords.
Index Fields
The Index Fields module allows you to see, how many documents you have in which Solr core and which fields those documents have.
Core Optimization
Core optimization Module is responsive for managing the behaviour of cores. By modifying of following things, you can also change the ranking and/or the results.
Stop Words
With the stopwords module you can define a list of words that should be excluded from the search.
Common usecases are:
Very often occurring words like "the", "and" ... are excluded are filtered out because they are more or less "noize words".
You can add words that you want to avoid from indexing.
Synonyms
With the synonyms module you can allow to find documents by words that do not occur in the document but have the same meaning:
E.g. smartphone, cellphone, mobile, mobilephone
Note: The word that you want replace with a synonym needs to appear on both sides when you want to find it with the term itself later
Example
smartphone => smartphone, cellphone, mobile, mobilephone will match "smartphone, cellphone, mobile, mobilephone", when smartphone is missing on the right side, you will not find the document for smartphone anymore!
Index Queue
The Index Queue module is the most important module. It allows you to do the following things:
Select item types and add them for indexing to the indexing queue.
See the fill state of the indexing queue.
Check the indexing queue for errors when the indexing of an items failed.
Start an instant indexing run, directly from the module.
Clear the indexing queue and re-queue items.
Index Administration (earlier Index Maintenance)
The Index Administration module allows you, to do the following administrative operations on your Solr index:
Reload the Solr configuration.
Empty your Solr index. This removes all documents from the index of the current selected site.
Clear the indexing queue.
Index Inspector
Beside the own backend module, EXT:solr provides a feature called "Search Index Inspector". This Tool allows you to select a page or sysfolder and check
what data is stored for this entity in the Solr index.
The "Search Index Inspector" can be opened with the TYPO3 Info Module ("Web > Info > Search Index Inspector") it shows the stored data in Solr from the page or sysfolder that is selected in the pagetree:
The EXT:solr Search Index Inspector
Page properties
EXT:solrs indexing behaviour depends on page properties.
Depending on the state of page properties the pages and or sub pages may be indexed or not.
Behaviour
The Behaviour tab of the page properties
Include in Search (no_search)
By default, every allowed page is indexed. Use this flag to
exclude the current page from being indexed.
Include sub entries in Search (no_search_sub_entries)
Use this flag, to be able to exclude sub pages from index recursively.
By unsetting this flag, the sub pages/entries of desired page will be excluded from index.
Tip
To exclude the page and its sub pages disable no_search and no_search_sub_entries at same time.
Scheduler
When you want to index content from TYPO3 into Solr automatically EXT:solr ships scheduler tasks, that can be called at a configured time or directly from the backend.
Index Queue Worker
Changes that are done in the backend by an editor are written into a queue. This queue is processed asynchronously with a scheduler task and each item in the queue is indexed into solr.
The "Index Queue Worker" task has the following custom properties:
Site: Here you select the Solr site, that you want to index with this task instance.
Number of documents to Index: Here you can configure how many documents you want to index in one run. Depending on the performance of your system and the expected update time of the search you can choose a realistic number here.
Forced webroot: The scheduler task can be executed in the cli context, because no webserver is used there, TYPO3 is unable to detect your webroot. As assumption we use PATH_site as default here. When you need to configure something else, you can do it with this option. You can use the marker ###PATH_site### and ###PATH_typo3### to define relative pathes here, to be independent from the concrete instance.
The EXT:solr Index Queue Worker - Scheduler Task
Force Re-Indexing of a site
This task allows you to force the re-indexing of a site & indexing configuration at a planned time.
The Force Re-Indexing of a site task has the following custom properties:
Site: Here you select the Solr site, that you want to index with this task instance.
Index Queue configurations to re-index: Here you can limit the set of indexing configurations that should be Re-Indexed.
The EXT:solr Force Re-Indexing of a site - Scheduler Task
Optimizing cores of a site
This task allows you to optimize the indexes of given cores for a site at a planned time.
The Optimize index of a site task has the following custom properties:
Site : Here you select the Solr site, that you want to index with this task instance.
Cores to optimize index for : Here you can select the cores that should be optimized by the task.
The EXT:solr Optimize index of a site - Scheduler Task
Plugins
EXT:solr provides the following plugin instances that can be configured in the backend:
The results plugin is the most important plugin of the extension. It is responsible to render a search form
and the results.
Flexform Configuration
All configuration can be done with TypoScript and the settings from EXT:solr are used. For some settings it makes sence to overwrite them with the flexform in the plugin settings.
The following settings can be overwritten by instance with the flexform:
"Target Page":
Target page that should be used when a search is submitted. This can be usefull
when you want to show the results on another page.
When nothing is configured the current page will be used.
Overwritten TypoScript Path
plugin.tx_solr.search.targetPage
Type:
Integer
"Initialize search with empty query":
If enabled, the results plugin issues a "get everything" query during initialization. This is useful, if you want to create a page that shows all available facets although no search has been issued by the user yet.
Note: Enabling this option alone will not show results of the get everything query. To also show the results of the query, see option Show results of initial empty query below.
Overwritten TypoScript Path
plugin.tx_solr.search.initializeWithEmptyQuery
Type:
Boolean
"Show results of initial empty query":
Requires "Initialize search with empty query" (above) to be enabled to have any effect. If enabled together with "Initialize search with empty query" the results of the initial "get everything" query are shown. This way, in combination with a filter you can easily list a predefined set of results.
This configuration can be used to configure an initial query string that is triggered when the plugin is rendered.
Overwritten TypoScript Path
plugin.tx_solr.search.initializeWithQuery
Type:
String
"Show results of initial query":
This option is used to configure if the results of an initial query should be shown.
Overwritten TypoScript Path
plugin.tx_solr.search.showResultsOfInitialQuery
Type:
Boolean
"Filters":
This flexform element allows you to define custom filters by selecting a Solr field and a value:
Overwritten TypoScript Path
plugin.tx_solr.search.query.filter.
Type:
Array
"Sorting":
When you want to sort initially by a field value and not by relevance this can be configured here.
Overwritten TypoScript Path
plugin.tx_solr.search.query.sortBy
Type:
String
Example:
title desc
"Boost Function":
A boost function can be useful to influence the relevance calculation and boost some documents to appear more at the beginning of the result list.
Technically the parameter will be mapped to the "bf" parameter in the Solr query.
Use cases for example could be:
"Give never documents a higher priority":
This could be done with a recip function:
recip(ms(NOW,created),3.16e-11,1,1)
Copied!
"Give documents with a certain field value a higher priority":
The boostQuery is a query that can be used for boosting. Technically it is mapped to the "bq" parameter of the Solr query. Compared to boost a function a boost query provides less use cases.
An example could be to boost documents based on a certain field value:
Since EXT:solr 7.0.0 the old templating of EXT:solr was droppend and rendering with fluid was added.
Along with this change some concepts have changed:
Until EXT:solr 7.0.0 EXT:solr css and javascript was loaded by EXT:solr automatically. In most cases you want to use custom css or you have custom javascript and the integrator want to decide which css or javascript to use. Therefore EXT:solr does not load it by default anymore and the integrator can load it with TypoScript. EXT:solr provides a lot of example TypoScript templates that load the default css or load the javascript that is needed to use a specific feature. Maybe take the time to explore the TypoScript templates that are shipped with the extension to see how they are implemented.
The were some old typosript settings that manipulate the data before it was passed to the view. With fluid this can also be done with core viewhelpers or custom viewhelpers. Therefore the following settings have been removed:
plugin.tx_solr.search.results.fieldProcessingInstructions (The fieldProcessingInstructions still exist at index time since there it is still needed)
Beside that there were some template related settings in the TypoScript, that can be solved just with fluid:
plugin.tx_solr.search.faceting.facetLinkATagParams or plugin.tx_solr.search.faceting.[facetName].facetLinkATagParams When you need something like this, you can just change the partials or render a facet with a custom partial (partialName = MyPartial) and add the properties there.
plugin.tx_solr.search.faceting.removeFacetLinkText This can be done just be rendering the text in the partial, that you need.
The setting faceting.facets.[facetName].selectingSelectedFacetOptionRemovesFilter has been removed, since it is possible to build this functionality just with Fluid ViewHelpers. The file "EXT:solr/Resources/Private/Templates/Partials/Facets/OptionsToggle.html" shows how f:if together with `option.selected` can be used to have this behaviour.
Fluid Template Structure
First we start with a short overview of the template structure. This is just to get an rought overview. The templates will be explained in
detail in the template where they belong to:
Layouts: Layouts that are used in the search and the faceting.
Partials:
Facets: Partials that are use to render the specific facet types.
Result: Partials that are used during the result rendering (e.g. to render the result document, sorting or perPage selector)
Search: Partials that are used for the search also when no search was executed.
Templates:
Search: All templates that are used to render the actions in the SearchController
ViewHelper: All templates that are use in the widgets (FrequentSearched, LastSearches)
All backend related files are in a "Backend" folder. Everything else is frontend related.
Result List
The most important part of a search are the results. The rendering of the results is done in the "Results.html" template
(Located in Templates/Search/Results.html)
The following part of the default template iterates over the results and renders every document with the Document.html partial (Partials/Frontend/Result/Document.html)
This structure allows you to use e.g. the fluid if ViewHelper to render a result with a different partial, based on a field value.
But as you see in the template above, by default the partial "Result/Document" is used.
The "document" partial is getting the document object. In our case this is an instance of "ApacheSolrForTypo3SolrfluidDomainSearchResultSetSearchResult"
the api of this object allows to get the Solr field content with "Document->getFieldName()" that can be used as "document.fieldName" in fluid.
Facets
The goal of a good search is, that the user will find what he is looking for as fast as possible.
To support this goal you can give information from the results to the user to "drill down" or "filter" the results
up to a point where he exactly finds what he was looking for. This concept is called "faceting".
Imagine a user in an online shoe shop is searching for the term "shoe", wouldn't it be useful to allow
the user to filter by "gender", "color" and "brand" to find exactly the model where he is looking for?
In the following paragraphs we will get an overview about the different facet types that can be created on a Solr field
just by adding a few lines of configuration.
Facet Types
A Solr field can contain different type of data, where different facets make sence. The simplest facet is an option "facet".
The "options facet" just contains a list of values and the user can choose one or many of them. A more complex type
could be a "range facet" on a price field. A facet like this needs to allow to filter on a range of a minimum and a maximum value.
The "type" of a facet can be controlled with the "type" property. When nothing is configured there, the facet will be threated
as option facet.
In the following paragraphs we will introduce the available facet types in EXT:solr and show how to configure them.
Option
The simplest and most often used facet type is the options facet. It renders the items that could be filtered as a simple list.
To setup an simple options facet you can use the following TypoScript snipped:
plugin.tx_solr.search {
faceting = 1
faceting {
facets {
contentType {
label = Content Type
field = type
}
}
}
}
Copied!
By using this configuration you create an options facet on the Solr field "type" with the name "contentType". This field represents the record type, that was
indexed into solr. Shown in the frontend it will look like this:
When you have an option facet with very much options you might want to group the options by an prefix of an option. This can be used e.g. to group the options alphabetically.
The following example shows how options can be grouped by prefix (from EXT:solr/Configuration/TypoScript/Examples/Facets/OptionsPrefixGrouped/setup.typoscript):
The query group facet renders an option list, compareable to the options facet, but the single options are not created from
plain Solr field values. They are created from dynamic queries.
A typical usecase could be, when you want to offer the possiblity to filter on the creation date and want to offer options like "yesterday", "last year" or "more then five years".
With the following example you can configure a query facet:
plugin.tx_solr.search {
faceting = 1
faceting {
facets {
age {
label = Age
field = created
type = queryGroup
queryGroup {
week.query = [NOW/DAY-7DAYS TO *]
old.query = [* TO NOW/DAY-7DAYS]
}
}
}
}
}
Copied!
The example above will generate an options facet with the output "week" (for items from the last week) and "old" (for items older then one week).
The output in the frontend will look like this:
Solr queryGroup facet
An more complex example is shipped with this extension and can be enabled by including the template "Search - (Example) Fluid queryGroup facet on the field created",
this example makes also use of renderingInstructions to render nice labels for the facet.
The example above just shows a simple example tree that is just rendering the uid's of the rootline as a tree:
Hierachy facet
A more complex example, that is rendering the pagetree with titles is shipped in the extension. You can use it by
including the example TypoScript "Search - (Example) Fluid hierarchy facet on the rootline field":
Technically the hierarchical facet for Solr is the same as a flat options facet. The support of hierarchies is implemented,
by writing and reading the facet options by a convention:
[depth]-Level1Label/Level2Label/
Copied!
When you follow this convention by writing date into a Solr field you can render it as hierarchical facet. As example you can check indexing configuration in EXT:solr (EXT:solr/Configuration/TypoScript/Solr/setup.typoscript)
When you want to provide a range filter on a date field in EXT:solr, you can use the type "dateRange".
The default partial generates a markup with all needed values in data attributes. Together with the provided jQuery UI implementation you can
create an out-of-the-box date range facet.
With the following TypoScript you create a date range facet:
plugin.tx_solr.search {
faceting = 1
faceting.facets {
creationDateRange {
label = Created Between
field = created
type = dateRange
}
}
}
Copied!
In the extension we ship the TypoScript example "Search - (Example) dateRange facet with jQuery UI datepicker on created field" that shows how to
configure a dateRange facet and load all required javascript files.
When you include this template a date range facet will be shown in the frontend that we look like this:
EXT:solr dateRange facet
As described before for the date range facet markup and javascript code is required, looking at the example template "Search - (Example) dateRange facet with jQuery UI datepicker on created field"
in "Configuration/TypoScript/Examples/DateRange" you see that for the jQueryUi implementation the following files are included:
Beside dates ranges are also usefull for numeric values. A typical usecase could be a price slider for a products page.
With the user interface you should be able to filter the documents for a certain price range.
In the default partial, we also ship a partial with data attributes here to support any custom implementation.
By default we will use the current implementation from EXT:solr based on jQueryUi.
The following example configures a numericRange facet for the field "pid":
plugin.tx_solr.search {
faceting = 1
faceting.facets {
pidRangeRange {
field = pid
label = Pid Range
type = numericRange
numericRange {
start = 0
end = 100
gap = 1
}
}
}
}
Copied!
The numeric range facet requires beside the template also a javascript library to render the slider. The example TypoScript template "Search - (Example) Fluid numericRange facet with jQuery UI slider on pid field"
can be used to see the range slider with jQuery UI for the Solr field pid by example.
When you configure a facet on the pid field like this, the frontend will output the following facet:
Numeric range facet
Beside the implementation with jQueryUi you are free to implement a range slider with any other javascript framework.
Rendering with fluid
Rendering facets with fluid is very flexible, because you can use existing ViewHelpers and implement your own logic in ViewHelpers
to support your custom rendering logic.
In the default template the main faceting area on the left side, is done in the following file:
Looking at the code above we see to important details that are important for solr.
Facet Grouping (Areas)
The first important part if the facet.area.group ViewHelper. By default all facets in the group main will be rendered.
This value is the default value.
When you now want to render the facet at another place you can change the group with the following TypoScript configuration:
plugin.tx_solr.search {
faceting = 1
faceting.facets {
contentType {
field = type
label = Content Type
groupName = bottom
}
}
}
Copied!
Now the facet belongs to another group and will not be rendered in the "main" area anymore.
Default Partials
Another important fact is that Facet->getPartialName() is used to render the detail partial.
The default implementation of a facet will return the default partial, that is able to render this facet.
If you need another rendering for one facet you can overwrite the used partial within the configuration:
plugin.tx_solr.search {
faceting = 1
faceting.facets {
contentType {
field = type
label = Content Type
partialName = mySpecialFacet
}
}
}
Copied!
Combining all of these concepts together with the flexibility of fluid you are able to render facets in a very
flexible way.
Autosuggest
A user of the search typically want to find the results a fast as possible. To support the user and avoid to much typing
solr can create a drop down list of common suggested search terms right after the search input box.
This feature can be easily configured with the following TypoScript setting:
If you want to configure an the autosuggest by example, you can include the TypoScript example template "(Example) Suggest/autocomplete with jquery".
When everything is configured the frontend will show you a drop down of suggestions when you are typing in the search field:
Autocomplete with jQuery
Beside search term suggestions the autocomplete an also show you the top search results for the user input.
This feature can be enabled with:
plugin.tx_solr.suggest.showTopResults = 1
Copied!
If you want to change the number of proposed top results you can also configure them:
plugin.tx_solr.suggest.numberOfTopResults = 5
Copied!
Sorting
When no sorting is selected the search will order the results by "relevance". This relevance is calculated by many factors and has the goal to deliver
the best result for the query on the first position. That's what you expect from a search :)
For some usecases you want to change the sorting of the results by a certain field. In an onlineshop a user might want to order the results
by the price to find the cheapest product that is matching his query.
A simple sorting can be configured with the following TypoScript snipped:
plugin.tx_solr.search.sorting >
plugin.tx_solr.search {
sorting = 1
sorting {
defaultOrder = asc
options {
relevance {
field = relevance
label = Relevance
}
title {
field = sortTitle
label = Title
}
}
}
}
Copied!
With the configuration above the possibility to sort by title is introduced. At the same time the sort by relevance link can be used to
reset the sorting to sort by the natural Solr relevance.
Search with sorting
Templating
The rendering of the sorting is done on "Resources/Private/Partials/Results/Sorting.html" this partial is using the configuration and
the view helpers to generate sorting links with the same behaviour as in ext:solr. For sure you can modifiy this template and use the ViewHelpers
in the way how you want to implement your custom sorting.
Results per Page
EXT:solr allows you to configure how many result per page will be shown and at the same time the user can also change
this value to an allowed value.
The following configuration can be used to configured the results per page:
When you apply the configuration above, the frontend will show 6 search results by default and show the options 12, 18 and 24
to the user to change the amount of visible results
Results per page
Templating
The rendering of the "perPage selector" is done on "Resources/Private/Partials/Frontend/Results/PerPage.html". This partial is build in a way that the behaviour of the perPage selector is the same as in EXT:solr. If you want to do your custom rendering for example with links instead of a for, you can customize the rendering there.
Ajaxified Results
To improve the use experience and the performance it is possible to load most of the sub requests with ajax.
To activate the "ajaxification" you need to include the TypoScript template "Search - Ajaxify the searchresults with jQuery"
Include EXT:solr ajaxify template
How it works?
For all links with the css class "solr-ajaxified" the javascript search controller triggers the request against the same search page but with the type "7383" which is just rendering the search request.
The response is replacing everything in the container "div.tx_solr" with the content of the response.
Limitations
Since the example only renders the plugin on a specific page type and not the same plugin instance as on the page, settings from the flexform are not taken into account by now.
Customize
Use custom Fluid Templates
After these steps solrfluid is usable and using the default Templates, Layouts and Partials. If you want to overwrite them, you can change the TypoScript configuration:
The next step is to configure a path segment or to mask the facets inside of the query.
Facets in path segments
This chapter describe how you can use a facet as part of your url path.
Imagine you have an online shop and use Solr to display the products. It would be convenient for the customer if can call an url path like /products/candy.
The first step is to create a site /products and place the Solr plugin as content.
Configure the path segment
In our example we are using facet productType to filter the search result for a specific product type.
The first step is to define a route path. Because we have already a side called /products only need to define a path contain the facet.
The name itself needs no connection to the facet, so you can choose something different.
The solution is to define a part of a query path and attach the facet you want to place inside of the url.
This is done by separating the query path and the facet by a dash.
routeEnhancers:products:# Note: All arguments inside of namespace tx_solr. See -> extensionKey# Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'_arguments:productType:filter-productTyperequirements:productType:'.*'
Copied!
The route enhancer now knows to place values of facet productType as path segment.
This example shows the all configuration done above
routeEnhancers:products:type:SolrFacetMaskAndCombineEnhancerlimitToPages:-42extensionKey:tx_solrroutePath:'/{productType}'# Note: All arguments inside of namespace tx_solr. See -> extensionKey# Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'_arguments:productType:filter-productType# Important: Configure requirement for fields! If you wand to allow empty values, set .*requirements:productType:'.*'solr:multiValueSeparator:';'replaceCharacters:' ':'-'
Copied!
Simplify facets inside of the query
Sometimes you or your customer want to share links to a search result. The link itself could really hard to read, a specially if it contains a lot of filters.
There are two options that you can apply on the query string:
Concat multiple values of facet into a single string
Mask the filter inside of the query
All query related configurations will be placed below node query below node solr:
routeEnhancers:products:solr:query:
Copied!
Concat multiple values into a string
To activate the concatenation of multiple values add the concat entry and set the value to true.
By default the values will concat with a colon. If you need a different separator, add the entry valueSeparator and set a value that fits your need.
By enabling the concat of multiple values, it would change into:
taste=matcha,sour,sweet
Tip
The facet and the "target" can be equal, you may use same names.
Note
By enabling the mask option, the concat option applied automatically!
Note
The mask only apply to facet you configure inside of the map.
Important
Some facet names are excluded from mask until you configure a different parameter name. This are the internal/reserved parameters of TYPO3.
type
id
no_cache
cHash
* MP
Full example
This example shows all configurations together, which were done above in step by step:
routeEnhancers:products:solr:query:# To reduce the amount of parameters you can force Solr to concat the values.# For example you have following filter:# tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha## Concat will:# 1. collect all filters of the same type# 2. will sort all filter values alpha numeric# 3. join the values together## As a result the query will modified into:# tx_solr[filter][0]=taste:matcha,sour,sweet## Note: If you active the mask option, the concat feature turn on automatically#concat:true# valueSeparator: ','# You can tell Solr to mask query facets. This feature require the map below## For example you have following filter:# tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha# Mask will:# 1. implode all values into a single string and sort it -> green,red,yellow# 2. replace tx_solr[filter][]=color: with color=## As a result the query will modified into:# taste=matcha,sour,sweet#mask:true# In order to simplify a filter argument, you have to define a corresponding map value# There is no automatically reduction of filter parameters at the moment available.# The key is the name of your facet, the value what use instead.## Important:# There are some restrictions for the values. The use of TYPO3 core parameters is prohibited.# This contains at the moment following strings: no_cache, cHash, id, MP, typemap:taste:taste
Copied!
Example
routeEnhancers:Products:type:SolrFacetMaskAndCombineEnhancerlimitToPages:-10extensionKey:tx_solrsolr:multiValueSeparator:'+'replaceCharacters:' ':'_'query:# To reduce the amount of parameters you can force Solr to concat the values.# For example you have following filter:# tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha## Concat will:# 1. collect all filters of the same type# 2. will sort all filter values alpha numeric# 3. join the values together## As a result the query will modified into:# tx_solr[filter][0]=taste:matcha,sour,sweet## Note: If you active the mask option, the concat feature turn on automatically#concat:true# valueSeparator: ','# You can tell Solr to mask query facets. This feature require the map below## For example you have following filter:# tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha# Mask will:# 1. implode all values into a single string and sort it -> green,red,yellow# 2. replace tx_solr[filter][]=color: with color=## As a result the query will modified into:# taste=matcha,sour,sweet#mask:true# In order to simplify a filter argument, you have to define a corresponding map value# There is no automatically reduction of filter parameters at the moment available.# The key is the name of your facet, the value what use instead.## Important:# There are some restrictions for the values. The use of TYPO3 core parameters is prohibited.# This contains at the moment following strings: no_cache, cHash, id, MP, typemap:color:colortaste:tasteproduct:productroutePath:'/{type}'# Note: All arguments inside of namespace tx_solr. See -> extensionKey# Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'_arguments:type:filter-typerequirements:type:'.*'
Copied!
Logging
When you need to debug something in EXT:solr the extension allows you to write several information to the TYPO3 logging framework.
This can be helpful to analyze e.g.:
What is happening during the indexing?
What is going when a search in the frontend is triggered?
The configuration can be found in the folder "Resources/Private/Solr".
This folder contains:
The folder "configsets": This folder a set of configuration files that can be deployed into a Solr server, as a template.
It contains the "solrconfig.xml" file, the "schema.xml" files for all languages and the accessfilter libary that belongs to
this version as a jar file. This configSet needs to be in place on a Solr server to create cores that are compatible to the EXT:solr
extension.
The folder "cores": This folder ships an example "core.properties" file for all languages that are compatible with EXT:solr.
A "core.properties" file references a "configSet" that should be used. The path to the schema that is bound to a core is configured as "schema" relative to the root folder of the "configSet".
By example a "core.properties" file looks like this:
The solr.xml file: This file configures Solr as required for the used Apache Solr version.
The EXT:solr configSet
As mentioned before the configSet is one single package, that contains all to customize a plain Apache Solr Server, to an Apache Solr Server, ready
for EXT:solr & TYPO3.
The configset of the current release is located in "Resources/Private/Solr/configsets/ext_solr_<release>".
Inside the configSet you find the following folders:
conf: This folder contains the Solr configuration and all schemata files. There is one directory for each language which ships the schema.xml file for this language.
The schema.xml file contains the language specific adaptions and includes all general schema fields and types with an XInclude statement.
typo3lib: This folder ships the compiled access filter jar file, that should be used with this EXT:solr version.
The solrconfig (conf/solrconfig.xml) is configured to load all jar files from typo3lib.
Setup steps
With the extension we ship an installer for development and a Docker images that can be used to install Solr.
When you want to install Solr on your system in another way the following steps are required.
Install the Solr server
Copy the configsets into the configset folder (by default $SOLR_HOME/server/solr/configsets)
Make sure that the solr.xml file ($SOLR_HOME/server/solr/solr.xml) is in place and fits to your Solr version
Create an init script that starts Solr on boottime.
Secure your Solr port from outside.
Make sure that Solr is running with an own user.
Backup your data folders
Hint: Apache Solr ships an install script in newer version that might cover your requirements for production
($SOLR_HOME/bin/install_solr_service.sh). We don't use it in EXT:solr because there are currently problems when using it with Ubuntu Xenial (16.04)
Index Maintenance
Solr offers a lot of request handlers to do maintenance tasks.
There are several parameters in the solrconfig.xml that can be used to tune your Solr server.
Our solrconfig.xml is designed to ship a reasonable configuration for the most standard use cases.
For use cases with very large indexes or high performance requirements it makes sence to tune those parameters
indexConfig.useCompoundFile
This value is "true" by default in our configuration. By setting this value to true Solr only writes one file
for indexes instead of many. This is a little bit slower but more robust to prevent errors with "Too many open files".
A switch to completely turn on / off EXT:solr. Comes in handy with multi site
installations where you want to enable EXT:solr only for certain sites, but
still have the extension's configuration at a single place and include that for
each site. Just set enabled = 0 for each site's root TS template or use
conditions where you do not want EXT:solr.
Important
This also influences the connection manager; connections will be registered /
detected only for enabled = 1
If enabled, the debugQuery query parameter is added to the Solr queries. Solr
will then return additional information explaining the the query, scoring,
timing, and other information.
Defines the format that is used for dates throughout the extension like in
view helpers for example. The format uses the strftime() php function syntax,
please consult the php documentation for available options.
tx_solr.view
All view related settings, these settings might also be relevant for Fluid.
pluginNamespace
Type
String
TS Path
plugin.tx_solr.view.pluginNamespace
Since
7.0
Default
tx_solr
Plugin namespace. Can be used to change the plugin namespace and can be changed by instance in the flexform.
templateFiles
By convention the templates is loaded from EXT:solr/Resources/Private/Templates/Frontend/Search/(ActionName).html.
If you want to define a different entry template, you can do this here to overwrite the conventional default template.
If you want to use FLUID fallbacks you can just configure the template name, otherwise you could also use a full reference EXT:/.../.
The templates that you configure in availableTemplate can be used in the flexform by the editor to select a template for the concrete plugin instance.
By convention the "Results" template from you configured FLUID template path will be used As alternative you can configure a different template name here (e.g. MyResults or a full path to an entry template here).
By convention the "Form" template from you configured FLUID template path will be used . As alternative you can configure a different template name here (e.g. MyForm or a full path to an entry template here).
By convention the "FrequentlySearched" template from you configured FLUID template path will be used . As alternative you can configure a different template name here (e.g. FrequentlySearched or a full path to an entry template here).
tx_solr.index
This part contains all configuration that is required to setup your indexing configuration. You can use EXT:solr
to easily index pages or any kind of records of your TYPO3 CMS.
Allows to prevent frontend indexing of pages when a backend editor is logged in and browsing the website.
additionalFields (deprecated)
Type
String, cObject (since 1.1)
TS Path
plugin.tx_solr.index.additionalFields
Since
1.0
Deprecated
2.0
A mapping of Solr field names to additional string values to be indexed with page documents. Use dynamic fields to index additional data, this way you don't have to modify the schema.xml
Example:
plugin.tx_solr.index.additionalFields {
myFirstAdditionalField_stringS = some string
mySecondAdditionalField_stringS = TEXT
mySecondAdditionalField_stringS {
value = some other value that can be constructed using any TypoScript cObject
case = upper
// more processing here as needed
}
}
Copied!
Since version 1.1 you can use cObjects to generate the value for the field. The only thing to observe is that you generate strings. Other values may work, but haven't been tested yet.
Deprecated since 2.0, please use the Index Queue indexing configurations instead as it allows you to define more precisely for which types of documents you want which fields to be indexed.
Assigns processing instructions to Solr fields during indexing (Syntax: Solr index field = processing instruction name). Before documents are sent to the Solr server they are processed by the field processor service. Currently you can make a filed's value all uppercase, convert a UNIX timestamp to an ISO date, or transform a path into a hierarchy for hierarchical facets (2.0 only). Currently you can use only one processing instruction at a time.
Custom processors have to implement interface ApacheSolrForTypo3\Solr\FieldProcessor\FieldProcessor.
queue
The Index Queue is a powerful feature introduced with version 2.0. It allows you to easily index any table in your TYPO3 installation by defining a mapping of SolrFieldName = DatabaseTableFieldNameOrContentObject. The table must be configured / described in TCA, though. To index other, external data sources you might want to check out Solr's Data Import Handler (DIH).
The Index Queue comes preconfigured to index pages (enabled by default) and an example configuration for tt_news (provided as a separate TypoScript template).
Type
Array
TS Path
plugin.tx_solr.index.queue
Since
2.0
Default
pages
Defines a set of table indexing configurations. By convention the name of the indexing configuration also represents the table name. You can name the indexing configuration differently though by explicitly defining the table as a parameter within the indexing configuration. That's useful when indexing records from one table with different configuration - different single view pages / URLs for example.
Example:
// enables indexing of tt_news records
plugin.tx_solr.index.queue.news = 1
plugin.tx_solr.index.queue.news.fields {
abstract = short
author = author
description = short
title = title
// the special SOLR_CONTENT content object cleans HTML and RTE fields
content = SOLR_CONTENT
content {
field = bodytext
}
// the special SOLR_RELATION content object resolves relations
category_stringM = SOLR_RELATION
category_stringM {
localField = category
multiValue = 1
}
// the special SOLR_MULTIVALUE content object allows to index multivalue fields
keywords = SOLR_MULTIVALUE
keywords {
field = keywords
}
// build the URL through typolink, make sure to use returnLast = url
url = TEXT
url {
typolink.parameter = {$plugin.tt_news.singlePid}
typolink.additionalParams = &tx_ttnews[tt_news]={field:uid}
typolink.additionalParams.insertData = 1
typolink.returnLast = url
typolink.useCacheHash = 1
}
sortAuthor_stringS = author
sortTitle_stringS = title
}
Copied!
queue.[indexConfig]
Type
Boolean, Array
TS Path
plugin.tx_solr.index.queue.[indexConfig]
Since
2.0
Default
pages
An indexing configuration defines several parameters about how to index records of a table. By default the name of the indexing configuration is also the name of the table to index.
By setting plugin.tx_solr.index.queue.[indexConfig] = 1 or 0 you can en- / disable an indexing configuration.
Note: you could add L={field:__solr_index_language} in the additionalParams of the typolink to link to the correct language version (this was removed from the example above to simplify the example)
A WHERE clause that is used when initializing the Index Queue, limiting pages that goes into the Queue.
This filter is applied prior to the plugin.tx_solr.index.queue.[indexConfig].additionalWhereClause
filter and hence provides an even stronger filter mechanism - since it can be used to filter away page
ID's that shouldn't be processed at all.
// Filter away pages that are "spacer" and have no_search, hidden and nav_hide set to zero
plugin.tx_solr.index.queue.pages.initialPagesAdditionalWhereClause = doktype <> 199 AND no_search = 0 AND hidden = 0 AND nav_hide = 0
Defines additional pages to take into account when indexing records for example. Especially useful for indexing DAM records or if you have your news outside your site root in a shared folder to use for multiple sites.
Additional page IDs can be provided as comma-separated list.
queue.[indexConfig].type
Type
String
TS Path
plugin.tx_solr.index.queue.[indexConfig].type
Since
11.5.1
Defines the type to index, which is usally the database table. Sometimes you may want to index records from a table with different configurations, f.e., to generate different single view URLs for tt_news records depending on their category or storage page ID. In these cases you can use a distinct name for the configuration and define the table explicitly.
plugin.tx_solr.index.queue.generalNews {
type = tt_news
fields.url = URL for the general news
// more field configurations here ...
}
// extends the general news configuration
plugin.tx_solr.index.queue.pressNews < plugin.tx_solr.index.queue.generalNews
plugin.tx_solr.index.queue.pressNews {
fields.url = overwriting URL for the press announcements
// may overwrite or unset more settings from the general configuration
}
// completely different configuration
plugin.tx_solr.index.queue.productNews {
type = tt_news
fields.url = URL for the product news
}
When initializing the Index Queue through the search backend module the queue tries to determine what records need to be indexed. Usually the default initializer will be enough for this purpose, but this option allows to define a class that will be used to initialize and add records to the Index Queue in special ways.
The extension uses this option for initializing the pages and more specifically to resolve Mount Page trees so they can be indexed too, although only being virtual pages.
queue.[indexConfig].indexer
Type
String, Array
TS Path
plugin.tx_solr.index.queue.[indexConfig].indexer
Since
2.0
When configuring tables to index a default indexer is used that comes with the extensions. The default indexer resolves the Solr field to database table field mapping as configured. However, in some cases you may reach the limits of TypoScript, when this happens you can configure a specialized indexer using this setting.
The indexer class is loaded using TYPO3's auto loading mechanism, so make sure your class is registered properly. The indexer must extend tx_solr_indexqueue_Indexer.
Example, pages use a specialized indexer:
plugin.tx_solr.index.queue.pages {
indexer = tx_solr_indexqueue_PageIndexer
indexer {
// add options for the indexer here
}
}
Copied!
Within the indexer configuration you can also define options for the specialized indexer. These are then available within the indexer class in $this->options.
Example, the TypoScript settings are available in PHP:
TypoScript:
plugin.tx_solr.index.queue.myIndexingConfiguration {
indexer = tx_myextension_indexqueue_MyIndexer
indexer {
someOption = x
someOtherOption = y
}
}
Allows to define the order in which Index Queue items of different kinds are indexed. Items with higher priority are indexed first.
queue.[indexConfig].fields
Type
Array
TS Path
plugin.tx_solr.index.queue.[indexConfig].fields
Since
2.0
Mapping of Solr field names on the left side to database table field names or content objects on the right side. You must at least provide the title, content, and url fields. TYPO3 system fields like uid, pid, crdate, tstamp and so on are added automatically by the indexer depending on the TCA information of a table.
You can also use a virtual field called __solr_content, which holds the content of the current page.
Example:
plugin.tx_solr.index.queue.[indexConfig].fields {
content = bodytext
title = title
url = TEXT
url {
typolink.parameter = {$plugin.tx_extensionkey.singlePid}
typolink.additionalParams = &tx_extenionkey[record]={field:uid}
typolink.additionalParams.insertData = 1
typolink.returnLast = url
}
}
Comma-separated list of fields that hold files. Using this setting allows to tell the file indexer in which fields to look for files to index from records.
Allows to define a list of additional fields from the pages table that will trigger a recursive update.
plugin.tx_solr.index.queue.pages.recursiveUpdateFields = title
Copied!
The example above will trigger a recursive update of pages if the title is changed.
Please note that the following columns should NOT be configured as recursive update fields: "hidden" and "extendToSubpages".
These fields are handled by EXT:solr already internally and thus they will have not effect.
The example above will remove the content of items in the page that have the css class "removeme".
queue.pages.allowedPageTypes
Type
List of Integers
TS Path
plugin.tx_solr.index.queue.pages.allowedPageTypes
Since
3.0
Default
1,7
Allows to set the pages types allowed to be indexed.
Even if you have multiple queues for pages, e.g. via different additionalWhereClause's, you have
to set this value to allow further doktype's. Restrict the pages to be indexed by each queue via
additionalWhereClause.
To make life even easier the Index Queue provides some indexing helpers. These helpers are content objects that perform cleanup tasks or content transformations.
SOLR_CONTENT
Since
2.0
Cleans a database field in a way so that it can be used to fill a Solr document's content field. It removes HTML markup, Javascript and invalid utf-8 chracters.
The helper supports stdWrap on its configuration root.
Example:
content = SOLR_CONTENT
content {
field = bodytext
}
Usually the label field to retrieve from the related records is determined automatically using TCA, using this option the desired field can be specified explicitly. To specify the label field for recursive relations, the field names can be separated by a dot, e.g. for a category hierarchy to get the name of the parent category one could use "parent.name" (since version:2.9).
Defines how to build the label for indexing, stdWrap is applied. Can be used to overrule foreignLabelField. Referencing to "field" e.g. inside dataWrap will resolve to resolved record.
Whether to return related records suitable for a multi value field. If this is disabled the related values will be concatenated using the following singleValueGlue.
When not using multiValue, the related records need to be concatenated using a glue string, by default this is ", " (comma followed by space). Using this option a custom glue can be specified. The custom value must be wrapped by pipe (|) characters to be able to have leading or trailing spaces.
Allows to classify documents based on a configured pattern
Example:
topic_stringM = SOLR_CLASSIFICATION
topic_stringM {
field = __solr_content
classes {
programming {
matchPatterns = php, java, javascript, go
class = programming
}
cms {
matchPatterns = TYPO3, joomla
class = cms
}
database {
matchPatterns = mysql, MariaDB, postgreSQL
class = database
}
}
}
Copied!
The `matchPatterns` can be used to configure pattern that can occure in the content to add that class. In addition `unmatchPatterns` can be configured to define patterns that should not occure in the content.
Patterns are regular expressions. You configure everything that is possible with regular expressions.
Example:s
The pattern `\ssmart[a-z]*\s` will match everything, that starts with a space followed by smart ending with any lowercase letter and ending by space. This would match e.g. smartphone, smarthome and every other word that starts with `smart`.
Note:
The configuration `patterns` is deprecated with 10.0.0 and will be removed in EXT:solr 11. Please use `matchPatterns` and `unmatchPatterns now.
Name of the database field, that should be used to as content to classify. The special field __solr_content can
be used during indexing to classify the content of the page or file or any other record that fills the content field before.
Array of classification configurations. Each configuration needs to have the property "patterns", that is a list of patters that need to match and "class", that is the mapped class that will be indexed then.
Note:
The output field needs to be a multivalue field since an indexed item can have multiple classes.
enableCommits
Type
Boolean
TS Path
plugin.tx_solr.index.enableCommits
Since
6.1
Default
true
This setting controls whether ext-solr will implicitly cause Solr commits as part of its operation.
If this settings is set to false, you need to ensure that something else will periodically call
commits. The Solr daemons AutoCommit feature would be a natural choice.
This feature is mainly useful, when you have many installations in the same Solr core.
Note: Calling some APIs may still cause commits, but these can always be explicitly disabled.
tx_solr.search
The search section, you probably already guessed it, provides configuration options for the all things related to actually searching the index, setting query parameters, formatting and processing result documents and the result listing.
targetPage
Type
Integer
TS Path
plugin.tx_solr.search.targetPage
Default
Since
1.0
Sets the target page ID for links. If it is empty or 0, the current page ID will be used.
Note: This setting can be overwritten by the plugins flexform.
trustedFields
Type
String
TS Path
plugin.tx_solr.search.trustedFields
Default
url
Since
3.1
The data in EXT:solr is escaped right after the retrieval from Solr. In rare cases when you need to store HTML in Solr documents you can use this configuration to mark these fields as trusted fields and skip the escaping. Typically this is needed when you want to retrieve html from solr.
The following example shows how to avoid html in the content field:
If enabled, the results plugin (pi_results) issues a "get everything" query during initialization. This is useful, if you want to create a page that shows all available facets although no search has been issued by the user yet. Note: Enabling this option alone will not show results of the get everything query. To also show the results of the query, see option showResultsOfInitialEmptyQuery below.
Requires initializeWithEmptyQuery (above) to be enabled to have any effect. If enabled together with initializeWithEmptyQuery the results of the initial "get everything" query are shown. This way, in combination with a filter you can easily list a predefined set of results.
When doing a new search, existing parameters like filters will be carried over to the new search. This is useful for a scenario where you want to list all available documents first, then allow the user to filter the documents using facets and finally allow him to specify a search term to refine the search.
ignoreGlobalQParameter
Type
Boolean
TS Path
plugin.tx_solr.search.ignoreGlobalQParameter
Default
Options
0,1
Since
7.0
In some cases you want EXT:solr to react on the parameter "q" in the url. Normally plugins are bounded to a namespace to allow multiple instances of the search on the same page. In this case you might want to disable this and let EXT:solr only react on the namespaced query parameter (tx_solr[q] by default).
Comma-separated list of additional argument names, that should be added to the persistent arguments that are kept for sub request, like the facet and sorting urls. Hard coded argument names are q, filter and sort.
Till Solr version 6.5.x all parameters of the plugin namespace was added to the url again. With this setting you could enable this behavior again, but only with a whitelist of argument names.
query
The query sub-section defines a few query parameters for the query that will be sent to the Solr server later on. Some query parameters are also generated and set by the extension itself, f.e. when using facets.
query.allowEmptyQuery
Type
Boolean
TS Path
plugin.tx_solr.search.query.allowEmptyQuery
Default
Options
0,1
Since
1.4
If enabled, empty queries are allowed.
query.allowedSites
Type
String
TS Path
plugin.tx_solr.search.query.allowedSites
Since
2.2
Default
__solr_current_site
When indexing documents (pages, records, files, ...) into the Solr index, the Solr extension adds a "siteHash". The siteHash is used to allow indexing multiple sites into one index and still have each site only find its own documents. This is achieved by adding a filter on the siteHash.
Sometimes though, you want to search across multiple domains, then the siteHash is a blocker. Using the allowedSites setting you can set a comma-separated list of domains who's documents are allowed to be included in the current domain's search results. The default value is __solr_current_site which is a magic string/variable that is replaced with the current site's domain when querying the Solr server.
Since
3.0
Version 3.0 introduced a couple more magic keywords that get replaced:
__current_site same as __solr_current_site
__all Adds all domains as allowed sites
* (asterisk character) Everything is allowed as siteHash (same as no siteHash check). This option should only be used when you need a search across multiple system and you know the impact of turning of the siteHash check.
query.getParameter
Type
String
TS Path
plugin.tx_solr.search.query.getParameter
Since
2.2
Default
tx_solr|q
The GET query parameter name used in URLs. Useful for cases f.e. when a website tracking tool does not support the default array GET parameters.
The option expects a string, you can also define an array in the form of arrayName|arrayKey.
query.fields has been renamed to query.queryFields in version 3.0
Defines what fields to search in the index. Fields are defined as a comma separated list. Each field can be given a boost by appending the boost value separated by the ^ character, that's Lucene query language. The boost value itself is a float value, pay attention to using a dot as the separator for the fractions. Use this option to add more fields to search.
The boost take influence on what score a document gets when searching and thus how documents are ranked and listed in the search results. A higher score will move documents up in the result listing. The boost is a multiplier for the original score value of a document for a search term.
By default if a search term is found in the content field the documents gets scored / ranked higher as if a term was found in the title or keywords field. Although the default should provide a good setting, you can play around with the boost values to find the best ranking for your content.
query.returnFields
Type
String
TS Path
plugin.tx_solr.search.query.returnFields
Since
3.0
Default
*, score
Limits the fields returned in the result documents, by default returns all field plus the virtual score field.
Sets the minimum match mm query parameter.
By default the mm query parameter is set in solrconfig.xml as 2<-35%. This means that for queries with less than three words they all must match the searched fields of a document. For queries with three or more words at least 65% of them must match rounded up.
Please consult the link to the Solr wiki for a more detailed description of the mm syntax.
A boost function can be useful to influence the relevance calculation and boost some documents to appear more at the beginning of the result list.
Technically the parameter will be mapped to the "bf" parameter in the Solr query.
Use cases for example could be:
"Give newer documents a higher priority":
This could be done with a recip function:
recip(ms(NOW,created),3.16e-11,1,1)
Copied!
"Give documents with a certain field value a higher priority":
Allows to further manipulate the score of a document by using Lucene syntax queries. A common use case for boost queries is to rank documents of a specific type higher than others.
Please consult the link to the Solr wiki for a more detailed description of boost functions.
This parameter ties the scores together. Setting is to "0" (default) uses the maximum score of all computed scores.
A value of "1.0" adds all scores. The value is a number between "0.0" and "1.0".
Allows to predefine filters to apply to a search query. You can add multiple filters through a name to Lucene filter mapping. The filters support stdWrap.
This is a magic/reserved filter (thus the double underscore). It limits the query and the results to certain branches/sections of the page tree. Multiple starting points can be provided as a comma-separated list of page IDs.
Since version 11.5.6 it's possible to apply a stdWrap to it.
Example:
plugin.tx_solr.search.query.filter {
__pageSections = TEXT
__pageSections.data = leveluid:0
}
Copied!
query.sortBy
Type
String
TS Path
plugin.tx_solr.search.query.sortBy
Since
1.0
Allows to set a custom sorting for the query. By default Solr will sort by relevance, using this setting you can sort by any sortable field.
Needs a Solr field name followed by asc for ascending order or desc for descending.
This parameter enables the phrase search feature from Apache Solr. Setting is to "0" (default) does not change behaviour from Apache Solr if user searches for two and more words.
Enabling phrase search feature influences the document set and/or the scores of documents.
This parameter defines what fields should be used to search in the given phrase. Matched documents will be boosted according to fields boost value.
Fields are defined as a comma separated list and same way as queryFields.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter defines the "phrase slop" value, which represents the number of positions one word needs to be moved in relation to another word in order to match a phrase specified in a query.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter defines the "phrase slop" value, which represents the number of positions one word needs to be moved in relation to another word in order to match a phrase specified in a explicit phrase search query.
Note: On explicit("double quoted" phrase) phrase search Apache Solr searches in "qf" queryFields
Note: The value of this setting has no influence on implicit phrase search.
On explicit phrase search the Solr searches in qf (plugin.tx_solr.search.query.queryFields) defined fields.
This parameter enables the bigram phrase search feature from Apache Solr. Setting is to "0" (default) does not change behaviour from Apache Solr if user searches for three and more words.
Enabling bigram phrase search feature influences the scores of documents with phrase occurrences.
This parameter defines what fields should be used to search in the given sentence(three+ words). Matched documents will be boosted according to fields boost value.
Fields are defined as a comma separated list and same way as queryFields.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter defines the "bigram phrase slop" value, which represents the number of positions one word needs to be moved in relation to another word in order to match a phrase specified in a query.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter enables the phrase search feature from Apache Solr. Setting is to "0" (default) does not change behaviour from Apache Solr if user searches for two and more words.
Enabling phrase search feature influences the scores of documents with phrase occurrences.
This parameter defines what fields should be used to search in the given phrase. Matched documents will be boosted according to fields boost value.
Fields are defined as a comma separated list and same way as queryFields.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter defines the "trigram phrase slop" value, which represents the number of positions one word needs to be moved in relation to another word in order to match a phrase specified in a query.
Note: The value of this setting has NO influence on explicit phrase search.
En-/disables search term highlighting on the results page.
Note: The FastVectorHighlighter is used by default (Since 4.0) if fragmentSize is set to at least 18 (this is required by the FastVectorHighlighter to work).
Note: The highlighting in Solr (based on FastVectorHighlighter requires a field datatype with termVectors=on, termPositions=on and termOffsets=on which is the case for the content field).
If you add other fields here, make sure that you are using a datatype where this is configured.
When highlighting is activated Solr highlights the fields configured in highlightFields and can return multiple fragments of fragmentSize around the highlighted search word. These fragments are used as teasers in the results list. fragmentSeparator allows to configure the glue string between those fragments.
Activates TYPO3's highlighting of search words on the actual pages. The words a user searched for will be wrapped with a span and CSS class csc-sword
Highlighting can be styled using the CSS class csc-sword, you need to add the style definition yourself for the complete site.
This setting can be used to trigger a new search automatically when the previous search had no results but
suggestions from the spellchecking. In this case the user can directly see the results of the best correction option.
lastSearches
lastSearches
Type
Boolean
TS Path
plugin.tx_solr.search.lastSearches
Since
1.3-dkd
Default
Set plugin.tx_solr.search.lastSearches = 1 to display a list of the last searches.
lastSearches.limit
Type
Integer
TS Path
plugin.tx_solr.search.lastSearches.limit
Since
1.3-dkd
Default
10
Defines the number of last searches, that should get minded.
lastSearches.mode
Type
String
TS Path
plugin.tx_solr.search.lastSearches.mode
Since
1.3-dkd
Default
disabled
Options
user, global, disabled
If mode is user, keywords will get stored into the session. If mode is global keywords will get stored into the database. If mode is disabled, then keywords are not stored in the database.
frequentSearches
frequentSearches
Type
Boolean
TS Path
plugin.tx_solr.search.frequentSearches
Since
1.3-dkd, 2.8
Default
Set plugin.tx_solr.search.frequentSearches = 1 to display a list of the frequent / common searches.
When enabled, keywords are written to the statistics table in lower case.
frequentSearches.minSize
Type
Integer
TS Path
plugin.tx_solr.search.frequentSearches.minSize
Since
1.3-dkd, 2.8
Default
14
The difference between frequentSearches.maxSize and frequentSearches.minSize is used for calculating the current step.
frequentSearches.maxSize
Type
Integer
TS Path
plugin.tx_solr.search.frequentSearches.maxSize
Since
1.3-dkd, 2.8
Default
32
The difference between frequentSearches.maxSize and frequentSearches.minSize is used for calculating the current step.
frequentSearches.limit
Type
Integer
TS Path
plugin.tx_solr.search.frequentSearches.limit
Since
1.3-dkd, 2.8
Default
20
Defines the maximum size of the list by frequentSearches.select.
frequentSearches.select
Type
cObject
TS Path
plugin.tx_solr.search.frequentSearches.select
Since
1.3-dkd, 2.8
Defines a database connection for retrieving statistics.
sorting
sorting
Type
Boolean
TS Path
plugin.tx_solr.search.sorting
Since
1.0
Default
Set plugin.tx_solr.search.sorting = 1 to allow sorting of results.
sorting.defaultOrder
Type
String
TS Path
plugin.tx_solr.search.sorting.defaultOrder
Since
1.0
Default
asc
Options
asc, desc
Sets the default sort order for all sort options.
sorting.options
This is a list of sorting options. Each option has a field and label to be used. By default the options title, type, author, and created are configured, plus the virtual relevancy field which is used for sorting by default.
Example:
plugin.tx_solr.search {
sorting {
options {
relevance {
field = relevance
label = Relevance
}
title {
field = sortTitle
label = Title
}
}
}
}
Copied!
Note: As mentioned before relevance is a virtual field that is used to reset the sorting. Sorting by relevance means to have the order provided by the scoring from solr. That the reason why sorting descending on relevance is not possible.
Defines how facet options are sorted, by default they are sorted by count of results, highest on top. count, 1, true are aliases for each other.
Facet options can also be sorted alphabetically (lexicographic by indexed term) by setting the option to index. index, 0, false, alpha (from version 1.2 and 2.0), and lex (from version 1.2 and 2.0) are aliases for index.
faceting.limit
Type
Integer
TS Path
plugin.tx_solr.search.faceting.limit
Since
1.0
Default
10
Number of options to display per facet. If more options are returned by Solr, they are hidden and can be expanded by clicking a "show more" link. This feature uses a small javascript function to collapse/expand the additional options.
Defines the output of the "Show more" link, that is rendered if there are more facets given than set by faceting.limit.
faceting.showEmptyFacets
Type
Boolean
TS Path
plugin.tx_solr.search.faceting.showEmptyFacets
Since
1.3
Default
Options
0, 1
By setting this option to 1, you will allow rendering of empty facets. Usually, if a facet does not offer any options to filter a result-set of documents, the facet header will not be shown. Using this option allows the header still to be rendered when no filter options are provided.
faceting.urlParameterStyle
Type
Option/String (index or assoc)
TS Path
plugin.tx_solr.search.faceting.urlParameterStyle
Since
11.1
Default
index
Allows to change the URL style of facets.
Possible values:
index: Index style (default)
tx_solr[filter][0]=type:pages, the legacy style URL
assoc: Associative style
tx_solr[filter][type:pages]=1, the more modern associative style URL.
Note: The setting faceting.urlParameterSort will be enabled and can not be disabled.
On faceting.urlParameterStyle = assoc, this setting can not be disabled.
Allows to enable sorting of url parameters, so the single state of facets is associated with same url, no matter in which order the facets were selected
Defines which fields you want to use for faceting. It's a list of facet configurations.
plugin.tx_solr.search.faceting.facets {
type {
field = type
label = Content Type
}
category {
field = category_stringM
label = Category
}
}
Copied!
faceting.facets.[facetName] - single facet configuration
You can add new facets by simply adding a new facet configuration in TypoScript. [facetName] represents the facet's name and acts as a configuration "container" for a single facet. All configuration options for a facet are defined within that "container".
A facet will use the values of a configured index field to offer these values as filter options to your site's visitors. You need to make sure that the facet field's type allows to sort the field's value; like string, int, and other primitive types.
To configure a facet you only need to provide the label and field configuration options, all other configuration options are optional.
The settings `keepAllOptionsOnSelection` and `keepAllFacetsOnSelection` are used internally to build exclude tags for facets in order to exclude the filters from the facet counts.
This helps to keep the counts of a facet as expected by the user, in some use-cases (Read also: http://yonik.com/multi-select-faceting/).
With the setting `additionalExcludeTags` you can add tags of facets that should be excluded from the counts as well.
Note: This setting is only available for option facets by now.
When you want to add fields as `additionalExcludeTags` for a facet a tag for this facet needs to exist. You can use this setting to force the creation of a tag for this facet in the Solr query.
Used as a headline or title to describe the options of a facet.
Used in flex forms of plugin for filter labels. Can be translated with LLL: and consumed and translated in Partial/Facets/* with f:translate ViewHelper.
Metrics can be use to collect and enhance facet options with statistical data of the faceted documents. They can
be used to render useful information in the context of an facet option.
Example:
plugin.tx_solr.search.faceting.facets {
category {
field = field
label = Category
metrics {
downloads = sum(downloads_intS)
}
}
}
Copied!
The example above will make the metric "downloads" available for all category options. In this case it will be the sum of all downloads
of this category item. In the frontend you can render this metric with "<facetoptions.>.metrics.downloads" and use it for example to show it instead of the normal option count.
Normally, when clicking any option link of a facet this would result in only that one option being displayed afterwards. By setting this option to one, you can prevent this. All options will still be displayed.
This is useful if you want to allow the user to select more than one option from a single facet.
When configuring a facet to allow selection of multiple options, you can use this option to decide whether multiple selected options should be combined using AND or OR.
Sets how a single facet's options are sorted, by default they are sorted by count of results, highest on top.
Facet options can also be sorted alphabetically by setting the option to alpha.
Note: Since 9.0.0 it is possible to sort a facet by a function. This can be done by defining a metric and use that metric in the sortBy configuration. As sorting name you then need to use by convention "metrics_<metricName>"
Example:
pid {
label = Content Type
field = pid
metrics {
newest = max(created)
}
sortBy = metrics_newest desc
}
By default facet options are sorted by the amount of results they will return when applied. This option allows to manually adjust the order of the facet's options. The sorting is defined as a comma-separated list of options to re-order. Options listed will be moved to the top in the order defined, all other options will remain in their original order.
Define an alternative delimiter instead of the default comma (,) for the manualSortOrder option.
This is especially useful if the , is part of a facet option value.
Set's the minimumCount for a single facet. This can be useful e.g. to set the minimumCount of a single facet to 0,
to have the options available even when there is result available.
Note: This setting is only available for facets that are using the json faceting API of solr. By now this
is only available for the options facets.
Allows you to display a facet even if it does not offer any options (is empty) and although you have set plugin.tx_solr.search.faceting.showEmptyFacets = 0.
Defines the type of the facet. By default all facets will render their facet options as a list. PHP Classes can be registered to add new types. Using this setting will allow you to use such a type and then have the facet's options rendered and processed by the registered PHP class.
Allows to define requirements for a facet to be rendered. These requirements are dependencies on values of other facets being selected by the user. You can define multiple requirements for each facet. If multiple requirements are defined, all must be met before the facet is rendered.
Each requirement has a name so you can easily recognize what the requirement is about. The requirement is then defined by the name of another facet and a list of comma-separated values. At least one of the defined values must be selected by the user to meet the requirement.
There are two magic values for the requirement's values definition:
__any: will mark the requirement as met if the user selects any of the required facet's options
__none: marks the requirement as met if none of the required facet's options is selected. As soon as any of the required facet's options is selected the requirement will not be met and thus the facet will not be rendered
Example of a category facet showing only when the user selects the news type facet option:
plugin.tx_solr {
search {
faceting {
facets {
type {
label = Content Type
field = type
}
category {
label = Category
field = category_stringS
requirements {
typeIsNews {
# typeIsNews is the name of the requirement, c# choose any so you can easily recognize what it does
facet = type
# The name of the facet as defined above
values = news
# The value of the type facet option as# it is stored in the Solr index
}
}
}
}
}
}
}
Overwrites how single facet options are rendered using TypoScript cObjects.
Example: (taken from issue #5920)
plugin.tx_solr {
search {
faceting {
facets {
type {
renderingInstruction = CASE
renderingInstruction {
key.field = optionValue
pages = TEXT
pages.value = Pages
pages.lang.de = Seiten
tx_solr_file = TEXT
tx_solr_file.value = Files
tx_solr_file.lang.de = Dateien
tt_news = TEXT
tt_news.value = News
tt_news.lang.de = Nachrichten
}
}
language {
renderingInstruction = CASE
renderingInstruction {
key.field = optionValue
0 = TEXT
0.value = English
0.lang.de = Englisch
1 = TEXT1.value = German
1.lang.de = Deutsch
}
}
}
}
}
}
Copied!
EXT:solr provides the following renderingInstructions that you can use in your project:
FormatDate:
This rendering instruction can be used in combination with a date field or an integer field that hold a timestamp. You can use this rendering instruction to format the facet value on rendering.
A common use-case for this is, when the datatype in Solr needs to be sortable (date or int) but you need to render the date as readable date option in the frontend:
plugin.tx_solr.search.faceting.facets {
created {
field = created
label = Created
sortBy = alpha
reverseOrder = 1
renderingInstruction = TEXT
renderingInstruction {
field = optionValue
postUserFunc = ApacheSolrForTypo3\Solr\Domain\Search\ResultSet\Facets\RenderingInstructions\FormatDate->format
}
}
}
Copied!
elevation
elevation
Type
Boolean
TS Path
plugin.tx_solr.search.elevation
Since
3.0
Default
Set plugin.tx_solr.search.elevation = 1 to enable content elevation in search results.
If enabled, elevated results are marked with CSS class "results-elevated".
variants
By using variants you can shrink down multiple documents with the same value
in one field into one document and make similar documents available in
the variants property.
By default the field variantId is used as Solr collapsing criteria.
This can be used e.g. as one approach of deduplication to group similar
documents into on "root" SearchResult.
To use the different variants of the documents you can
access "document.variants" to access the expanded documents.
This can be used for example for de-duplication to list variants of
the same document below a certain document.
Note: Internally this is implemented with Solr field collapsing
Warning
If you're additionally using the grouping feature, the variants
feature will be deactivated completely.
Type
Boolean
TS Path
plugin.tx_solr.search.variants
Since
6.0
Default
Set plugin.tx_solr.search.variants = 1 to enable the variants in search results.
variants.expand
Used to expand the document variants to the document.variants property.
Type
Boolean
TS Path
plugin.tx_solr.search.variants.expand
Since
6.0
Default
1
variants.variantField
Used to expand the document variants to the document.variants property.
Note:: The field must be a numeric field or a string field! Not a
text field!
Type
String
TS Path
plugin.tx_solr.search.variants.variantField
Since
6.0
Default
variantId
variants.limit
Limit of expanded documents.
Though this setting limits the returned variants, you still can get the number
of existing variants, it's set in "document.variantsNumFound" (since
EXT:solr 10)
Type
Integer
TS Path
plugin.tx_solr.search.variants.limit
Since
6.0
Default
10
grouping
The Solr grouping feature can be used to group documents based on a Solr field
or a set of Solr queries.
Note
The grouping feature has a higher value than the collapsing/variant
feature. So if you use the grouping feature, the variant feature
is deactivated.
The following example shows how to group documents based on the "type" field:
plugin.tx_solr {
search {
grouping = 1
grouping {
numberOfGroups = 5
numberOfResultsPerGroup = 5
allowGetParameterSwitch = 0
groups {
typeGroup {
field = type
}
}
}
}
}
Copied!
The next example shows how to group documents based on queries:
Allows to set a custom sorting for the group. Useful especially if you have already set plugin.tx_solr.search.query.sortBy. By default Solr will sort within a group by relevance, using this setting you can sort by any sortable field.
Needs a Solr field name followed by asc for ascending order or desc for descending.
tx_solr.suggest
This feature allows you to show a suggest layer that suggest terms that start with the letters that have been typed into the search field
numberOfSuggestions
Type
Integer
TS Path
plugin.tx_solr.suggest.numberOfSuggestions
Since
1.1
Default
10
Sets the number of suggestions returned and displayed in the layer attached to the search field.
suggestField
Type
String
TS Path
plugin.tx_solr.suggest.suggestField
Since
1.1
Default
spell
Sets the Solr index field used to get suggestions from. A general advice is to use a field without stemming on it. For practical reasons this is currently the spell checker field.
Note: With EXT:solr 11.1.0 ASCII folding and language depending normalization filters were introduced, but due to the special behaviour of the auto suggestions ascii-terms were not treated correctly. So with 11.1.3 the untouched tokens are also kept, as this might lead to duplicate
suggestions, a new field for exact suggestions is introduced, if you want to avoid duplicates and use stricter suggestions, just configure spellExact as suggest field.
forceHttps
Type
Boolean
TS Path
plugin.tx_solr.suggest.forceHttps
Since
1.1
Default
Options
0,1
If enabled, HTTPS will be used for querying suggestions. Otherwise HTTP will be used.
When a user types multiple words into your search field they usually are split up into full keywords used in the query's q parameter and the last part being the partial keyword in the facet.prefix parameter. Enabling this setting moves everything into the facet.prefix parameter. This is usually only useful when using a string field as suggest / auto-complete source field.
Example - Here "Hello Solr" are the full keywords and the user started typing "World" so that "Wo" is used as the partial keyword:
When this setting is enabled, the top results are shown in the suggest layer. The top results are build from the first search match,
or when the first search delivers no hit, the results from the first suggestion are shown.
numberOfTopResults
Type
Integer
TS Path
plugin.tx_solr.suggest.numberOfTopResults
Since
8.0
Default
5
Defines the number of top results that will be shown.
additionalTopResultsFields
Type
String
TS Path
plugin.tx_solr.suggest.additionalTopResultsFields
Since
9.0
Comma-separated list of fields that should be added to the top results response json.
tx_solr.statistics
This section allows you to configure the logging for statistics.
Note: The statistics are logged into a mysql table. This might not make sense for high frequently used searches. In this case you should think about to connect a dedicated tracking tool.
statistics
Type
Boolean
TS Path
plugin.tx_solr.statistics
Since
2.0
Default
Set plugin.tx_solr.statistics = 1 to log statistics.
statistics.anonymizeIP
Type
Integer
TS Path
plugin.tx_solr.statistics.anonymizeIP
Since
2.0
Default
Allowed values are 0 (masking disabled), 1 (mask host), 2 (mask host and subnet).
statistics.addDebugData
Type
Boolean
TS Path
plugin.tx_solr.statistics.addDebugData
Since
6.1
Default
Adds debug data to the columns time_total, time_preparation and time_processing in the table tx_solr_statistics
from the result of the search query.
Note: Enabling addDebugData can have performance impact since debugMode is appended to queries.
statistics.topHits.days
Type
Integer
TS Path
plugin.tx_solr.statistics.topHits.days
Since
12.0
Default
30
Number of days to read out the search top hits.
statistics.topHits.limit
Type
Integer
TS Path
plugin.tx_solr.statistics.topHits.limit
Since
12.0
Default
5
Number of records to read out the search top hits.
statistics.noHits.days
Type
Integer
TS Path
plugin.tx_solr.statistics.noHits.days
Since
12.0
Default
30
Number of days to read out non-search hits.
statistics.noHits.limit
Type
Integer
TS Path
plugin.tx_solr.statistics.noHits.limit
Since
12.0
Default
5
Number of records to read out non-search hits.
statistics.queries.days
Type
Integer
TS Path
plugin.tx_solr.statistics.queries.days
Since
12.0
Default
30
Number of days to read out search queries.
statistics.queries.limit
Type
Integer
TS Path
plugin.tx_solr.statistics.queries.limit
Since
12.0
Default
100
Number of records to read out search queries.
tx_solr.logging
This section defines logging options. All loggings will be available in the TYPO3 logging framework.
If enabled the written log entries will be printed out as debug message in the frontend or to the TYPO3 debug console in the backend.
This setting replaces the previous setting plugin.tx_solr.logging.debugDevLogOutput which was needed, when the devLog was used.
exceptions
Type
Boolean
TS Path
plugin.tx_solr.logging.exceptions
Default
1
Options
0,1
Since
1.0
If enabled, thrown exceptions are logged.
indexing
Type
Boolean
TS Path
plugin.tx_solr.logging.indexing
Default
1
Options
0,1
Since
1.0
If enabled, logs when pages / documents are indexed.
If enabled, the requested data will be logged. Request data includes item, url, parameters, headers, data, decodedData and report.
indexing.pageIndexed
Type
Boolean
TS Path
plugin.tx_solr.logging.indexing.pageIndexed
Default
Options
0,1
Since
2.0
If enabled, it will be logged if a page was indexed successful or not. Additionally, an error is logged if indexing is disabled (via config.index_enable)
query.filters
Type
Boolean
TS Path
plugin.tx_solr.logging.query.filters
Default
1
Options
0,1
Since
1.0
If enabled, filters will be logged when they get added to the Solr query.
query.searchWords
Type
Boolean
TS Path
plugin.tx_solr.logging.query.searchWords
Default
1
Options
0,1
Since
1.0
If enabled, received search queries will be logged.
query.queryString
Type
Boolean
TS Path
plugin.tx_solr.logging.query.queryString
Default
1
Options
0,1
Since
1.0
If enabled, query string parameters and the respective response will be logged.
query.rawPost
Type
Boolean
TS Path
plugin.tx_solr.logging.query.rawPost
Default
1
Options
0,1
Since
1.0
If enabled, POST requests against the Solr server will be logged.
query.rawGet
Type
Boolean
TS Path
plugin.tx_solr.logging.query.rawGet
Default
1
Options
0,1
Since
1.0
If enabled, GET requests against the Solr server will be logged.
Extension Configuration
The following settings can be defined in the "Settings":>"Extension Configuration":>"solr"
Basic
pluginNamespaces
Type
String
Since
11.1
Default
tx_solr
A list of white listed plugin namespaces (Required by cacheHash/excludedParameters and plugin flex form).
Note
This list only is available in Plugin -> Options -> Plugin Namespace.
includeGlobalQParameterInCacheHash
Type
Boolean
Since
11.1
Default
Include/Exclude global q parameter in/from cacheHash.
useConfigurationFromClosestTemplate
Type
Boolean
Since
6.1
Default
When this setting is active the closest page with a TypoScript template will be used to fetch the configuration.
This improves the performance but limits also the possibilities. E.g. conditions can not be used that are related to a certain page.
Monitoring
useConfigurationTrackRecordsOutsideSiteroot
Type
Boolean
Since
6.1
Default
1
A common scenario is to have a site and a storage folder for records parallel to it
on the same level (f.e.)
If you don't want this behaviour - it should be set to false.
useConfigurationMonitorTables
Type
String
Since
6.1
Default
Monitor tables - explicitly monitor these (still requires TypoScript configuration)
monitoringType
Type
Int
Since
11.2
Default
Defines how data updates should be monitored
By default (=0) recognized updates will be processed directly and the Solr index queue will be directly updated, also
the Solr index where appropriate. As in huge instances the monitoring can slow down the TYPO3 backend, two more monitoring
options are available:
1: Delayed: Record update events will be queue and processed later, the scheduler task "Event Queue Worker" is required for processing.
2: No monitoring: Monitoring is completely disabled, please note that you have to take care of Solr index updates yourself.
Route Enhancer
enableRouteEnhancer
Type
Boolean
Since
12.0
Default
To use the EXT:solr possibility to create speaking URLs for Solr facets, activate this option.
As this feature requires additional configuration and costly processing, it's disabled by default.
Advanced
allowSelfSignedCertificates
Type
Boolean
Since
6.1
Default
Can be used to allow self signed certificates - when using the SSL protocol.
Solr Connection
EXT:solr uses the TYPO3 PSR-7 client to connect to the Solr server.
You can apply additional configuration for proxy settings using $GLOBALS['TYPO3_CONF_VARS']['HTTP'].
Tip
For more details about Guzzle settings and TYPO3 implementation see
Some of the SQL statements performed on the pages table in TYPO3 perform extensive operations while copying
page-trees. These operations can be speed up by by adding 2 indexes to the standard table pages.
The indexes are:
content_from_pid_deleted (content_from_pid, deleted),
doktype_no_search_deleted (doktype, no_search, deleted)
It is not required that these indexes are created by in above scenarios considerable performance gains can be achieved.
Development
There are many ways to extend and hook into EXT:solr to customize EXT:solr for your needs.
If external data should be indexed or the RecordIndexer is not required, it is possible to fill the index with an extension as well. The class can then be called e.g. by a CLI call.
<?phpnamespaceVendor\ExtensionName\Import;
useApacheSolrForTypo3\Solr\ConnectionManager;
useApacheSolrForTypo3\Solr\Domain\Site\SiteRepository;
useApacheSolrForTypo3\Solr\System\Solr\Document\Document;
useApacheSolrForTypo3\Solr\System\Solr\SolrConnection;
useTYPO3\CMS\Core\Utility\GeneralUtility;
classIndexer{
protected ConnectionManager $connectionManager;
publicfunction__construct(){
$this->connectionManager = GeneralUtility::makeInstance(ConnectionManager::class);
}
/**
* Send data to Solr index
*
* @param array $rows Data to be indexed, e.g. multiple DB rows
* @param int $pageId root page
* @param int $language language id
*/publicfunctionindex(array $rows, int $pageId = 1, int $language = 0){
$documents = [];
foreach ($rows as $row) {
$documents[] = $this->createDocument($row, $pageId);
}
$connection = $this->connectionManager->getConnectionByPageId($pageId, $language);
$connection->getWriteService()->addDocuments($documents);
}
/**
* Remove all from index
*
* @throws \ApacheSolrForTypo3\Solr\NoSolrConnectionFoundException
*/publicfunctionclearIndex(){
$connections = $this->connectionManager->getAllConnections();
foreach ($connections as $connectionLanguage => $connection) {
/** @var SolrConnection */
$connection->getWriteService()->deleteByType('custom_type');
}
}
/**
* Create a Solr document which then is sent to Solr
*
* @param array $row
* @param int $pageId
* @return Document
*/protectedfunctioncreateDocument(array $row, int $pageId): Document{
$document = $this->getBaseDocument($row, $pageId);
$solrFieldMapping = [
'title' => 'title',
'summary' => 'abstract',
'information' => 'content',
'keywords' => 'keywords',
'area' => 'area_stringS',
'category' => 'category_stringS'
];
foreach ($row as $key => $value) {
if (isset($solrFieldMapping[$key])) {
$document->setField($solrFieldMapping[$key], $value);
}
}
// url generation
$document->setField('url', 'todo'); // custom implementation requiredreturn $document;
}
/**
* Creates a Solr document with the basic / core fields set already.
*
* @param array $itemRecord The record to use to build the base document
* @param int $rootPageId root page id
* @return Document A basic Solr document
*/protectedfunctiongetBaseDocument(array $itemRecord, int $rootPageId): Document{
$siteRepository = GeneralUtility::makeInstance(SiteRepository::class);
$site = $siteRepository->getSiteByRootPageId($rootPageId);
/** @var Document $document */
$document = GeneralUtility::makeInstance(Document::class);
// required fields
$document->setField('id', 'custom_type_' . $itemRecord['uid']);
$document->setField('variantId', 'custom_type' . $itemRecord['uid']);
$document->setField('type', 'custom_type');
$document->setField('appKey', 'EXT:solr');
$document->setField('access', ['r:0']);
// site, siteHash
$document->setField('site', $site->getDomain());
$document->setField('siteHash', $site->getSiteHash());
// uid, pid
$document->setField('uid', $itemRecord['uid']);
$document->setField('pid', 1);
return $document;
}
}
Copied!
Developing Backend Modules
Backend Components
EXT:solr provides UI components for backend modules. Some components hold (GUI)state and some not, but all components calling actions(changing the extension and/or GUI state!),
and then redirecting to the actions within the component was used(referrer also) or to the defined by callers action uri, if that is required by UX.
Below are all available components listed and their responsibility.
CoreSelector
Renders menu in backends doc header with available Solr cores for selected Site and changes the Solr core by clicking on option in drop down menu.
Provides following methods, which must be called inside the initializeView(...) method in your controller to render this component in Backend:
generateCoreSelectorMenuUsingSiteSelector()
Use this method together with SiteSelectorMenu component.
generateCoreSelectorMenuUsingPageTree()
Use this method if you are using original page tree from CMS.
Provides following Actions for changing state, must be added to actions list of your controller:
switchCore
Provides following fully initialized properties in utilizing action controller:
$selectedSolrCoreConnection from type \ApacheSolrForTypo3\Solr\SolrService
If you need the possibility to switch the core, you can extends the AbstractModuleController (in ApacheSolrForTypo3SolrControllerBackendSearch).
FAQ
Why should I add some action name to my action controller?
To allow calling this action within your controller, component can use own controller for changing state only if that is hardcoded with some module
and allowed by ACL for all(or almost all) be users/groups, but this is a bag approach. Therefore allow changing something, only if that is needed.
What do I need to do for using Backend Components?
By extending ApacheSolrForTypo3SolrControllerBackendSearchAbstractModuleController your module has the pagetree (to select the side) and the core selector, to
select the needed Solr core.
Best Practice
Optimize Page Indexing
Indexing of pages is usually done via this extensions PageIndexer which
determines a pages content by crawling its URL and extracting everything between
the two magic markers <!--TYPO3SEARCH_begin--> and <!--TYPO3SEARCH_end-->.
When fetching the URL the header X-Tx-Solr-Iq is added to the request, which
is (among other things) disabling all caches.
To reduce crawling runtimes you might wanna disable the rather expensive
generation of navigation structures such as main and footer menus that you've
defined for your page object in TypoScript.
You can do so by creating a TypoScript condition that unsets all sorts of
non-content elements. Like this for example:
or if EXT:solrconsole is installed on your system:
bin/typo3 solr:queue:index -n -a 1 -s <site>
Copied!
If not: go to the TYPO3 backend and check the page object definition in the
TypoScript object browser with that condition being activated to see what it
looks like now.
Development Environment
To simplify the development for TYPO3 and Solr related components we provide a development environment based on Docker and the DDEV of the TYPO3 core.
You can find the development box in the following git repository:
When you start the box, you will find a pre-configured environment for the support TYPO3 version and the Solr installation that is needed for the installed EXT:solr version.
Testing and Continues Integration
The goal during the development of EXT:solr is, to test most of the components with unit and integration tests.
Unit Tests
For the single classes we've added unit tests whenever we though it is usefull. The unit tests should run very quickly, to git a quick response.
Integration Tests
As in EXT:solr the integration tests are more complex and test the integration of the different components. Since a database server and a Solr server is required this is needed to run the integration test suite. During the bootstrap of the test environment, we use the TYPO3 core functionality for database tests and we install a local Solr server with out install script.
To simplify the local usage of the unit and integration tests, we ship a few bash script that support you to get everything started.
Bootstrapping the Test Environment
When you want to start the testrunner in your shell you need to bootstrap it once:
source ./Build/Test/bootstrap.sh --local
Copied!
The bootstrapper will prompt for some values:
When the boostrapper was finished successful the following was done:
Environment variables for the TYPO3 testing framework have been exported
Test database was created
Test Solr instance was created
Afterwards you can run the ci suite in your shell
Running the ci Suite
When the test environment was boostrapped correctly you can start the test runner:
./Build/Test/cibuild.sh
Copied!
When everything is configured correctly all tests should run through and you should get a green bar:
Code Structure
The components of EXT:solrfluid have been developed with the domain driven design (DDD) approach (https://de.wikipedia.org/wiki/Domain-driven_Design) for our extension we tried to separate the code by the following layers:
Domain: Everything that is related to the "search" domain should be implemented here.
System: Everything that is related to the "system" (e.g. TYPO3 specific) should be implemented here.
Domain Layer & Domain Model
The classes of the domain layer are located in "Classes/Domain" and should contain everything that is related to the "search domain".
ResultSet
The "SearchResultSet" is the main entity that you get passed to the view. It can be used to access all search related objects on your result page.
UML diagram of the SearchResutSet
The SearchResultSet can be used e.g. to get facets and spelling suggestions. A focus for the first release was a new domain model for facets,
that can be rendered with fluid or any other template engine.
Facets
The following UML diagram shows the implemented facets in EXT:solrfluid. Every facet has one or more facet items attached.
For the OptionsFacet the FacetItem is an Option, for the NumericRangeFacet a NumericRange.
Rendering of a facet:
Based on the "type" TypoScript configuration the "FacetRegistry" chooses the responsible facet package class that
is used to create the object structure from the Solr repsonse. Each facet type is shipped with a default fluid partial, that is able to render such a facet.
A facet package consists of the following parts:
The facet parser: Responsible to parse the Apache Solr response into a domain object structure
The url decoder: Responsible to decode the url fragment of the facet in EXT:solr
The query builder: Responsible to build the needed query fragment for Apache Solr to apply the facet
The TypoScript configuration "partialName" can be used to force the rendering with another fluid partial.
For advanced use cases you can use the "FacetRegistry" to register your own facet type or overwrite the facet behaviour for a certain facet type.
UML diagram of the *Facet classes
As you see in the diagram above Solr ships a clean object structure of the facets, that you can render in your custom templates as you need them.
Events
In version 11 we started with the implementation of events, using the EventDispatcher (PSR-14 Events).
Step by step the events will replace older hooks and signals. In the following you will find a description of the available events
Tip
If you miss any feature useful to the general public, please create a feature request
in our issue tracker.
Monitoring
Observing and processing of data updates is controlled by the following events:
ContentElementDeletedEvent, fired if a content element is deleted
PageMovedEvent, fired if a page is moved
RecordDeletedEvent, fired if a record is deleted
RecordGarbageCheckEvent, fired if a record garbage check is triggered
RecordMovedEvent, fired if a record is moved
VersionSwappedEvent, fired if a version is swapped
All events implement the DataUpdateEventInterface and at least provide information about the elements uid, table and modified fields.
Data update processing
ProcessingFinishedEvent
The ProcessingFinishedEvent indicates that the processing of a data update event is finished, if you want to implement an own monitoring you
can use the ProcessingFinishedEvent to start the processing. The event provides the originally fired data update events listed above, so that
you have all the required information about the update to process.
If you're using the event you're indepent of the monitoring setting, as this event if fired in the immediate and delayed monitoring mode as
soon as an event is processed.
If you're using the delayed processing (see "monitoringType"), you can use one of the following events:
DelayedProcessingQueuingFinishedEvent
This event is fired as soon as the update event is queued in the event queue (tx_solr_eventqueue_item), you can e.g. use this event to
register own events or to implement processing that has to executed immediately even in delayed monitoring mode.
DelayedProcessingFinishedEvent
The Scheduler task "Event Queue Worker" is required to process the data updates in delayed monitoring mode and will fire the DelayedProcessingFinishedEvent
as soon as an event has been processed. If you require to perform actions only during delayed processing, the event can be used.
SearchUriBuilder
The SearchUriBuilder is responsible to build uris, that are used in the searchContext. Since the route enhancer is introduced you can use the following
event to influence the build uris:
BeforeVariableInCachedUrlAreReplacedEvent
BeforeCachedVariablesAreProcessedEvent
AfterUriIsProcessedEvent
Facets
Currently EXT:solr provides following events for Facets-Component modification:
AfterFacetIsParsedEvent
ViewHelpers
Beside the controllers, the domain objects and the templates we ship a few useful view helpers. To avoid a strong coupling between the extension and fluid as template engine we tried to keep all ViewHelpers as "slim" as possible. Whenever it was possible we moved the logic into custom service classes and just use them in the ViewHelper.
Since everything belongs to the "SearchResultSet" and we wanted to avoid the need ob passing this object around from "template to template" and "partial to partial" we decided to provide an own "ControllerContext" that referenced the "SearchResultSet". With this approach, it is possible to access the "SearchResultSet" in every ViewHelper.
With the current release we ship the following concrete ViewHelpers:
|
Path
Description
s:debug.documentScoreAnalyzer
Used to render the score analysis.
s:debug.query
Shows the Solr query debug information.
s:document.highlightResult
Performs the highlighting on a document.
s:document.relevance
Shows the relevance information for a document.
s:facet.area.group
Filters the facets in the rendering scope to one group.
s:facet.options.group.prefix.labelPrefixes
Provides an array of available label prefixes that can be used
to filter with s:facet.options.group.prefix.labelFilter.
s:facet.options.group.prefix.labelFilter
Filters the options of a facet by a given prefix.
s:uri.facet.addFacetItem
Add's a facet item to the current url.
s:uri.facet.removeAllFacets
Removes all facet items from the current url.
s:uri.facet.removeFacet
Removes all options from one facet.
s:uri.facet.removeFacetItem
Removes a single facet item from the url.
s:uri.facet.setFacetItem
Sets one single item for a facet (and removes other setted)
s:uri.paginate.resultPage
Creates a link to a result page of the current search.
s:uri.search.currentSearch
Creates a link to the current search (with facets, sorting...)
s:uri.search.startNewSearch
Creates a link for a new search by a term.
s:uri.sorting.removeSorting
Creates a link to the current search and removes the sorting.
s:uri.sorting.setSorting
Creates a link to the current search and sets a new sorting.
s:pageBrowserRange
Provides the range data for the pagination.
s:searchForm
Renders the searchForm.
s:translate
Custom translate ViewHelper (uses translations from ext:solr)
FAQ - Frequently Asked Questions
What does the term "Core" mean?
This term relates to Apache Solr indexes and means a single distinct part of an index. It is possible to use multiple cores on one single Apache Solr instance.
Good examples are using a different Apache Solr core for each language or of course a separate core for each website.
For more information please refer to the Apache Solr documentation for "Core" explanation.
|
Where can I report a bug?
Please make sure that this bug is not reported already, use also the search function of our issue tracker.
Our issue tracker is on GitHub.
|
Where can I report a security issue?
If you have found a security issue in our extension, please do not post about it in a public channel.
Please send an email to the TYPO3 security team with detailed description of found vulnerability. For more details about security issue handling see https://typo3.org/teams/security/contact-us/
|
Is there some chat/irc channel for EXT:solr available?
Join us on the official Slack for TYPO3 and get answers related to EXT:solr in the #ext-solr channel immediately!
|
Which plugins(TYPO3 Frontend) are available?
Search: Form only
Search: Form, Result, Additional Components
Search: Frequent Searches
Just insert one of this plugins on corresponding page to fade in the search form and/or supply the front end with a search results.
|
When i open the search page i see the message 'Search is currently not available. ', whats wrong?
Did you configure your Solr connection as required?
Please read "Configure Extension" and check if you have configured everything
Did you configure Solr server and port and does the scheme and path match?
Did you click "Initialize connection" after configuring the Solr server?
Can you access the Solr server with wget or curl from the command line?
Is the system report of EXT:solr green?
|
In which cases do I want to trigger indexing manually?
after changing any configuration file.
after modifying synonyms, stop words, protected words in TYPO3 Backend -> Search
Moreover by changing core/index configuration you need to reload the core to make the changes become active.
To reload configuration you can either restart the whole Solr server or simply reload a specific core.
|
I want to index files with EXT:solr. How can i do that?
We provide an addon called EXT:solrfal, that allows you to index files from FAL into Solr. This addon is currently available for partner only.
|
The indexer does not generate URLs to restricted pages. How can I force the generation of URLs?
If you have a detail page for e.g. news records that is restricted, please use the typolink attribute linkAccessRestrictedPages :
|
default = TEXT
default {
// ... other typolink settings
typolink.linkAccessRestrictedPages = 1
}
Copied!
|
How can i use Fluid templates with EXT:solr < v7.0.0?
For the Fluid rendering in EXT:Solr >= 5.0 <= 6.1 we provide the addon EXT:solrfluid, that allows you to render your search results with Fluid.
Since EXT:Solr 7.0 Fluid is the default templating engine.
|
Which versions of EXT:solr / EXT:solrfal and EXT:solrfluid work together?
Pages are not indexed. I did everything by the book.
You forgot to set config.index_enable = 1 in your TypoScript setup: Enable indexing
|
My indexed documents are empty, i can not find the content of a page?
Did you configure the search markers ( "<!-- TYPO3SEARCH_begin -->" and "<!-- TYPO3SEARCH_end -->") on your page? Check the paragraph Search Markers and make sure your page renders them.
|
When i change a record, no update is detected. What's wrong?
Are your records inside of your site root? EXT:solr record monitor processes records that belong to your site, which means they need to be below your site root.
If you want to index records that are outside your sideroot, you need to configure the page id's of the sysfolder as additionalPageIds:
There are two data-types for text stringS and textS. When should i choose which datatype?
String data types like stringS store the raw string. No processing, like stemming, splitting etc. is applied. The processing is useful when you want to search in the field and support more then exact matches. When you just want to display the content you should choose a stringS type, when you want to search in the field you should choose textS.
|
I am adding content to a dynamic field but when i search for the content i can not find the document. What's wrong?
Beside the indexing part you need to configure the query part. Make sure that all relevant fields are configured as query fields:
I don't find the expected document on the first position. What can i do?
:) That's a good question. In the end, Solr is a search and the sorting depends on the score, not as in a database on one or two simple criterion.
In the end Solr provides a lot of settings that influence the score calculation and you need to tune the results to you needs. The following settings are helpful to tune your results.
Check your data
The quality of you data is important. Maybe a document is on the first position because, the search term is really relevant for it? Maybe it is an option to change the content?
Adjust the query field boost factors
For each query field there is a boost value after the ^ sign. To increase the factor of a single field for the whole query, you can increase the number in the query fields.
For use cases like "news are always more important then pages" or "Newer documents should be at the beginning" you can use boostFunctions (query.boostFunction) or boostQueries (query.boostQuery)
The search term only exists as a synonym
You can use the backend module synonyms (Synonyms) to maintain synonyms and configure Solr to retrieve documents by a term that is not naturally inside the document.
Ask DKD support
Beside that, there are more options to tune. The DKD support can help you, to analyze and tune your search results. Call +49 (0)69 - 247 52 18-0.
Non ASCII characters like german umlauts do not work when i search, how do I fix that?
To allow search with umlauts Tomcat needs to be configured to use UTF-8 encoded urls. Go to apache-tomcat/conf/server.xml and change the URIEncoding parameter:
How can I change Solr's schema and add custom fields?
Please do not change the shipped Solr schema. There are a lot of dynamic fields (Appendix - Dynamic Fields) that can be used to index any kind of datatype.
I am using varnish before my site. How can i index pages properly?
SOLR Indexer might have some issues, when the page to index is behind a Varnish Proxy. We have collected two ways of solving this issue
Bypassing varnish:
Bypass when X-Tx-Solr-Iq is present
The Solr indexer request send the header X-Tx-Solr-Iq.
To have bypass the Varnish caching, put this into your sub vcl_recv part of the configuration
|
if (req.http.X-Tx-Solr-Iq) {
return(pipe);
}
Copied!
Using Cache-Control:
Put this into your sub vcl_fetch part of the configuration
|
if (req.http.Cache-Control ~ "no-cache") {
set beresp.ttl = 0s;
# Make sure ESI includes are processed!
esi;
set beresp.http.X-Cacheable = "NO:force-reload";
# Make sure that We remove all cache headers, so the Browser does not cache it for us!
remove beresp.http.Cache-Control;
remove beresp.http.Expires;
remove beresp.http.Last-Modified;
remove beresp.http.ETag;
remove beresp.http.Pragma;
return (deliver);
}
Copied!
I want to build the Dockerfile_full image on my mac with a local volume, how can i do that?
The following example shows how to build the Dockerfile image and start a container with a mapped local volume (only for the data).
This was tested with "Docker for Mac" (not Docker Toolbox). Before executing the example, make sure, that you have added " /solrdata" as allowed volume in the Docker configuration.
# build the image
docker build -t typo3-solr -f Docker/SolrServer/Dockerfile .
# create volume directory locally
mkdir -p ~/solrdata
# add Solr group to volume directory
sudo chown :8983 ~/solrdata
# run Docker container from image with volume
docker run -d -p 127.0.0.1:8282:8983 -v ~/solrdata:/var/solr/data/data typo3-solr
Copied!
Can i index a https (SSL) site?
Yes. You need a ssl certificate (can be self signed) and change the following setting:
How can i use a configuration from AdditionalConfiguration.php when i deploy my application on several instances?
The configuration of the connection is done with TypoScript. When you want to use a configuration from TYPO3_CONF_VARS or from the system environment,
you can apply an stdWrap on the configuration that reads from these configurations.
The following example shows how a host can be configured in the AdditionalConfiguration.php and used in your TypoScript to connect to solr:
The following line is added to AdditionalConfiguration.php
To use this configuration for the host, you can use a TEXT element in the configuration and use override.data to use the
value from the AdditionalConfiguration.php
plugin.tx_solr.solr {
host = TEXT
host {
value = localhost
override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|host
}
}
Copied!
|
I want to index extension records, what do i need to do?
EXT:solr provides a flexible indexing for TYPO3 pages and records. You can add a custom indexing configuration for your own records with a valid TCA configuration.
The extension ships several examples in the Folder "Configuration/TypoScript/Examples", read them and try to undestand them.
EXT:solr can not know the business logic of an extension to generate a link to a detail view. You need to use typolink to build an url that points to a valid, existing detail page.
When you index records, e.g. news it these records are indexed in Solr and point to a news details page. That's the reason why it makes sence to exclude the news detail page from the normal page indexing. Otherwise the indexing of this page will produce an error message, because only a url with a valid news uid produces a valid output.
|
Are in EXT:solr some cli commands available?
Yes, currently(v. 6.1) only one for initializing Solr connections.
But check for new ones with bin/typo3 list command.
|
I want to overwrite the type field, why is this not possible?
The type field is a system field that EXT:solr uses to keep the system in sync. Overwritting this field might result in inconsistency.
However, if you need something like a custom type you can also write the information to a dynamic Solr field and use that one as a type.
The following example shows, how to fill the field "mytype_stringS" and build a facet on this field:
plugin.tx_solr {
index{
queue{
news = 1
news {
type = tt_news
fields {
mytype_stringS = TEXT
mytype_stringS.value = news
}
}
}
}
search.faceting.facets.mytype_stringS {
label = Type
field = mytype_stringS
}
}
Copied!
|
I want to implement a toggle functionality for facet options as previously possible with selectingSelectedFacetOptionRemovesFilter. How can i do that?
This is completely possible with Fluid core ViewHelpers and the domain model. The following steps are required.
I want to store HTML in solr, how can i retrieve that?
In general it is not recommend to allow html in the Solr field. Especially when you index content that can be changed by the user.
However, if you want to allow html in a Solr field, you need to add the field as trusted field and the content will not be escaped during the retrieval from solr.
The following example shows how to avoid html in the content field:
Note: When you allow html in the content please make sure that the usage of crop ViewHelpers or a limit of the field length does not break your markup.
I want to use two instances of the search plugin on the same page, how can i do that?
If you want to use two search plugins on the same page you can add two instances and assign a different "Plugin Namespace" in the flexform. If you want to avoid, that both plugins react on the global "q" parameter, you can disable this also in the flexform. Each instance is using the querystring from <pluginNamespace>[q] then.
How can i configure switchable templates for the results plugin?
The following example shows, how you can configure a custom switchable entry template for the Results plugin:
plugin.tx_solr {
solr {
host = TEXT
host {
value = {$plugin.tx_solr.solr.host}
override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|sites|###rootPageId###|solrhost
}
port = TEXT
port {
value = {$plugin.tx_solr.solr.port}
override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|sites|###rootPageId###|solrport
}
}
}
Copied!
I want to use faceting.facets.[facetName].singleOptionMode why was it removed?
This setting belongs to the rendering and not to the facet itself. You can implement the same behaviour just with the given ViewHelpers.
The behaviour is the same, when you just call the ViewHelper s:uri.facet.setFacetItem instead of s:uri.facet.addFacetItem, which semantically just overwrites the current value.
We've added an example partial "OptionsSinglemode" that shows this behaviour. The example TypoScript template "Search - (Example) Options with singlemode (only one option at a time)" shows how to use this partial in combination with the setting "keepAllOptionsOnSelection".
I want to build a tab facet where all options remain, even with an option count of 0. How can i do that?
This can be done with the combination of several settings:
The example above changes the minimumCount to 0, the default value i 1. Setting it to zero allows to have options without any results.
The setting "keepAllFacetsOnSelection" let all facets remain and with keepAllOptionsOnSelection the options in the type facet remain.
How can i add a search-box on every page?
In most projects you want to add a searchbox on every content page. To support this, the default EXT:solr TypoScript template provides the TypoScript template path "plugin.tx_solr_PiSearch_Search" that contains a configured TypoScript code to render the searchbox. When you want to add that to your project in the most cases you would need to refer to a search result page.
The following example shows how you can build a TypoScript lib object that configures the target page for this plugin instance:
By adding the snippet to a generic tempate you could render the searchbox on every page.
How can I index protected pages (htaccess protection)?
Protected pages can be accessed by passing the htpasswd username and password to the indexing queue.
You can set the credentials by the following configuration:
As credentials are stored as plain text, go for sure that your web server does not serve your TypoScript files publicly (protect the directory or by file endings).
If you don't want to store plain text passwords, you can configure your web server to allow access from a specific domain (see below).
If you have multiple domains to index, the webserver requires the credentials for each domain accessed by the Solr indexer. The extension passes the credentials only once, so you will run into errors on a multi domain environment.
Solution: Instead of passing the credentials as shown above, configure your webserver directory protection to allow access from the Solr IP:
AuthType Basic
AuthUserFile /path/to/.htpasswd
<RequireAny>
Require ip XXX.XX.XX.XX (the IP of the Solr server)
Require valid-user
</RequireAny>
Copied!
Be aware, that this will allow all accesses by given IP.
How can I use different host / port configurations in Solr v10 (e.g. for local environments)?
While you could use TypoScript conditions to change the configuration for different project evironments in the past, you can now use environment variables in the config.yaml like shown below.
How can i register a custom statistic writer processor?
If the TypoScript option plugin.tx_solr.statistics = 1 is activated, a StatisticsWriterProcessor is used by EXT:solr, which writes the queries into the database.
If there are more requirements for the statistics or if you want to write the data into another system, you can register your own StatisticWriterProcessor in the ext_localconf.php of the extension:
Solr claims to be not configured in backend context, although I did it well. What can be the reason?
Solr cannot read the configuration from access restricted pages, e. g. "only visible for logged in users".
Although access to your root page is not restricted, this can occur if the root page is a shortcut to a access restricted page.
Does this make sense?
Yes. If user is not logged in one can redirect to a login page with the help of an error handler. And after login one can re-redirect to a restricted page again. Please omit such scenarios.
The reason for that:
With TYPO3 10 and solr 11.0.1 it was possible for solr to read it's configuration from shortcut pages directly. With TYPO3 11 and solr 11.5 these shortcuts are followed. And if the access to the destination of the shortcut is restricted, solr cannot read the configuration because solr is not logged in.
The solution:
If the root page is a shortcut, the destination MAY NOT be access restricted!
In addition a login page itself cannot get hidden after login. This would lead to errors and no redirections will take place.
This can e. g. be achieved in the following way.
The root page is a shortcut to the 1st subpage.
The 1st subpage is a shortcut to an unprotected login page, and will be hidden after successfull login.
The 2nd subpage is an pagetree with the access restricted content.
And last but not least there is the login page (e. g. 3rd subpage) in an unprotected area, which will never get hidden. So ext-solr can read its configuration here.
This way already logged in users will be redirected directly to the restricted page / sub-tree, because the 1st subpage is hidden and so the restricted page is the first sub-page now.
The solr container in DDEV setup keeps quitting?
The issue was experienced on a OsX M2 ARM64 setup, but might be relevant for others.
We have experienced that using Colimas as docker provider in DDEV, caused the solr container to quit upon first HTTP request. This could happen when clicking the "Index Queue" backend module.
The container log did not contain any output relevant to the container quitting.
Changing to Docker Desktop as provider keep the container alive.
Mount points are supported in general and the mounted pages will be indexed like standard pages.
But there is a point to consider: Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs.
The behavior is intentionally designed this way in TYPO3 core, the background is that it is not possible to specify the languages of the mounted page tree without Site Configuration.
[BUGFIX] Fix notice exception in ScoreCalculationService a7b819cbd
[FEATURE] Add an event for modifying the domain used for a site 2b848a77e
[FIX] Docker tests suite does not contain all logs a89de2f97
!!![FIX] Docker execution order issue for as-sudo tweaks 77c3c0f44
[BUGFIX] Docker tweaks as-sudo do not preserve the Docker image ENV eb8546858
[TASK] Use relative path to typo3lib in Apache Solr config dbbf4c5b4
[DOCS] Improve Solr core creation via API and other deployment parts 761894713
!!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814
Updates EXT:solr to Apache Solr 9.8.0.
Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default,
which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality.
All Apache Solr libs, modules or plugins must be configured within the main server configuration files.
See: https://issues.apache.org/jira/browse/SOLR-16781
Impact:
Docker
By using our official Docker image from https://hub.docker.com/r/typo3solr/ext-solr,
you want to pull the image v. 13.0.1+ and restart the container with that image, which will run a migrations-script
automatically to secure the configuration in used volume automatically.
No other steps are required.
Alternatively you can wipe the volume and start the container with v. 13.0.1+ image, but that method will wipe the index as well.
See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh
Other server setups
You have 2 possibilities to fix that issue in your Apache Solr Server:
(PREFERRED) Migrate the EXT:solr's Apache Solr configuration
Or do that in other ways to set the solr.config.lib.enabled=true to sys-props of Apache Solr Server.
!!![FIX] Docker execution order issue for as-sudo tweaks
This change renames the file
from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh
and moves the folder
from /docker-entrypoint-initdb.d/as-sudo/
to /docker-entrypoint-initdb.d-as-sudo/
to fix the execution order issue when setting the correct file permissions
when starting the docker container, leading to a Operation not permitted errors.
/docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh instead of
/docker-entrypoint-initdb.d/as-sudo-tweaks.sh
/docker-entrypoint-initdb.d-as-sudo instead of
/docker-entrypoint-initdb.d/as-sudo/
Release 13.0.0
This is a new major release for TYPO3 13.4 LTS.
New in this release
!!! Upgrade to Apache Solr 9.7.0
This release requires Apache Solr v 9.7.0+.
Adjust mount point indexing
Mount point indexing and corresponding tests have been adjusted for TYPO3 13. Mount points are supported in general and the mounted pages will be indexed like standard pages.
But there is a point to consider: Mounted pages from a pagetree without a site configuration cannot be indexed, in fact TYPO3 currently can't mount a page from a page tree without a site configuration and an exeception occurs.
The behavior is intentionally designed this way in TYPO3 core, the background is that it is not possible to specify the languages of the mounted page tree without Site Configuration.
Note
We require at least TYPO3 13.4.2, as this version contains some bugfixes that address problems with the determination of TypoScript and the site configuration of mounted pages.
All Changes
[TASK] Prepare main branch for TYPO3 13 by Rafael Kähm (eaec73806)
[TASK] Set Apache Solr configsets to ext_solr_13_0_0 2024.05.13 by Rafael Kähm (460f919be)
[BUGFIX] Fix TYPO3 coding standards by Rafael Kähm (80cfe91dc)
[TASK] Simple blocker:: come through typo3 extension:setup command by Rafael Kähm (12de6ef21)
[BUGFIX] Set solr configSet to ext_solr_13_0_0 by Thomas Löffler (c3c317ffe)
[TASK] Update navigationComponent for page tree in v13 by Thomas Löffler (64673fd0f)
[TASK] Adapt Unit Tests for TYPO3 v13 by Benni Mack (c0baedbaa)
!!![TASK] Change default to not track last searches by Christoph Lehmann (e1f607a88)
[TASK] Beautify backend modules for v13 by Thomas Löffler (e51bd8286)
[TASK] Adapt Unit Tests for TYPO3 v13 by Benni Mack (1c3c35105)
[TASK] Make TSFE resolving work again by Benni Mack (6e2b3f3b4)
[TASK] Clean up usages of TSFE mocking by Benni Mack (87630a289)
[BUGFIX] Fix remaining integration tests by Benni Mack (f859f0c5b)
[TASK] fix PhpStan errors for TYPO3 13 by Rafael Kähm (89d9f0d27)
[TASK] Disable progress % within Github Actions by Rafael Kähm (396396979)
[FIX] tests for TYPO3 13 @ 2024.07.02 by Rafael Kähm (20b35ea21)
[FIX] tests for TYPO3 13 @ 2024.07.09 by Rafael Kähm (c02a3ebbc)
[FIX] IntegrationSearchTest for TYPO3 13 by Rafael Kähm (3415e9871)
[FIX] require PHP 8.2 for TER version by Rafael Kähm (3d1092b84)
[FIX] Dependabot not working by Rafael Kähm (dfcdd98bb)
[TASK] Don't store build solrci-image longer as 1 days by Rafael Kähm (ed561a654)
[FIX] GitHub scheduled Actions for daily tests by Rafael Kähm (556b5d3df)
[FIX] deprecations for Fluid viewHelpers and stack by Rafael Kähm (216319eed)
[FIX] IntegrationExtbasePersistenceEventListenerTest errors by Rafael Kähm (97156bf19)
[FIX] Restore BE Modules functionality for TYPO3 13 by Rafael Kähm (cdd979018)
[TASK] migrate to typo3fluid/fluid v4 as required by TYPO3 13 by Rafael Kähm (064ce710d)
[TASK] Remove deprecated queue.[indexConfig].table TypoScript setting by Rafael Kähm (1a426a6dc)
[FIX] Translation handling by delegating requered context objects/values by Rafael Kähm (c3d9db33b)
[FIX] follow-up for removed queue.[indexConfig].table TypoScript setting by Rafael Kähm (7fee9368e)
[FIX] wrong Schema version in status checks by Rafael Kähm (590b34e8d)
[TASK] skip tests for mount-pages temporary #4160 by Rafael Kähm (32906dccf)
[TASK] skip tests for acces restrictions stack temporary #4161 by Rafael Kähm (f8eeaad03)
[BUGFIX] PhpStan Variable $parameters in empty() always exists and is not falsy by Rafael Kähm (2a8596519)
[FIX] Tests for TYPO3 dev-main @2024.09.23 by Rafael Kähm (ff7e038f7)
[BUGFIX] Failed to resolve module specifier '@apache-solr-for-typo3/solr//FormModal.js' by Rafael Kähm (3c86a707f)
[BUGFIX] @typo3/backend/tree/page-tree-element does not work in BE-Modules by Rafael Kähm (111f68404)
[FIX] access restrictions stack for TYPO3 13 by Rafael Kähm (dc7162b25)
[FIX] #[Group('frontend')] attribute has comment in SearchControllerTest by Rafael Kähm (0514886b4)
[TASK] Adjust configuration check and fallbacks in MultiValue CO by Markus Friedrich (ea883ce33)
[TASK] Adapt simulated environment for TYPO3 13 by Markus Friedrich (fb9fdd8c8)
Update TxSolrSearch.rst by Florian Seirer (f8d330082)
[TASK] Update dependencies by Rafael Kähm (01e5387c0)
[TASK] fix CS issues for newest typo3/coding-standards by Rafael Kähm (8c1e28850)
!!![TASK] Upgrade to Apache Solr 9.7.0 by Markus Friedrich (323b1f0c2)
[RELEASE] 13.0.0-alpha-1 by Markus Friedrich (3bd453d09)
[FIX] allow tags/releases from main branch by Rafael Kähm (26e38f8b7)
[TASK] migrate plugin subtype "list_type" by Rafael Kähm (0c0f2b953)
[TASK] Upgrade typo3/testing-framework to dev-main 2024.10.15 by Rafael Kähm (a4596d49e)
[TASK] Use TYPO3 13.4+ and 13.4.x-dev after TYPO3 13 LTS release by Rafael Kähm (0fd63e172)
[TASK] Remove JSONP callback in suggest by Benni Mack (094b4e5b2)
[FEATURE] Introduce method to unset the query string (#4136) by Ayke Halder (b0ddab00e)
Update ExtensionSettings.rst by Jon Echeveste González (d79c92c9d)
[FEATURE] Make Node->depth actually initialized and usable by snk-spo (b530a2f03)
[TASK] Update version matrix by Markus Friedrich (b6bfad8f1)
[TASK] 13.0.x-dev Update solarium/solarium requirement by dependabot[bot] (64e978646)
[TASK] improve exception handling by Rafael Kähm (8f1597b4d)
[FIX] Garbage collector does not get configuration by Rafael Kähm (f73de9da2)
[FIX] CS in Configuration/Backend/Modules.php by Rafael Kähm (08f717129)
[FIX] deprecations in Dockerfile by Rafael Kähm (af1e8cdcd)
[BUGFIX] Ensure index document is deleted by Markus Friedrich (10c0fde3c)
[DOCs] for release 12.0.4 by Rafael Kähm (7b61833ad)
[DOCs] Update EXT:solr 12.0.x line in version matrix by Rafael Kähm (ac1ff3663)
[FIX] phpstan: Method UrlHelper::withQueryParameter() has parameter $value with no type specified by Rafael Kähm (588564f27)
[TASK] Remove Scrutinizer integrations on release-12.0.x by Rafael Kähm (c2558c1d3)
[FIX] Re-added template variables for SearchFormViewHelper by thomashohn (f7ad16ae4)
[DOCs] for release 12.0.5 by Rafael Kähm (ec97b6fd1)
[TASK] Remove Implicitly nullable parameter declarations deprecated by Thomas Hohn (207a0e5fa)
Update composer requirement by Thomas Hohn (43f3baa94)
[TASK] CS change to multiline parameters with comma on last by Rafael Kähm (9aa403a65)
[TASK] Clean and improve ConnectionManagerTest by Markus Friedrich (edf482457)
[TASK] Adjust mount point indexing by Markus Friedrich (bf446c032)
[BUGFIX] Fix record monitoring if site is missing by Markus Friedrich (0dfd4b454)
[TASK] Evaluate all entries in Services.yaml regarding to shared setting by Rafael Kähm (f8083a616)
[TASK] Add int cast for sys_language_uid by Guido Schmechel (de7d7efa7)
[TASK] Add int cast for sys_language_uid by Guido Schmechel (5d659dd3a)
[DOCS] Switch documentation rendering to PHP-based rendering by Rafael Kähm (4f7b9a73e)
[DOCS] workaround for version matrix by Rafael Kähm (bc5bf0b6f)
[FEATURE] Add timeframe filter to statistics module by Bastien Lutz (0fc8d7cbd)
[BUGFIX] Respect foreignLabel in related items from mm table by Till Hörner (f5271b049)
[BUGFIX] Make getHasChildNodeSelected recursive by Tobias Wojtylak (a128c3018)
[BUGFIX] Add StartTimeRestriction to ConfigurationAwareRecordService by Amir Arends (27f36af68)
[FEATURE] Use PHP generator to prevent processing of all available site by Stefan Frömken (7fec14dc4)
[FIX] Indexing fails with SOLR_* cObj in TypoScript by Rafael Kähm (bcb252197)
[FIX] missing TypoScript configuration on RecordMonitor stack by Rafael Kähm (31199d2a1)
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Albrecht Köhnlein
Amir Arends
Ayke Halder
Bastien Lutz
Benni Mack
Bernd Wilke
Christoph Lehmann
@chrrynobaka
@derMatze82
Florian Seirer
Guido Schmechel
Hendrik vom Lehn
Jon Echeveste González
Lars Tode
Markus Friedrich
Rafael Kähm
@snk-spo
Stefan Frömken
Thomas Hohn
Thomas Löffler
Till Hörner
Tobias Wojtylak
Torben Hansen
Wolfgang Wagner
Also a big thank you to our partners who have already concluded one of our development participation packages such
as Apache Solr EB for TYPO3 13 LTS (Feature):
+Pluswerk AG
.hausformat
711media websolutions GmbH
Amt der Oö Landesregierung
Autorité des marchés financiers
Berlin-Brandenburgische Akademie der Wissenschaften
Bestellung EB13 SOS Software GmbH für Telekom
CS2 AG
F7 Media GmbH
Fachhochschule Erfurt
Getdesigned GmbH
Groupe Toumoro inc
KONVERTO AG
Kassenärztliche Vereinigung Rheinland-Pfalz
Kreis Euskirchen
LOUIS INTERNET GmbH
Leuchtfeuer Digital Marketing GmbH
LfdA - Labor für digitale Angelegenheiten GmbH
MOSAIQ GmbH
Marketing Factory Digital GmbH
ProPotsdam GmbH
SITE'NGO
Snowflake Productions GmbH
Stämpfli AG
THE BRETTINGHAMS GmbH
b13 GmbH
clickstorm GmbH
cron IT GmbH
graphodata GmbH
i-kiu motion
in2code GmbH
internezzo ag
jweiland.net e.K.
mehrwert intermediale kommunikation GmbH
network.publishing Möller-Westbunk GmbH
plan2net GmbH
queo GmbH
visol digitale Dienstleistungen GmbH
werkraum Digitalmanufaktur GmbH
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
You are on docs for EXT:solr main version.
This file may be outdated if you are on wrong release "branch" version.
To get the most recent changelog, please choose on the top left dropdown menu the version you are looking for.
[TASK] CS change to multiline parameters with comma on last b5b22eb3b
[TASK] Evaluate all entries in Services.yaml regarding to shared setting a3da67ac2
[FEATURE] Add timeframe filter to statistics module 9e8d69772
[BUGFIX] Respect foreignLabel in related items from mm table fc67246a5
[BUGFIX] Make getHasChildNodeSelected recursive 5d3ab6b0b
[BUGFIX] Add StartTimeRestriction to ConfigurationAwareRecordService 7369adf95
[FEATURE] Use PHP generator to prevent processing of all available site fe71a4f48
[BUGFIX] Fix monitoring of mounted pages f4924d144
[DOCS] Switch documentation rendering to PHP-based rendering 82773617f
[BUGFIX] Add check if generator is valid before traversing it 1c45b5318
[FIX] docker image tests do not fail if core can not start f52a50ec2
!!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814 71ac3ee1a
[DOCS] Actually mention the values of monitoringType 7f72a9fab
!!![TASK] Remove JSONP callback in suggest fca9069ed
[BUGFIX] Fix notice exception in ScoreCalculationService 52800162b
[FIX] Docker tests suite does not contain all logs 9ce209824
!!![FIX] Docker execution order issue for as-sudo tweaks 6ac30c124
[BUGFIX] Docker tweaks as-sudo do not preserve the Docker image ENV c88254506
[TASK] Use relative path to typo3lib in Apache Solr config ad32c9905
[DOCS] Improve Solr core creation via API and other deployment parts 387826fca
!!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814
Updates EXT:solr to Apache Solr 9.8.0.
Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default,
which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality.
All Apache Solr libs, modules or plugins must be configured within the main server configuration files.
See: https://issues.apache.org/jira/browse/SOLR-16781
Impact:
Docker
By using our official Docker image from https://hub.docker.com/r/typo3solr/ext-solr,
you want to pull the image v. 12.0.6+ and restart the container with that image, which will run a migrations-script
automatically to secure the configuration in used volume automatically.
No other steps are required.
Alternatively you can wipe the volume and start the container with v. 12.0.6+ image, but that method will wipe the index as well.
See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh
Other server setups
You have 2 possibilities to fix that issue in your Apache Solr Server:
(PREFERRED) Migrate the EXT:solr's Apache Solr configuration
In own implementation of autosuggest JS parts with usage of JSONP, the action must be migrated to non-JSONP calls.
!!![FIX] Docker execution order issue for as-sudo tweaks
This change renames the file
from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh
and moves the folder
from /docker-entrypoint-initdb.d/as-sudo/
to /docker-entrypoint-initdb.d-as-sudo/
to fix the execution order issue when setting the correct file permissions
when starting the docker container, leading to a Operation not permitted errors.
The Search Component system, which is used to enrich the search query (e.g.
by faceting, boosting, debug analysis), has been completely reworked by
utilizing the PSR-14 event system.
At the same time the Search Query Modifiers have been merged into the
Query Component systems.
All built-in components are now reworked and utilize the
\ApacheSolrForTypo3\Solr\Event\Search\AfterSearchQueryHasBeenPreparedEvent
PSR-14 event.
The interface
\ApacheSolrForTypo3\Solr\Domain\Search\SearchRequestAware has been removed.
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifySearchQuery']
and the interfaces
\ApacheSolrForTypo3\Solr\Query\Modifier as well
as
\ApacheSolrForTypo3\Solr\Search\QueryAware and
\ApacheSolrForTypo3\Solr\Search\SearchAware
have been removed. The modifiers have been merged into Components.
Registration does not happen in ext_localconf.php anymore via
\ApacheSolrForTypo3\Solr\Search\SearchComponentManager
which has been removed, but now happens in Configuration/Services.yaml
as documented in TYPO3 Core's PSR-14 Registration API.
Related hooks around this system have been moved to PSR-14 events as well:
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['beforeSearch'] has been replaced by
\ApacheSolrForTypo3\Solr\Event\Search\AfterInitialSearchResultSetHasBeenCreatedEvent
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] has been replaced by
\ApacheSolrForTypo3\Solr\Event\Search\AfterSearchHasBeenExecutedEvent
SignalSlots replaced by PSR-14 events
The previously available Extbase Signals have been removed from EXT:solr in favor of PSR-14 Events.
The signal
ApacheSolrForTypo3\Solr\Domain\Index\IndexService::beforeIndexItems
has been replaced by
\ApacheSolrForTypo3\Solr\Event\Indexing\BeforeItemsAreIndexedEvent
The signal
ApacheSolrForTypo3\Solr\Domain\Index\IndexService::beforeIndexItem
has been replaced by
\ApacheSolrForTypo3\Solr\Event\Indexing\BeforeItemIsIndexedEvent
The signal
ApacheSolrForTypo3\Solr\Domain\Index\IndexService::afterIndexItem
has been replaced by
\ApacheSolrForTypo3\Solr\Event\Indexing\AfterItemHasBeenIndexedEvent
The signal
ApacheSolrForTypo3\Solr\Domain\Index\IndexService::afterIndexItems
has been replaced by
\ApacheSolrForTypo3\Solr\Event\Indexing\AfterItemsHaveBeenIndexedEvent
The signal
ApacheSolrForTypo3\Solr\Domain\Search\ResultSet\Facets\OptionBased\Options\OptionFacetParser::optionsParsed
has been replaced by
\ApacheSolrForTypo3\Solr\Event\Parser\AfterFacetIsParsedEvent
The signal
ApacheSolrForTypo3\Solr\Controller\SearchController::resultsAction
has been replaced by
\ApacheSolrForTypo3\Solr\Event\Search\BeforeSearchResultIsShownEvent
The signal
ApacheSolrForTypo3\Solr\Controller\SearchController::formAction
has been replaced by
\ApacheSolrForTypo3\Solr\Event\Search\BeforeSearchFormIsShownEvent
The signal
ApacheSolrForTypo3\Solr\Controller\SearchController::frequentlySearchedAction
has been replaced by
\ApacheSolrForTypo3\Solr\Event\Search\AfterFrequentlySearchHasBeenExecutedEvent
The signal
ApacheSolrForTypo3\Solr\Controller\SearchController::beforeSearch
has been removed (see the new PSR-14 events below)
Hooks replaced by PSR-14 events
The previously available hooks and their respective interfaces have been removed from EXT:solr.
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['Indexer']['indexPageAddDocuments'] and its
interface
\ApacheSolrForTypo3\Solr\AdditionalPageIndexer are now superseded
by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\Indexing\BeforePageDocumentIsProcessedForIndexingEvent.
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifyVariantId'] and its
interface
\ApacheSolrForTypo3\Solr\Variants\IdModifier are now superseded
by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\Variants\AfterVariantIdWasBuiltEvent.
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['IndexQueueIndexer']['preAddModifyDocuments'] and its
interface
\ApacheSolrForTypo3\Solr\PageIndexerDocumentsModifier are now superseded
by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\Indexing\BeforeDocumentIsProcessedForIndexingEvent.
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['IndexQueueIndexer']['indexItemAddDocuments'] and its
interface
\ApacheSolrForTypo3\Solr\AdditionalIndexQueueItemIndexer are now superseded
by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\Indexing\BeforeDocumentsAreIndexedEvent.
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['Indexer']['indexPageSubstitutePageDocument'] and its
interface
\ApacheSolrForTypo3\Solr\SubstitutePageIndexer are now superseded
by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\Indexing\AfterPageDocumentIsCreatedForIndexingEvent.
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['postProcessIndexQueueInitialization'] and its
interface
\ApacheSolrForTypo3\Solr\IndexQueue\InitializationPostProcessor are now superseded
by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\IndexQueue\AfterIndexQueueHasBeenInitializedEvent
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['postProcessFetchRecordsForIndexQueueItem'] is now superseded
by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\IndexQueue\AfterRecordsForIndexQueueItemsHaveBeenRetrievedEvent
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['IndexQueuePageIndexer']['dataUrlModifier']
and the according interface
\ApacheSolrForTypo3\Solr\IndexQueue\PageIndexerDataUrlModifier
is now superseded by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\Indexing\AfterFrontendPageUriForIndexingHasBeenGeneratedEvent
The hook
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['postProcessIndexQueueUpdateItem']
is now superseded by the PSR-14 event
\ApacheSolrForTypo3\Solr\Event\Indexing\AfterIndexQueueItemHasBeenMarkedForReindexingEvent
PSR-14 events renamed
Previous PSR-14 events have been renamed to be consistent with other PSR-14 Events in EXT:solr.
\ApacheSolrForTypo3\Solr\Event\Routing\PostProcessUriEvent is now named
\ApacheSolrForTypo3\Solr\Event\Routing\AfterUriIsProcessedEvent
\ApacheSolrForTypo3\Solr\Event\Routing\BeforeProcessCachedVariablesEvent is now named
\ApacheSolrForTypo3\Solr\Event\Routing\BeforeCachedVariablesAreProcessedEvent
\ApacheSolrForTypo3\Solr\Event\Routing\BeforeReplaceVariableInCachedUrlEvent is now named
\ApacheSolrForTypo3\Solr\Event\Routing\BeforeVariableInCachedUrlAreReplacedEvent
!!! Shortcut pages not indexed anymore
Currently there is no important reason to index the shortcut pages,
because the target pages are indexed as expected and the shortcuts are 307-redirected to their targets.
So contents can be found in search results as expected.
!!! Deprecated Node class removed
Former EXT:solr versions used an own node implementation for Solr endpoints, this implementation (ApacheSolrForTypo3SolrSystemSolrNode) is now removed in favor of the Endpoint implementation of Solarium.
If you've used this class or the SolrConnection directly, you have to adapt your PHP code:
use SolariumCoreClientEndpoint instead of ApacheSolrForTypo3SolrSystemSolrNode
call ApacheSolrForTypo3SolrSystemSolrSolrConnection->getEndpoint() instead of ApacheSolrForTypo3SolrSystemSolrSolrConnectiongetNode(), method will return Solarium Endpoint
Node could be converted to string to get the core base URI, getCoreBaseUri() can be used instead.
Note: With dropping the Node implementation we also dropped the backwards compatibility that allows to define the Solr path segment "/solr" within "solr_path_read" or "solr_path_write". Be sure your configuration doesn't contain this path segment!
!!! Changed visibility of ApacheSolrForTypo3SolrIndexQueueFrontendHelperPageIndexer methods
For testing purposes some methods of the PageIndexer were defined as public, these methods are now protected. The tests are adapted accordingly, so that there is no need to declare the methods as public.
If you have used one of this methods, you have to adapt your code. Affected methods:
setupConfiguration
index
indexPage
!!! Solr route enhancer disabled by default
EXT:solr offers the possibility to create speaking URLs for Solr facets, but as this feature requires additional configuration and costly processing this feature is now disabled by default.
If you've already used the route enhancer you must set option "enableRouteEnhancer":
The FrontendHelper logic revolving around PageIndexer has been reduced to
a minimum by only having two methods available:
ApacheSolrForTypo3\Solr\IndexQueue\FrontendHelper\FrontendHelper::activate() - used to register hooks and PSR-14 event listeners
ApacheSolrForTypo3\Solr\IndexQueue\FrontendHelper\FrontendHelper::deactivate(PageIndexerResponse $response) - used to populate data into the PageIndexerResponse object
The actual PageIndexerRequest object is now available as a property of TYPO3's
Request object as attribute named "solr.pageIndexingInstructions".
!!!Complex query in FlexForm filter value
It is now possible to use complex query in FlexForm filter value.
If the value contains space and no special characters, the value is always automatically escaped.
The old behaviour is still working,
so if a string value contains space(s) and no special characters of the solr query parser,
the string is always wrapped with double quotes.
But if the string contains special characters no wrapping happen
special characters are:
+ - && || ! ( ) { } [ ] ^ " ~ * ? : \
There is some cases where this change can break,
for example if the filter value is something like
toto AND tata or
music (rock) or my "flow" is.
Here the wrapping and the escaping of the inner double quote have to be manually updated like this
"toto AND tata" or
"music (rock)" and
"my \"flow\" is".
[TASK] Sync with EXT:solr addons :: Fix Type-Hinting issues aee51e499 on @2023-08-11 (thanks to Rafael Kähm)
[TASK] Sync with EXT:solr addons :: Fix Type-Hinting for AfterIndexQueueItemHasBeenMarkedForReindexingEvent d96e4ebf0 on @2023-08-14 (thanks to Rafael Kähm)
[TASK] Sync with EXT:solr addons :: fix old linter issues b15df830e on @2023-08-14 (thanks to Rafael Kähm)
[BUGFIX] prevent Exception when create Event Queue Worker Task ee4b19cda on @2023-08-25 (thanks to Achim Fritz)
[DOC] Add FAQ how to generate URLs to restricted pages 42e938ba4 on @2023-10-12 (thanks to Sascha Schieferdecker)
[BUGFIX] Prevent indexing error on missing 'foreignLabelField' ca39ec607 on @2023-10-12 (thanks to Rafael Kähm)
[DOC] Solr claims to be not configured in backend context, although I did it well. What can be the reason? (#3708) 6c6952beb on @2023-10-12 (thanks to haraldwitt)
!!![FEATURE] Allow using complex filter values in FlexForm 61d1a92ba on @2023-10-12 (thanks to Eric Chavaillaz)
[TEST] Handle float values in options facet parser cb5cdb7d7 on @2023-10-12 (thanks to Rafael Kähm)
[BUGFIX] Handle float values in options facet parser b9299f531 on @2023-07-11 (thanks to Sascha Nowak)
Also a big thank you to our partners who have already concluded one of our new development participation packages such
as Apache Solr EB for TYPO3 12 LTS (Feature or Maintenance):
+Pluswerk AG
.hausformat
3m5. Media GmbH
4eyes GmbH
711media websolutions GmbH
ACO Ahlmann SE & Co. KG
AVM Computersysteme Vertriebs GmbH
AmedickSommer Neue Medien GmbH
Ampack AG
Amt der Oö Landesregierung
Autorité des Marchés Financiers (Québec)
Bandesinstitut für Schule und Medien Berlin-Brandenburg
Beeeh IT
Bytebetrieb GmbH & Co. KG
CARL von CHIARI GmbH
CDG 59
CPS GmbH
CS2 AG
Columbus Interactive GmbH
Connecta AG
DGB Rechtsschutz GmbH
DMK E-BUSINESS GmbH
DP-Medsystems AG
DSCHOY GmbH
Davitec GmbH
Deutsches Literaturarchiv Marbach
Die Medialen GmbH
Digitale Offensive GmbH
EB-12LTS-FEATURE
Eidg. Forschungsanstalt WSL
F7 Media GmbH
FTI Touristik GmbH
Fachagentur Nachwachsende Rohstoffe fnr.de
Forte Digital Germany GmbH
GPM Deutsche Gesellschaft für Projektmanagement e. V.
Gernot Leitgab
Getdesigned GmbH
Groupe Toumoro inc
HEAD acoustics GmbH
HSPV NRW
Hochschule Koblenz Standort Remagen
INOTEC Sicherheitstechnik GmbH
IW Medien GmbH
Internezzo
Intersim AG
KONVERTO AG
Kassenzahnärztliche Vereinigung Bayerns (KZVB)
Kassenärztliche Vereinigung Rheinland-Pfalz
Kreis Euskirchen
Kwintessens B.V.
L.N. Schaffrath DigitalMedien GmbH
LOUIS INTERNET GmbH
La Financière agricole du Québec
Land Tirol
Landeskriminalamt Thüringen
Leuchtfeuer Digital Marketing GmbH
Lingner Consulting New Media GmbH
MEDIENHAUS der Evangelischen Kirche in Hessen und Nassau GmbH
Macaw Germany Cologne GmbH
Marketing Factory Consulting GmbH
NEW.EGO GmbH
OST Ostschweizer Fachhochschule
ProPotsdam GmbH
Provitex GmbH Provitex GmbH
Québec.ca gouv.qc.ca
Randstad Digital
Rechnungshof Österreich
Red Dot GmbH & Co. KG
SIWA Online GmbH
SUNZINET GmbH
Sandstein Neue Medien GmbH
Schoene neue kinder GmbH
Serviceplan Suisse AG
Snowflake Productions GmbH
Stadtverwaltung Villingen-Schwenningen
Statistik Österreich
Studio 9 GmbH
Stämpfli AG
Systime/Gyldendal A/S
Südwestfalen IT
THE BRETTINGHAMS GmbH
Talleux & Zöllner GbR
Typoheads GmbH
UEBERBIT GmbH
Universität Regensburg
VisionConnect.de
WACON Internet GmbH
WIND Internet BV
Webtech AG
Werbeagentur netzpepper
XIMA MEDIA GmbH
b13 GmbH
bgm websolutions GmbH & Co. KG
chiliSCHARF GmbH
clickstorm GmbH
cosmoblonde GmbH
cron IT GmbH
cyperfection GmbH
digit.ly
gedacht
graphodata GmbH
grips IT GmbH
helhum.io
in2code GmbH
jweiland.net
keeen GmbH
medien.de mde GmbH
mehrwert intermediale kommunikation GmbH
mellowmessage GmbH
morbihan.fr
ochschule Furtwangen
pietzpluswild GmbH
plan2net GmbH
rms. relationship marketing solutions GmbH
rocket-media GmbH & Co KG
sgalinski Internet Services
studio ahoi Weitenauer Schwardt GbR
visuellverstehen GmbH
webconsulting business services gmbh
werkraum Digitalmanufaktur GmbH
wow! solution
zimmer7 GmbH
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
You are on docs for EXT:solr main version.
This file may be outdated if you are on wrong release "branch" version.
To get the most recent changelog, please choose on the top left dropdown menu the version you are looking for.
Release 11.6.3 ELTS
This is a non-public maintenance release for TYPO3 11.5 ELTS, containing:
New in this release
Apache Solr 9.8.1 support
EXT:solr 11.6.3 has been tested with Apache Solr 9.8.1, this version can be used without any update steps to consider.
Full list of changes
[TASK] Allow Apache Solr 9.8.1 by @dkd-friedrich
[BUGFIX:P:11.6] Add check if generator is valid before traversing it by @jacobsenj and @dkd-friedrich
[FEATURE:P:11.6] Use PHP generator to prevent processing of all available site @sfroemkenjw and @dkd-friedrich
Release 11.6.2 ELTS
This is a non-public security release for TYPO3 11.5 ELTS, containing:
!!![SECURITY] Update to Apache solr 9.8.0 : CVE-2025-24814
Updates EXT:solr to Apache Solr 9.8.0.
Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default,
which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality.
All Apache Solr libs, modules or plugins must be configured within the main server configuration files.
See: https://issues.apache.org/jira/browse/SOLR-16781
Impact:
Docker
You can wipe the volume and start the container with v. 11.6.2+ image, but that method will wipe the index as well.
See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh
Other server setups
You have 2 possibilities to fix that issue in your Apache Solr Server:
(PREFERRED) Migrate the EXT:solr's Apache Solr configuration
Or do that in other ways to set the solr.config.lib.enabled=true to sys-props of Apache Solr Server.
!!![FIX] Docker execution order issue for as-sudo tweaks
This change renames the file
from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh
and moves the folder
from /docker-entrypoint-initdb.d/as-sudo/
to /docker-entrypoint-initdb.d-as-sudo/
to fix the execution order issue when setting the correct file permissions
when starting the docker container, leading to a Operation not permitted errors.
/docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh instead of
/docker-entrypoint-initdb.d/as-sudo-tweaks.sh
/docker-entrypoint-initdb.d-as-sudo instead of
/docker-entrypoint-initdb.d/as-sudo/
Minor changes & bugfixes
[DOCS] Improve Solr core creation via API and other deployment parts by @dkd-kaehm & @dkd-friedrich in #41
[TASK] Use relative path to typo3lib in Apache Solr config by @dkd-kaehm & @dkd-friedrich in #41
[BUGFIX] Docker twaks as-sudo do not preserve the Docker image ENV by @dkd-kaehm & @dkd-friedrich in #41
[BUGFIX] Docker tests suite does not contain all logs by @dkd-kaehm & @dkd-friedrich in #41
[BUGFIX] docker image tests do not fail if core can not start by @dkd-kaehm & @dkd-friedrich in #41
[TASK] Replace "Publish to TER" to release by @dkd-kaehm in #38
Release 11.6.1 ELTS
This is a non-public maintenance release for TYPO3 11.5.
Full list of changes
[TASK] Integrate TYPO3 11.5 ELTS by @dkd-kaehm & @dkd-friedrich
[TASK] Prepare release-11.6.x ELTS by @dkd-friedrich
[BUGFIX] NPE in Index Queue module when no site is selected by @adamkoppede
[FEATURE] Add timeframe filter to statistics module by @BastiLu
Release 11.6.0
We are happy to release EXT:solr 11.6.0.
The focus of this release has been on Apache Solr upgrade to v9.7.0.
Important: This version is installable with TYPO3 11 LTS on v11.5.14+ only and contains some breaking changes, see details below.
New in this release
!!! Upgrade to Apache Solr 9.7.0
This release requires Apache Solr v9.7.0+.
Along with the compatibility to Solr 9.7 the dependency to SOLR_ENABLE_STREAM_BODY is removed.
Full list of changes
!!![BUGFIX:BP:11.6] Queue check considers indexing configuration by @dkd-friedrich
!!![TASK:BP:11.6] Introduce queue and queue item interfaces by @dkd-friedrich
[TASK:BP:11.6] Consider queue initialization status by @dkd-friedrich
!!![TASK:BP:11.6] Introduce specific EXT:solr exceptions by @dkd-friedrich
[TASK] Prepare release-11.6.x for TYPO3 11.5 LTS by @dkd-kaehm
[TASK] sync the CI stuff from main branch into 11.6.x by @dkd-kaehm
[TASK] Activate PHPStan by @dkd-friedrich
[TASK] Use Apache Solr 9.2 for EXT:solr 11.6 by @dkd-friedrich
[FEATURE] Introduce TYPO3_SOLR_ENABLED_CORES docker env variable by Christoph Lehmann
[TASK] Make it possible to change solr unix GID:UID on docker image… by @dkd-kaehm
[TASK] revert unnecessary changes on Apache Solr 9.2 upgrade by @dkd-kaehm
!!![TASK] Upgrade to Apache Solr 9.3.0 by @dkd-friedrich
[BUGFIX] Fix result highlighting fragment size by @dkd-friedrich
[TASK] Update to Solr 9.5 by @dkd-friedrich
New Crowdin updates 2024.07.03 by @dkd-kaehm
[TASK] Upgrade to Apache Solr 9.7 by @dkd-friedrich
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Adam Koppe
@BastiLu
Christoph Lehmann
Markus Friedrich
Rafael Kähm
Also a big thank you to our partners who have already concluded one of our new development participation packages such as Apache Solr EB for TYPO3 11 LTS:
.hausformat GmbH
3m5. Media GmbH
abteilung_digital GmbH
ACO Ahlmann SE & Co. KG
Agence E-magineurs
Agenda d.o.o.
AgenturWebfox GmbH
Amedick & Sommer Neue Medien GmbH
Ampack AG
Atol CD
Autorité des Marchés Financiers (Québec)
avenit AG
b13 GmbH
brandung GmbH Oliver Krause
Bundesanstalt Statistik Österreich
Bytebetrieb GmbH & Co. KG
Canton de Neuchâtel - SIEN
CARL von CHIARI GmbH
chiliSCHARF GmbH
clickstorm GmbH
co-operate Wegener & Rieke GmbH
Columbus Interactive GmbH
cosmoblonde GmbH
creativ clicks GmbH
cyperfection GmbH
digit.ly
DMK E-BUSINESS GmbH
Earlybird GmbH & Co KG
elancer-team GmbH
eulenblick Kommunikation und Werbung
Fa. .hausformat
Fa. Ausy
Fachagentur Nachwachsende Rohstoffe fnr.de
Fachhochschule Erfurt
Fourdegrees GbR
FTI Touristik GmbH
für novotegra GmbH (BayWa r.e. AG)
Getdesigned GmbH
GFE Media GmbH
graphodata GmbH
gressourcenmangel integral gmbh
grips IT GmbH
hiroki digital GmbH
Hirsch & Wölfl GmbH
Hochschule Furtwangen
Hochschule Koblenz - Standort Remagen
HSPV NRW
in2code
INOTEC Sicherheitstechnik GmbH
Institut national d'excellence en santé et en services sociaux inesss.qc.ca
internezzo ag
IW Medien GmbH
jweiland
Kassenärztliche Vereinigung Rheinland-Pfalz
Kreis Euskirchen
L.N. Schaffrath DigitalMedien GmbH
La Financière agricole du Québec
Landeskriminalamt Thüringen
Leuchtfeuer Digital Marketing GmbH
Lingner Consulting New Media GmbH
LOUIS INTERNET GmbH
Marketing Factory
media::essenz
medien.de mde GmbH
mehrwert intermediale kommunikation GmbH
Meridium Technologies
MOSAIQ GmbH
NEW.EGO GmbH
Overlap GmbH & Co KG
Patrick Gaumond Québec.ca gouv.qc.ca
peytz.dk
pietzpluswild GmbH
Pixelant / Resultify
Plan Software GmbH
Plan.Net France
ProPotsdam GmbH
Provitex GmbH
queo GmbH
rms. relationship marketing solutions GmbH
RR Anwendungsentwicklung
Sandstein Neue Medien GmbH
Schoene neue kinder GmbH
seam media group gmbh
Shop F7 Media GmbH
SITE'NGO
Snowflake Productions GmbH
SOS Software Service GmbH
Stadtverwaltung Villingen-Schwenningen
Stämpfli AG
systime.dk
Talleux & Zöllner GbR
tirol.gv.at
toumoro.com
Typoheads GmbH
UEBERBIT GmbH
unternehmen online GmbH Co. KG (UO)
Verband der Vereine Creditreform e.V.
VisionConnect.de
visol digitale Dienstleistungen GmbH
visuellverstehen GmbH
WACON Internet GmbH
webconsulting business services gmbh
Webtech AG
Werbeagentur netzpepper
werkraum Digitalmanufaktur GmbH
wow! solution
zimmer7 GmbH
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
You are on docs for EXT:solr main version.
This file may be outdated if you are on wrong release "branch" version.
To get the most recent changelog, please choose on the top left dropdown menu the version you are looking for.
Release 11.2.5 ELTS
This is a non-public security release for TYPO3 10.4 ELTS, containing:
!!![SECURITY] Update to Apache Solr 9.8.0 : CVE-2025-24814
Updates EXT:solr to Apache Solr 9.8.0.
Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default,
which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality.
All Apache Solr libs, modules or plugins must be configured within the main server configuration files.
See: https://issues.apache.org/jira/browse/SOLR-16781
Impact:
Docker
You can wipe the volume and start the container with v. 11.2.5+ image, but that method will wipe the index as well.
See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh
Other server setups
You have 2 possibilities to fix that issue in your Apache Solr Server:
(PREFERRED) Migrate the EXT:solr's Apache Solr configuration
Or do that in other ways to set the solr.config.lib.enabled=true to sys-props of Apache Solr Server.
!!![FIX] Docker execution order issue for as-sudo tweaks
This change renames the file
from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh
and moves the folder
from /docker-entrypoint-initdb.d/as-sudo/
to /docker-entrypoint-initdb.d-as-sudo/
to fix the execution order issue when setting the correct file permissions
when starting the docker container, leading to a Operation not permitted errors.
/docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh instead of
/docker-entrypoint-initdb.d/as-sudo-tweaks.sh
/docker-entrypoint-initdb.d-as-sudo instead of
/docker-entrypoint-initdb.d/as-sudo/
Upgrade to Apache Solr 9.7.0
This release requires Apache Solr v9.7.0+.
Along with the compatibility to Solr 9.7 the dependency to SOLR_ENABLE_STREAM_BODY is removed.
Minor changes & bugfixes
[TASK] Update GitHub actions by @dkd-friedrich in #42
[DOCS] Improve Solr core creation via API and other deployment parts by @dkd-kaehm & @dkd-friedrich in #42
[TASK] Use relative path to typo3lib in Apache Solr config by @dkd-kaehm & @dkd-friedrich in #42
[BUGFIX] Docker twaks as-sudo do not preserve the Docker image ENV by @dkd-kaehm & @dkd-friedrich in #42
[BUGFIX] Docker tests suite does not contain all logs by @dkd-kaehm & @dkd-friedrich in #42
[BUGFIX] docker image tests do not fail if core can not start by @dkd-kaehm & @dkd-friedrich in #42
Release 11.2.4 ELTS
Note
Non public ELTS release, you can find more details on typo3-solr.com
Support of Apache Solr 9.5.0
Compatibility with Apache Solr 9.5.0 is checked and EXT:solr now accepts the following Apache Solr versions:
9.3.0
9.4.0
9.4.1
9.5.0
Small improvements and bugfixes
[BUGFIX] Handle float values in options facet parser by @dkd-kaehm in #11
[BUGFIX:11.2] Exception with tx_solr_statistics after latest TYPO3 security update by @dkd-kaehm in #14
!!![TASK] Update to Apache Solr 9.2 for TYPO3 10.4 ELTS by @dkd-kaehm in #19
[TASK] Remove unused field cookie in tx_solr_statistics by @dkd-kaehm in #21
[BUGFIX] Fix autosuggest with non-ascii terms by @dkd-friedrich in #28
[TASK] Prepare release-11.2.x ELTS by @dkd-kaehm in #17
[BUGFIX] Fix branch alias by @dkd-friedrich in #30
[TASK] Allow custom ELTS repository by @dkd-friedrich in #30
[BUGFIX:P:11.2] prevent undefined array key warning if filter is empty by Achim Fritz in #32
[TASK] Allow Apache Solr 9.5 by @dkd-friedrich in #31
Release 11.2.3 - Last non ELTS release
This is a maintenance release for TYPO3 10.4 and the last non ELTS release, as TYPO3 10 LTS reaches the ELTS phase on April 30, 2023.
EXT:solr release-11.2.x will not be maintained in TYPO3-Solr/ext-solr repository any more. The maintenance and builds will be moved to a private
repository and ELTS versions, EXT:solr 11.2.4+ for TYPO3 10 ELTS versions, can be obtained through the dkd EB program.
This release contains:
[BUGFIX:P:11.2] make CE search form in backend editable again by @dkd-kaehm in #3640
[DOC] Fix wrong type for boostQuery in the docs and example by @rr-it and @dkd-kaehm in a997a2f4
Release 11.2.2
This is a maintenance release for TYPO3 10.4.
EXT:solr release-11.2.x will not be maintained in TYPO3-Solr/ext-solr repository any more. The maintenance and builds will be moved to a private
repository and ELTS versions, EXT:solr 11.2.3+ for TYPO3 10 ELTS versions, can be obtained through the dkd EB program.
This release contains:
[BUGFIX] Type-hinting for SiteUtility::getConnectionProperty() by @dkd-kaehm in #3395
[TASK:Security:P:11.2] Update jQuery and its plugin libs by @dkd-kaehm in #3430
[FEATURE] Add signal before search in resultsAction by @stat1x in #3391
[BUGFIX:BP:11.2] Don't use minimum-stability dev on TYPO3 stable in build/CI by @dkd-kaehm in #3466
[TASK] Docker version check on docker image build by @dkd-kaehm in #3524
[BUGFIX:P:11.2] Use ConfigurationManager to get typscript in plugin FlexForm by @dkd-kaehm in #3497
[TASK] Use PCOV for coverage instead of xDebug :: Upgrade to phpunit 8 by @dkd-kaehm in #3528
[BUGFIX:BP:11.2] Make API eID script compatible with TYPO3 v11.5 by @dkd-friedrich in #3550
[BUGFIX:BP:11.2] Use plugin namespace as label for flexforms by @dkd-friedrich in #3553
[BUGFIX:BP:11.2] Respect indexingPriority in QueueItemRepository by @dkd-friedrich in #3557
[BUGFIX:BP:11.2] use siteLanguage TypoScript object to get language id by @dkd-friedrich in #3555
[TASK:11.2] equalize CI/Github-Actions for release-11.0|2|5 and main by @dkd-kaehm in #3467
[BUGFIX:BP:11.2] Sanitize frequent and last searches output by @dkd-friedrich in #3592
[BUGFIX:BP:11.2] changed from always picking the 0 array value to pic… by @dkd-friedrich in #3594
[BUGFIX] Enforce visibility context in Tsfe by @saitho in #3050
[TASK:BP:11.2] Disable sql handler by @dkd-friedrich in #3604
Release 11.2.1
This is a maintenance release for TYPO3 10.4, containing:
[BUGFIX] Prepend wrong fe language on empty cache (#3375)
[TASK] Use TYPO3 IpAnonymizationUtility (#3262)
[BUGFIX:BP:11.2] Shortcircuit work in SolrRoutingMiddleware (#3202)
[TASK] Fix scrutinizer on release-11.2.x
[BUGFIX:BP:11.2] Do not handle page updates on new page with uid 0 (#3338)
[TASK] Remove not used strptime() adaption for windows.
[BUGFIX] Respect indexing configuration for new and updated subpages (#3276)
[BUGFIX] Ensure keywords string does not exceed database field length (#3321)
[TASK:BP:11.2] Adapt column arrangement within sites config (#3295)
[DOCS:BP:11.2] add missing doc for ..pageIndexed
[BUGFIX] Fix PSR-4 Namespaces and Paths (#3285)
[BUGFIX] Silence DebugWriter for PageIndexerRequest (#3030)
[BUGFIX] AbstractSolrTask::setRootPageId(): Argument #1 must be of type int, string given (#3267)
[BUGFIX:11.2] Update GarbageCollector.php (#3249)
[DOCS] Align with new TYPO3 documentation standards (#3242)
[DOCS] Align README.md with other extensions (#3218)
[BUGFIX] Missing dot in configuration in numberOfResultsPerGroup method (#3098
[TASK] Added info about using page content in fields
[TASK] Added info about the virtual field __solr_contents
[BUGFIX:BP:11.2] Fix write connection (#2916)
Release 11.2.0
We are happy to release EXT:solr 11.2.0.
The focus of this release has been on supporting the latest Apache Solr version 8.11.1 and on optimizing the data update monitoring.
Apache Solr 8.11.1 support
With EXT:solr 11.2.0 we support Apache Solr 8.11.1, the latest release of Apache Solr.
To ensure the Solr index is up-to-date an extensive monitoring is done, in huge sites this may slow down the TYPO3 backend, as many records and
pages have to be checked and updated. With EXT:solr 11.2 you can configure how EXT:solr will monitor and handle data updates, by default EXT:solr
acts as in all former versions, but you can now configure a delayed update handling or even turn the monitoring off.
Small improvements and bugfixes
Beside the major changes we did several small improvements and bugfixes:
Special thanks to ACO Ahlmann SE & Co. KG for sponsoring the improved data update handling, #3153!
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Georg Ringer
Lars Tode
Mario Lubenka
Markus Friedrich
Marc Bastian Heinrichs
Michael Wagner
Rafael Kähm
Also a big thank you to our partners who have already concluded one of our new development participation packages such as Apache Solr EB for TYPO3 11 LTS (Feature), Apache Solr EB for TYPO3 10 LTS (Maintenance)
or Apache Solr EB for TYPO3 9 ELTS (Extended):
ACO Ahlmann SE & Co. KG
avenit AG
b13 GmbH
cyperfection GmbH
in2code GmbH
Leitgab Gernot
medien.de mde GmbH
TOUMORØ
WIND Internet
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
You are on docs for EXT:solr main version.
This file may be outdated if you are on wrong release "branch" version.
To get the most recent changelog, please choose on the top left dropdown menu the version you are looking for.
Release 11.5.5
This is a maintenance release for TYPO3 11.5 and the last release that supports Apache Solr 8.11.
Next EXT:solr release for TYPO3 11.5 will be 11.6.0, it
will contain support for Apache Solr 9 and some breaking improvements.
Note: This change requires the database schema update, due of database schema change from pull-request #3880
11.5.5 contains the following changes:
!!![BUGFIX:11.5] Exception with tx_solr_statistics after latest TYPO3 security update by @dkd-kaehm in #3880
Release 11.5.4
11.5.4 contains the following changes:
[TASK] Fix CI 2023.09.11 on release-11.5.x by @dkd-kaehm in #3777
[BUGFIX:BP:11.5] Fix EXT:solr route enhancer by @dkd-friedrich in #3743
[BUG] Fix detection of "draft records" in workspaces by @baschny in #3642
[BUGFIX] Do not index translations on default language in languages free mode by @dkd-kaehm in #3786
[BUGFIX:BP:11.5] Retry Uri Building after exception by @dkd-friedrich in #3789
[BUGFIX] Delete index documents without available site by @dkd-kaehm in #3778
[TASK:BP:11.5] Ensure recursive page update on page movement by @dkd-friedrich in #3771
[FEATURE:BP:11.5] Add index queue indices by @dkd-friedrich in #3791
[TASK:BP:11.5] Migrate top.fsMod by @dkd-friedrich in #3795
[BUGFIX:BP:11.5] Return value getPageItemChangedTime() must be of the type int by @dkd-kaehm in #3813
[TASK:BP:11.5] Remove duplicate withHeader() by @dkd-kaehm in #3817
[BUGFIX:BP:11.5] Do not list cores twice in Index Inspector by @dkd-kaehm in #3818
[BUGFIX] Fixes multiple sortings by @BastiLu in #3762
[BUGFIX:BP:11.5] Fix missing frontend.typoscript request attribute while indexing by @dkd-kaehm in #3822
[BUGFIX] Prevent indexing error on missing 'foreignLabelField' by @kitzberger in #3740
[BUGFIX:BP:11.5] Force score to float by @dkd-kaehm in #3824
[BUGFIX:BP:11.5] Fix possible notice by @dkd-kaehm in #3825
[DOC:BP:11.5] Add FAQ how to generate URLs to restricted pages by @dkd-kaehm in #3826
[BUGFIX:BP:11.5] Handle float values in options facet parser by @dkd-kaehm in #3827
[BUGFIX:BP:11.5] handle localizations with un-available tsfe more gracefully by @dkd-kaehm in #3832
[TASK] Update the version matrix by @dkd-friedrich in #3860
Release 11.5.3
This is a maintenance release for TYPO3 11.5, containing:
[BUGFIX] make CE search form in backend editable again by @rr-it in #3626
[DOC] Fix wrong type for boostQuery in the docs and example by @dkd-kaehm in #3e7ff72
[TASK] Fix unit tests for 2023.06.07 by @dkd-kaehm in #3695
Release 11.5.2
This is a maintenance release for TYPO3 11.5, containing:
[BUGFIX:BP:11.5] Fix error when indexing pages with field processing instruction categoryUidToHierarchy by @dkd-kaehm in #3462
[BUGFIX:BP:11.5] Custom doktype is deleted from Solr after saving with custom queue configuration by @dkd-friedrich in #3468
[BUGFIX:BP:11.5] Don't use minimum-stability dev on TYPO3 stable in build/CI by @dkd-kaehm in #3464
[BUGFIX:BP:11.5] Fix value resolution in SOLR_RELATION by @dkd-friedrich in #3484
[FEATURE:P:11.5] Add new option manualSortOrderDelimiter for facets by @dkd-kaehm in #3494
[BUGFIX:P:11.5] Casting after check by @dkd-kaehm in #3495
[TASK] Docker version check on docker image build by @dkd-kaehm in #3525
[BUGFIX:P:11.5] Use ConfigurationManager to get typscript in plugin FlexForm by @dkd-kaehm in #3496
[BUGFIX:P:11.5] Exception on search plugin if no Solr connection is configured by @dkd-kaehm in #3498
[BUGFIX:BP:11.5] Fix handling of non existing pages on deletions by @dkd-friedrich in #3520
[TASK:BP:11] Verify the record before accessing the pid by @3l73 in #3537
[TASK:BP:11.5] Handle Solr connection exception by @dkd-friedrich in #3542
[BUGFIX:BP:11.5] Fix Solr response handling during indexing by @dkd-friedrich in #3544
[BUGFIX:BP:11.5] Use plugin namespace as label for flexforms by @dkd-friedrich in #3552
[BUGFIX:BP:11.5] Respect indexingPriority in QueueItemRepository by @dkd-friedrich in #3556
[BUGFIX:BP:11.5] add empty string as fallback by @dkd-friedrich in #3559
[BUGFIX:BP:11.5] use siteLanguage TypoScript object to get language id by @dkd-friedrich in #3554
[BUGFIX:BP:11.5] Sanitize frequent and last searches output by @dkd-friedrich in #3591
[BUGFIX:BP:11.5] changed from always picking the 0 array value to pic… by @dkd-friedrich in #3593
[BUGFIX:BP:11.5] Avoid getSolrConfiguration() on null by @dkd-friedrich in #3599
[TASK:BP:11.5] Disable sql handler by @dkd-friedrich in #3603
[BUGFIX:BP:11.5] Avoid PHP 8 warning when page indexing fails by @dkd-friedrich in #3609
[BUGFIX:BP:11.5] Ensure record exists before asserting if draft by @dkd-friedrich in #3610
[BUGFIX:BP:11.5] Fix usage of null coalescing operator by @dkd-friedrich in #3611
[BUGFIX:BP:11.5] return empty string for renderStatic if there is no content … by @dkd-friedrich in #3612
Release 11.5.1
We are happy to publish EXT:solr 11.5.1 maintenance release
New in this release
[BUGFIX] Do not include removed strptime() by @dkd-kaehm in #3335
[BUGFIX:BP:11.5] Do not handle page updates on new page with uid 0 by @rr-it in #3344
[BUGFIX:BP:11.5] Shortcircuit work in SolrRoutingMiddleware by @christophlehmann in #3341
!!![TASK] Use preAddModifyDocuments hook for pages by @christophlehmann in #3076
[BUGFIX] Fix array key access in ext_getSetup (Backport 11.5) by @saitho in #3361
[TASK:BP:11.5] Indexing configuration icon fallback by @dkd-friedrich in #3371
[BUGFIX:BP:11.5] Do not index missing fields by @dkd-friedrich in #3372
[TASK:BP:11.5] Introduce index queue type setting by @dkd-friedrich in #3370
[TASK:BP:11.5] Do not index language with unconfigured core by @christophlehmann in #3373
[BUGFIX] Make API eID script compatible with TYPO3 v11.5 by @peterkraume in #3350
[BUGFIX] Type-hinting for SiteUtility::getConnectionProperty() by @dkd-kaehm in #3396
[TASK:BP:11.5] Introduce generic EXT:solr exception by @dkd-friedrich in #3422
[BUGFIX:BP:11.5] Fix frontend Solr connection initialization by @dkd-friedrich in #3425
[ACTIONS:2022.12.22] Use fixed typo3/coding-standards 0.6.x < 0.7.0 for TYPO3 11.5 by @dkd-kaehm in #3429
[TASK:Security] Update jQuery and its plugin libs by @dkd-kaehm in #3428
[BUGFIX:P:11.5] Proper check for config.index_enable by @georgringer in #3433
[BUGFIX:P:11.5] Typecast $timestamp to int in TimestampToUtcIsoDate by @derhansen in #3434
[BUGFIX:P:11.5] prevent undefined array key warning if filter is empty by @achimfritz in #3435
[FEATURE] Add signal before search in resultsAction by @stat1x in #3392
[BUGFIX] Fix php warning undefined array key no_search_sub_entries by @DrWh0286 in #3381
We are happy to release EXT:solr 11.5.0.
The focus of this release has been on TYPO3 11 LTS compatibility.
#standwithukraine #nowar
Attention
This version is installable with TYPO3 11 LTS on v11.5.14+ only and contains some breaking changes, see details below.
New in this release
Support of TYPO3 11 LTS
With EXT:solr 11.5 we provide the support of TYPO3 11 LTS.
Please note that we require at least TYPO3 11.5.14, as this version contains some change concerning the usage of local TypoScriptFrontendController objects that are solving some issues during indexing.
Bootstrap 5.1
The default templates provided by EXT:solr were adapted for Bootstrap 5.1.
The templates are also prepared to display some icons with Bootstrap Icons, but the usage is optional and the icons are no longer provided with EXT:solr as the former Glyphicons were.
Custom field processors
fieldProcessingInstructions can be used for processing values during indexing, e.g. timestampToIsoDate or uppercase. Now you can register and use your own field processors via:
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['fieldProcessor']['yourFieldProcessor'] = ACustomFieldProcessor::class;
Custom processors have to implement interface ApacheSolrForTypo3SolrFieldProcessorFieldProcessor.
N-Gram Filter for strings
Provides a new field type and dynamic fields for strings
with enabled Edge-N-Gram filter.
Now the following fields can be used:
- *_stringEdgeNgramS
- *_stringEdgeNgramM
Improve and Fix TSFE Initialization
The initialization of TSFE within indexing and Backends modules contexts is refactored.
In this change the setting and usage of $GLOBALS['TSFE'] is removed and replaced by TYPO3s Core Context API.
The "Context" is always cloned instead of using its singleton instance.
The "Context", "Language", "TSFE" and "ServerRequest", which are required for TypoScript parsing in BE-modules and indexing contexts,
are highly isolated/capsuled and not visible anymore for all things not belonging to EXT:solr internals.
Byside of isolation/encapsulation of TSFE, the language handling is restored
to pure and default EXT:solr "fallback" mode approach.
So all page records, which are involved in indexing(All page types and above all all with sys_template or records-to-index), must be translated.
Otherwise the translation records will be indexed in default language.
Note: Since TYPO3 11 LTS does not allow to instantiate TSFE for sys folders and spacer,
the initialization of TSFE will be done for first and closest page(not spacer or folder) within the site rootline.
Get "free content mode" working
In previous releases of EXT:solr the language handling for "free-content-mode" inconsistent.
The behavior of "free-content-mode" related records varied in RecordMonitor, Initializing and Indexing contexts,
which was the source of troubles for mixing overlay records in wrong cores/languages.
This change brings the RecordMonitor, Initializing and Indexing contexts for "free-content-mode" related records
into the same line, so the "free-content-mode" records are processed the same way.
Make pageRangeFirst and pageRangeLast accessible in fluid
With these two additional getters it is possible to access the variables
in fluid templates. See: #3254
[TASK:BP:11.5] Adapt column arrangement within sites config bd628be99
[FEATURE:BP:11.5] Add custom field processors 173c7a5d4
[TASK:11.5] Fix TYPO3 coding standards issues after upgrade to v0.5.5 55830f209
Ensure keywords string does not exceed database field length 9f2c81768
[BUG] make sure that $currentPageNumber in resultsAction is always >= 1 (#3324) be8cc90b6
[FEATURE] add logging for failed http requests f9edd8bc4
[BUGFIX] fix infinite loop in Tsfe::getPidToUseForTsfeInitialization() 3a2b8d0e8
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Achim Fritz
Andreas Beutel
Andreas Kießling
ayacoo
Christoph Lehmann
Christopher Schnell
Daniel Koether
dev-rke
Dmitry Dulepov
dsone
FearFreddy
Georg Ringer
garfieldius
Guido Schmechel
Henrik Elsner
Jan Delius
Jens Jacobsen
Lars Tode
leslawp
Marc Bastian Heinrichs
Mario Lubenka
Marcus Balasch
Marcus Schwemer
Markus Friedrich
Markus Kobligk
Michael Kettel
Michael Wagner
Michiel Roos
Nicola Widmer
Pascal Hofmair
Peter, CyberForum e.V
Peter Kraume
Philipp Kitzberger
Rafael Kähm
René Maas
Rudy Gnodde
rr-it
Sascha Egerer
Sebastian Hofer
Sebastian Michaelsen
Soren Malling
stat1x
Stefan Frömken
Stefano Kowalke
twojtylak
Thomas Löffler
Tobias Kretschmann
Tobias Schmidt
Torben Hansen
Also a big thank you to our partners who have already concluded one of our new development participation packages such as Apache Solr EB for TYPO3 11 LTS (Feature), Apache Solr EB for TYPO3 10 LTS (Maintenance)
or Apache Solr EB for TYPO3 9 ELTS (Extended):
.hausformat GmbH
ACO Ahlmann SE & Co. KG
AgenturWebfox GmbH
Amedick & Sommer Neue Medien GmbH
avenit AG
b13 GmbH
Bytebetrieb GmbH & Co. KG
Cobytes B.V.
Connetation Web Engineering GmbH
cosmoblonde GmbH
creativ clicks GmbH
cyperfection GmbH
DVT - Daten-Verarbeitung-Tirol GmbH
Earlybird GmbH & Co KG
elancer-team GmbH
eulenblick Kommunikation und Werbung
FONDA GmbH
GFE Media GmbH
graphodata GmbH
Hirsch & Wölfl GmbH
Hochschule Niederrhein
i-fabrik GmbH
in2code GmbH
internezzo ag
Intersim AG
IW Medien GmbH
Jochen Weiland
Kassenärztliche Vereinigung Rheinland-Pfalz
Kreis Euskirchen
Landeskriminalamt Thüringen
L.N. Schaffrath DigitalMedien GmbH
Leitgab Gernot
LOUIS INTERNET GmbH
Marketing Factory Consulting GmbH
medien.de mde GmbH
MEDIA::ESSENZ
mehrwert intermediale kommunikation GmbH
Neue Medien GmbH
NEW.EGO GmbH
novotegra GmbH
Overlap GmbH & Co KG
Pädagogische Hochschule Karlsruhe
peytz.dk
ProPotsdam GmbH
Proud Nerds
Provitex GmbH
queo GmbH
Québec.ca
rms. relationship marketing solutions GmbH
Sandstein Neue Medien GmbH
Schoene neue kinder GmbH
seam media group gmbh
SITE'NGO
Snowflake Productions GmbH
SOS Software Service GmbH
Stämpfli AG
Studio 9 GmbH
systime.dk
techniConcept Sàrl
TOUMORØ
Typoheads GmbH
UEBERBIT GmbH
visol digitale Dienstleistungen GmbH
WACON Internet GmbH
we.byte GmbH
wegewerk GmbH
werkraum Digitalmanufaktur GmbH
WIND Internet
zimmer7 GmbH
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
You are on docs for EXT:solr main version.
This file may be outdated if you are on wrong release "branch" version.
To get the most recent changelog, please choose on the top left dropdown menu the version you are looking for.
Release 11.1.0
We are happy to release EXT:solr 11.1.0.
The focus of this release has been on URL and SEO optimizations.
Attention
This version is installable with TYPO3 10 LTS only and contains some breaking changes, see details below.
New in this release
Route enhancers
Introduce the TYPO3s route enhancer functionality for facets.
This feature allows to mask facets inside the query string or as part of the path segment.
Associative keys for tx_solr[filter] facet URL parameters
Introduce a new style how the facet array represented inside of the url with a combination of key and value in order to be able to keep a specific order.
This feature allows to change the url parameters from index based to associative keys for facets.
Using associative keys, the value of a facet will be replaced with 1. A value of 1 means, that the facet is active. A value of 0 means, the value is inactive.
Additionaly a new sort option for url parameters is available. The sort of parameters is mandatory for associative keys.
By default Solr behaves as before and will be changed in future releases.
To improve the search behaviour we introduce folding filters, e.g. allowing to skip accents in search terms. The following languages are
now using the ASCII folding filter:
dutch
english
finish
french
german
hungarian
irish
italian
polish
portuguese
serbian (for fields that don't include the Serbian Normalization Filter)
spanish
turkish
For the Scandinavian languages, Norwegian, Swedish and Danish, a similiar approach is used, but we're using the more specialized Scandinavian Normalization
and Scandinavian Folding Filters.
Folding process usally takes place at a late stage, so your configurations shouldn't be affected. But for the Scandinavian languages the Scandinavian Normalization
Filter processes the terms earlier, so your protected words for the Snowball Porter Filter, e.g. danish/protwords.txt, might be affected, please be sure to use the
right spelling (see https://solr.apache.org/guide/8_8/language-analysis.html#scandinavian-normalization-filter).
EXT:solrs components like range facets can not be properly handled by cHash stack, because the amount of possible range-combinations is infinite, therefore they must be excluded from cHash calculation.
This change makes it possible to exclude all EXT:solr parameters from cache hash. To prevent misconfigurations, the new extension configuration setting "pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters. This setting makes it impossible to chose invalid/unhandled EXT:solr plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)
Please follow the following migration instructions
Plugin namespaces:
Needed only if other as default (tx_solr) plugin namespace is used in instance. Add the used namespace[s] to
$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] or via backend
"Settings" -> "Extension Configuration" -> "solr" -> "A list of white listed plugin namespaces"
Global q parameter:
Needed only if global "q" parameter without plugin namespace is used and wants to be included in cache hash calculation. Set the setting
$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] = '1'
or enable it via backend "Settings" -> "Extension Configuration" -> "solr" -> "Include/Exclude global q parameter in/from cacheHash"
Makes it possible to manage resources per core instead of previously used per schema approach.
Now is it possible to maintain the stopwords and synonyms for each core/site separately,
and avoid mixing the synonyms between sites using same core.
You are on docs for EXT:solr main version.
This file may be outdated if you are on wrong release "branch" version.
To get the most recent changelog, please choose on the top left dropdown menu the version you are looking for.
Release 11.0.9 ELTS
We are happy to release EXT:solr 11.0.9 maintenance release, this release is a non-public ELTS release.
Note
Non public ELTS release, you can find more details on typo3-solr.com
Support of Apache Solr 9.5.0
Compatibility with Apache Solr 9.5.0 is checked and EXT:solr now accepts the following Apache Solr versions:
9.2.0
9.2.1
9.3.0
9.4.0
9.4.1
9.5.0
To avoid breaking changes EXT:solr 11.0.9 doesn't require Solr 9.5.0, but it's strongly recommended to use the latest supported version.
Note that due to CVE-2023-50290 you should at least check your configuration or update to at least Apache Solr 9.3.0.
Small improvements and bugfixes
[TASK:BP:11.0] Prepend wrong fe language on empty cache by @goldi42
[BUGFIX:BP:11.0] Fix connection initialization by @dkd-friedrich in #10
[BUGFIX] Handle float values in options facet parser by @dkd-kaehm in #12
[BUGFIX:11.0] Exception with tx_solr_statistics after latest TYPO3 security update by @dkd-kaehm in #13
[TASK] Remove unused field cookie in tx_solr_statistics by @dkd-kaehm in #22
[TASK] Allow Apache Solr 9.4 by @dkd-friedrich in #25
[BUGFIX] Fix result highlighting fragment size by @dkd-friedrich in #26
[TASK] Allow Apache Solr 9.5 by @dkd-friedrich in #27
Release 11.0.8 ELTS
We are happy to release EXT:solr 11.0.8 maintenance release, this release is a non-public ELTS release.
Note
Non public ELTS release, you can find more details on typo3-solr.com
Support of Apache Solr 9.2.0
Apache Solr 8.5.1 is outdated, thus we now provide support of Apache Solr 9.2. Technically an update is not required, but recommended to avoid possible
security issues in no longer supported Apache Solr versions. If you can't update your Apache Solr server right now, EXT:solr 11.0.8 should still work with 8.5,
but you'll see warnings in the reports module.
Note: With Apache Solr 9 the following components are no longer available and you have to adapt the configuration if needed. No longer available components are:
1) Data Import Handler (DIH)
DIH is an independent project now; it is no longer a part of Solr
2) VelocityResponseWriter
VelocityResponseWriter is an independent project now; it is no longer a part of Solr. This encompasses all previously included /browse and wt=velocity examples.
Small improvements and bugfixes
[TASK:Security:P:ELTS_9.5] Update jQuery and its plugin libs by @dkd-kaehm in #3
[BUGFIX:BP:11.0] Respect indexingPriority in QueueItemRepository by @dkd-friedrich in #5
[TASK] Integrate in packagist.com : conflict with non ELTS EXTsolr by @dkd-kaehm in #4
[BUGFIX:BP:11.0] Sanitize frequent and last searches output by @dkd-friedrich in #6
[BUGFIX] Fix TypeError in StatisticsWriterProcessor by @dkd-friedrich in #6
[TASK] Remove unneeded GitHub actions by @dkd-friedrich in #6
[TASK] Remove unneeded TYPO3 version from matrix by @dkd-friedrich in #6
[BUGFIX] Try to solve conflicts issue on composer on CI by @dkd-kaehm in #7
[TASK] Prepare configsets for 11.0 ELTS by @dkd-friedrich in #8
[BUGFIX:BP:11.0] Fix expected variant results by @dkd-friedrich in #8
[TASK] Update version matrix by @dkd-friedrich in #8
Release 11.0.7 - Last non ELTS release
TYPO3 9 LTS reached the ELTS stage: free community support for TYPO3 9 LTS ended on 30 sept. 2021.
We'll join the TYPO3s ELTS regiment and provide EXT:solr support for TYPO3 9 ELTS upwardly via our EB program.
Therefore the EXT:solr release-11.0.x will not be maintained in TYPO3-Solr/ext-solr repository any more. The maintenance and builds will be moved to other place.
The new EXT:solr 11.0.8+ for TYPO3 9 ELTS versions will be provided via dkds EB program.
Release 11.0.6
This is a bugfix-only release that contains only bugfixes
This is a bugfix-only release that contains:
[BUGFIX:BP:11-0] Respect TCA setting of 'tstamp' field (#3037)
[DOCS] Use *_PORT variable for setting the port fca6f68 (#2759)
[BUGFIX] Correct Content-Type header for suggest response e843b44 (#2783)
[TASK] Change configuration files to TYPO3 file extensions 6d513e6 (#2813)
[BUGFIX] garbage collector on translations 9adcc40 (#2797)
[BUGFIX] Quote field within score calculation 3969340 (#2824)
[BUGFIX] disabled Solr Sites e7bc3ab (#2795)
[BUGFIX] Enable unicode when fetching pages eb33376 (#2810)
[TASK] Disable cache time information for ajax request f54213f (#2834)
[TASK] Adjust composer TYPO3 version constrains for EXT:Solr 11.0.4+ 50df86a (#2844)
[FEATURE] Allow stdWrap on sorting label 5f2cee2 (#2339)
[BUGFIX] Fix handling of case sensitive variant ids (#2865)
[FEATURE] Store number of existing variants 9c88401 (#2870)
[BUGFIX] Function call with non existing variable 0a69d45 (#2842 / #2520)
[BUGFIX:BACKPORT:11] Fix missing variant field value 8e0c648 (#2878)
[BUGFIX] Exception on Cached state of TranslateViewHelper 1765751 (#2830)
[BUGFIX] Check if $recordUid is non-numeric before substitution a9cf555 (#2836)
[TASK] Remove usages of Prophet by all occurrences within TYPO3 API 3bbf25a, 45b1703, 4f2b37a (#2862)
[TASK] Remove TYPO3 long time ago deprecated cache class 79cafe9 (#2782)
[BUGFIX] Change filter for workspace 5408889 (#2847)
[BUGFIX] Use Iconfactory to retrieve record icons fa77962 (#2900)
[BUGFIX] Language overlay for records is not retrieved since solr Version 11.x (#2788)
[BUGFIX] Temporary free mode fix d5e936b
[BUGFIX] Content id in language aspect c84ce1b
[BUGFIX] Language aspect for indexer 9af09f3
[BUGFIX] remove escaping on suggestion prefix f70829e (#2917)
[FEATURE] Exclude sub entries of page/storage recursively 4151a25 (#2934)
[BUGFIX] Make relevance sorting option markable as active bc813c8 (#2922)
Release 11.0.3
This is a bugfix-only release that contains only a few bugfixes
[TASK] Use minor version of solr docker image (#2740)
[BUGFIX] Make sure HtmlContentExtractor::cleanContent() is UTF-8 safe (#2514)
[BUGFIX] Database exception in RecordMonitor for records no-"enable" columns (#2512)
[BUGFIX] Indexing of records fails with solr 10.x (#2521)
[BUGFIX] Hard codes plugin namespace (#2732)
[BUGFIX] Restricted pages are not being indexed in Typo3 10 (#2695)
[BUGFIX] Prevent duplicate urls for page 0 (#2718)
[BUGFIX] Fix assignment for page uid variable (#2664)
[BUGFIX] Use num_found in static db table (#2668)
[BUGFIX] Build core base path right, when path is slash only (#2692)
[BUGFIX] Fix missing renderType attribute in flexform for search plugin (#2669)
[BUGFIX] Add option to override 'port' in frontend indexing URL (#2618)
[BUGFIX] Reset uriBuilder before building a new uri (#2658)
[DOCS] Multiple improvements to the docs
Release 11.0.2
This is a bugfix-only release that contains only a few bugfixes
[TASK] Add warning in the docs that a fqdn is required for the sitehandling
[BUGFIX] Re-enable Integration Tests for TYPO3 v10
[BUGFIX] Fix unit tests with new controller context check
[BUGFIX] Fix tests and add groups for tests
[BUGFIX] Remove mocks in TYPO3 v10 Integration tests
[BUGFIX] Remove unneeded constant
[BUGFIX] Fix travis.yml to use correct stable versions
[BUGFIX] Ensure to hand in PSR-7 Request to TSFE->getPageAndRootlineWithDomain
[BUGFIX] Remove unneeded is_siteroot flag in nested storage folder
[BUGFIX] Always return array on non-mounted sites
[BUGFIX] Fix multiple rootpages in nested sites
[BUGFIX] Prevent SiteNotFoundException in reports module
Release 11.0.1
This is a bugfix-only release that contains only a few bugfixes
[BUGFIX] Fix documentation and Versionmatrix
[BUGFIX] Fix failing build on docker hub
Release 11.0.0
We are happy to release EXT:solr 11.0.0.
The focus of this release was the support of TYPO3 10 LTS.
Important: This version is installable with TYPO3 9 and 10 LTS. For TYPO3 9 LTS at least version 9.5.16 is required.
EXT:solr 11 requires the usage of the TYPO3 site handling for the configuration of solr.
The `legacyMode` that allows the usage of domain records and configuration of Solr cores in TypoScript was dropped with EXT:solr 11.
New in this release
With EXT:solr 11 we provide the support of TYPO3 10 LTS. If you update to EXT:solr 11, make sure, that you are using the TYPO3 site management to manage your Apache Solr endpoints.
Thanks to: Achim Fritz & b13 for the support on that topic
Support of Apache Solr 8.5.1
With EXT:solr 11 we support Apache Solr 8.5.1, the latest release of Apache Solr.
You are on docs for EXT:solr main version.
This file may be outdated if you are on wrong release "branch" version.
To get the most recent changelog, please choose on the top left dropdown menu the version you are looking for.
Release 10.0.0
We are happy to release EXT:solr 10.0.0.
The focus of EXT:solr 10.0.0 was, to support the site handling that was introduced in TYPO3 9 LTS .
Important: This version is installable with TYPO3 9 LTS only.
Since the development budget is limited at one side and we have other project requests at the other side we could spend less time on the development of EXT:solr.
If you want to support us please consider to sponsor us.
New in this release
TYPO3 9 LTS site handling support
The major new feature of EXT:solr 10 is the support of the TYPO3 site handling. You can now configure the Apache Solr connections along with your TYPO3 site.
To get a typical Solr site running you now need only a few settings and can use the site handling user interface for that.
Apache Solr 8.2 support
EXT:solr 10 ships Apache Solr 8.2.
Note: The data volume of the Apache Solr Docker container was changed from `/opt/solr/server/solr/data` to `/var/solr/data/data` this might require changes in your infrastructure configuration.
Extensionscanner cleanup
Most (not all) of the extension scanner violations have been fixed. We hope to get the extension scanner violations cleaned in the next releases.
Drop TYPO3 8 compatibility
To simplify the development we've dropped the compatibility for TYPO3 8 LTS. If you need to use TYPO3 8 please use the 9.0.x branch.
Add default field for image and price
To allow external applications to index common information for product's we've added a field for price and image. Along with that we've changed the suggest to render the content of the "image" field instead of "previewImage_stringS", this might require changes in your index configuration.
Migration from EXT:solr 9 to EXT:solr 10
Important things on sites with activated Site Handling
By default EXT:solr 10 expects the configuration of Solr connections and cores in the site handling module, along with your TYPO3 site.
The configuration of the Solr connections with your site are available immediatly, so now initialization of connections is required anymore.
The old fashioned setup (domain records and Solr connections in TypoScript) are now the legacy mode. If you want or need to use that still, you can enable the legacy mode,
with your extension configuration by setting `allowLegacySiteMode = 1`.
The following things will become standard, and should be preferred and activated/configured as close as possible and in some cases immediately,
otherwise the things wil not work or break the whole setup:
TypoScript
plugin.tx_solr.solr
This TypoScript configurations for Constants and for Setup are at least partially obsolete and are ignored on Site Handling activated sites.
All Apache Solr connections must be stored in Site Handling "config.yaml" file for each language.
Important: The goal of the configuration with your TYPO3 site and the site module UI was to simplify the setup, especially for new users. Therefore, not all options are still possible as before, but the most of them are still possible by editing the yaml file.
There are the following known and wanted limitations:
It is not possible to configure a username and a password for the solr
server with the UI. You still have the possibility to add that configuration
in the yaml file, in that case make sure that this file is not readable from
the web!. Another approach is to configure the usage from the environment
variables. To configure a username the setting solr_username_read or
solr_password_read can be used in the yaml file.
It is not possible to configure a different Solr hostname with the UI.
If you need that you can still configured that in the yaml file, by using the
fallback mechanism.
The fallback mechanism work like that:
Each setting has the following structure solr_{$setting}_{$scope}"`. The scope can be read or write. Every setting can be overwritten for the scope write, if nothing is configured it will fallback to the read setting. Every setting can be defined on the language level, if it is not configured on the language level it fallsback to the global setting.
The example above shows that you are able to define the setting
solr_host_read on the language level. Since this is a more advanced
configuration and the user interface should be kept simple, this can only be
configured in the yaml.
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Achim Fritz
Georg Ringer
Helmut Hummel
Marc Bastian Heinrichs
Marco Pfeiffer
Markus Kobligk
Netcoop
Nicole Cordes
Rafael Kähm
Rémy DANIEL
Sascha Egerer
Stefan Frömken
Stephan Jorek
Timo Hund
Yann Weyer
Gerald Aistleitner
Also a big thanks to our partners that have joined the EB2019 program:
21TORR GmbH
3m5, Media GmbH
Absolut Research GmbH
AgenturWebfox GmbH
Amedick & Sommer Neue Medien GmbH
arndtteunissen GmbH
Arrabiata Solutions GmbH
artif GmbH & Co. KG
Atol Conseils & Développements
b13 GmbH
bgm business websolutions GmbH & Co KG
Bitmotion GmbH
BIBUS AG Group
Bitmotion GmbH
Columbus Interactive GmbH
Consulting Piezunka und Schamoni - Information Technologies GmbH
cosmoblonde GmbH
CS2 AG
datamints GmbH
Diesel Technic AG
Die Medialen GmbH
Direction des Systèmes d’Information - Département du Morbihan
dörler engineering services
E-Magineurs
Fachhochschule für öffentliche Verwaltung NRW Zentralverwaltung
fixpunkt werbeagentur gmbh
Flowd GmbH
Frequentis Comsoft GmbH
GAYA - La Nouvelle Agence
Gernot Leitgab
Getdesigned GmbH
.hausformat GmbH
Haute école de travail social et de la santé - EESP
Hirsch & Wölfl GmbH
Hochschule Furtwangen
Hypo Tirol Bank AG
Intera Gesellschaft für Software-Entwicklung mbH
interactive tools GmbH - Agentur für digitale Medien
internezzo ag
iresults gmbh
ITK Rheinland
LOUIS INTERNET GmbH
Kassenärztliche Vereinigung Bayerns (KZVB)
KONVERTO AG
kraftwerk Agentur für neue Kommunikation GmbH
Landesinstitut für Schule und Medien Berlin-Brandenburg
Libéo
LINGNER CONSULTING NEW MEDIA GMBH
MaxServ B.V.
McLicense GmbH
MeinEinkauf AG
NEW.EGO GmbH
medien.de mde GmbH
mehrwert intermediale kommunikation GmbH
mellowmessage GmbH
mentronic . Digitale Kommunikation
MOSAIQ GmbH
pietzpluswild GmbH
plan2net GmbH
plan.net - agence conseil en stratégies digitales
Proud Nerds
+Pluswerk AG
punkt.de GmbH
Redkiwi
ressourcenmangel dresden GmbH
rrdata
RKW Rationalisierungs- und Innovationszentrum der Deutschen Wirtschaft e.V.
Site’nGo
SIWA Online GmbH
Stadt Wien - Wiener Wohnen Kundenservice GmbH
Stadtverwaltung Villingen-Schwenningen
Stefan Galinski Internetdienstleistungen
Studio Mitte Digital Media GmbH
TOUMORO
Ueberbit Gmbh
WACON Internet GmbH
webconsulting business services gmbh
webschuppen GmbH
Webstobe GmbH
webit! Gesellschaft für neue Medien mbH
wegewerk GmbH
werkraum Digitalmanufaktur GmbH
XIMA MEDIA GmbH
Special thanks to our premium EB 2019 partners:
jweiland.net
sitegeist media solutions GmbH
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
We are happy to release EXT:solr 9.0.0. The focus of EXT:solr 9.0.0 was, to support the latest version of Apache Solr (7.6.0) and to drop the usage of the solrphpclient and use the solarium php API instead.
Attention
This version is installable with TYPO3 9 LTS, but does not support all features of TYPO3 9 yet. Especially the site handling needs further development in EXT:solr to fully support it with TYPO3 9 LTS. Beside the open tasks in EXT:solr there are also parts left in the TYPO3 core (e.g. when using language fallbacks). In the next release of EXT:solr we want to improve the integration with the site management in TYPO3. Since the development budget is limited at one side and we have other project requests at the other side we could spend less time on the development of EXT:solr by the end of the year. If you want to support us please consider to sponsor us in 2019.
New in this release
Support of Apache Solr 7.6
EXT:solr 9.0.0 ships a ready to use docker container with Apache Solr 7.6. This makes new features of Apache Solr available to EXT:solr.
Replaced solrphpclient with solarium php api
For the communication between EXT:solr we've used the solrphpclient library. This library was not maintained anymore and had several custom modifications. Therefore we made the decision to move to the solarium php api.
This brings us the following advantages:
Use a common, robust, maintained library
Join the forces with other PHP projects to improve solarium and benefit from that
The migration to solarium required several changes in EXT:solr and all add-on's and we will provide compabtility releases for them as well.
With the move to solarium we donated some parts to the solarium API (e.g. the Solr core handling). This allows us to remove some redundant logic in EXT:solr in the future.
Thanks:
Thanks to the whole solarium team (https://github.com/orgs/solariumphp/people) and Markus Kalkbrenner for the work on solarium and the support during the integration into EXT:solr
By now we use the Queries and Httpclient of solarium, but not the domain classes because this requires additional changes in EXT:solr and solarium.
In the future we want to get rid of redundant code and use the API where we can and it makes sence and support solarium with the features that we need for EXT:solr.
TYPO3 9 compatibility
The current release is installable and useable with TYPO3 9 LTS but not all features are supported.
Currently it is supported to:
Use EXT:solr with sites that do have a domain record or domain configuration from EXT:solr
The following parts require additional work and are not supported:
SiteHandling
Extensionscanner proofed
Since a backward compatibility to TYPO3 8.7 LTS make it harder to support the previous mentioned topics. We will drop the support for TYPO3 8 LTS
in the next version and improve the support of those TYPO3 9 LTS features.
Nevertheless a lot of work was allready done for the basic support of TYPO3 9 LTS in the following pull requests:
Support to differ between read and write connections
By now each site had one Solr connection for reading and writing. In most of the cases this good enough when you want to index and search in the same core.
Some setups require a more flexible approach:
E.g. when you want to clean a core and re-index that data your index is not complete for some time on the live site
When you want to do a primary/replica setup for performance reasons(e.g. by using a replica node on the web server) this was not possible by now
With a separation of read and write connections this is now possible. With these building blocks you could e.g.
Index into a shadow core (that is the write core) and swap read and write core when your re-index is done
Install a replica server on your frontend server and index into a dedicated primary node that act's as a Solr primary server
Allow to configure additionalExcludeTags for option facets
When you want to exclude facets from the counts of another facets, Apache Solr uses tags and excludeTags to realize that.
With the setting `additionalExcludeTags` you can add custom exclude tags for a facet and `addFieldAsTag` allows you, to force the creation of a tag for a certain facet.
Thanks to Marc Bastian Heinrichs for creating a patch for that and to in2code for paying for the finalization and documentation.
We ship Apache Solr 7.5.0, you need to install that Version with our configSet.
The argument "hasSearched" was removed from the searchAction and is no longer passed. You can now retrieve this
information by calling "SearchResultSet::getHasSearch" or "{resultSet.hasSearched}" in the FLUID template.
When you access this argument in your FLUID Template, you need to change that as well.
EXT:solr 9 differs between read and write connections now. As fallback the old configuration is still supported and used for reading and writing.
Nevertheless you need to re-initialize the Solr connections that the data in the registry is rewritten. If you want to make use of the new configuration
you can configure the connections like that:
The following code parts have been removed as announced in previous versions of EXT:solr:
SearchResultSetService::getHasSearched Please use SearchResultSet::getHasSearched now
SortingHelper::getSortFields
SortingHelper::getSortOptions
Queue::initialize
Queue::initializeIndexingConfigurations
Search::hasSearched
Search::getResultDocumentsRaw
Search::getResultDocumentsEscaped
Search::getFacetCounts
Search::getFacetFieldOptions
Search::getFacetQueryOptions
Search::getFacetRangeOptions
Search::getSpellcheckingSuggestions
Util::isLocalizedRecord
Outlook
In the next release we will drop the support of TYPO3 8 and focus on the integration into TYPO39. Depending on the funding we would like to support
the integration into the TYPO3 site management and want to allow to configure you Solr site with the TYPO3 site management module.
With the move to the solarium php api, we take the first step of the integration. In the next releases we want to use more parts of the solarium API and also contribute to that API to share the improvements with other PHP projects.
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Aljoscha Weber
Benni Mack
Felix Nagel
Florian Wessels
Helmut Hummel
Jens Jacobsen
Marc Bastian Heinrichs
Patrick Gaumond
Rafael Kähm
Sasche Egerer
Thomas Hohn
Timo Hund
Also a big thanks to our partners that have joined the EB2019 program:
Amedick & Sommer Neue Medien GmbH
BIBUS AG Group
Bitmotion GmbH
CS2 AG
Gernot Leitgab
Getdesigned GmbH
Hirsch & Wölfl GmbH
ITK Rheinland
Kassenärztliche Vereinigung Bayerns (KZVB)
TOUMORO
Ueberbit Gmbh
XIMA MEDIA GmbH
b13 GmbH
bgm business websolutions GmbH & Co KG
datamints GmbH
medien.de mde GmbH
mehrwert intermediale kommunikation GmbH
mellowmessage GmbH
plan2net GmbH
punkt.de GmbH
Special thanks to our premium EB 2019 partners:
jweiland.net
sitegeist media solutions GmbH
In addition i want to thank Markus Kalkbrenner and the whole solarium team for the support.
Thanks to everyone who helped in creating this release!
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
This patch extends the hook PageModuleSummary displaying information about the plugins flexform settings in the page module. Now the plugin name will be displayed and linked to the edit form, similar to the default behavior.
The implementation of solrconsole required several API changes for the implementation of the commands. These changes have been added with the following pull requests:
When terms get removed because they are stopwords this might have an unwanted impact when the mm condition is evaluated. Setting mm.autoRelax to true fixes this.
The type could not be passed before, now you can pass the type to the SearchFormViewHelper as "suggestPageType" argument.
The filters have been passed as global url arguments "filter" as json encoded array, what was hard to debug and inconsistent to the other parameters. We changed the parameter to the name "additionalFilters" and each filter is passed as an array item.
Impact: If you just use the default ViewHelpers and templates you need to change nothing. When you add custom filters to the suggest you now need to pass each filter as an array item of the argument "additionalFilters"
With 8.1.0 we do not officially support TYPO3 9. You can install it on TYPO3 9.3.99 for development and we tried to fix the most important issues during the development but there are already
a few known issues:
The following methods have been marked as deprecated and will be removed in EXT:solr 9.0.0:
SearchResultSetService::getHasSearched() please use SearchResultSet::getHasSearched along with that the global template variable "hasSearched" will be removed with 9.0.0 as well.
Search::getHasSearched() please use SearchResultSet::getHasSearched instead along with that the global template variable "hasSearched" will be removed with 9.0.0 as well.
Util::isLocalizedRecord() please use TCAService::isLocalizedRecord instead
Queue::initialize() please use Queue::initializeBySiteAndIndexConfiguration instead
Queue::initializeIndexingConfigurations() please use Queue::initializeBySiteAndIndexConfiguration instead
SortingHelper::getSortFields() please use the SearchResultSet to get the parsed sorting
SortingHelper::getSortOptions() please use the SearchResultSet to get the parsed sorting
Outlook
In the next release, we want to focus on the move to solarium and the support of the latest Apache Solr version.
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Andreas Lappe
Benni Mack
Florian Schöppe
Frans Saris
Jens Jacobsen
Marc Bastian Heinrichs
Markus Friedrich
Michael Telgkamp
Olivier Dobberkau
Rafael Kähm
Rémy DANIEL
Thomas Löffler
Timo Hund
Thomas Hohn
Also a big thanks to our partners that have joined the EB2018 program:
4eyes GmbH
Albervanderveen
Agentur Frontal AG
AlrweNWR Internet BV
Amedick & Sommer
AUSY SA
Bibus AG
Bitmotion GmbH
bgm Websolutions GmbH
bplusd interactive GmbH
Centre de gestion de la Fonction Publique Territoriale du Nord (Siège)
Citkomm services GmbH
Consulting Piezunka und Schamoni - Information Technologies GmbH
We are happy to release EXT:solr 8.0.0. The focus of EXT:solr 8.0.0 was, to improve the user experience in the frontend and backend.
New in this release
In the following paragraphs we want to summarize the new features that will be shipped with EXT:solr 8.0.0
New suggest
We've replaced the old jQuery UI based autosuggest with a new suggest (https://github.com/devbridge/jQuery-Autocomplete). The advanced suggest can not only show the suggestions, it can also show a configurable amount of top search results.
When the user clicks on the result, he can directly jump to the result page without opening the search results page.
Thanks:
Frans Saris and http://www.beech.it for sharing the codebase of the initial patch!
Apache Solr offers a JSON API for faceting since several versions. Starting with the options facet we've added the support to use this JSON faceting API in EXT:solr.
The support of the JSON API, in general, allows us to build new features on top of that API, that was impossible before. With the first implementation we've added the following features:
By now an option was simply the value and the count, that reflects the number of documents that belong to that option. At EXT:solr 8.0.0 we've added a TypoScript option that is called "metrics", that allows us to collect and show several metrics from documents that belong to a facet option. Examples of metrics are e.g "sum of downloads", "average price",... These metrics will be available in Option model in the FLUID template and can also be used to sort the facet options.
The following example shows an configured options facet with a configured metric:
Thanks to Jens Jacobsen and UEBERBIT for sponsoring Jens work on that feature at our code sprint.
Since we'replaced the whole internal communication from EXT:solr to Apache Solr when options facets are used we are very happy to get your feedback and bug reports when you use the options facets with EXT:solr
When you have option facets with a lot of options, it would be nice to group those options by a prefix. An example is that you group all options by the starting letter to organize them in tabs:
With EXT:solr 8 we ship the following components that allow grouping your facet options to arrange them as you need them in your template:
LabelFilterViewHelper: Can be used to filter options based on a prefix of the label.
LabelPrefixesViewHelper: Can be used to access all available prefixes of the facet options.
TypoScript example template "(Example) Options grouped by prefix" that configures a grouped facet on the author field
In the previous section, the facets get grouped by prefix to organize a large number of options. Another way that you also often see on the web is to allow to filter the options with an additional input box above the facet.
The implementation of that feature is possible just with a partial and a few JavaScript components. To simplify the integration of that feature in a project we ship
Example FLUID partial that uses the filter for options
Example JavaScript that implements the filter functionality
Example TypoScript "Search - (Example) Options filterable by option value" that uses the partials and javascript for a facet
The old templating was created with custom CSS that was shipped with the extension. Since we want to decrease the effort that is required to create a mobile search and many integrators use bootstrap.css we decided to ship bootstrap templates by default. If you want to use another framework or your own custom CSS you are still able to do that with custom templates.
Nevertheless, the mobile search in a TYPO3 introduction installation with bootstrap is much better than before and your effort to adopt it should be reduced.
In EXT:solr 7.x and below a ping request was done before each search. In EXT:solr 8.0.0 we just catch a failed search and handle the unavailability. This saves up to 30% time because we just need one HTTP request to Apache Solr instead of 2.
In the previous versions, we've introduced own backend modules that can also be used by regular TYPO3 users to perform several tasks. With EXT:solr 8.0.0 the index inspector will be moved from the common info module to our info module:
Besides the move, we also added the functionality to ReQueue a single document from the index inspector when you have permissions on the index queue module.
When you index a lot of documents you might want to create facets based on patterns that occur in the content.
The cObject SOLR_CLASSIFICATION allows you to do a lightweight classification based on regex patterns that you configure in the index configuration.
The following example shows how SOLR_CLASSIFICATION can be used to map patterns on classes that are indexed into a Solr field that could be used for faceting:
plugin.tx_solr.index.queue.pages.businessarea_stringM = SOLR_CLASSIFICATION
plugin.tx_solr.index.queue.pages.businessarea_stringM {
field = __solr_content
classes {
automotive {
patterns = car,jeep,SUV
class = automotive
}
pharma {
patterns = pharma,doc,medicine
class = pharma
}
}
}
Copied!
With the configuration above Solr documents get the value "automotive" assigned in the Solr field "businessarea_stringM" when the content contains the term "car", "jeep" or "SUV".
Thanks: Thanks to http://www.bibus.ch who sponsored the implementation of this feature.
With plugin.tx_solr.search.query.(phrase/bigramPhrase/trigramPhrase).fields you can control what is passed to Solr with the ps,ps2 and ps3 value.
With these phrase fields, you can boost documents where phrases occur in close proximity. This can be very handy when you want to tune your search in terms of relevancy.
With plugin.tx_solr.search.query.tieParameter you can now configure the tie value that is passed to Apache Solr.
This value allows you to configure the impact of low scoring fields to the overall score. 0.0 means, that only high score fields will matter, 0.99 means that all fields have the same impact
TYPO3 8 introduced Doctrine DBAL for database queries and the old API will be removed in TYPO3 9. Since we've used a lot of repositories with custom SQL queries, we had to rewrite a lot of queries.
In EXT:solr we've used the chance to restructure the SQL related code and move them to repositories whenever this was possible.
With EXT:solr 8 every usage of the old database API is removed and we are prepared in that way to be ready for TYPO3 9.
Fluent API for Queries with the QueryBuilder
Many parts of the code of EXT:solr deal with queries for Apache Solr that's no surprise :). The corresponding parts in the code especially the Query class had grown over time and reached a huge complexity.
This has several drawbacks:
It is hard to integrate new features (e.g the tiebreaker or bigram features)
TYPO3 specific logic and common Apache Solr logic is mixed and makes it hard to switch to frameworks like e.g. Solarium
The Query class does multiple things: Build the query, initialize the query from the configuration,... This could be split into multiple components.
To get better in that regards our goal is to split the Query into:
Query: Aggregate that is responsible to build the Solr query string based on the options
QueryBuilder: Builder class that is responsible to build an initialized Query object e.g. based on TypoScript configuration and user input.
With the current state the QueryBuilder does the following to build a Query from the user input:
Allow devs to compose own queries that use or ignore several aspects of EXT:solr
Simplify the switch or integration of a generic Solr API that is independent of TYPO3 (e.g. Solarium)
On the way to TYPO3 9
With EXT:solr 8.0.0 we will not officially support TYPO3 9 since it is not an LTS release! Nevertheless, we want to stay close to the TYPO3 core and allow the usage in 9 already.
By now we mainly fix Doctrine and Composer related issues and support the dropped "pageslanguageoverlay" table.
So to sum up... EXT:solr 8.0.0 will mainly support TYPO3 8 LTS and we will support TYPO3 9.x a good as we can without losing the backward compatibility to TYPO3 8 LTS.
In the long run we want to be able to use other PHP frameworks for Apache Solr e.g. solarium(http://www.solarium-project.org/). To make this possible, we
need to split the pure Solr query related logic from the TYPO3Solr specific query logic (e.g. accessFilter,...). To get a step closer into this direction, we've extracted
the logic that is required to build a TYPO3 specific Solr query into the QueryBuilder. The pure Solr related query logic remains in the Query class.
Impact:
Whenever you create or modify queries you should use the QueryBuilder class for that. In one of the next releases we will support to create solarium queries with this QueryBuilder.
Beside the query refactoring, that required to remove and change several methods, the following code has been removed:
Hooks:
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifySearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['processSearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
Deprecated Code
The following methods have been marked as deprecated and will be removed in EXT:solr 9.0.0
ApacheSolrForTypo3SolrSearch::getResultDocumentsRaw - Use the SearchResultsSet::getSearchResults now
ApacheSolrForTypo3SolrSearch::getResultDocumentsEscaped - Use the SearchResultsSet::getSearchResults now
ApacheSolrForTypo3SolrSearch::getFacetCounts - Use the SearchResultSet::getFacets now
ApacheSolrForTypo3SolrSearch::getFacetFieldOptions - Use the SearchResultSet::getFacets now
ApacheSolrForTypo3SolrSearch::getFacetQueryOptions - Use the SearchResultSet::getFacets now
ApacheSolrForTypo3SolrSearch::getFacetRangeOptions - Use the SearchResultSet::getFacets now
ApacheSolrForTypo3SolrSearch::getSpellcheckingSuggestions - Use SearchResultSet::getSpellcheckingSuggestions
ApacheSolrForTypo3SolrQuery is deprecated, use ApacheSolrForTypo3SolrDomainSearchQueryQuery now
ApacheSolrForTypo3SolrSuggestQuery is deprecated, use ApacheSolrForTypo3SolrDomainSearchQuerySuggestQuery now
Outlook
In the next release we want to focus on the move to Solarium and the support of the lastest Apache Solr version.
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors for this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Andreas Lappe
Andri Steiner
Benni Mack
Daniel Diesenreither
Daniel Mann
Daniel Ruf
Georg Ringer
Hannes Lau
Jeffrey Nellissen
Jens Jacobsen
Marco Bresch
Marcus Schwemer
Markus Friedrich
Markus Kobligk
Markus Sommer
Nicole Cordes
Patrick Schriner
Golmann
Rafael Kähm
Sascha Egerer
Simon Schmidt
Thomas Löffler
Timo Hund
Tomas Norre Mikkelsen
Also a big thanks to our partners that have joined the EB2018 program:
Albervanderveen
Amedick & Sommer
AUSY SA
bgm Websolutions GmbH
Citkomm services GmbH
Consulting Piezunka und Schamoni - Information Technologies GmbH
We are happy to release EXT:solr 7.0.0. This release brings several smaller and some bigger changes
New in this release
FLUID Templating
One and a half years ago we started to implement FLUID templating for EXT:solr. This project was initially started as the addon solrfluid. Solrfuid was only available for our partners.
With EXT:solr 7.0.0 the new templating is the default templating in EXT:solr. A lot of code was added and several old stuff was removed. Since some things are conceptional different in FLUID and you also have a lot of possibilities we also dropped some parts, that can be build with fluid itself or do not make sence to do them before rendering the result in the view.
Most of the things just work like before but in the following parts we made conceptional changes (for good reasons):
No css or javascript will be added to the page automatically with the page renderer! Because the integrator wants to have control on that and TYPO3 allows to add this with TypoScript we propose to add these things via TypoScript. EXT:solr offers a lot of example TypoScript templates e.g. to add the default css or to add the javascript for a range facet.
The following TypoScript settings have been removed because they can be implemented with FLUID:
You can add them in your project partials. If you need it just for one facet, please overwrite the render partial with facet.partialName and render the attributes different there
You can add them in your project partials. If you need it just for one facet, please overwrite the render partial with facet.partialName and render the attributes different there
This can be implemented with FLUID logic. Please check the example "Search - (Example) Options with on/off toggle" that implements that (by using the partial Facets/OptionsToggle.html)
Please use custom ViewHelpers or the cObject ViewHelper for that.
Attention
The support of fluid templating would not have been possible without the financial support of all partners! If you want to support us with the implementation of features like this, please think about to join the EB 2017 or 2018. Special thanks also to Frans Saris and beech.it for working on solrfluid together!
In EXT:solr 7.0.0 the backend modules are structured into multiple backend modules. This makes the user experience in the TYPO3 backend more consistent and allows you, to give different permissions on each module.
When you login into the backend, you now have the following modules available:
Info: Gives information of your Solr system, index fields and search usage.
Core Optimization: This module can be used to maintain the synonyms and stopwords in the Apache Solr server.
Index Queue: Gives an overview on indexed records and can be used to requeue records for indexing.
Index Administration: This module can be used for administrator tasks on your Solr system (clear index, index queue or reload a core)
The Query class is one of the biggest classes in EXT:solr and grown over time. This class has now been splitted into several classes.
Along with that a concept of "ParameterBuilder" has been introduced. A ParameterBuilder is responsible to build a parameter part of the query.
E.g. the Grouping ParameterBuilder is responsible to build all parameters of the Solr query for the grouping.
Move FilterEncoder and FacetBuilder to Facet Package
In Solrfluid there was one folder for each facet, that contains the facet class and a parser that parsers the Solr response into the facet object.
The opposite part(parse the url, build the Solr query) was previously done in EXT:solr, with a FilterEncoder that was registered in the FacetRendererFactory.
Now because solrfluid and Solr have been merged, this logic can also be streamlined. Every facet is now structured in a FacetPackage.
A FacetPackage describes:
Which parser should be used to parse the Solr response
Which url decoder should be used to parse the EXT:solr query data
Which query builder should be used to build the faceting query part
You can also implement custom facet types by registering an on FacetPackage with the FacetRegistry.
Migration:
When you have implemented an own FacetParser for solrfluid, you should add a FacetPackage, that references a UrlDecoder and QueryBuilder.
If you have used a custom FacetParser without registring a custom facet type in EXT:solr (ApacheSolrForTypo3SolrFacetFacetRendererFactory::registerFacetType) you can just reference DefaultUrlDecorder and DefaultFacerQueryBuilder in your FacetPackage.
Before solrfluid was merged there were several parts in EXT:solr where the data was read using GeneralUtility::_GET. The drawback of this approach is that the structure of the urls is hard to change and it is not possible to have custom namespaces for each instance of a plugin.
With solrfuid a SearchRequest object was introduced. This object holds all data from the user request. Now this object is used, whenever data from the user action is read. This allows us to make the request namespace changeable. You can now add your custom plugin namespace to a search plugin instance.
As an ongoing task, we started with the migration of database queries to doctrine. Since the database is used in many parts of the extension there are still many parts open.
If you want to work on that, your help is very welcome.
By now EXT:solr had the dependency on an existing domain record. This can be a problem, when you domain is dynamic or
you need to be able to generate it.
Now you can configure a domain by the rootPageId in the TYPO3_CONF_VARS, the domain record is still used, when nothing is configured here.
There might be an approach to support this in TYPO3 Version 9 by the core and we will adopt this then.
During the implementation of this the logic to retrieve the SiteHash and get the SolrConfiguration was moved to the SiteRepository,
this requires an update of the scheduler instances because the scheduler saves a serialized task. Please run the shipped migration to
update scheduler tasks created with 6.1.x.
The following code has been removed since it is not used anymore:
Classes:
ScriptViewHelper
StyleViewHelper
AbstractSolrBackendViewHelper
StringUtility
Methods:
Util::camelize
Util::camelCaseToLowerCaseUnderscored
Util::underscoredToUpperCamelCase
Util::pageExists
Deprecated Code
Methods:
Query::setQueryFieldsFromString use setQueryFields(QueryFields::fromString('foo')) with QueryFields instead, will be removed in 8.0
Query::getQueryFieldsAsString use getQueryFields()->toString() now if needed, will be removed in 8.0
Query::setQueryField use getQueryFields()->set() now, will be removed in 8.0
Query::escape Use EscapeService::escape now, when needed
Query::addReturnField use getReturnFields()->add() now, will be removed in 8.0
Query::removeReturnField use getReturnFields()->remove() now, will be removed in 8.0
Query::getFieldList use getReturnFields()->getValues() now, will be removed in 8.0
Query::setFieldList use setReturnFields() now, will be removed in 8.0
Query::escapeMarkers not needed anymore, use your own implementation when needed
Query::setNumberOfGroups use getGrouping()->setNumberOfGroups() instead, will be removed in 8.0
Query::getNumberOfGroups use getGrouping()->getNumberOfGroups() instead, will be removed in 8.0
Query::addGroupField use getGrouping()->addField() instead, will be removed in 8.0
Query::getGroupFields use getGrouping()->getFields() instead, will be removed in 8.0
Query::addGroupSorting use getGrouping()->addSorting() instead, will be removed in 8.0
Query::getGroupSortings use getGrouping()->getSortings() instead, will be removed in 8.0
Query::addGroupQuery use getGrouping()->addQuery() instead, will be removed in 8.0
Query::getGroupQueries use getGrouping()->getQueries() instead, will be removed in 8.0
Query::setNumberOfResultsPerGroup use getGrouping()->setResultsPerGroup() instead, will be removed in 8.0
Query::getNumberOfResultsPerGroup use getGrouping()->getResultsPerGroup() instead, will be removed in 8.0
Query::setFacetFields use getFaceting()->setFields() instead, will be removed in 8.0
Query::addFacetField use getFaceting()->addField() instead, will be removed in 8.0
Query::removeFilter use getFilters()->removeByFieldName() instead, will be removed in 8.0
Query::removeFilterByKey use getFilters()->removeByName() instead, will be removed in 8.0
Query::removeFilterByValue use getFilters()->removeByValue() instead, will be removed in 8.0
Query::addFilter use getFilters()->add() instead, will be removed in 8.0
Method Arguments:
Query::setGrouping now expects the first argument to be a Grouping object, compatibility for the old argument (bool) will be dropped in 8.0
Query::setHighlighting now expects the first argument to be a Highlighting object, compatibility for the old arguments (bool, int) will be dropped in 8.0
Query::setFaceting now expects the first argument to be a Faceting object, compatibility for the old arguments (bool) will be dropped in 8.0
Hooks:
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifySearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['processSearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
Outlook
In the next release we want to focus on the user experience in the backend and in the frontend. As preparation we collected several tasks.
The goal of some of them (e.g. bootstrap templating, checkbox facets, filterable options partial) is to make more things possible out of the box and make the extension more user friendly:
Close to the release of TYPO3 8 LTS we are happy to announce EXT:solr 6.1 that is compatible with TYPO3 8 LTS.
New in this release
In this release we've merged over 160 pull requests! With these pull requests several new features and bugfixes were added.
Compatible with TYPO3 8 LTS and PHP 7.1 ready
EXT:solr 6.1 is ready for TYPO3 8 LTS and php 7.1 while keeping the compatibility to TYPO3 7.6 LTS and PHP 5.5 and 5.6.
Backend Performance Improvements
During the development of version 6.1 there have been a lot of smaller and bigger performance improvements.
The following changes might be interesting.
Allow to use the closest configuration in the page tree
When a record is saved in the backend by now the whole TypoScript configuration is evaluated for the page where the record is located. In many setups it is enough to just use the closest template in the rootline to parse the configuration.
Since there are cases, where this method does not work (e.g. when you use conditions based on page ids) you need to switch this behaviour explicitly on, by enable "useConfigurationFromClosestTemplate".
To optimize the performance the RecordMonitor and GarbageCollector class have been splitted into several components. These components use the TYPO3 caching framework to cache result that do not need to be retrieved multiple times.
If a record is relevant can only be decided in the context of a Solr site because the TypoScript configuration defines the indexing configuration.
When you want to tweek the backend performance you can define a global list of monitoredTables. Other tables will then be ignored and also the parsing of the TypoScript configuration is not needed then.
Since extjs will be removed more and more from the TYPO3 core we decided to migrate the index inspector to fluid.
You can use the index inspector as usual from "Web > Info > Search Index Inspector" to analyze which documents are in the
solr server for a specific page.
When the property "additionalPageIds" is configured for an indexing configuration this configuration is now taken into account in the record monitor to re-queue these elements when a change in the backend is done.
Since this is an expensive operation, you can disable this feature by configuring the extension setting (useConfigurationTrackRecordsOutsideSiteroot) in the extension manager.
By now the devLog from TYPO3 was used to log messages in the extension. Since several years there is a more flexible logging framework available in the TYPO3 core.
In EXT:solr 6.1 we make use of this and use the TYPO3 logging framework to write the logs.
Support partial matches with N-Gram and Edge N-Gram
For some search use cases it makes sence to support partial matches e.g. when you search for "boch" and want to mach a field with the content "bochum".
EXT:solr 6.1.0 shippes new dynamic field types that make "N-Gram" and "Edge N-Gram" processed fields available:
Allow to disable siteHash check by using allowedSites = *
The siteHash is unique for a Solr site in the TYPO3 system. When Solr does a query the allowedSites setting can be used to control the set of documents that are queried in a Solr core.
This is useful when you want to split the content from multiple sites in a single Solr core. In some cases it is useful to disable this limitation.
E.g. when:
You have data in Solr that comes from another system
When you want to search across multiple sites
Before the extension "solr_disablesitehash" was required to turn this sitehash check off. With EXT:solr 6.1.0 we've changed the meaning of the allowedSites:
Before: * was the same as __all, which means all sites in the system
After: __all is still handled as __all sites in the system, but * now means every site (same as no check at all)
Migration: When you are using * for query.allowedSites change the setting to __all.
The Solr access filter plugin has been optimized to use a Solr post filter. By using a post filter the performance of this plugin is much better, because less documents need to be evaluated.
In this release we ship this new version 2.0.0 of the access filter with the default configSet and in our docker container.
In the next release we will drop the support of TYPO3 7 LTS and only support 8 LTS. Since in the TYPO3 core the database access was migrated to Doctrine, we will use Doctrine as well.
To simplify this, we have centralized a lot of database queries in this release and also dropped duplicate queries to reduce to amount of work that needs to be done for this in the next releaese.
Not every project can use officially signed SSL certificates for all stages. Because of that we allowed for the page indexing to index a host, with a self signed certificate.
You can enable this feature by configuring the extension setting (allowSelfSignedCertificates) in the extension manager to true.
When you use EXT:solr in a deployment scenario (e.g. platform.sh) you maybe want to define the Solr endpoints by environment variables or from variables in TYPO3_CONF_VARS. Both approaches are supported by the TypoScript TEXT object.
Therefore it makes sence for the Solr endpoint settings in plugin.tx_solr.solr to support the usage of cObjects there. This allows you to define connections like this:
For the continues improvement of the codestructure and preparation for upcoming tasks, the following methods have been marked as deprecated and will be removed in EXT:solr 7.0 (next release):
Please check your deprecation log and replace the usages in your code.
Breaking changes
In this release some breaking changes have been required. Please follow the notes below when you migrate from 6.0 to 6.1
Default variantId changed
The default variantId was table/uid before since this id is not unique across multiple TYPO3 system, a system hash was added before.
The scheme of the new variantId is systemhash/table/uid and allows to use grouping when data from multiple TYPO3 systems get indexed and searched.
"Only variables should be assigned by reference" in IndexQueueIndexer::preAddModifyDocuments()
Along we the removal of the reference GeneralUtility::getUserObj was replaced with GeneralUtility::makeInstance, because the usage with ":" is deprecated since TYPO3 8 and will be removed.
Migration:
When you reference custom indexer (in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['IndexQueueIndexer']['indexItemAddDocuments']) you should reference only an autoloadable classname.
The following steps are required:
Move your Indexer to an own extension with proper autoloading configuration
We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 6.0.0.
With this release we ship support for Apache Solr 6.3 and provide a Docker file, that
can be used to spin up a Solr server configured and ready to run within a few steps.
Besides that the release contains many other features and bugfixes.
New in this release
In this release we've merged over 120 pull requests!
With these pull requests several new features and bugfixes were added.
Apache Solr 6.3 Support
The focus of this release was adding support for the latest version of Apache Solr
and making setup easier. Configuration of Apache Solr is now provided via configSets.
A configSet contains everything you need - solrconfig.xml, schema.xml & custom
access filter JAR files - to configure and run a Solr server for use with TYPO3.
Access Filter Support for multi-value Access Fields
The next release of EXT:solrfal will allow merging of duplicates into one document.
The implementation of this feature requires that the permissions are stored in a multi-value
field because different references to a file can have different permissions.
Thus the access field is now a multi-value field and the access filter plugin
is able to evaluate multiple access values.
Support for New Languages (Irish, Serbian and Latvian)
With the new Apache Solr version we have added languages Irish, Serbian, and Latvian.
You can now use them to index sites in these languages by creating a Solr core using the
configuration provided by the extension.
Our goal is to be able to support TYPO3 CMS v8 as soon as version 8 LTS will be released.
Many patches from the community have been merged to improve the compatibility for version 8 already.
Thanks to everyone who worked on that and provided contributions!
Until now the Solr schema was fetched in the frontend even when it was not needed.
Now we only fetch the schema in the backend when it's really needed (when updating stopswords or synonyms).
Reduce ping requests:
The amount of ping requests has been reduced to only do one ping when the plugin is rendered.
Improved configuration caching:
Configuration object, are now cached in an in memory cache. This gives an improvement when pages in the backend are copied.
Until now you could only limit the facet options shown when they get rendered in the frontend.
Sometimes however you want to reduce the available facet options at an earlier time when
retrieving them from Solr.
The following setting now allows you to configure this limit:
To ensure changes don't reduce the quality of the code we have introduced Scrutinizer-CI to
provide feedback on code quality. After adding Scrutinizer-CI we also started to resolve
initial issues reported step by step and to increase the coverage of automated tests.
You can have a look at the results of the inspections and see the impact there:
Our goal is to improve the code step by step and don't get worse.
Use of Compound Index Format
To avoid an error in Apache Solr with "too many open files", it is possible
to use the compound index format. Apache Solr is then writting to only one
index file instead of many. The downside however, is that the performance may
be slightly reduced but should not be significant for most projects.
We decided to change the default configuration here to use a more robust setting.
If you have very high performance requirements you can still change this behavior
by setting <useCompoundFile>false</useCompoundFile> in your solrconfig.xml.
Updating from version 5.1.1 or earlier there are some breaking changes that
your need to keep in mind when updating.
Apache Solr 6 and Access Filter
Since version 6 Apache Solr comes with an embedded Jetty server which completely
changes the setup compared to Apache Solr 4.10. It is now required to install a
new instance of Apache Solr Server. This can be done with our install script for
development or you can use our Docker image to start a new Solr server as a container.
Both approaches are described in our documentation. The embedded Solr Jetty server also uses a new default port (8983) to serve connections.
Migration to Command Controllers for CLI
We are now using a command controller to update the Solr server connections.
Please use the following command to update your connections:
In the next release (6.1) our focus will be to prepare support for TYPO3 CMS
version 8 LTS as well as possible. The next release (6.1) will also be the
last release for TYPO3 CMS version 7.6 LTS.
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors for this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Anjey
Claus Due
Daniel Siepmann
Dominique Kreemers
Georg Ringer
Hendrik Putzek
Ingo Renner
Josef Glatz
Markus Friedrich
Markus Kasten
Michiel Roos
Olivier Dobberkau
Patrick Oberdorf
Peter Kraume
Philipp Gampe
Pierrick Caillon
Sascha Egerer
Thomas Hohn
Timo Hund
Tomas Norre Mikkelsen
Also a big thanks to our partners that have joined the EB2016 program:
Arrabiata Solutions GmbH & Co. KG
avonis
Bank CIC AG
Bitmotion GmbH
Citkomm services GmbH
cron IT
CS2 AG
Cosmoblonde GmbH
Daniz online markting
datenwerk innovationsagentur gmbh
Die Medialen GmbH
die_schnittsteller GmbH
E-magineurs
Fernando Hernáez Lopez
Future Connection AG
Gernot Leitgab
.hausformat
Hirsch & Wölfl GmbH
hs-digital GmbH
IHK Neubrandenburg
internezzo AG
jweiland.net
L.N. Schaffrath DigitalMedien GmbH
mehrwert intermediale kommunikation GmbH
netlogix GmbH & Co. KG
Pixel Ink
Pixelpark AG
pixolith GmbH & Co. KG
polargold GmbH
portrino GmbH
Q3i GmbH & Co. KG
raphael gmbh
RUAG Corporate Services AG
sitegeist media solutions GmbH
ST3 Elkartea
Star Finanz-Software Entwicklung und Vertriebs GmbH
Stefan Galinski Interndienstleistungen
Speedpartner GmbH
sunzinet AG
Systime A/S
SYZYGY Deutschland GmbH
tecsis GmbH
web-vision GmbH
websedit AG - Internetagentur
Webstobe GmbH
werkraum GmbH
WIND Internet
wow! solution
zdreicon AG
Thanks also to our partners who already singed up for a 2017 partnership (EB2017):
Amedick & Sommer Neue Medien GmbH
cron IT GmbH
b:dreizehn GmbH
Die Medialen GmbH
Leibniz Universität IT Services, Hannover
LOUIS INTERNET
polargold GmbH
Mercedes-AMG GmbH
Triplesense Reply GmbH
zdreicom AG
Thanks to everyone who helped in creating this release!
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 5.1.0. This release ships a few bugfixes and smaller features
New in this Release
The focus of this release was to provide the new features from the codesprint and bugfixes from the issue tracker. Beside that, we've added a variants feature that could be used together with the upcomming solrfal release 3.2 as one approach to avoid duplicated files in the search frontend.
Document Variants
This feature adds a new optional field variantId that is filled with type/uid and can be used for collapsing by default.
The following example shows the required TypoScript configuration that is needed:
During the codesprint in Frankfurt a new statistics module was added for that backend that allows you to get some usefull insights
of your search installation.
Which terms are the most frequent search terms?
Which terms are frequently searched but deliver no result?
UX Improvements
During the codesprint several UX improvements in the backend module have been done:
Replace content element wizard icon with svg icon:
Bugfix: absRefPrefix auto in cli mode with webroot different from PATH_site
When you are using absRefPrefix auto and index in cli mode and you webroot differs from PATH_site you have the possibility to configure a
webroot in the scheduler task to make absRefPrefix=auto work in this case
In the next release (6.0) we will focus on the integration of the latest Apache Solr Version (currently 6.1.0)
Contributors
Like always this release would not have been possible without the help from our
awesome community. These are the contributors for this release.
(patches, comments, bug reports, review, ... in alphabetical order)
Daniel Siepmann
Ingo Renner
Steffen Ritter
Timo Hund
Thomas Hohn
Thomas Scholze
Also a big thanks to our partners that have joined the EB2016 program:
Bank CIC AG
Bitmotion GmbH
Citkomm services GmbH
CS2 AG
Cosmoblonde GmbH
Daniz online markting
datenwerk innovationsagentur gmbh
die_schnittsteller GmbH
E-magineurs
Fernando Hernáez Lopez
Future Connection AG
Gernot Leitgab
Hirsch & Wölfl GmbH
hs-digital GmbH
IHK Neubrandenburg
L.N. Schaffrath DigitalMedien GmbH
mehrwert intermediale kommunikation GmbH
netlogix GmbH & Co. KG
pixolith GmbH & Co. KG
Q3i GmbH & Co. KG
RUAG Corporate Services AG
ST3 Elkartea
Star Finanz-Software Entwicklung und Vertriebs GmbH
Stefan Galinski Interndienstleistungen
Systime A/S
SYZYGY Deutschland GmbH
web-vision GmbH
websedit AG - Internetagentur
Webstobe GmbH
WIND Internet
Thanks to everyone who helped in creating this release!
How to get involved
There are many ways to get involved with Apache Solr for TYPO3:
Submit bug reports, and feature requests on GitHub
Ask or help answering questions in our Slack channel
Provide patches through Pull Request or review and comment on existing Pull Requests
Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3
Releases 5.0
Release 5.0.0
Apache Solr for TYPO3 version 5.0 released
We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 5.0.0. With this release we ship the requirements to use our new addon EXT:solrfluid, that allows you to render your search results with the fluid templating engine.
Beside the adaptions for solrfluid we mainly did bugfixes and cleanup in the code base. If you want to use solrfluid in your projects, you should signup a partnership with dkd (visit typo3-solr.com or call dkd +49 (0)69 - 247 52 18-0 for more information).
New in this Release
Preparing fluid support
Adding a SearchRequest object
Improving the SearchResultSetService
Dropping deprecated code
As announced in Version 4.0 the following methods have been removed:
Util::getTypoScriptObject please use TypoScriptConfiguration::getObjectByPath() instead.
Util::isValidTypoScriptPath please use TypoScriptConfiguration::isValidPath() instead.
Util::getTypoScriptValue please use TypoScriptConfiguration::getValueByPath() instead.
IndexQueueQueue::getTableToIndexByIndexingConfigurationName please use TypoScriptConfiguration::getIndexQueueTableNameOrFallbackToConfigurationName instead.
IndexQueueQueue::getTableIndexingConfigurations please use TypoScriptConfiguration::getEnabledIndexQueueConfigurationNames instead.
PluginPluginBase::search / PluginBase::getSearch / PluginBase::setSearch please use $pi->getSearchResultSetService()->getSearch() instead.
PluginResultsResults::getAdditionalFilters please use $pi->getSearchResultSetService()->getAdditionalFilters() instead.
PluginResultsResults::getNumberOfResultsPerPage use $pi->getSearchResultSetService()->getNumberOfResultsPerPage() instead.
PluginResultsResults::getAdditionalFilters please use $pi->getSearchResultSetService()->getAdditionalFilters() instead.
TypoScriptConfiguration::offsetGet / offsetExists / offsetSet please use TypoScriptConfiguration::getObjectByPath / isValidPath / getValueByPath instead. These functions have only been implemented for backwards compatibility in will be removed in 5.0
Dropping the class maps
Since EXT:solr 5.0 is only running on TYPO3 7.6 LTS and classmap files for backwards compatibility have been dropped. This also gives am small performance improvement for the current version.
Unify xliff files in one file per language
Because we want to use the labels in EXT:solr and EXT:solrfluid, and extbase requires to use one xliff file per language, we take the chance to streamline the language files into one file per language.
Performance improvement for TypoScriptConfiguration
Because TypoScriptConfiguration is used quite frequently, we did some performance improvements by optimizing the access to the underling array structure.
Add additionalWhereClause to SOLR_RELATION
You can now add an additionalWhereClause for SOLR_RELATION items.
Example:
plugin.tx_solr.index.queue {
record = 1
record {
table = tx_extension_domain_model_record
fields {
title = title
category_stringM = SOLR_RELATION
category_stringM {
localField = category
multiValue = 1
additionalWhereClause = pid=2
}
}
}
}
Show devLog entries as debug message or in the TYPO3 console
You can now show the written log messages as output of the debug console in the backend or debug messages in the frontent, when you enable the following setting:
plugin.tx_solr.logging.debugDevlogOutput = 1
Copied!
Attention
$GLOBALS['TYPO3_CONF_VARS']['SYS']['devIPmask'] needs to be configured and match.
RST Documentation
We've ported the TypoScript reference from the forge wiki into an rst format. At the same time we've enhanced the documentation with chapters about:
Getting Started
The Backend
Logging
FAQ section
If you want to contribute and bring the documentation one step further feel free to proof-read or provide additional parts as a pull request.
Deprecations
Typo3Environment: The usage of ApacheSolrForTypo3SolrTypo3Environment is deprecated and should be removed
Outlook
In the next release (6.0) we will focus on our codesprint (11. to 13. august in frankfurt). Beside that we will focus
on issues in the issue tracker and general improvements of the codebase and documentation.
Contributors
Like always this release would not have been possible without the help from our
awesome community. These are the contributors for this release.
(patches, comments, bug reports, review, ... in alphabetical order)
Daniel Siepmann
Frans Saris
Gerald Spreer
Hendrik Nadler
Ingo Renner
Marc Bastian Heinrichs
Markus Friedrich
Olivier Dobberkau
Patrick Oberdorf
Peter Kraume
Sascha Löffler
Timo Hund
Also a big thanks to our partners that have joined the EB2016 program:
Bank CIC AG
CS2 AG
Cosmoblone GmbH
Daniz online markting
datenwerk innovationsagentur gmbh
die_schnittsteller GmbH
E-magineurs
Fernando Hernáez Lopez
Future Connection AG
Hirsch & Wölfl GmbH
hs-digital GmbH
L.N. Schaffrath DigitalMedien GmbH
pixolith GmbH & Co. KG
Q3i GmbH & Co. KG
RUAG Corporate Services AG
ST3 Elkartea
Star Finanz-Software Entwicklung und Vertriebs GmbH
Stefan Galinski Interndienstleistungen
Systime A/S
websedit AG - Internetagentur
Webstobe GmbH
web-vision GmbH
Thanks to everyone who helped in creating this release!
How to get involved
There are many ways to get involved with Apache Solr for TYPO3:
Submit bug reports, and feature requests on GitHub
Ask or help answering questions in our Slack channel
Provide patches through Pull Request or review and comment on existing Pull Requests
Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3
Releases 4.0
Release 4.0.0
Apache Solr for TYPO3 version 4.0 released
We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr)
version 4.0.0. With this release we now support TYPO3 CMS versions 7.6LTS
together with Apache Solr version 4.10.
At the same time we also started making the extension work with TYPO3 8.
New in this release
Support of TYPO3 7.6 LTS and TYPO3 8.0 (Dropping Support of 6.2 LTS)
To stay up-to-date with the TYPO3 core we decided to drop the support of 6.2 LTS
and support 7.6 LTS only. We also made sure that EXT:solr works
with TYPO3 8.0, this will be an ongoing effort until the LTS release of TYPO3 8.
Until then you may experience occasional bugs as we try to keep up with the
core's development.
Improve the Test Coverage for Fluid Integration and Refactoring
During the preparation for the upcoming fluid integration we used the chance to
refactor the frontend part of the extension.
What we did here:
Move the logic from the Result/Search and FrequentSearches plugins to service
classes in order to have the possibility to use them in different places
(e.g. other controllers)
Introduce a SearchResultSet and SearchResultSetService to improve the object
model of search results
TypoScriptConfiguration Object and Streamlining of Configuration usage
TypoScript configuration is used in many places and accessing it with an object
has some advantages:
Merging configuration with Flexforms and other sources is easier
We have a single point of access for configuration
We can move and remove isset()/is_array()/if..else logic to read and evaluate
configuration into this object and make the code that is using it more readable
Since the implementation is a cross cutting concern for the whole extension,
it can be used to improve the coverage of integration tests for all parts
By reading the annotations later it could be used to automatically generate
a TypoScript reference
During the preparation of the current release we implemented this
TypoScriptConfiguration object and replaced all parts in the scope of EXT:solr
Migration:
For backwards compatibility the TypoScriptConfiguration object can be used with
array access operators. If you want to read the configuration you can use the
speaking method name for a configuration path or use the methods
getValueByPath() or getObjectByPath().
PHP 7.0 ready
During the development we made sure that the code of EXT:solr works with PHP 7.0.
General Changes
Beside the major changes that have been mentioned before, there are a few minor
changes that might be interesting:
ExcludeContentByClass
You can now use a new TypoScript setting plugin.tx_solr.index.queue.pages.excludeContentByClass.
Since solrconfig.xml contained a default value for the mm (MinimumMatch) and
q.op parameters and it does not make sense to have both at the same time for
eDismax we removed the q.op parameter from the default configuration to avoid
confusion.
Allow Indexing from Backend Module
To avoid switching from the backend module to scheduler and back, we added a
button to do a simple indexing run with just 10 documents.
Resolving Overlays of SOLR_RELATION objects
The SOLR_RELATION object is now taking an overlay into account when relations
are resolved. This allows you to resolve different relations from a translated
entity for an original entity.
Auto-Correct in Default Template
The goal of the search is to keep the visitor on the site and provide the best
results for them with the smallest possible interaction.
There was a feature in EXT:solr (searchUsingSpellCheckerSuggestion) that could
be used to automatically trigger a search using the spellchecker's suggestions
when a search with the original term did not retrieve any results. To simplify
the integration we added the needed markup to the default templates so that you
now just need to enable the feature and use it with the shipped default template.
Use the shipped default templates or add the following snippet to your template (between
<!-- ###SOLR_SEARCH_NO_RESULTS### begin --> and <!-- ###SOLR_SEARCH_NO_RESULTS### end -->:
<divclass="tx-solr-search-suggestion_results"><!-- ###IF:###NO_RESULTS.HAS_SUGGESTION_RESULTS###|==|1### begin -->
###NO_RESULTS.SUGGESTION_RESULTS###
<!-- ###IF:###NO_RESULTS.HAS_SUGGESTION_RESULTS###|==|1### end --></div>
Copied!
Dynamic Field Type for Spellchecking
For spellchecking we had no dynamic field type before. Since 4.0.0 we now ship
the dynamic field types *_textSpellS and *_textSpellM to allow you to
easily define fields for spell checking.
Usage of Styles and Modals from TYPO3 7 LTS
To keep the styling of the backend up-to-date and use the new JavaScript modals
we updated the backend module to use these new features.
The FastVectorHighlighter will now be used by default if fragmentSize is set
to at least 18 (this is required by the FastVectorHighlighter to work).
Use Caching Framework in Util::getConfigurationFromPageId
Configuration is retrieved and evaluated many times during indexing. Thus it
made sense to use the TYPO3 caching framework to cache these expensive
operations and improve their performance.
Use Xliff as Translation Format
Translation files have been migrated to the xliff format. The EXT:solr team
would be happy if you contribute missing translations.
Resolve affected pages when "extendToSubpages" is set and "hidden" flag is changed
The Record Monitor is now able to evaluate the "hidden" flag on subpages when
the extendToSubpages page property is set or unset.
Add signal after IndexQueueWorkerTask::indexItems
You can now register for a new signal "afterIndexItems".
Bugfixes
Beside the improvements mentioned before many smaller bugs have been fixed in
this release as well.
Installation and Updating
This version of EXT:solr is based on the same Apache Solr Version (4.10) as the
last version so an update for the Solr server itself is not needed.
However, a few modifications to the schema and Solr configuration where made.
Please make sure to use the latest solrconfig.xml and schema versions.
You can always check TYPO3's system status report to see whether your setup
meets the requirements and is up-to-date to be used with EXT:solr.
Deprecations
The following functions and methods have been marked as deprecated and will be
removed in version 5.0
Util::getTypoScriptObject please use TypoScriptConfiguration::getObjectByPath() instead.
Util::isValidTypoScriptPath please use TypoScriptConfiguration::isValidPath() instead.
Util::getTypoScriptValue please use TypoScriptConfiguration::getValueByPath() instead.
IndexQueueQueue::getTableToIndexByIndexingConfigurationName please use TypoScriptConfiguration::getIndexQueueTableNameOrFallbackToConfigurationName instead.
IndexQueueQueue::getTableIndexingConfigurations please use TypoScriptConfiguration::getEnabledIndexQueueConfigurationNames instead.
PluginPluginBase::search / PluginBase::getSearch / PluginBase::setSearch please use $pi->getSearchResultSetService()->getSearch() instead.
PluginResultsResults::getAdditionalFilters please use $pi->getSearchResultSetService()->getAdditionalFilters() instead.
PluginResultsResults::getNumberOfResultsPerPage use $pi->getSearchResultSetService()->getNumberOfResultsPerPage() instead.
PluginResultsResults::getAdditionalFilters please use $pi->getSearchResultSetService()->getAdditionalFilters() instead.
TypoScriptConfiguration::offsetGet / offsetExists / offsetSet please use TypoScriptConfiguration::getObjectByPath / isValidPath / getValueByPath instead. These functions have only been implemented for backwards compatibility in will be removed in 5.0
Outlook
The next release (5.0) will focus on the needed refactorings to allow implementing
fluid template rendering. At the same time we will ship the first version of our
new extension "solrfluid" to our EAP partners.
Contributors
Like always this release would not have been possible without the help from our
awesome community. These are the contributors for this release.
(patches, comments, bug reports, review, ... in alphabetical order)
Dmitry Dulepov
Florian Seirer
Frank Nägler
Frans Saris
Ingo Pfennigstorf
Ingo Renner
Marc Bastian Heinrichs
Markus Friedrich
Markus Kobligk
Mickael Vanclooster
Olivier Dobberkau
Sascha Egerer
Timo Schmidt
Thanks to everyone who helped in creating this release!
How to get involved
There are many ways to get involved with Apache Solr for TYPO3:
Submit bug reports, and feature requests on GitHub
Ask or help answering questions in our Slack channel
Provide patches through Pull Request or review and comment on existing Pull Requests
Go to www.typo3-solr.com or call dkd to sponsor the ongoing development of Apache Solr for TYPO3
Releases 3.1
Release 3.1.0
Apache Solr for TYPO3 version 3.1 released
We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 3.1.0. With this release we now support TYPO3 CMS versions 6.2LTS and 7.6LTS together with Apache Solr version 4.10.
New in this Release
Support of TYPO3 7.6 LTS:
We added support for TYPO3 CMS 7.6LTS while still supporting version 6.2LTS to allow an easy upgrade. Support for TYPO3 version 4.5LTS has been dropped with this release. The next release will likely require at least TYPO3 7.6LTS.
On the code side we now use PSR-2 coding style like in TYPO3 7.6LTS. This way it is easy for contributors and developers to stick with a common coding style. For contributions PSR-2 style is also enforced through automatic checks now.
Overall we resolved over 40 issues and added almost 700 commits.
Apache Solr 4.10
The extension comes with an installer that installs Apache Tomcat 8.0.29 and Apache Solr 4.10.4. Please make sure to update to Apache Solr 4.10.4 and the new Solr schema and configuration provided by the extension to make sure the extension works as intended.
Manage Stopwords from the Backend
With EXT:solr 3.0 we introduced an all new backend module to manage the Index Queue, check index data, and manage synonyms. With this release we're adding a new sub module to allow editing stopwords.
Stopwords are usually used to discard common words when indexing. This can increase relevance of documents.
Introducing PHP Namespaces
Our complete codebase is now namespaced and uses root namespace "ApacheSolrForTypo3Solr" for all classes. We also added a class map to provide backwards compatibility. Nevertheless we suggest updating your extensions to use the namespaced class names if you use them.
Introducing Travis CI
For a long time we already had a small unit test suite. However, it was not kept up-to-date, tests weren't executed automatically and generally in a state that left room for improvement. Starting with this release we will focus on increasing test coverage and keeping them up-to-date.
The unit test suite is now executed automatically for each Pull Request through Travis CI (https://travis-ci.org/TYPO3-Solr/ext-solr). Currently we run tests against TYPO3 version 6.2 LTS, 7.6 LTS, and dev-main with the same PHP versions as TYPO3 CMS, PHP 5.3-5.6.
Additionally Travis CI also checks for PSR-2 compliance and will fail a PR in case the code does not fit that coding standard.
In case a Pull Request is rejected by Travis CI you can follow the link to the build and inspect its output to see what needs to be fixed.
General Changes
HTML escaping after retrieval from Solr
The data ist now escaped right after the retrieval from Solr. In rare cases when you need to store HTML in Solr documents you can use the following configuration to mark these fields as trusted fields and skip the escaping:
Synonym handling (Lowercasing and Evaluation at Index- and Querytime)
Synonyms are now handled at query and index time. The advantage is, that when editing synonyms from the backend module those changes become effective immediately.
Removed page browser dependency
As we were preparing to be compatible with TYPO3 7.6LTS we found that there is no compatible version of EXT:pagebrowse. We used that extension to provide the page browser in results listings. To become compatible with TYPO3 7.6LTS we now have integrated the page browser into EXT:solr itself. So if you are not using EXT:pagebrowse for anything else you can safely remove the extension from your installation.
Installation and Updating
To install EXT:solr version 3.1.0 you will need either TYPO3 6.2LTS or TYPO3 7.6LTS. On the Solr side we require Apache Solr 4.10.4, which requires Java 7.
When updating an existing installation to EXT:solr 3.1.0 please make sure to update to Apache Solr 4.10.4 including the new schema and configuration files. The extension comes with a script that will install a working setup for you automatically. The script can be found in EXT:solr/Resources/Install/.
In rare cases you may have to re-initialize the Solr connections. You can do so from TYPO3's clear cache menu in the top toolbar.
Because of the changes made to the schema you will need to re-index your site.
Make sure to check the system status report for any warnings or errors.
### TypoScript Paths changed
During the migration to namespace we also had to change the TypoScript paths:
If you're building custom TypoScript content objects referencing Solr plugin configurations you should make to update these as well.
Outlook
With this release out the door we will focus on updating the add-on extensions like EXT:solrgrouping, EXT:tika, and EXT:solrfal for file indexing, these should be available soon, too.
The next release within the 3.x series will likely require at least TYPO3 7.6LTS and might as well arrive soon as there are no bigger changes planned yet besides dropping support for TYPO3 6.2LTS.
After that we will start working on the long-awaited move to Fluid as the template engine for version 4.0.
Contributors
Like always this release would not have been possible without the help from our awesome community. These are the contributors for this release.
(patches, comments, bug reports, review, ... in alphabetical order)
Alexander Stehlik
Andreas Allacher
Andreas Fernandez
Andreas Wolf
Daniel Siepmann
Dmitry Dulepov
Frank Nägler
Frans Saris
Gordon Brüggemann
Hans Höchtl
Heiko Hardt
Hendrik Nadler
Ingo Renner
Javn Wagner
Jens Jacobsen
Jigal van Hemert
Joschi Kuphal
Josef Florian Glatz
Kevin von Spiczak
Marc Bastian Heinrichs
Markus Friedrich
Markus Günther
Markus Sommer
Markus Kobligk
Maxime Lafontaine
Michael Knabe
Michiel Roos
Tillmann
Olivier Dobberkau
Peter Kraume
Phuong Doan
Romain Canon
Sascha Affolter
Sascha Egerer
Sascha Nowak
Sebastian Enders
Soren Malling
Stefan Galinski
Stefan Neufeind
Steffen Müller
Steffen Ritter
Thomas Heilmann
Thomas Janke
Timo Schmidt
Witali Rott
Thomas Heilmann
Thanks to everyone who helped in creating this release!
Introducing Timo Schmidt
Last but not least we would like to introduce Timo Schmidt. Timo recently joined dkd to work on Apache Solr for TYPO3 in support and development. If you've been active in our Slack channel or followed our GitHub commits you may have been in contact with Timo already. We're looking forward to working with Timo.
How to get involved
There are many ways to get involved with Apache Solr for TYPO3:
Go to http://www.typo3-solr.com/ or call dkd to sponsor the ongoing development of Apache Solr for TYPO3
Releases 3.0
Release 3.0.0
Apache Solr for TYPO3 version 3.0 released
We're happy to announce the release of Apache Solr for TYPO3 (EXT:solr) version 3.0.0. With this release we now support TYPO3 CMS versions 4.5 to 6.2, and Apache Solr version 4.8.
A couple weeks ago we released Apache Solr for TYPO3 version 3.0 to the TYPO3 Extension Repository. Even without this announcement we have seen quite a few downloads already and have also been able to help projects update to the current version even during the development phase.
New in this Release
TYPO3 CMS 6.2 LTS
We added support for TYPO3 CMS 6.2 LTS while still supporting version 4.5 LTS to allow an easy upgrade. Which means you can now use system categories to build hierarchical facets, and on the code side we moved a lot of code around to match the file and folder structure of extbase extensions.
All New Search Backend Module
If you are running TYPO3 CMS version 6.0 or newer you will be pleased to find a completely new backend module to get an overview of your Solr installation, the Index Queue, and execute index maintenance tasks.
The new backend module is designed so that it can easily be extended with sub-modules. One such sub-module is the new synonyms module allowing you to manage synonyms for your site keywords. Further modules are planned and we also encourage others to add functionality using the new API.
Apache Solr 4.8
In the background we also updated Apache Solr to version 4.8 which brings new features and allows us to make things even easier for integrators.
New features resulting from the Solr update include highlighting elevated search results (editorial or paid content).
When indexing content before, you had to wait for Solr to commit newly added documents before seeing the changes show up in the search results. This is not necessary anymore as Solr 4.0 added near real time search and automatic soft commits, meaning that documents added to the Solr index now become searchable immediately.
General Changes
Overall we have resolved close to 80 issues adding up to almost 400 changes. Other than the changes listed above we also added support for limiting a search to certain branches of the page tree. Of course the usual bug fixes and performance improvements were added as well, too.
Updating
During the reorganization of the files and folders we also moved the TypoScript configuration files. Because of this the include path for these files will not match anymore. You can simply re-add the TypoScript in the Include Static section of your template record. We also added an upgrade script that does that for you automatically.
Two scheduler tasks have been removed; the commit scheduler task was meant for development purposes only anyway and is not needed anymore thanks to Solr's automatic soft commits. The optimize task has been removed since its name was a bit misleading and in certain situations it could do more harm than any good.
Outlook
The release of version 3.0 lays the foundation for further changes to come in keeping the extension up to date with developments in both TYPO3 CMS and Apache Solr, providing the best search experience available for TYPO3 CMS.
New add-ons like FAL file indexing and updated versions of existing add-on extensions will be provided through the Early Access Program.
There may be one or two more minor release in the 3.x branch to deliver features that did not make it in 3.0. After that we will be moving on to the development of 4.0 which will have a minimum requirement of TYPO3 CMS 6.2 LTS. The main goal for version 4 will be switching to Fluid for templating and adding namespaces.
Contributors
Like always this release would not have been possible without the help from our awesome community. These are the contributors for this release.
(patches, comments, bug reports, ... no particular order)
Sascha Egerer
John Foushee
Tolleiv Nietsch
Dmitry Dulepov
Stefan Galinski
Franz G. Jahn
Hauke Meyer
Jonas Götze
Joschi Kuphal
Kay Strobach
Stefan Sprenger
Marc Bastian Heinrichs
Lienhart Woitok
Kai Vogel
Jochen Rieger
Irene Höppner
Tomita Militaru
Jan Kiesewetter
Lars Peipmann
Markus Kobligk
Michael Knabe
Michiel Roos
Peter Kraume
Gerrit Venema
Michel Tremblay
Lucas Jenss
Steffen Ritter
Hans Höchtl
Stefan Neufeind
Soren Malling
Witali Rott
Bernhard Kraft
Phuong Doan
Stephan Schuler
Tim Werdin
Rik Willems
Rémy Daniel
Andreas Lappe
Sascha Nowak
Alexander Stehlik
Hans Höchtl
Cyrel Wolfangel
Georg Kuehnberger
Jan-Erik Revsbech
Gabe Blair
Bart Gijswijt
Arjen Hoekema
Thanks to everyone who helped in creating this release!
Sitemap
Appendix - Docker Tweaks
Beside of original Apache Solr Docker image, on which is our image based on we provide some tweaks to make our and your work simpler.
Disable unnecessary cores on container start
By defining env TYPO3_SOLR_ENABLED_CORES with a space separated list of languages/cores to enable, only those cores will be initialized on start-up.
This allows to save memory usage of Apache Solr server instance.
Usage:
docker run -e 'TYPO3_SOLR_ENABLED_CORES=english german' -it typo3solr/ext-solr
Copied!
Appendix - Dynamic Fields
Dynamic fields allow you to add custom fields to Solr documents. That said, you never need to modify Solr's schema (which could cause problems or at least unnecessary additional work when updating the Solr extension).
The following sections describe how to use dynamic fields with your Solr for TYPO3 installation.
Usage of dynamic fields
You can use dynamic fields by following a special naming convention for document fields.
E.g. to create a dynamic field that is a string the field name should end with _stringS. So if you want to
create a field for storing a title you would name it title_stringS. We suggest you use lower camel case for the field name followed by an underscore followed by the dynamic field type "extension".
We've predefined the following dynamic fields:
Extension
Type
Multivalue
Comment
*_stringS
String
No
*_stringM
String
Yes
*_stringCollatedS
string_collated
No
*_stringCollatedM
string_collated
Yes
*_binS
binary
No
Stored but not indexed
*_binM
binary
Yes
Stored but not indexed
*_boolS
Boolean
No
*_boolM
Boolean
Yes
*_intS
Integer
No
*_intM
Integer
Yes
*_tIntS
Integer
No
Deprecated use _intS removed in EXT:solr 10
*_tIntM
Integer
Yes
Deprecated use _intM removed in EXT:solr 10
*_longS
Long
No
*_longM
Long
Yes
*_tLongS
Long
No
Deprecated use _longS removed in EXT:solr 10
*_tLongM
Long
Yes
Deprecated use _longM removed in EXT:solr 10
*_floatS
Float
No
*_floatM
Float
Yes
*_tFloatS
Float
No
Deprecated use _floatS removed in EXT:solr 10
*_tFloatM
Float
Yes
Deprecated use _floatM removed in EXT:solr 10
*_doubleS
Double
No
*_doubleM
Double
Yes
*_tDoubleS
Double
No
Deprecated use _doubleS removed in EXT:solr 10
*_tDoubleM
Double
Yes
Deprecated use _doubleS removed in EXT:solr 10
*_tDouble4S
Double
No
Deprecated use _double4S removed in EXT:solr 10
*_tDouble4M
Double
Yes
Deprecated use _double4M removed in EXT:solr 10
*_dateS
Date
No
*_dateM
Date
Yes
*_tDateS
Date
No
Deprecated use _dateS removed in EXT:solr 10
*_tDateM
Date
Yes
Deprecated use _dateM removed in EXT:solr 10
*_random
Random
No
*_textS
Text
No
*_textM
Text
Yes
*_textTS
Text Tight
No
*_textTM
Text Tight
Yes
*_textSortS
Sortable Text
No
*_textSortM
Sortable Text
Yes
*_textWstS
Whitespace tokenized Text
No
*_textWstM
Whitespace tokenized Text
Yes
*_textEdgeNgramS
Edge Ngram (hello => hello, hell..)
No
*_textEdgeNgramM
Edge Ngram (hello => hello, hell..)
Yes
*_textNgramS
Ngram (hello => he,ll,lo,hel,llo)
No
*_textNgramM
Ngram (hello => he,ll,lo,hel,llo)
Yes
*_textPath
textPath
No
*_textExactS
textExact
No
*_textExactM
textExact
Yes
*_textSpellS
textSpell
No
*_textSpellM
textSpell
Yes
*_textSpellExactS
textSpellExact
No
*_textSpellExactM
textSpellExact
Yes
*_phoneticS
Phonetic
No
*_phoneticM
Phonetic
Yes
*_point
point
No
*_location
location
No
*_coordinate
double
*_locationRpt
locationRpt
No
*_currency
currency
No
Appendix - Version Matrix
Supported versions
List of EXT:solr versions and the matching versions of Apache Solr and TYPO3 that are supported:
$↺ - Published funding contribution. Previously as ($), but merged in EXT:solr core
↻ - Open Source and financed by ($)
Ø - not yet available
ᾱ - non stable alpha release
β - non stable beta release
rc - release candidate available
¹ - recommended Apache Solr version, check version matrix in composer.json (composer info:solr-versions) for full list
Important
Non-stable releases are not available in TER, but
via Composer or as a ZIP file attachment on GitHub release page.
Extended Long Term Support (ELTS)
Since January 2022, we have been following the TYPO3 release cycles and actively support the last two TYPO3 versions; in addition, we offer ELTS support for
selected older versions. The following table illustrates the offers and available and upcoming versions:
TYPO3 ELTS
EXT:solr ($)
EXT:tika
EXT:solrfal ($)
EXT:solrconsole ($)
EXT:solrdebugtools ($)
Apache Solr
Configset
11.5
11.6.3+
11.0
11.0
11.0
11.0
9.8.1¹
ext_solr_11_6_0_elts
10.4
11.2.5+
10.0
10.0
10.0
10.0
9.8.1¹
ext_solr_11_2_0_elts
Our Apache Solr for TYPO3 EB-partners newsletter will keep you updated!