Apache Solr for TYPO3 is the search engine you were looking for with special
features such as Faceted Search or Synonym Support and an incredibly fast
response times of results within milliseconds.
When development started, the primary goal was to create a replacement for
Indexed Search. With the initial public release at T3CON09 in Frankfurt, Germany
that goal was reached and even passed by adding features which Indexed Search
does not support.
With EXT:solr it is simple to build a search for a TYPO3 website, that allows you to index any kind of TYPO3 records with TCA configuration and pages.
The results can be rendered with flexible fluid templates, to render the results as you need them.
Feature List
Faceted Search
Spellchecking / Did you mean
Multi Language Support
Search word highlighting
Field Boosting for fine tuning the importance of certain index fields
Frontend User Group Access Restrictions Support
Stop word Support
Synonym Support
Auto complete / Auto suggest
Language Analysis / Support for inflected word forms
Content indexing through a near instant backend Index Queue
Auto correction (search for the first suggestion)
and more...
Development and Partnership
The extension is developed in an open source way and the source code is completely available on github. Releases to the TYPO3 TER are done in regular time frames (Usually every 3 months).
To make the development possible you can join a partner ship with dkd. By joining the partner program you have the following benefits:
You support the further development on EXT:solr
You get access to addon's that are not public available
Indexing of files from FAL (TYPO3 File Abstraction Layer)
You can included support based on your subscription
By the financial invest of our partners it is possible to keep this extension uptodate and integrate great new features.
If you like this extension please think about to become a partner as well!
|
If you are interested to become a partner visit http://www.typo3-solr.com or call dkd +49 (0)69 - 247 52 18-0.
Getting Started
In this chapter we would like to give you a quick introduction into the very first steps with EXT:solr.
After reading this chapter you will know:
How to...
Start a solr server
Install the extension and connect your TYPO3 system to your solr server
Index a few pages and see the search results on your website
First you need to install Solr itself. There are several ways to do so:
Using Hosted-solr.com
If you want to start simple and just create a solr core with a click. You can use hosted-solr.com. For a small fee you get your own solr core in seconds, configured to be used with EXT:solr.
Docker
You can use our official docker image to start and maintain solr server with a small effort.
Our and official Apache Solr image exports a volume /var/solr for persistent data.
This volume will be mounted to persist the index and other resources from Apache Solr server.
Following paths inside the exported volume are relevant for backups.
the managed stop words and synonyms of corresponding core
Tip
To be save for other scenarios(e.g. SVC of modified Solr Schemas and managed resources), simply backup the whole "data/" folder.
Start container with anonymous volume
To run the container with anonymous volume, simply type the following in command line:
docker run --name=typo3s-solr-server -d -p 8983:8983 typo3solr/ext-solr
Copied!
This will create a docker anonymous volume and store the data inside of it.
To find out the path of used anonymous volume, simply type the following in command line:
To check whether Solr is up and running head over to:
http://<ip>:8983/solr/#/core_en/query.
You should see the web interface of Solr to run queries:
Important: The image ships a default cores for all languages. The data of the cores is stored on an exported volume.
When you want to update the container, you can just start a new container using the data volume of the old container.
But at the same time this has the limitation, that you should only use this image with the default cores!
If you want to create custom cores with a different configuration please read the section "Advanced Docker Usage"
Please note: The steps above show how to build the image from the Dockerfile. You can also download and use our compiled images from dockerhub:
Our image has the intension to create running cores out of the box. This implies, that the schema is inside the container.
The intension in our integration was to stay as close as possible to the official Apache Solr docker images. Sometimes it might make
sense that you use the official image directly instead of our image.
An example could be when you want to have the solrconfig, schema and data outside of the container.
The following example shows how you can run our configuration with the official Apache Solr Docker container by mounting the configuration
and data from a volume (When using Docker on macOS make sure you've added the volume folder to "Preferences -> File Sharing").
With the extension we ship and install script that can be used for a development context or as inspiration for own deployments. It creates a solr server with a core for all languages.
This script is located in "Resources/Private/Install" an it installs a configured solr server that is usable with EXT:solr.
By default this script is not executable and you need to add the execute permissions to your user to run it.
The example below shows how to install a solr server to /home/developer
After running the script you are able to open a solr server with over the loopback address. Which means, when you want to access it from outside, you need to create an ssh tunnel.
Other Setup
Beside the install script and Docker there are various possibilities to setup solr. All of these possibilities are not
officially supported, but the simplify the setup i want to mention them shortly here and summarize the needed steps.
Known Installers
All of these installers can be used to setup a plain, reboot save solr server:
Use the installer shipped with Solr itself bin/install_solr_service.sh:
Allows to install Solr on many distributions including init scripts.
Use chef / ansible / whatever dev ops tool:
Allows you to setup a solr server with your DevOps tool.
Deployment of EXT:solr configuration into Apache Solr
Since EXT:solr 6.0.0 the configuration and all JAR files can be found in EXT:solr/Resources/Private/Solr folder.
The goal of this approach is to make the deployment much easier.
All you need to do is, you need to copy the configSet directory into your prepared solr installation and replace the solr.xml file. In the installer we do it like this:
After this, you can decide which cores you want to boot and delete not relevant cores from ${SOLR_INSTALL_DIR}/server/solr/cores/ directory.
If you want to create other cores, you can create them by adding a <core_name>/core.properties file like provided by EXT:solr.
This is the recommended workflow, so you can deploy the Solr server in an Infrastructure as Code (IaC) manner.
Alternatively Create a core with the rest api (not recommended):
After installing the solr server and deploying all schemata, the TYPO3 reports module helps you to verify if your setup fits to the requirements of EXT:solr
You now have a fully working, pre configured Solr running to start with
No you can continue with installing the extension Install EXT:solr.
Install EXT:solr
Install from TER using the TYPO3 Extension Manager
You can simply install stable versions of EXT:solr using the Extension Manager
from the TYPO3 backend.
Go to the Extension Manager, select Get Extensions and search for
"solr".
Install the extension.
The Extension Manager will also install EXT:scheduler if not installed already
for running the indexing tasks
While developing we recommend installing devlog for easier error detection, too.
Install from git
Alternatively, you can also get the latest development version from GitHub:
Underlying settings can be found in the extension configuration, though the default settings are appropriate for most sites you should check and adapt this settings if necessary:
Static TypoScript
The extension already comes with basic TypoScript configuration that will work for small pages out of the box. For now create or
edit an existing TypoScript Template record in your page tree and add the provided static TypoScript:
Search Markers
EXT:solr is indexing everything on a page between <!-- TYPO3SEARCH_begin --> and <!-- TYPO3SEARCH_end --> to ensure this is the case, check the output of you website and add the markers to your template.
If the markers are missing, you should add them to your template. To increase the quality of the search results the markes should only wrap the relevant content of a page and exclude e.g. menus, because they are same on each page.
The markers can be used multiple times, but make sure each "TYPO3SEARCH_begin" is properly closed with "TYPO3SEARCH_end".
EXT:Solr needs a proper domain as base for the site. If the base is set to "/" or a similar relative path, the Indexing will throw an error, so please make sure that "base" is a fully qualified domain like "https://my.project.tld/"
Configure Solr Connections
Important
The Site-Handling GUI provides the common setup scenario for most users only.
There is no possibility in the GUI to setup different things for each language in site configuration except Corename(core path).
Tip
All in sites config.yaml global defined thing for EXT:solr can be configured for each language differently by editing this file manually.
As first step the authority part of URI to Apache Solr machine must be configured.
The Corename(aka path to core) for each available language must be assigned respectively.
Index the first time
After everything is setup, you need to index the contents of TYPO3 to enable searching in Solr.
To do so open the Apache Solr module and navigate to the Index Queue. Select the contents to index
and Queue Selected Content for Indexing.
Now the records are added to the index queue and ready to index. The indexing can be triggered from the backend module, or
can be triggered by the TYPO3 scheduler.
Switch to the Scheduler module. If the module is not available, make sure to enable the extension
first. It comes bundled with TYPO3 CMS but is not enabled by default.
Create a new scheduler task to run the indexing:
After the task was created, run it manually. The page will indicate a reload but won't reload after
the task was run. Therefore you can reload the module to see the progress bar indicating the current
progress of indexing:
The duration depends on things like the number of records to index and the number of languages
configured in your system. Also whether caching is enabled and warmed up.
The extension will now index all records in the queue and send them to Solrs index.
After Solr has some documents inside his index, you can insert the plugin to provide a search with
results from Solr. To do so create a new content record of type Search on a page:
Select Search: Form, Result, Additional Components if not already selected inside the content
element:
Open the page and search for *, you should see all currently indexed records from Solr:
That's it. You now have a working TYPO3 Installation with Solr integration. You are able to queue
items for indexing, index them and provide an interface for visitors to search the indexed records.
You can now adjust the fluid templates to your needs.
Backend
The chapter before gives you a short introduction about how to setup a solr server, the extension and index a few documents into solr.
In this chapter we want to go deeper and learn how to write more complex indexing configurations and see what other possibilities the backend of EXT:solr provides.
You can configure all solr connection properties along with your TYPO3 site in the site module.
The configuration in the UI is limited to the following settings to keep the setup as simple as possible:
Global:
Scheme
Host
Port
Path to cores
Languagelevel:
Solr core name
If you have a more complex setup or need to configure username and password please configure solr by editing the yaml file.
The fallback mechanism allows you to overwrite settings on the language level.
The fallback mechanism work like that:
Each setting has the following structure solr_{$setting}_{$scope}"`. The scope can be read or write. Every setting can be overwritten for the scope write, if nothing is configured it will fallback to the read setting. Every setting can be defined on the language level, if it is not configured on the language level it fallsback to the global setting.
The example above shows that you are able to define the setting
solr_host_read on the language level. Since this is a more advanced
configuration and the user interface should be kept simple, this can only be
configured in the yaml.
Legacy Mode
Warning
The ability to use this mode will be removed in EXT:solr 11.0.
In EXT:solr all the configuration, including options affecting backend functions, are done in TypoScript. The clear cache menu provides an entry to initialize the Solr connections configured in TypoScript.
Configure the Solr connection in TypoScript under plugin.tx_solr.solr, providing host, port, and path.
On your site's root page set the flag Use as Root Page on the Behaviour tab.
Initialize the Solr connections through the clear cache menu
Initialize all solr connections
When initializing the Solr connections the extensions looks for all the pages with the root flag set, generates the TypoScript configuration for that page like in the frontend and reads the Solr connection parameters.
The extension also repeats that process for each language configured on the installation's root (uid = 0). This way you can configure different Solr cores for each language by using regular conditions that change the path of the Solr connection depending on the currently selected language.
Once all the configured Solr connections in the installation are found, they're stored in TYPO3's registry so that they can easily be retrieved without needing to reevaluate the TypoScript configuration every time we connect to Solr.
All that magic happens in class source:Classes/ConnectionManager.php. The connection manager and it's public API actually must be used whenever a Solr connection is needed.
IndexQueue Configuration
As you already learned you can index pages very easy with EXT:solr and setup a search for pages in seconds. Beside pages, there might be other records in your TYPO3 CMS that you want to have available in your search results.
Indexing custom records
As a core feature EXT:solr allows you to write custom typoscript configuration to index records from any extension just with configuration. To see how this is working, we open the content of the TypoScript example "Search - Index Queue Configuration for news" that can be found in "Configuration/TypoScript/Examples/IndexQueueNews":
By reading the example above you might recognize the following facts:
The indexing configuration is done in the TypoScript path 'plugin. tx_solr. index. queue. [configName]' and there can be multiple queue configurations.
The database table is configured in the property 'plugin.tx_solr.index.queue.[configName].table'. This allows you to have multiple index queue configurations for the same database table. This can be helpful when you have multiple queue configurations for news (e.g. if you have a press & corporate news section on your website).
The solr fields are configured in 'plugin.tx_solr.index.queue.[configName].fields'. This allows you to flexibly fill any solr field. The combination of dynamic fields (Appendix - Dynamic Fields) and the queue configuration allows you to write any kind of data into solr without adapting the solr schema.
There are custom TypoScript objects from EXT:solr that are used in the index queue configuration
When the index queue configuration of your custom record is ready, you can check the index queue in the backend module and add the news items to the queue.
Custom records - links and detail page
In the example above typolink is used to build a link to the detail view of the news. This is required, because EXT:solr can not know the business logic of the news extension to build a detail link.
The typoscript constant "plugin.tx_news.settings.detailPid" is used to configure the target pageId of the news single view. This has two important impacts:
The constant (plugin.tx_news.settings.detailPid) need to point to a valid news single page.
The page with the news single view, should be configured with "Include in Search => Disable" because indexing this page with the normal page indexing without a news id will produce an error page.
Include in Search - Disable
Sysfolders outside the siteroot
The page with a domain record act as a siteroot in EXT:solr. It is a good practice not to nest the siteroots and do the configuration on the root page.
Changes on records that are done in the TYPO3 backend are detected and the solr document will be readded to the index queue when something was changed.
By default only records are monitored for a site that are in the tree of the site. If you want to index records and detect changes on records in a different siteroot, the
index queue configuration needs to contains "additionalPageIds" (e.g.: 'plugin.tx_solr.index.queue.<queueName>.additionalPageIds = 45,48').
Since the monitoring of changes in these records is expensive from performance perspective, you need to enable this feature in the extension configuration:
Enable tracking of records outside siteroot
Backend Modules
The backend modules are in the "APACHE SOLR" section available and can be unlocked for BE-users or/and groups. The modules help you to do maintenance tasks and get an overview on the system status:
During the next paragraphs we will go over the modules and explain, what could be done with them.
Info Module
Info Module shows you important infos about TYPO3 CMS and Solr state.
It contains different info tabs described below:
Connections
It lists all the for the site configured connections and their status.
Statistics
The Search Statistics module allows you to see Top Search Phrases with and without results. In addition it possible to see a
complete listing with hits etc. ranked by Top search keywords.
Index Fields
The Index Fields module allows you to see, how many documents you have in which solr core and which fields those documents have.
Core Optimization
Core optimization Module is responsive for managing the behaviour of cores. By modifying of following things, you can also change the ranking and/or the results.
Stop Words
With the stopwords module you can define a list of words that should be excluded from the search.
Common usecases are:
Very often occurring words like "the", "and" ... are excluded are filtered out because they are more or less "noize words".
You can add words that you want to avoid from indexing.
Synonyms
With the synonyms module you can allow to find documents by words that do not occur in the document but have the same meaning:
E.g. smartphone, cellphone, mobile, mobilephone
Note: The word that you want replace with a synonym needs to appear on both sides when you want to find it with the term itself later
Example
smartphone => smartphone, cellphone, mobile, mobilephone will match "smartphone, cellphone, mobile, mobilephone", when smartphone is missing on the right side, you will not find the document for smartphone anymore!
Index Queue
The Index Queue module is the most important module. It allows you to do the following things:
Select item types and add them for indexing to the indexing queue.
See the fill state of the indexing queue.
Check the indexing queue for errors when the indexing of an items failed.
Start an instant indexing run, directly from the module.
Clear the indexing queue and re-queue items.
Index Administration (earlier Index Maintenance)
The Index Administration module allows you, to do the following administrative operations on your solr index:
Reload the solr configuration.
Empty your solr index. This removes all documents from the index of the current selected site.
Clear the indexing queue.
Index Inspector
Beside the own backend module, EXT:solr provides a feature called "Search Index Inspector". This Tool allows you to select a page or sysfolder and check
what data is stored for this entity in the solr index.
The "Search Index Inspector" can be opened with the TYPO3 Info Module ("Web > Info > Search Index Inspector") it shows the stored data in solr from the page or sysfolder that is selected in the pagetree:
The EXT:solr Search Index Inspector
Page properties
EXT:solrs indexing behaviour depends on page properties.
Depending on the state of page properties the pages and or sub pages may be indexed or not.
Behaviour
The Behaviour tab of the page properties
Include in Search (no_search)
By default, every allowed page is indexed. Use this flag to
exclude the current page from being indexed.
Include sub entries in Search (no_search_sub_entries)
Use this flag, to be able to exclude sub pages from index recursively.
By unsetting this flag, the sub pages/entries of desired page will be excluded from index.
Tip
To exclude the page and its sub pages disable no_search and no_search_sub_entries at same time.
Scheduler
When you want to index content from TYPO3 into solr automatically EXT:solr ships scheduler tasks, that can be called at a configured time or directly from the backend.
Index Queue Worker
Changes that are done in the backend by an editor are written into a queue. This queue is processed asynchronously with a scheduler task and each item in the queue is indexed into solr.
The "Index Queue Worker" task has the following custom properties:
Site: Here you select the solr site, that you want to index with this task instance.
Number of documents to Index: Here you can configure how many documents you want to index in one run. Depending on the performance of your system and the expected update time of the search you can choose a realistic number here.
Forced webroot: The scheduler task can be executed in the cli context, because no webserver is used there, TYPO3 is unable to detect your webroot. As assumption we use PATH_site as default here. When you need to configure something else, you can do it with this option. You can use the marker ###PATH_site### and ###PATH_typo3### to define relative pathes here, to be independent from the concrete instance.
The EXT:solr Index Queue Worker - Scheduler Task
Force Re-Indexing of a site
This task allows you to force the re-indexing of a site & indexing configuration at a planned time.
The Force Re-Indexing of a site task has the following custom properties:
Site: Here you select the solr site, that you want to index with this task instance.
Index Queue configurations to re-index: Here you can limit the set of indexing configurations that should be Re-Indexed.
The EXT:solr Force Re-Indexing of a site - Scheduler Task
Optimizing cores of a site
This task allows you to optimize the indexes of given cores for a site at a planned time.
The Optimize index of a site task has the following custom properties:
Site : Here you select the solr site, that you want to index with this task instance.
Cores to optimize index for : Here you can select the cores that should be optimized by the task.
The EXT:solr Optimize index of a site - Scheduler Task
Plugins
EXT:solr provides the following plugin instances that can be configured in the backend:
The results plugin is the most important plugin of the extension. It is responsible to render a search form
and the results.
Flexform Configuration
All configuration can be done with TypoScript and the settings from EXT:solr are used. For some settings it makes sence to overwrite them with the flexform in the plugin settings.
The following settings can be overwritten by instance with the flexform:
"Target Page":
Target page that should be used when a search is submitted. This can be usefull
when you want to show the results on another page.
When nothing is configured the current page will be used.
Overwritten TypoScript Path
plugin.tx_solr.search.targetPage
Type:
Integer
"Initialize search with empty query":
If enabled, the results plugin issues a "get everything" query during initialization. This is useful, if you want to create a page that shows all available facets although no search has been issued by the user yet.
Note: Enabling this option alone will not show results of the get everything query. To also show the results of the query, see option Show results of initial empty query below.
Overwritten TypoScript Path
plugin.tx_solr.search.initializeWithEmptyQuery
Type:
Boolean
"Show results of initial empty query":
Requires "Initialize search with empty query" (above) to be enabled to have any effect. If enabled together with "Initialize search with empty query" the results of the initial "get everything" query are shown. This way, in combination with a filter you can easily list a predefined set of results.
This configuration can be used to configure an initial query string that is triggered when the plugin is rendered.
Overwritten TypoScript Path
plugin.tx_solr.search.initializeWithQuery
Type:
String
"Show results of initial query":
This option is used to configure if the results of an initial query should be shown.
Overwritten TypoScript Path
plugin.tx_solr.search.showResultsOfInitialQuery
Type:
Boolean
"Filters":
This flexform element allows you to define custom filters by selecting a solr field and a value:
Overwritten TypoScript Path
plugin.tx_solr.search.query.filter.
Type:
Array
"Sorting":
When you want to sort initially by a field value and not by relevance this can be configured here.
Overwritten TypoScript Path
plugin.tx_solr.search.query.sortBy
Type:
String
Example:
title desc
"Boost Function":
A boost function can be useful to influence the relevance calculation and boost some documents to appear more at the beginning of the result list.
Technically the parameter will be mapped to the "bf" parameter in the solr query.
Use cases for example could be:
"Give never documents a higher priority":
This could be done with a recip function:
recip(ms(NOW,created),3.16e-11,1,1)
Copied!
"Give documents with a certain field value a higher priority":
The boostQuery is a query that can be used for boosting. Technically it is mapped to the "bq" parameter of the solr query. Compared to boost a function a boost query provides less use cases.
An example could be to boost documents based on a certain field value:
Since EXT:solr 7.0.0 the old templating of EXT:solr was droppend and rendering with fluid was added.
Along with this change some concepts have changed:
Until EXT:solr 7.0.0 EXT:solr css and javascript was loaded by EXT:solr automatically. In most cases you want to use custom css or you have custom javascript and the integrator want to decide which css or javascript to use. Therefore EXT:solr does not load it by default anymore and the integrator can load it with typoscript. EXT:solr provides a lot of example typoscript templates that load the default css or load the javascript that is needed to use a specific feature. Maybe take the time to explore the typoscript templates that are shipped with the extension to see how they are implemented.
The were some old typosript settings that manipulate the data before it was passed to the view. With fluid this can also be done with core viewhelpers or custom viewhelpers. Therefore the following settings have been removed:
plugin.tx_solr.search.results.fieldProcessingInstructions (The fieldProcessingInstructions still exist at index time since there it is still needed)
Beside that there were some template related settings in the typoscript, that can be solved just with fluid:
plugin.tx_solr.search.faceting.facetLinkATagParams or plugin.tx_solr.search.faceting.[facetName].facetLinkATagParams When you need something like this, you can just change the partials or render a facet with a custom partial (partialName = MyPartial) and add the properties there.
plugin.tx_solr.search.faceting.removeFacetLinkText This can be done just be rendering the text in the partial, that you need.
The setting faceting.facets.[facetName].selectingSelectedFacetOptionRemovesFilter has been removed, since it is possible to build this functionality just with Fluid ViewHelpers. The file "EXT:solr/Resources/Private/Templates/Partials/Facets/OptionsToggle.html" shows how f:if together with `option.selected` can be used to have this behaviour.
Fluid Template Structure
First we start with a short overview of the template structure. This is just to get an rought overview. The templates will be explained in
detail in the template where they belong to:
Layouts: Layouts that are used in the search and the faceting.
Partials:
Facets: Partials that are use to render the specific facet types.
Result: Partials that are used during the result rendering (e.g. to render the result document, sorting or perPage selector)
Search: Partials that are used for the search also when no search was executed.
Templates:
Search: All templates that are used to render the actions in the SearchController
ViewHelper: All templates that are use in the widgets (FrequentSearched, LastSearches, ResultPaginate)
All backend related files are in a "Backend" folder. Everything else is frontend related.
Result List
The most important part of a search are the results. The rendering of the results is done in the "Results.html" template
(Located in Templates/Search/Results.html)
The following part of the default template iterates over the results and renders every document with the Document.html partial (Partials/Frontend/Result/Document.html)
This structure allows you to use e.g. the fluid if ViewHelper to render a result with a different partial, based on a field value.
But as you see in the template above, by default the partial "Result/Document" is used.
The "document" partial is getting the document object. In our case this is an instance of "ApacheSolrForTypo3SolrfluidDomainSearchResultSetSearchResult"
the api of this object allows to get the solr field content with "Document->getFieldName()" that can be used as "document.fieldName" in fluid.
Facets
The goal of a good search is, that the user will find what he is looking for as fast as possible.
To support this goal you can give information from the results to the user to "drill down" or "filter" the results
up to a point where he exactly finds what he was looking for. This concept is called "faceting".
Imagine a user in an online shoe shop is searching for the term "shoe", wouldn't it be useful to allow
the user to filter by "gender", "color" and "brand" to find exactly the model where he is looking for?
In the following paragraphs we will get an overview about the different facet types that can be created on a solr field
just by adding a few lines of configuration.
Facet Types
A solr field can contain different type of data, where different facets make sence. The simplest facet is an option "facet".
The "options facet" just contains a list of values and the user can choose one or many of them. A more complex type
could be a "range facet" on a price field. A facet like this needs to allow to filter on a range of a minimum and a maximum value.
The "type" of a facet can be controlled with the "type" property. When nothing is configured there, the facet will be threated
as option facet.
In the following paragraphs we will introduce the available facet types in EXT:solr and show how to configure them.
Option
The simplest and most often used facet type is the options facet. It renders the items that could be filtered as a simple list.
To setup an simple options facet you can use the following TypoScript snipped:
plugin.tx_solr.search {
faceting = 1
faceting {
facets {
contentType {
label = Content Type
field = type
}
}
}
}
Copied!
By using this configuration you create an options facet on the solr field "type" with the name "contentType". This field represents the record type, that was
indexed into solr. Shown in the frontend it will look like this:
When you have an option facet with very much options you might want to group the options by an prefix of an option. This can be used e.g. to group the options alphabetically.
The following example shows how options can be grouped by prefix (from EXT:solr/Configuration/TypoScript/Examples/Facets/OptionsPrefixGrouped/setup.typoscript):
The query group facet renders an option list, compareable to the options facet, but the single options are not created from
plain solr field values. They are created from dynamic queries.
A typical usecase could be, when you want to offer the possiblity to filter on the creation date and want to offer options like "yesterday", "last year" or "more then five years".
With the following example you can configure a query facet:
plugin.tx_solr.search {
faceting = 1
faceting {
facets {
age {
label = Age
field = created
type = queryGroup
queryGroup {
week.query = [NOW/DAY-7DAYS TO *]
old.query = [* TO NOW/DAY-7DAYS]
}
}
}
}
}
Copied!
The example above will generate an options facet with the output "week" (for items from the last week) and "old" (for items older then one week).
The output in the frontend will look like this:
Solr queryGroup facet
An more complex example is shipped with this extension and can be enabled by including the template "Search - (Example) Fluid queryGroup facet on the field created",
this example makes also use of renderingInstructions to render nice labels for the facet.
The example above just shows a simple example tree that is just rendering the uid's of the rootline as a tree:
Hierachy facet
A more complex example, that is rendering the pagetree with titles is shipped in the extension. You can use it by
including the example TypoScript "Search - (Example) Fluid hierarchy facet on the rootline field":
Technically the hierarchical facet for solr is the same as a flat options facet. The support of hierarchies is implemented,
by writing and reading the facet options by a convention:
[depth]-Level1Label/Level2Label/
Copied!
When you follow this convention by writing date into a solr field you can render it as hierarchical facet. As example you can check indexing configuration in EXT:solr (EXT:solr/Configuration/TypoScript/Solr/setup.typoscript)
In this case the "fieldProcessingInstruction" "pageUidToHierarchy" is used to create the rootline for solr in the conventional way.
Date Range
When you want to provide a range filter on a date field in EXT:solr, you can use the type "dateRange".
The default partial generates a markup with all needed values in data attributes. Together with the provided jQuery ui implementation you can
create an out-of-the-box date range facet.
With the following typoscript you create a date range facet:
plugin.tx_solr.search {
faceting = 1
faceting.facets {
creationDateRange {
label = Created Between
field = created
type = dateRange
}
}
}
Copied!
In the extension we ship the TypoScript example "Search - (Example) dateRange facet with jquery ui datepicker on created field" that shows how to
configure a dateRange facet and load all required javascript files.
When you include this template a date range facet will be shown in the frontend that we look like this:
EXT:solr dateRange facet
As described before for the date range facet markup and javascript code is required, looking at the example template "Search - (Example) dateRange facet with jquery ui datepicker on created field"
in "Configuration/TypoScript/Examples/DateRange" you see that for the jQueryUi implementation the following files are included:
Beside dates ranges are also usefull for numeric values. A typical usecase could be a price slider for a products page.
With the user interface you should be able to filter the documents for a certain price range.
In the default partial, we also ship a partial with data attributes here to support any custom implementation.
By default we will use the current implementation from EXT:solr based on jQueryUi.
The following example configures a numericRange facet for the field "pid":
plugin.tx_solr.search {
faceting = 1
faceting.facets {
pidRangeRange {
field = pid
label = Pid Range
type = numericRange
numericRange {
start = 0
end = 100
gap = 1
}
}
}
}
Copied!
The numeric range facet requires beside the template also a javascript library to render the slider. The example typoscript template "Search - (Example) Fluid numericRange facet with jquery ui slider on pid field"
can be used to see the range slider with jQuery ui for the solr field pid by example.
When you configure a facet on the pid field like this, the frontend will output the following facet:
Numeric range facet
Beside the implementation with jQueryUi you are free to implement a range slider with any other javascript framework.
Rendering with fluid
Rendering facets with fluid is very flexible, because you can use existing ViewHelpers and implement your own logic in ViewHelpers
to support your custom rendering logic.
In the default template the main faceting area on the left side, is done in the following file:
Looking at the code above we see to important details that are important for solr.
Facet Grouping (Areas)
The first important part if the facet.area.group ViewHelper. By default all facets in the group main will be rendered.
This value is the default value.
When you now want to render the facet at another place you can change the group with the following TypoScript configuration:
plugin.tx_solr.search {
faceting = 1
faceting.facets {
contentType {
field = type
label = Content Type
groupName = bottom
}
}
}
Copied!
Now the facet belongs to another group and will not be rendered in the "main" area anymore.
Default Partials
Another important fact is that Facet->getPartialName() is used to render the detail partial.
The default implementation of a facet will return the default partial, that is able to render this facet.
If you need another rendering for one facet you can overwrite the used partial within the configuration:
plugin.tx_solr.search {
faceting = 1
faceting.facets {
contentType {
field = type
label = Content Type
partialName = mySpecialFacet
}
}
}
Copied!
Combining all of these concepts together with the flexibility of fluid you are able to render facets in a very
flexible way.
Autosuggest
A user of the search typically want to find the results a fast as possible. To support the user and avoid to much typing
solr can create a drop down list of common suggested search terms right after the search input box.
This feature can be easily configured with the following typoscript setting:
If you want to configure an the autosuggest by example, you can include the typoscript example template "(Example) Suggest/autocomplete with jquery".
When everything is configured the frontend will show you a drop down of suggestions when you are typing in the search field:
Autocomplete with jQuery
Beside search term suggestions the autocomplete an also show you the top search results for the user input.
This feature can be enabled with:
plugin.tx_solr.suggest.showTopResults = 1
Copied!
If you want to change the number of proposed top results you can also configure them:
plugin.tx_solr.suggest.numberOfTopResults = 5
Copied!
Sorting
When no sorting is selected the search will order the results by "relevance". This relevance is calculated by many factors and has the goal to deliver
the best result for the query on the first position. That's what you expect from a search :)
For some usecases you want to change the sorting of the results by a certain field. In an onlineshop a user might want to order the results
by the price to find the cheapest product that is matching his query.
A simple sorting can be configured with the following typoscript snipped:
plugin.tx_solr.search.sorting >
plugin.tx_solr.search {
sorting = 1
sorting {
defaultOrder = asc
options {
relevance {
field = relevance
label = Relevance
}
title {
field = sortTitle
label = Title
}
}
}
}
Copied!
With the configuration above the possibility to sort by title is introduced. At the same time the sort by relevance link can be used to
reset the sorting to sort by the natural solr relevance.
Search with sorting
Templating
The rendering of the sorting is done on "Resources/Private/Partials/Results/Sorting.html" this partial is using the configuration and
the view helpers to generate sorting links with the same behaviour as in ext:solr. For sure you can modifiy this template and use the ViewHelpers
in the way how you want to implement your custom sorting.
Results per Page
EXT:solr allows you to configure how many result per page will be shown and at the same time the user can also change
this value to an allowed value.
The following configuration can be used to configured the results per page:
When you apply the configuration above, the frontend will show 6 search results by default and show the options 12, 18 and 24
to the user to change the amount of visible results
Results per page
Templating
The rendering of the "perPage selector" is done on "Resources/Private/Partials/Frontend/Results/PerPage.html". This partial is build in a way that the behaviour of the perPage selector is the same as in EXT:solr. If you want to do your custom rendering for example with links instead of a for, you can customize the rendering there.
Ajaxified Results
To improve the use experience and the performance it is possible to load most of the sub requests with ajax.
To activate the "ajaxification" you need to include the typoscript template "Search - Ajaxify the searchresults with jQuery"
Include EXT:solr ajaxify template
How it works?
For all links with the css class "solr-ajaxified" the javascript search controller triggers the request against the same search page but with the type "7383" which is just rendering the search request.
The response is replacing everything in the container "div.tx_solr" with the content of the response.
Limitations
Since the example only renders the plugin on a specific page type and not the same plugin instance as on the page, settings from the flexform are not taken into account by now.
Customize
Use custom Fluid Templates
After these steps solrfluid is usable and using the default Templates, Layouts and Partials. If you want to overwrite them, you can change the TypoScript configuration:
The next step is to configure a path segment or to mask the facets inside of the query.
Facets in path segments
This chapter describe how you can use a facet as part of your url path.
Imagine you have an online shop and use Solr to display the products. It would be convenient for the customer if can call an url path like /products/candy.
The first step is to create a site /products and place the Solr plugin as content.
Configure the path segment
In our example we are using facet productType to filter the search result for a specific product type.
The first step is to define a route path. Because we have already a side called /products only need to define a path contain the facet.
The name itself needs no connection to the facet, so you can choose something different.
The solution is to define a part of a query path and attach the facet you want to place inside of the url.
This is done by separating the query path and the facet by a dash.
routeEnhancers:products:# Note: All arguments inside of namespace tx_solr. See -> extensionKey# Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'_arguments:productType:filter-productTyperequirements:productType:'.*'
Copied!
The route enhancer now knows to place values of facet productType as path segment.
This example shows the all configuration done above
routeEnhancers:products:type:SolrFacetMaskAndCombineEnhancerlimitToPages:-42extensionKey:tx_solrroutePath:'/{productType}'# Note: All arguments inside of namespace tx_solr. See -> extensionKey# Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'_arguments:productType:filter-productType# Important: Configure requirement for fields! If you wand to allow empty values, set .*requirements:productType:'.*'solr:multiValueSeparator:';'replaceCharacters:' ':'-'
Copied!
Simplify facets inside of the query
Sometimes you or your customer want to share links to a search result. The link itself could really hard to read, a specially if it contains a lot of filters.
There are two options that you can apply on the query string:
Concat multiple values of facet into a single string
Mask the filter inside of the query
All query related configurations will be placed below node query below node solr:
routeEnhancers:products:solr:query:
Copied!
Concat multiple values into a string
To activate the concatenation of multiple values add the concat entry and set the value to true.
By default the values will concat with a colon. If you need a different separator, add the entry valueSeparator and set a value that fits your need.
By enabling the concat of multiple values, it would change into:
taste=matcha,sour,sweet
Tip
The facet and the "target" can be equal, you may use same names.
Note
By enabling the mask option, the concat option applied automatically!
Note
The mask only apply to facet you configure inside of the map.
Important
Some facet names are excluded from mask until you configure a different parameter name. This are the internal/reserved parameters of TYPO3.
type
id
no_cache
cHash
* MP
Full example
This example shows all configurations together, which were done above in step by step:
routeEnhancers:products:solr:query:# To reduce the amount of parameters you can force Solr to concat the values.# For example you have following filter:# tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha## Concat will:# 1. collect all filters of the same type# 2. will sort all filter values alpha numeric# 3. join the values together## As a result the query will modified into:# tx_solr[filter][0]=taste:matcha,sour,sweet## Note: If you active the mask option, the concat feature turn on automatically#concat:true# valueSeparator: ','# You can tell Solr to mask query facets. This feature require the map below## For example you have following filter:# tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha# Mask will:# 1. implode all values into a single string and sort it -> green,red,yellow# 2. replace tx_solr[filter][]=color: with color=## As a result the query will modified into:# taste=matcha,sour,sweet#mask:true# In order to simplify a filter argument, you have to define a corresponding map value# There is no automatically reduction of filter parameters at the moment available.# The key is the name of your facet, the value what use instead.## Important:# There are some restrictions for the values. The use of TYPO3 core parameters is prohibited.# This contains at the moment following strings: no_cache, cHash, id, MP, typemap:taste:taste
Copied!
Example
routeEnhancers:Products:type:SolrFacetMaskAndCombineEnhancerlimitToPages:-10extensionKey:tx_solrsolr:multiValueSeparator:'+'replaceCharacters:' ':'_'query:# To reduce the amount of parameters you can force Solr to concat the values.# For example you have following filter:# tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha## Concat will:# 1. collect all filters of the same type# 2. will sort all filter values alpha numeric# 3. join the values together## As a result the query will modified into:# tx_solr[filter][0]=taste:matcha,sour,sweet## Note: If you active the mask option, the concat feature turn on automatically#concat:true# valueSeparator: ','# You can tell Solr to mask query facets. This feature require the map below## For example you have following filter:# tx_solr[filter][0]=taste:sweet&tx_solr[filter][1]=taste:sour&tx_solr[filter][2]=taste:matcha# Mask will:# 1. implode all values into a single string and sort it -> green,red,yellow# 2. replace tx_solr[filter][]=color: with color=## As a result the query will modified into:# taste=matcha,sour,sweet#mask:true# In order to simplify a filter argument, you have to define a corresponding map value# There is no automatically reduction of filter parameters at the moment available.# The key is the name of your facet, the value what use instead.## Important:# There are some restrictions for the values. The use of TYPO3 core parameters is prohibited.# This contains at the moment following strings: no_cache, cHash, id, MP, typemap:color:colortaste:tasteproduct:productroutePath:'/{type}'# Note: All arguments inside of namespace tx_solr. See -> extensionKey# Example: Argument 'type' define as 'filter-type' will convert into 'tx_solr/filter-type'_arguments:type:filter-typerequirements:type:'.*'
Copied!
Logging
When you need to debug something in EXT:solr the extension allows you to write several information to the TYPO3 logging framework.
This can be helpful to analyze e.g.:
What is happening during the indexing?
What is going when a search in the frontend is triggered?
The configuration can be found in the folder "Resources/Private/Solr".
This folder contains:
The folder "configsets": This folder a set of configuration files that can be deployed into a solr server, as a template.
It contains the "solrconfig.xml" file, the "schema.xml" files for all languages and the accessfilter libary that belongs to
this version as a jar file. This configSet needs to be in place on a solr server to create cores that are compatible to the EXT:solr
extension.
The folder "cores": This folder ships an example "core.properties" file for all languages that are compatible with EXT:solr.
A "core.properties" file references a "configSet" that should be used. The path to the schema that is bound to a core is configured as "schema" relative to the root folder of the "configSet".
By example a "core.properties" file looks like this:
The solr.xml file: This file configures solr as required for the used Apache Solr version.
The EXT:solr configSet
As mentioned before the configSet is one single package, that contains all to customize a plain Apache Solr Server, to an Apache Solr Server, ready
for EXT:solr & TYPO3.
The configset of the current release is located in "Resources/Private/Solr/configsets/ext_solr_<release>".
Inside the configSet you find the following folders:
conf: This folder contains the solr configuration and all schemata files. There is one directory for each language which ships the schema.xml file for this language.
The schema.xml file contains the language specific adaptions and includes all general schema fields and types with an XInclude statement.
typo3lib: This folder ships the compiled access filter jar file, that should be used with this EXT:solr version.
The solrconfig (conf/solrconfig.xml) is configured to load all jar files from typo3lib.
Setup steps
With the extension we ship an installer for development and a docker images that can be used to install solr.
When you want to install solr on your system in another way the following steps are required.
Install the solr server
Copy the configsets into the configset folder (by default $SOLR_HOME/server/solr/configsets)
Make sure that the solr.xml file ($SOLR_HOME/server/solr/solr.xml) is in place an fits to your solr version
Create an init script that start solr on boottime.
Secure your solr port from outside.
Make sure that solr is running with an own user.
Backup your data folders
Hint: Apache Solr ships an install script in newer version that might cover your requirements for production
($SOLR_HOME/bin/install_solr_service.sh). We don't use it in EXT:solr because there are currently problems when using it with ubuntu xenial (16.04)
Index Maintenance
Solr offers a lot of request handlers to do maintenance tasks.
There are several parameters in the solrconfig.xml that can be used to tune your solr server.
Our solrconfig.xml is designed to ship a reasonable configuration for the most standard use cases.
For use cases with very large indexes or high performance requirements it makes sence to tune those parameters
indexConfig.useCompoundFile
This value is "true" by default in our configuration. By setting this value to true solr only writes one file
for indexes instead of many. This is a little bit slower but more robust to prevent errors with "Too many open files".
A switch to completely turn on / off EXT:solr. Comes in handy with multi site
installations where you want to enable EXT:solr only for certain sites, but
still have the extension's configuration at a single place and include that for
each site. Just set enabled = 0 for each site's root TS template or use
conditions where you do not want EXT:solr.
Important
This also influences the connection manager; connections will be registered /
detected only for enabled = 1
If enabled, the debugQuery query parameter is added to the Solr queries. Solr
will then return additional information explaining the the query, scoring,
timing, and other information.
Defines the format that is used for dates throughout the extension like in
view helpers for example. The format uses the strftime() php function syntax,
please consult the php documentation for available options.
tx_solr.view
All view related settings, these settings might also be relevant for Fluid.
pluginNamespace
Type
String
TS Path
plugin.tx_solr.view.pluginNamespace
Since
7.0
Default
tx_solr
Plugin namespace. Can be used to change the plugin namespace and can be changed by instance in the flexform.
templateFiles
By convention the templates is loaded from EXT:solr/Resources/Private/Templates/Frontend/Search/(ActionName).html.
If you want to define a different entry template, you can do this here to overwrite the conventional default template.
If you want to use FLUID fallbacks you can just configure the template name, otherwise you could also use a full reference EXT:/.../.
The templates that you configure in availableTemplate can be used in the flexform by the editor to select a template for the concrete plugin instance.
By convention the "Results" template from you configured FLUID template path will be used As alternative you can configure a different template name here (e.g. MyResults or a full path to an entry template here).
By convention the "Form" template from you configured FLUID template path will be used . As alternative you can configure a different template name here (e.g. MyForm or a full path to an entry template here).
By convention the "FrequentlySearched" template from you configured FLUID template path will be used . As alternative you can configure a different template name here (e.g. FrequentlySearched or a full path to an entry template here).
tx_solr.solr
Warning
The ability to use the legacy mode will be removed in EXT:solr 11.0. please configure your solr connections together with your TYPO3 site in the site module
This section defines the address of the Solr server. As the communication with
the Solr server happens over HTTP this is just a simple URL. Each of the URL's
components can be defined separately.
Note: Since version 9 of EXT:solr you can optionally configure different endpoints for reading and writing
Allows to set the connection scheme to "https" instead of the default "http".
host
Type
String
TS Path
plugin.tx_solr.solr.host
Default
localhost
cObject supported
yes
Since
1.0
Deprecated
10.0
Sets the host portion of the URL.
port
Type
Integer
TS Path
plugin.tx_solr.solr.port
Default
8983
cObject supported
yes
Since
1.0
Deprecated
10.0
Sets the port portion of the URL.
path
Type
String
TS Path
plugin.tx_solr.solr.path
Default
/
cObject supported
yes
Since
1.0
Deprecated
10.0
Sets the path portion of the URL. Make sure to have the path end with a slash (/).
username
Type
String
TS Path
plugin.tx_solr.solr.username
Since
6.0
cObject supported
yes
Deprecated
10.0
Sets the username required to access the solr server.
password
Type
String
TS Path
plugin.tx_solr.solr.password
Since
6.0
cObject supported
yes
Deprecated
10.0
Sets the password required to access the solr server.
timeout
Type
Float
TS Path
plugin.tx_solr.solr.timeout
Default
0.0
Since
1.0
cObject supported
no
Deprecated
in 10.0 removed in 11.1.
Now the $GLOBALS['TYPO3_CONF_VARS']['HTTP'] can be used to configure a connection timeout and other options.
tx_solr.index
This part contains all configuration that is required to setup your indexing configuration. You can use EXT:solr
to easily index pages or any kind of records of your TYPO3 CMS.
Allows to prevent frontend indexing of pages when a backend editor is logged in and browsing the website.
additionalFields (deprecated)
Type
String, cObject (since 1.1)
TS Path
plugin.tx_solr.index.additionalFields
Since
1.0
Deprecated
2.0
A mapping of Solr field names to additional string values to be indexed with page documents. Use dynamic fields to index additional data, this way you don't have to modify the schema.xml
Example:
plugin.tx_solr.index.additionalFields {
myFirstAdditionalField_stringS = some string
mySecondAdditionalField_stringS = TEXT
mySecondAdditionalField_stringS {
value = some other value that can be constructed using any TypoScript cObject
case = upper
// more processing here as needed
}
}
Copied!
Since version 1.1 you can use cObjects to generate the value for the field. The only thing to observe is that you generate strings. Other values may work, but haven't been tested yet.
Deprecated since 2.0, please use the Index Queue indexing configurations instead as it allows you to define more precisely for which types of documents you want which fields to be indexed.
Assigns processing instructions to Solr fields during indexing (Syntax: Solr index field = processing instruction name). Currently it is not possible to extend / add own processing instructions.
Before documents are sent to the Solr server they are processed by the field processor service. Currently you can make a filed's value all uppercase, convert a UNIX timestamp to an ISO date, or transform a path into a hierarchy for hierarchical facets (2.0 only). Currently you can use only one processing instruction at a time.
The Index Queue is a powerful feature introduced with version 2.0. It allows you to easily index any table in your TYPO3 installation by defining a mapping of SolrFieldName = DatabaseTableFieldNameOrContentObject. The table must be configured / described in TCA, though. To index other, external data sources you might want to check out Solr's Data Import Handler (DIH).
The Index Queue comes preconfigured to index pages (enabled by default) and an example configuration for tt_news (provided as a separate TypoScript template).
Type
Array
TS Path
plugin.tx_solr.index.queue
Since
2.0
Default
pages
Defines a set of table indexing configurations. By convention the name of the indexing configuration also represents the table name. You can name the indexing configuration differently though by explicitly defining the table as a parameter within the indexing configuration. That's useful when indexing records from one table with different configuration - different single view pages / URLs for example.
Example:
// enables indexing of tt_news records
plugin.tx_solr.index.queue.news = 1
plugin.tx_solr.index.queue.news.fields {
abstract = short
author = author
description = short
title = title
// the special SOLR_CONTENT content object cleans HTML and RTE fields
content = SOLR_CONTENT
content {
field = bodytext
}
// the special SOLR_RELATION content object resolves relations
category_stringM = SOLR_RELATION
category_stringM {
localField = category
multiValue = 1
}
// the special SOLR_MULTIVALUE content object allows to index multivalue fields
keywords = SOLR_MULTIVALUE
keywords {
field = keywords
}
// build the URL through typolink, make sure to use returnLast = url
url = TEXT
url {
typolink.parameter = {$plugin.tt_news.singlePid}
typolink.additionalParams = &tx_ttnews[tt_news]={field:uid}
typolink.additionalParams.insertData = 1
typolink.returnLast = url
typolink.useCacheHash = 1
}
sortAuthor_stringS = author
sortTitle_stringS = title
}
Copied!
queue.[indexConfig]
Type
Boolean, Array
TS Path
plugin.tx_solr.index.queue.[indexConfig]
Since
2.0
Default
pages
An indexing configuration defines several parameters about how to index records of a table. By default the name of the indexing configuration is also the name of the table to index.
By setting plugin.tx_solr.index.queue.[indexConfig] = 1 or 0 you can en- / disable an indexing configuration.
Note: you could add L={field:__solr_index_language} in the additionalParams of the typolink to link to the correct language version (this was removed from the example above to simplify the example)
A WHERE clause that is used when initializing the Index Queue, limiting pages that goes into the Queue.
This filter is applied prior to the plugin.tx_solr.index.queue.[indexConfig].additionalWhereClause
filter and hence provides an even stronger filter mechanism - since it can be used to filter away page
ID's that shouldn't be processed at all.
// Filter away pages that are "spacer" and have no_search, hidden and nav_hide set to zero
plugin.tx_solr.index.queue.pages.initialPagesAdditionalWhereClause = doktype <> 199 AND no_search = 0 AND hidden = 0 AND nav_hide = 0
Defines additional pages to take into account when indexing records for example. Especially useful for indexing DAM records or if you have your news outside your site root in a shared folder to use for multiple sites.
Additional page IDs can be provided as comma-separated list.
queue.[indexConfig].table
Type
String
TS Path
plugin.tx_solr.index.queue.[indexConfig].table
Since
2.0
Sometimes you may want to index records from a table with different configurations, f.e., to generate different single view URLs for tt_news records depending on their category or storage page ID. In these cases you can use a distinct name for the configuration and define the table explicitly.
plugin.tx_solr.index.queue.generalNews {
table = tt_news
fields.url = URL for the general news
// more field configurations here ...
}
// extends the general news configuration
plugin.tx_solr.index.queue.pressNews < plugin.tx_solr.index.queue.generalNews
plugin.tx_solr.index.queue.pressNews {
fields.url = overwriting URL for the press announcements
// may overwrite or unset more settings from the general configuration
}
// completely different configuration
plugin.tx_solr.index.queue.productNews {
table = tt_news
fields.url = URL for the product news
}
When initializing the Index Queue through the search backend module the queue tries to determine what records need to be indexed. Usually the default initializer will be enough for this purpose, but this option allows to define a class that will be used to initialize and add records to the Index Queue in special ways.
The extension uses this option for initializing the pages and more specifically to resolve Mount Page trees so they can be indexed too, although only being virtual pages.
queue.[indexConfig].indexer
Type
String, Array
TS Path
plugin.tx_solr.index.queue.[indexConfig].indexer
Since
2.0
When configuring tables to index a default indexer is used that comes with the extensions. The default indexer resolves the Solr field to database table field mapping as configured. However, in some cases you may reach the limits of TypoScript, when this happens you can configure a specialized indexer using this setting.
The indexer class is loaded using TYPO3's auto loading mechanism, so make sure your class is registered properly. The indexer must extend tx_solr_indexqueue_Indexer.
Example, pages use a specialized indexer:
plugin.tx_solr.index.queue.pages {
indexer = tx_solr_indexqueue_PageIndexer
indexer {
// add options for the indexer here
}
}
Copied!
Within the indexer configuration you can also define options for the specialized indexer. These are then available within the indexer class in $this->options.
Example, the TypoScript settings are available in PHP:
TypoScript:
plugin.tx_solr.index.queue.myIndexingConfiguration {
indexer = tx_myextension_indexqueue_MyIndexer
indexer {
someOption = x
someOtherOption = y
}
}
Allows to define the order in which Index Queue items of different kinds are indexed. Items with higher priority are indexed first.
queue.[indexConfig].fields
Type
Array
TS Path
plugin.tx_solr.index.queue.[indexConfig].fields
Since
2.0
Mapping of Solr field names on the left side to database table field names or content objects on the right side. You must at least provide the title, content, and url fields. TYPO3 system fields like uid, pid, crdate, tstamp and so on are added automatically by the indexer depending on the TCA information of a table.
You can also use a virtual field called __solr_content, which holds the content of the current page.
Example:
plugin.tx_solr.index.queue.[indexConfig].fields {
content = bodytext
title = title
url = TEXT
url {
typolink.parameter = {$plugin.tx_extensionkey.singlePid}
typolink.additionalParams = &tx_extenionkey[record]={field:uid}
typolink.additionalParams.insertData = 1
typolink.returnLast = url
}
}
Comma-separated list of fields that hold files. Using this setting allows to tell the file indexer in which fields to look for files to index from records.
Allows to define a list of additional fields from the pages table that will trigger a recursive update.
plugin.tx_solr.index.queue.pages.recursiveUpdateFields = title
Copied!
The example above will trigger a recursive update of pages if the title is changed.
Please note that the following columns should NOT be configured as recursive update fields: "hidden" and "extendToSubpages".
These fields are handled by EXT:solr already internally and thus they will have not effect.
The example above will remove the content of items in the page that have the css class "removeme".
queue.pages.allowedPageTypes
Type
List of Integers
TS Path
plugin.tx_solr.index.queue.pages.allowedPageTypes
Since
3.0
Default
1,4,7
Allows to set the pages types allowed to be indexed.
Even if you have multiple queues for pages, e.g. via different additionalWhereClause's, you have
to set this value to allow further doktype's. Restrict the pages to be indexed by each queue via
additionalWhereClause.
To make life even easier the Index Queue provides some indexing helpers. These helpers are content objects that perform cleanup tasks or content transformations.
SOLR_CONTENT
Since
2.0
Cleans a database field in a way so that it can be used to fill a Solr document's content field. It removes HTML markup, Javascript and invalid utf-8 chracters.
The helper supports stdWrap on its configuration root.
Example:
content = SOLR_CONTENT
content {
field = bodytext
}
Usually the label field to retrieve from the related records is determined automatically using TCA, using this option the desired field can be specified explicitly. To specify the label field for recursive relations, the field names can be separated by a dot, e.g. for a category hierarchy to get the name of the parent category one could use "parent.name" (since version:2.9).
Whether to return related records suitable for a multi value field. If this is disabled the related values will be concatenated using the following singleValueGlue.
When not using multiValue, the related records need to be concatenated using a glue string, by default this is ", " (comma followed by space). Using this option a custom glue can be specified. The custom value must be wrapped by pipe (|) characters to be able to have leading or trailing spaces.
Allows to classify documents based on a configured pattern
Example:
topic_stringM = SOLR_CLASSIFICATION
topic_stringM {
field = __solr_content
classes {
programming {
matchPatterns = php, java, javascript, go
class = programming
}
cms {
matchPatterns = TYPO3, joomla
class = cms
}
database {
matchPatterns = mysql, MariaDB, postgreSQL
class = database
}
}
}
Copied!
The `matchPatterns` can be used to configure pattern that can occure in the content to add that class. In addition `unmatchPatterns` can be configured to define patterns that should not occure in the content.
Patterns are regular expressions. You configure everything that is possible with regular expressions.
Example:s
The pattern `\ssmart[a-z]*\s` will match everything, that starts with a space followed by smart ending with any lowercase letter and ending by space. This would match e.g. smartphone, smarthome and every other word that starts with `smart`.
Note:
The configuration `patterns` is deprecated with 10.0.0 and will be removed in EXT:solr 11. Please use `matchPatterns` and `unmatchPatterns now.
Name of the database field, that should be used to as content to classify. The special field __solr_content can
be used during indexing to classify the content of the page or file or any other record that fills the content field before.
Array of classification configurations. Each configuration needs to have the property "patterns", that is a list of patters that need to match and "class", that is the mapped class that will be indexed then.
Note:
The output field needs to be a multivalue field since an indexed item can have multiple classes.
enableCommits
Type
Boolean
TS Path
plugin.tx_solr.index.enableCommits
Since
6.1
Default
true
This setting controls whether ext-solr will implicitly cause solr commits as part of its operation.
If this settings is set to false, you need to ensure that something else will periodically call
commits. The solr daemons AutoCommit feature would be a natural choice.
This feature is mainly useful, when you have many installations in the same solr core.
Note: Calling some APIs may still cause commits, but these can always be explicitly disabled.
tx_solr.search
The search section, you probably already guessed it, provides configuration options for the all things related to actually searching the index, setting query parameters, formatting and processing result documents and the result listing.
targetPage
Type
Integer
TS Path
plugin.tx_solr.search.targetPage
Default
Since
1.0
Sets the target page ID for links. If it is empty or 0, the current page ID will be used.
Note: This setting can be overwritten by the plugins flexform.
trustedFields
Type
String
TS Path
plugin.tx_solr.search.trustedFields
Default
url
Since
3.1
The data in EXT:solr is escaped right after the retrieval from Solr. In rare cases when you need to store HTML in Solr documents you can use this configuration to mark these fields as trusted fields and skip the escaping. Typically this is needed when you want to retrieve html from solr.
The following example shows how to avoid html in the content field:
If enabled, the results plugin (pi_results) issues a "get everything" query during initialization. This is useful, if you want to create a page that shows all available facets although no search has been issued by the user yet. Note: Enabling this option alone will not show results of the get everything query. To also show the results of the query, see option showResultsOfInitialEmptyQuery below.
Requires initializeWithEmptyQuery (above) to be enabled to have any effect. If enabled together with initializeWithEmptyQuery the results of the initial "get everything" query are shown. This way, in combination with a filter you can easily list a predefined set of results.
When doing a new search, existing parameters like filters will be carried over to the new search. This is useful for a scenario where you want to list all available documents first, then allow the user to filter the documents using facets and finally allow him to specify a search term to refine the search.
ignoreGlobalQParameter
Type
Boolean
TS Path
plugin.tx_solr.search.ignoreGlobalQParameter
Default
Options
0,1
Since
7.0
In some cases you want EXT:solr to react on the parameter "q" in the url. Normally plugins are bounded to a namespace to allow multiple instances of the search on the same page. In this case you might want to disable this and let EXT:solr only react on the namespaced query parameter (tx_solr[q] by default).
Comma-separated list of additional argument names, that should be added to the persistent arguments that are kept for sub request, like the facet and sorting urls. Hard coded argument names are q, filter and sort.
Till solr version 6.5.x all parameters of the plugin namespace was added to the url again. With this setting you could enable this behavior again, but only with a whitelist of argument names.
query
The query sub-section defines a few query parameters for the query that will be sent to the Solr server later on. Some query parameters are also generated and set by the extension itself, f.e. when using facets.
query.allowEmptyQuery
Type
Boolean
TS Path
plugin.tx_solr.search.query.allowEmptyQuery
Default
Options
0,1
Since
1.4
If enabled, empty queries are allowed.
query.allowedSites
Type
String
TS Path
plugin.tx_solr.search.query.allowedSites
Since
2.2
Default
__solr_current_site
When indexing documents (pages, records, files, ...) into the Solr index, the solr extension adds a "siteHash". The siteHash is used to allow indexing multiple sites into one index and still have each site only find its own documents. This is achieved by adding a filter on the siteHash.
Sometimes though, you want to search across multiple domains, then the siteHash is a blocker. Using the allowedSites setting you can set a comma-separated list of domains who's documents are allowed to be included in the current domain's search results. The default value is __solr_current_site which is a magic string/variable that is replaced with the current site's domain when querying the Solr server.
Since
3.0
Version 3.0 introduced a couple more magic keywords that get replaced:
__current_site same as __solr_current_site
__all Adds all domains as allowed sites
* (asterisk character) Everything is allowed as siteHash (same as no siteHash check). This option should only be used when you need a search across multiple system and you know the impact of turning of the siteHash check.
Security note (since fix for CVE-2026-56094): the siteHash filter is a security boundary, not a user-facing search option.
It cannot be set or overridden through request-provided tx_solr[additionalFilters] — the reserved names siteHash and access are stripped from request input before the query is built.
To search across sites, use the allowedSites setting above; passing a siteHash filter from the frontend has no effect.
Controls how much Solr/Lucene query syntax survives in tx_solr[q].
Selector, range, grouping and metacharacters (: [ ] ( ) { } ^ " ~ \ /) are always escaped at the user-input boundary regardless of this setting, so field-targeted enumeration and range injection cannot reach Solr.
1 (default) — the well-known Lucene operators + - && || ! * ? pass through, so the documented wildcard and boolean operator UX (apple*, +foo -bar) keeps working.
0 — strict mode; the additional SolrJ specials | & ; are also escaped. + - ! * ? and whitespace stay literal, so required/prohibited terms, NOT and the wildcard UX still function.
query.getParameter
Type
String
TS Path
plugin.tx_solr.search.query.getParameter
Since
2.2
Default
tx_solr|q
The GET query parameter name used in URLs. Useful for cases f.e. when a website tracking tool does not support the default array GET parameters.
The option expects a string, you can also define an array in the form of arrayName|arrayKey.
query.fields has been renamed to query.queryFields in version 3.0
Defines what fields to search in the index. Fields are defined as a comma separated list. Each field can be given a boost by appending the boost value separated by the ^ character, that's Lucene query language. The boost value itself is a float value, pay attention to using a dot as the separator for the fractions. Use this option to add more fields to search.
The boost take influence on what score a document gets when searching and thus how documents are ranked and listed in the search results. A higher score will move documents up in the result listing. The boost is a multiplier for the original score value of a document for a search term.
By default if a search term is found in the content field the documents gets scored / ranked higher as if a term was found in the title or keywords field. Although the default should provide a good setting, you can play around with the boost values to find the best ranking for your content.
query.userFields
Type
String
TS Path
plugin.tx_solr.search.query.userFields
Default
(empty — derived from query.queryFields)
Since
11.2.8
Whitelist of fields that a Solr field-selector (field:value) in tx_solr[q] may target.
By default the whitelist is derived from query.queryFields — only fields listed in qf are addressable via selector.
Selectors against fields outside the whitelist are treated as literal terms and silently miss.
A scalar value replaces the derived whitelist with a whitespace-separated list of field names:
plugin.tx_solr.search.query.userFields = title content fileExtension
Copied!
Alternatively the add and remove sub-keys apply comma-separated deltas on top of the qf-derived base list (used only when the scalar value is empty):
Sets the minimum match mm query parameter.
By default the mm query parameter is set in solrconfig.xml as 2<-35%. This means that for queries with less than three words they all must match the searched fields of a document. For queries with three or more words at least 65% of them must match rounded up.
Please consult the link to the Solr wiki for a more detailed description of the mm syntax.
A boost function can be useful to influence the relevance calculation and boost some documents to appear more at the beginning of the result list.
Technically the parameter will be mapped to the "bf" parameter in the solr query.
Use cases for example could be:
"Give newer documents a higher priority":
This could be done with a recip function:
recip(ms(NOW,created),3.16e-11,1,1)
Copied!
"Give documents with a certain field value a higher priority":
Allows to further manipulate the score of a document by using Lucene syntax queries. A common use case for boost queries is to rank documents of a specific type higher than others.
Please consult the link to the Solr wiki for a more detailed description of boost functions.
This parameter ties the scores together. Setting is to "0" (default) uses the maximum score of all computed scores.
A value of "1.0" adds all scores. The value is a number between "0.0" and "1.0".
Allows to predefine filters to apply to a search query. You can add multiple filters through a name to Lucene filter mapping. The filters support stdWrap.
This is a magic/reserved filter (thus the double underscore). It limits the query and the results to certain branches/sections of the page tree. Multiple starting points can be provided as a comma-separated list of page IDs.
query.sortBy
Type
String
TS Path
plugin.tx_solr.search.query.sortBy
Since
1.0
Allows to set a custom sorting for the query. By default Solr will sort by relevance, using this setting you can sort by any sortable field.
Needs a Solr field name followed by asc for ascending order or desc for descending.
This parameter enables the phrase search feature from Apache Solr. Setting is to "0" (default) does not change behaviour from Apache Solr if user searches for two and more words.
Enabling phrase search feature influences the document set and/or the scores of documents.
This parameter defines what fields should be used to search in the given phrase. Matched documents will be boosted according to fields boost value.
Fields are defined as a comma separated list and same way as queryFields.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter defines the "phrase slop" value, which represents the number of positions one word needs to be moved in relation to another word in order to match a phrase specified in a query.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter defines the "phrase slop" value, which represents the number of positions one word needs to be moved in relation to another word in order to match a phrase specified in a explicit phrase search query.
Note: On explicit("double quoted" phrase) phrase search Apache Solr searches in "qf" queryFields
Note: The value of this setting has no influence on implicit phrase search.
On explicit phrase search the Solr searches in qf (plugin.tx_solr.search.query.queryFields) defined fields.
This parameter enables the bigram phrase search feature from Apache Solr. Setting is to "0" (default) does not change behaviour from Apache Solr if user searches for three and more words.
Enabling bigram phrase search feature influences the scores of documents with phrase occurrences.
This parameter defines what fields should be used to search in the given sentence(three+ words). Matched documents will be boosted according to fields boost value.
Fields are defined as a comma separated list and same way as queryFields.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter defines the "bigram phrase slop" value, which represents the number of positions one word needs to be moved in relation to another word in order to match a phrase specified in a query.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter enables the phrase search feature from Apache Solr. Setting is to "0" (default) does not change behaviour from Apache Solr if user searches for two and more words.
Enabling phrase search feature influences the scores of documents with phrase occurrences.
This parameter defines what fields should be used to search in the given phrase. Matched documents will be boosted according to fields boost value.
Fields are defined as a comma separated list and same way as queryFields.
Note: The value of this setting has NO influence on explicit phrase search.
This parameter defines the "trigram phrase slop" value, which represents the number of positions one word needs to be moved in relation to another word in order to match a phrase specified in a query.
Note: The value of this setting has NO influence on explicit phrase search.
En-/disables search term highlighting on the results page.
Note
Since 11.2.8 the Unified Highlighter is used unconditionally, regardless of fragmentSize. If Solr cannot generate a highlighted snippet for a field, it returns a leading-text default summary of approximately hl.snippets * fragmentSize characters.
Note: The Unified Highlighter uses hl.offsetSource=ANALYSIS, so term vectors are not required.
Highlighted fields must be stored, and their analysis should be compatible with the queried fields.
When highlighting is activated Solr highlights the fields configured in highlightFields and can return multiple fragments of fragmentSize around the highlighted search word. These fragments are used as teasers in the results list. fragmentSeparator allows to configure the glue string between those fragments.
Activates TYPO3's highlighting of search words on the actual pages. The words a user searched for will be wrapped with a span and CSS class csc-sword
Highlighting can be styled using the CSS class csc-sword, you need to add the style definition yourself for the complete site.
This setting can be used to trigger a new search automatically when the previous search had no results but
suggestions from the spellchecking. In this case the user can directly see the results of the best correction option.
lastSearches
lastSearches
Type
Boolean
TS Path
plugin.tx_solr.search.lastSearches
Since
1.3-dkd
Default
Set plugin.tx_solr.search.lastSearches = 1 to display a list of the last searches.
lastSearches.limit
Type
Integer
TS Path
plugin.tx_solr.search.lastSearches.limit
Since
1.3-dkd
Default
10
Defines the number of last searches, that should get minded.
lastSearches.mode
Type
String
TS Path
plugin.tx_solr.search.lastSearches.mode
Since
1.3-dkd
Default
user
Options
user, global
If mode is user, keywords will get stored into the session. If mode is global keywords will get stored into the database.
frequentSearches
frequentSearches
Type
Boolean
TS Path
plugin.tx_solr.search.frequentSearches
Since
1.3-dkd, 2.8
Default
Set plugin.tx_solr.search.frequentSearches = 1 to display a list of the frequent / common searches.
When enabled, keywords are written to the statistics table in lower case.
frequentSearches.minSize
Type
Integer
TS Path
plugin.tx_solr.search.frequentSearches.minSize
Since
1.3-dkd, 2.8
Default
14
The difference between frequentSearches.maxSize and frequentSearches.minSize is used for calculating the current step.
frequentSearches.maxSize
Type
Integer
TS Path
plugin.tx_solr.search.frequentSearches.maxSize
Since
1.3-dkd, 2.8
Default
32
The difference between frequentSearches.maxSize and frequentSearches.minSize is used for calculating the current step.
frequentSearches.limit
Type
Integer
TS Path
plugin.tx_solr.search.frequentSearches.limit
Since
1.3-dkd, 2.8
Default
20
Defines the maximum size of the list by frequentSearches.select.
frequentSearches.select
Type
cObject
TS Path
plugin.tx_solr.search.frequentSearches.select
Since
1.3-dkd, 2.8
Defines a database connection for retrieving statistics.
sorting
sorting
Type
Boolean
TS Path
plugin.tx_solr.search.sorting
Since
1.0
Default
Set plugin.tx_solr.search.sorting = 1 to allow sorting of results.
sorting.defaultOrder
Type
String
TS Path
plugin.tx_solr.search.sorting.defaultOrder
Since
1.0
Default
asc
Options
asc, desc
Sets the default sort order for all sort options.
sorting.options
This is a list of sorting options. Each option has a field and label to be used. By default the options title, type, author, and created are configured, plus the virtual relevancy field which is used for sorting by default.
Example:
plugin.tx_solr.search {
sorting {
options {
relevance {
field = relevance
label = Relevance
}
title {
field = sortTitle
label = Title
}
}
}
}
Copied!
Note: As mentioned before relevance is a virtual field that is used to reset the sorting. Sorting by relevance means to have the order provided by the scoring from solr. That the reason why sorting descending on relevance is not possible.
Defines how facet options are sorted, by default they are sorted by count of results, highest on top. count, 1, true are aliases for each other.
Facet options can also be sorted alphabetically (lexicographic by indexed term) by setting the option to index. index, 0, false, alpha (from version 1.2 and 2.0), and lex (from version 1.2 and 2.0) are aliases for index.
faceting.limit
Type
Integer
TS Path
plugin.tx_solr.search.faceting.limit
Since
1.0
Default
10
Number of options to display per facet. If more options are returned by Solr, they are hidden and can be expanded by clicking a "show more" link. This feature uses a small javascript function to collapse/expand the additional options.
Defines the output of the "Show more" link, that is rendered if there are more facets given than set by faceting.limit.
faceting.showEmptyFacets
Type
Boolean
TS Path
plugin.tx_solr.search.faceting.showEmptyFacets
Since
1.3
Default
Options
0, 1
By setting this option to 1, you will allow rendering of empty facets. Usually, if a facet does not offer any options to filter a result-set of documents, the facet header will not be shown. Using this option allows the header still to be rendered when no filter options are provided.
faceting.urlParameterStyle
Type
Option/String (index or assoc)
TS Path
plugin.tx_solr.search.faceting.urlParameterStyle
Since
11.1
Default
index
Note
On assoc, the setting faceting.urlParameterSort will be enabled and can not be disabled.
Allows to change the url style of facets. This can be legacy index or more modern associative style.
Index style: tx_solr[filter][0]=type:pages
Associative style: tx_solr[filter][type:pages]=1
faceting.urlParameterSort
Type
Boolean
TS Path
plugin.tx_solr.search.faceting.urlParameterSort
Since
11.1
Default
Note
On faceting.urlParameterStyle = assoc, this setting can not be disabled.
Allows to enable sorting of url parameters, so the single state of facets is associated with same url, no matter in which order the facets were selected
Defines which fields you want to use for faceting. It's a list of facet configurations.
plugin.tx_solr.search.faceting.facets {
type {
field = type
label = Content Type
}
category {
field = category_stringM
label = Category
}
}
Copied!
faceting.facets.[facetName] - single facet configuration
You can add new facets by simply adding a new facet configuration in TypoScript. [facetName] represents the facet's name and acts as a configuration "container" for a single facet. All configuration options for a facet are defined within that "container".
A facet will use the values of a configured index field to offer these values as filter options to your site's visitors. You need to make sure that the facet field's type allows to sort the field's value; like string, int, and other primitive types.
To configure a facet you only need to provide the label and field configuration options, all other configuration options are optional.
The settings `keepAllOptionsOnSelection` and `keepAllFacetsOnSelection` are used internally to build exclude tags for facets in order to exclude the filters from the facet counts.
This helps to keep the counts of a facet as expected by the user, in some use-cases (Read also: http://yonik.com/multi-select-faceting/).
With the setting `additionalExcludeTags` you can add tags of facets that should be excluded from the counts as well.
Note: This setting is only available for option facets by now.
When you want to add fields as `additionalExcludeTags` for a facet a tag for this facet needs to exist. You can use this setting to force the creation of a tag for this facet in the solr query.
Used as a headline or title to describe the options of a facet.
Used in flex forms of plugin for filter labels. Can be translated with LLL: and consumed and translated in Partial/Facets/* with f:translate ViewHelper.
Metrics can be use to collect and enhance facet options with statistical data of the faceted documents. They can
be used to render useful information in the context of an facet option.
Example:
plugin.tx_solr.search.faceting.facets {
category {
field = field
label = Category
metrics {
downloads = sum(downloads_intS)
}
}
}
Copied!
The example above will make the metric "downloads" available for all category options. In this case it will be the sum of all downloads
of this category item. In the frontend you can render this metric with "<facetoptions.>.metrics.downloads" and use it for example to show it instead of the normal option count.
Normally, when clicking any option link of a facet this would result in only that one option being displayed afterwards. By setting this option to one, you can prevent this. All options will still be displayed.
This is useful if you want to allow the user to select more than one option from a single facet.
When configuring a facet to allow selection of multiple options, you can use this option to decide whether multiple selected options should be combined using AND or OR.
Sets how a single facet's options are sorted, by default they are sorted by count of results, highest on top.
Facet options can also be sorted alphabetically by setting the option to alpha.
Note: Since 9.0.0 it is possible to sort a facet by a function. This can be done by defining a metric and use that metric in the sortBy configuration. As sorting name you then need to use by convention "metrics_<metricName>"
Example:
pid {
label = Content Type
field = pid
metrics {
newest = max(created)
}
sortBy = metrics_newest desc
}
By default facet options are sorted by the amount of results they will return when applied. This option allows to manually adjust the order of the facet's options. The sorting is defined as a comma-separated list of options to re-order. Options listed will be moved to the top in the order defined, all other options will remain in their original order.
Set's the minimumCount for a single facet. This can be useful e.g. to set the minimumCount of a single facet to 0,
to have the options available even when there is result available.
Note: This setting is only available for facets that are using the json faceting API of solr. By now this
is only available for the options facets.
Allows you to display a facet even if it does not offer any options (is empty) and although you have set plugin.tx_solr.search.faceting.showEmptyFacets = 0.
Defines the type of the facet. By default all facets will render their facet options as a list. PHP Classes can be registered to add new types. Using this setting will allow you to use such a type and then have the facet's options rendered and processed by the registered PHP class.
Allows to define requirements for a facet to be rendered. These requirements are dependencies on values of other facets being selected by the user. You can define multiple requirements for each facet. If multiple requirements are defined, all must be met before the facet is rendered.
Each requirement has a name so you can easily recognize what the requirement is about. The requirement is then defined by the name of another facet and a list of comma-separated values. At least one of the defined values must be selected by the user to meet the requirement.
There are two magic values for the requirement's values definition:
__any: will mark the requirement as met if the user selects any of the required facet's options
__none: marks the requirement as met if none of the required facet's options is selected. As soon as any of the required facet's options is selected the requirement will not be met and thus the facet will not be rendered
Example of a category facet showing only when the user selects the news type facet option:
plugin.tx_solr {
search {
faceting {
facets {
type {
label = Content Type
field = type
}
category {
label = Category
field = category_stringS
requirements {
typeIsNews {
# typeIsNews is the name of the requirement, c# choose any so you can easily recognize what it does
facet = type
# The name of the facet as defined above
values = news
# The value of the type facet option as# it is stored in the Solr index
}
}
}
}
}
}
}
Overwrites how single facet options are rendered using TypoScript cObjects.
Example: (taken from issue #5920)
plugin.tx_solr {
search {
faceting {
facets {
type {
renderingInstruction = CASE
renderingInstruction {
key.field = optionValue
pages = TEXT
pages.value = Pages
pages.lang.de = Seiten
tx_solr_file = TEXT
tx_solr_file.value = Files
tx_solr_file.lang.de = Dateien
tt_news = TEXT
tt_news.value = News
tt_news.lang.de = Nachrichten
}
}
language {
renderingInstruction = CASE
renderingInstruction {
key.field = optionValue
0 = TEXT
0.value = English
0.lang.de = Englisch
1 = TEXT1.value = German
1.lang.de = Deutsch
}
}
}
}
}
}
Copied!
EXT:solr provides the following renderingInstructions that you can use in your project:
FormatDate:
This rendering instruction can be used in combination with a date field or an integer field that hold a timestamp. You can use this rendering instruction to format the facet value on rendering.
A common usecase for this is, when the datatype in solr needs to be sortable (date or int) but you need to render the date as readable date option in the frontend:
plugin.tx_solr.search.faceting.facets {
created {
field = created
label = Created
sortBy = alpha
reverseOrder = 1
renderingInstruction = TEXT
renderingInstruction {
field = optionValue
postUserFunc = ApacheSolrForTypo3\Solr\Domain\Search\ResultSet\Facets\RenderingInstructions\FormatDate->format
}
}
}
Copied!
elevation
elevation
Type
Boolean
TS Path
plugin.tx_solr.search.elevation
Since
3.0
Default
Set plugin.tx_solr.search.elevation = 1 to enable content elevation in search results.
If enabled, elevated results are marked with CSS class "results-elevated".
variants
By using variants you can shrink down multiple documents with the same value in one field into one document and make similar documents available in the variants property.
By default the field variantId is used as Solr collapsing criteria. This can be used e.g. as one approach of deduplication to group similar documents into on "root" SearchResult.
To use the different variants of the documents you can access "document.variants" to access the expanded documents.
This can be used for example for de-duplication to list variants of the same document below a certain document.
Note: Internally this is implemented with Solr field collapsing
Type
Boolean
TS Path
plugin.tx_solr.search.variants
Since
6.0
Default
Set plugin.tx_solr.search.variants = 1 to enable the variants in search results.
variants.expand
Used to expand the document variants to the document.variants property.
Type
Boolean
TS Path
plugin.tx_solr.search.variants.expand
Since
6.0
Default
1
variants.variantField
Used to expand the document variants to the document.variants property.
Note:: The field must be a numeric field or a string field! Not a text field!
Type
String
TS Path
plugin.tx_solr.search.variants.variantField
Since
6.0
Default
variantId
variants.limit
Limit of expanded documents.
Though this setting limits the returned variants, you still can get the number of existing variants, it's set in "document.variantsNumFound" (since EXT:solr 10)
Type
Integer
TS Path
plugin.tx_solr.search.variants.limit
Since
6.0
Default
10
tx_solr.suggest
This feature allows you to show a suggest layer that suggest terms that start with the letters that have been typed into the search field
numberOfSuggestions
Type
Integer
TS Path
plugin.tx_solr.suggest.numberOfSuggestions
Since
1.1
Default
10
Sets the number of suggestions returned and displayed in the layer attached to the search field.
suggestField
Type
String
TS Path
plugin.tx_solr.suggest.suggestField
Since
1.1
Default
spell
Sets the Solr index field used to get suggestions from. A general advice is to use a field without stemming on it. For practical reasons this is currently the spell checker field.
Note: With EXT:solr 11.1.0 ASCII folding and language depending normalization filters were introduced, but due to the special behaviour of the auto suggestions ascii-terms were not treated correctly. So with 11.1.3 the untouched tokens are also kept, as this might lead to duplicate
suggestions, a new field for exact suggestions is introduced, if you want to avoid duplicates and use stricter suggestions, just configure spellExact as suggest field.
forceHttps
Type
Boolean
TS Path
plugin.tx_solr.suggest.forceHttps
Since
1.1
Default
Options
0,1
If enabled, HTTPS will be used for querying suggestions. Otherwise HTTP will be used.
When a user types multiple words into your search field they usually are split up into full keywords used in the query's q parameter and the last part being the partial keyword in the facet.prefix parameter. Enabling this setting moves everything into the facet.prefix parameter. This is usually only useful when using a string field as suggest / auto-complete source field.
Example - Here "Hello Solr" are the full keywords and the user started typing "World" so that "Wo" is used as the partial keyword:
When this setting is enabled, the top results are shown in the suggest layer. The top results are build from the first search match,
or when the first search delivers no hit, the results from the first suggestion are shown.
numberOfTopResults
Type
Integer
TS Path
plugin.tx_solr.suggest.numberOfTopResults
Since
8.0
Default
5
Defines the number of top results that will be shown.
additionalTopResultsFields
Type
String
TS Path
plugin.tx_solr.suggest.additionalTopResultsFields
Since
9.0
Comma-separated list of fields that should be added to the top results response json.
tx_solr.statistics
This section allows you to configure the logging for statistics.
Note: The statistics are logged into a mysql table. This might not make sense for high frequently used searches. In this case you should think about to connect a dedicated tracking tool.
statistics
Type
Boolean
TS Path
plugin.tx_solr.statistics
Since
2.0
Default
Set plugin.tx_solr.statistics = 1 to log statistics.
statistics.anonymizeIP
Type
Integer
TS Path
plugin.tx_solr.statistics.anonymizeIP
Since
2.0
Default
Allowed values are 0 (masking disabled), 1 (mask host), 2 (mask host and subnet).
statistics.addDebugData
Type
Boolean
TS Path
plugin.tx_solr.statistics.addDebugData
Since
6.1
Default
Adds debug data to the columns time_total, time_preparation and time_processing in the table tx_solr_statistics
from the result of the search query.
Note: Enabling addDebugData can have performance impact since debugMode is appended to queries.
tx_solr.logging
This section defines logging options. All loggings will be available in the TYPO3 logging framework.
If enabled the written log entries will be printed out as debug message in the frontend or to the TYPO3 debug console in the backend.
This setting replaces the previous setting plugin.tx_solr.logging.debugDevLogOutput which was needed, when the devLog was used.
exceptions
Type
Boolean
TS Path
plugin.tx_solr.logging.exceptions
Default
1
Options
0,1
Since
1.0
If enabled, thrown exceptions are logged.
indexing
Type
Boolean
TS Path
plugin.tx_solr.logging.indexing
Default
1
Options
0,1
Since
1.0
If enabled, logs when pages / documents are indexed.
If enabled, the requested data will be logged. Request data includes item, url, parameters, headers, data, decodedData and report.
indexing.pageIndexed
Type
Boolean
TS Path
plugin.tx_solr.logging.indexing.pageIndexed
Default
Options
0,1
Since
2.0
If enabled, it will be logged if a page was indexed successful or not. Additionally, an error is logged if indexing is disabled (via config.index_enable)
query.filters
Type
Boolean
TS Path
plugin.tx_solr.logging.query.filters
Default
1
Options
0,1
Since
1.0
If enabled, filters will be logged when they get added to the Solr query.
query.searchWords
Type
Boolean
TS Path
plugin.tx_solr.logging.query.searchWords
Default
1
Options
0,1
Since
1.0
If enabled, received search queries will be logged.
query.queryString
Type
Boolean
TS Path
plugin.tx_solr.logging.query.queryString
Default
1
Options
0,1
Since
1.0
If enabled, query string parameters and the respective response will be logged.
query.rawPost
Type
Boolean
TS Path
plugin.tx_solr.logging.query.rawPost
Default
1
Options
0,1
Since
1.0
If enabled, POST requests against the Solr server will be logged.
query.rawGet
Type
Boolean
TS Path
plugin.tx_solr.logging.query.rawGet
Default
1
Options
0,1
Since
1.0
If enabled, GET requests against the Solr server will be logged.
Extension Configuration
The following settings can be defined in the "Settings":>"Extension Configuration":>"solr"
useConfigurationFromClosestTemplate
Type
Boolean
Since
6.1
Default
When this setting is active the closest page with a typoscript template will be used to fetch the configuration.
This improves the performance but limits also the possibilities. E.g. conditions can not be used that are related to a certain page.
useConfigurationTrackRecordsOutsideSiteroot
Type
Boolean
Since
6.1
Default
1
A common common scenario is to have a site and a storage folder for records parallel to it
on the same level (f.e.)
If you don't want this behaviour - it should be set to false.
allowSelfSignedCertificates
Type
Boolean
Since
6.1
Default
Can be used to allow self signed certificates - when using the SSL protocol.
allowLegacySiteMode
Type
Boolean
Since
10.0
Default
Can be used to allow using classic TypoScript Configuration for Sites.
monitoringType
Type
Int
Since
11.2
Default
Defines how data updates should be monitored
By default (=0) recognized updates will be processed directly and the Solr index queue will be directly updated, also
the Solr index where appropriate. As in huge instances the monitoring can slow down the TYPO3 backend, two more monitoring
options are available:
Delayed: Record update events will be queue and processed later, the scheduler task "Event Queue Worker" is required for processing.
No monitoring: Monitoring is completely disabled, please note that you have to take care of Solr index updates yourself.
Solr Connection
EXT:solr uses the TYPO3 PSR-7 client to connect to the Solr server.
You can apply additional configuration for proxy settings using $GLOBALS['TYPO3_CONF_VARS']['HTTP'].
Tip
For more details about Guzzle settings and TYPO3 implementation see
Some of the SQL statements performed on the pages table in TYPO3 perform extensive operations while copying
page-trees. These operations can be speed up by by adding 2 indexes to the standard table pages.
The indexes are:
content_from_pid_deleted (content_from_pid, deleted),
doktype_no_search_deleted (doktype, no_search, deleted)
It is not required that these indexes are created by in above scenarios considerable performance gains can be achieved.
Development
There are many ways to extend and hook into EXT:solr to customize EXT:solr for your needs.
If external data should be indexed or the RecordIndexer is not required, it is possible to fill the index with an extension as well. The class can then be called e.g. by a CLI call.
<?phpnamespaceVendor\ExtensionName\Import;
useApacheSolrForTypo3\Solr\ConnectionManager;
useApacheSolrForTypo3\Solr\Domain\Site\SiteRepository;
useApacheSolrForTypo3\Solr\System\Solr\Document\Document;
useApacheSolrForTypo3\Solr\System\Solr\SolrConnection;
useTYPO3\CMS\Core\Utility\GeneralUtility;
classIndexer{
/** @var ConnectionManager */protected $connectionManager;
publicfunction__construct(){
$this->connectionManager = GeneralUtility::makeInstance(ConnectionManager::class);
}
/**
* Send data to solr index
*
* @param array $rows Data to be indexed, e.g. multiple DB rows
* @param int $pageId root page
* @param int $language language id
*/publicfunctionindex(array $rows, int $pageId = 1, int $language = 0){
$documents = [];
foreach ($rows as $row) {
$documents[] = $this->createDocument($row, $pageId);
}
$connection = $this->connectionManager->getConnectionByPageId($pageId, $language);
$connection->getWriteService()->addDocuments($documents);
}
/**
* Remove all from index
*
* @throws \ApacheSolrForTypo3\Solr\NoSolrConnectionFoundException
*/publicfunctionclearIndex(){
$connections = $this->connectionManager->getAllConnections();
foreach ($connections as $connectionLanguage => $connection) {
/** @var SolrConnection */
$connection->getWriteService()->deleteByType('cutom_type');
}
}
/**
* Create a solr document which then is sent to solr
*
* @param array $row
* @param int $pageId
* @return Document
*/protectedfunctioncreateDocument(array $row, int $pageId): Document{
$document = $this->getBaseDocument($row, $pageId);
$solrFieldMapping = [
'title' => 'title',
'summary' => 'abstract',
'information' => 'content',
'keywords' => 'keywords',
'area' => 'area_stringS',
'category' => 'category_stringS'
];
foreach ($row as $key => $value) {
if (isset($solrFieldMapping[$key])) {
$document->setField($solrFieldMapping[$key], $value);
}
}
// url generation
$document->setField('url', 'todo'); // custom implementation requiredreturn $document;
}
/**
* Creates a Solr document with the basic / core fields set already.
*
* @param array $itemRecord The record to use to build the base document
* @param int $rootPageId root page id
* @return Document A basic Solr document
*/protectedfunctiongetBaseDocument(array $itemRecord, int $rootPageId): Document{
$siteRepository = GeneralUtility::makeInstance(SiteRepository::class);
$site = $siteRepository->getSiteByRootPageId($rootPageId);
/** @var $document Document */
$document = GeneralUtility::makeInstance(Document::class);
// required fields
$document->setField('id', 'cutom_type_' . $itemRecord['uid']);
$document->setField('variantId', 'cutom_type' . $itemRecord['uid']);
$document->setField('type', 'cutom_type');
$document->setField('appKey', 'EXT:solr');
$document->setField('access', ['r:0']);
// site, siteHash
$document->setField('site', $site->getDomain());
$document->setField('siteHash', $site->getSiteHash());
// uid, pid
$document->setField('uid', $itemRecord['uid']);
$document->setField('pid', 1);
return $document;
}
}
Copied!
Developing Backend Modules
Backend Components
EXT:solr provides UI components for backend modules. Some components hold (GUI)state and some not, but all components calling actions(changing the extension and/or GUI state!),
and then redirecting to the actions within the component was used(referrer also) or to the defined by callers action uri, if that is required by UX.
Below are all available components listed and their responsibility.
CoreSelector
Renders menu in backends doc header with available Solr cores for selected Site and changes the solr core by clicking on option in drop down menu.
Provides following methods, which must be called inside the initializeView(...) method in your controller to render this component in Backend:
generateCoreSelectorMenuUsingSiteSelector()
Use this method together with SiteSelectorMenu component.
generateCoreSelectorMenuUsingPageTree()
Use this method if you are using original page tree from CMS.
Provides following Actions for changing state, must be added to actions list of your controller:
switchCore
Provides following fully initialized properties in utilizing action controller:
$selectedSolrCoreConnection from type ApacheSolrForTypo3SolrSolrService
If you need the possibility to switch the core, you can extends the AbstractModuleController (in ApacheSolrForTypo3SolrControllerBackendSearch).
FAQ
Why should I add some action name to my action controller?
To allow calling this action within your controller, component can use own controller for changing state only if that is hardcoded with some module
and allowed by ACL for all(or almost all) be users/groups, but this is a bag approach. Therefore allow changing something, only if that is needed.
What do I need to do for using Backend Components?
By extending ApacheSolrForTypo3SolrControllerBackendSearchAbstractModuleController your module has the pagetree (to select the side) and the core selector, to
select the needed solr core.
Development Environment
To simplify the development for TYPO3 and solr related components we provide a development environment based on vagrant and the Homestead box of the TYPO3 core.
You can find the development box in the following git repository:
When you start the box, you will find a pre-configured environment for the support TYPO3 LTS version and the solr installation that is needed for the installed EXT:solr version.
Testing and Continues Integration
The goal during the development of EXT:solr is, to test most of the components with unit and integration tests.
Unit Tests
For the single classes we've added unit tests whenever we though it is usefull. The unit tests should run very quickly, to git a quick response.
Integration Tests
As in EXT:solr the integration tests are more complex and test the integration of the different components. Since a database server and a solr server is required this is needed to run the integration test suite. During the bootstrap of the test environment, we use the TYPO3 core functionality for database tests and we install a local solr server with out install script.
To simplify the local usage of the unit and integration tests, we ship a few bash script that support you to get everything started.
Bootstrapping the Test Environment
When you want to start the testrunner in your shell you need to bootstrap it once:
source ./Build/Test/bootstrap.sh --local
Copied!
The bootstrapper will prompt for some values:
When the boostrapper was finished successful the following was done:
Environment variables for the TYPO3 testing framework have been exported
Test database was created
Test solr instance was created
Afterwards you can run the ci suite in your shell
Running the ci Suite
When the test environment was boostrapped correctly you can start the test runner:
./Build/Test/cibuild.sh
Copied!
When everything is configured correctly all tests should run through and you should get a green bar:
Code Structure
The components of EXT:solrfluid have been developed with the domain driven design (DDD) approach (https://de.wikipedia.org/wiki/Domain-driven_Design) for our extension we tried to separate the code by the following layers:
Domain: Everything that is related to the "search" domain should be implemented here.
System: Everything that is related to the "system" (e.g. TYPO3 specific) should be implemented here.
Domain Layer & Domain Model
The classes of the domain layer are located in "Classes/Domain" and should contain everything that is related to the "search domain".
ResultSet
The "SearchResultSet" is the main entity that you get passed to the view. It can be used to access all search related objects on your result page.
UML diagram of the SearchResutSet
The SearchResultSet can be used e.g. to get facets and spelling suggestions. A focus for the first release was a new domain model for facets,
that can be rendered with fluid or any other template engine.
Facets
The following UML diagram shows the implemented facets in EXT:solrfluid. Every facet has one or more facet items attached.
For the OptionsFacet the FacetItem is an Option, for the NumericRangeFacet a NumericRange.
Rendering of a facet:
Based on the "type" TypoScript configuration the "FacetRegistry" chooses the responsible facet package class that
is used to create the object structure from the solr repsonse. Each facet type is shipped with a default fluid partial, that is able to render such a facet.
A facet package consists of the following parts:
The facet parser: Responsible to parse the apache solr response into a domain object structure
The url decoder: Responsible to decode the url fragment of the facet in EXT:solr
The query builder: Responsible to build the needed query fragment for Apache Solr to apply the facet
The typoscript configuration "partialName" can be used to force the rendering with another fluid partial.
For advanced use cases you can use the "FacetRegistry" to register your own facet type or overwrite the facet behaviour for a certain facet type.
UML diagram of the *Facet classes
As you see in the diagram above solr ships a clean object structure of the facets, that you can render in your custom templates as you need them.
Signals
Signals are currently only used in the search controller. Every action emits a signal which can be used to change its variables assigned to the view.
Example
As an example, the action resultsAction of the Classes/Controller/SearchController is used.
Notice: The values array needs to be returned as an single element of an array, because a slot method has to return the same number of arguments like it received.
Events
In version 11 we started with the implementation of events, using the EventDispatcher (PSR-14 Events).
Step by step the events will replace older hooks and signals. In the following you will find a description of the available events
Monitoring
Observing and processing of data updates is controlled by the following events:
ContentElementDeletedEvent, fired if a content element is deleted
PageMovedEvent, fired if a page is moved
RecordDeletedEvent, fired if a record is deleted
RecordGarbageCheckEvent, fired if a record garbage check is triggered
RecordMovedEvent, fired if a record is moved
VersionSwappedEvent, fired if a version is swapped
All events implement the DataUpdateEventInterface and at least provide information about the elements uid, table and modified fields.
Data update processing
ProcessingFinishedEvent
The ProcessingFinishedEvent indicates that the processing of a data update event is finished, if you want to implement an own monitoring you
can use the ProcessingFinishedEvent to start the processing. The event provides the originally fired data update events listed above, so that
you have all the required information about the update to process.
If you're using the event you're indepent of the monitoring setting, as this event if fired in the immediate and delayed monitoring mode as
soon as an event is processed.
If you're using the delayed processing (see "monitoringType"), you can use one of the following events:
DelayedProcessingQueuingFinishedEvent
This event is fired as soon as the update event is queued in the event queue (tx_solr_eventqueue_item), you can e.g. use this event to
register own events or to implement processing that has to executed immediately even in delayed monitoring mode.
DelayedProcessingFinishedEvent
The Scheduler task "Event Queue Worker" is required to process the data updates in delayed monitoring mode and will fire the DelayedProcessingFinishedEvent
as soon as an event has been processed. If you require to perform actions only during delayed processing, the event can be used.
SearchUriBuilder
The SearchUriBuilder is responsible to build uris, that are used in the searchContext. Since the route enhancer is introduced you can use the following
event to influence the build uris:
BeforeReplaceVariableInCachedUrlEvent
BeforeProcessCachedVariablesEvent
PostProcessUriEvent
ViewHelpers
Beside the controllers, the domain objects and the templates we ship a few useful view helpers. To avoid a strong coupling between the extension and fluid as template engine we tried to keep all ViewHelpers as "slim" as possible. Whenever it was possible we moved the logic into custom service classes and just use them in the ViewHelper.
Since everything belongs to the "SearchResultSet" and we wanted to avoid the need ob passing this object around from "template to template" and "partial to partial" we decided to provide an own "ControllerContext" that referenced the "SearchResultSet". With this approach, it is possible to access the "SearchResultSet" in every ViewHelper.
With the current release we ship the following concrete ViewHelpers:
|
Path
Description
s:debug.documentScoreAnalyzer
Used to render the score analysis.
s:debug.query
Shows the solr query debug information.
s:document.highlightResult
Performs the highlighting on a document.
s:document.relevance
Shows the relevance information for a document.
s:facet.area.group
Filters the facets in the rendering scope to one group.
s:facet.options.group.prefix.labelPrefixes
Provides an array of available label prefixes that can be used
to filter with s:facet.options.group.prefix.labelFilter.
s:facet.options.group.prefix.labelFilter
Filters the options of a facet by a given prefix.
s:uri.facet.addFacetItem
Add's a facet item to the current url.
s:uri.facet.removeAllFacets
Removes all facet items from the current url.
s:uri.facet.removeFacet
Removes all options from one facet.
s:uri.facet.removeFacetItem
Removes a single facet item from the url.
s:uri.facet.setFacetItem
Sets one single item for a facet (and removes other setted)
s:uri.paginate.resultPage
Creates a link to a result page of the current search.
s:uri.search.currentSearch
Creates a link to the current search (with facets, sorting...)
s:uri.search.startNewSearch
Creates a link for a new search by a term.
s:uri.sorting.removeSorting
Creates a link to the current search and removes the sorting.
s:uri.sorting.setSorting
Creates a link to the current search and sets a new sorting.
s:pageBrowserRange
Provides the range data for the pagination.
s:searchForm
Renders the searchForm.
s:translate
Custom translate ViewHelper (uses translations from ext:solr)
FAQ - Frequently Asked Questions
What does the term `"Core"<https://cwiki.apache.org/confluence/display/solr/Solr+Cores+and+solr.xml>`_ mean?
This term relates to Apache Solr indexes and means a single distinct part of an index. It is possible to use multiple cores on one single Apache Solr instance.
Good examples are using a different Apache Solr core for each language or of course a separate core for each website.
For more informations please refer to the Apache Solr documentation.
|
Where can I report a bug?
Please make sure that this bug is not reported already, use also the search function of our issue tracker.
Our issue tracker is on GitHub.
|
Where can I report a security issue?
If you have found a security issue in our extension, please do not post about it in a public channel.
Please send an email to the TYPO3 security team with detailed description of found vulnerability. For more details about security issue handling see https://typo3.org/teams/security/contact-us/
|
Is there some chat/irc channel for EXT:solr available?
Join us on the official Slack for TYPO3 and get answers related to EXT:solr in the #ext-solr channel immediately!
|
Which plugins(TYPO3 Frontend) are avalable?
Search: Form only
Search: Form, Result, Additional Components
Search: Frequent Searches
Just insert one of this plugins on corresponding page to fade in the search form and/or supply the front end with a search results.
|
When i open the search page i see the message 'Search is currently not available. ', whats wrong?
Did you configure your solr connection as required?
Please read "Configure Extension" and check if you have configured everything
Did you configure solr server and port and does the scheme and path match?
Did you click "Initialize connection" after configuring the solr server?
Can you access the solr server with wget or curl from the command line?
Is the system report of EXT:solr green?
|
In which cases do I want to trigger indexing manually?
after changing any configuration file.
after modifying synonyms, stop words, protected words in TYPO3 Backend -> Search
Moreover by changing core/index configuration you need to reload the core to make the changes become active.
To reload configuration you can either restart the whole Solr server or simply reload a specific core.
|
I want to index files with EXT:solr. How can i do that?
We provide an addon called EXT:solrfal, that allows you to index files from FAL into Solr. This addon is currently available for partner only.
|
How can i use Fluid templates with EXT:solr < v7.0.0?
For the Fluid rendering in EXT:Solr >= 5.0 <= 6.1 we provide the addon EXT:solrfluid, that allows you to render your search results with Fluid.
Since EXT:Solr 7.0 Fluid is the default templating engine.
|
Which versions of EXT:solr / EXT:solrfal and EXT:solrfluid work together?
Pages are not indexed. I did everything by the book.
You forgot to set config.index_enable = 1 in your TypoScript setup: Enable indexing
|
My indexed documents are empty, i can not find the content of a page?
Did you configure the search markers ( "<!-- TYPO3SEARCH_begin -->" and "<!-- TYPO3SEARCH_end -->") on your page? Check the paragraph Search Markers and make sure your page renders them.
|
I have languages in TYPO3 that are not relevant for the search. How can i exclude them?
You need to enable the search just for the relevant languages.
The extension is indexing into the wrong core for multi-language sites. What's wrong?
When indexing pages the page indexer retrieves the core from the TypoScript configuration. That configuration is determined by the language (GET L parameter). However, when the indexer tries to index a page that has not been translated TYPO3 will by default still render the page but falling back to the default language page. By that TYPO3 will also use the TypoScript configuration for the default language which usually points to a different Solr core.
Solution: Make sure you have configured config.sys_language_mode to use content_fallback. This way TYPO3 will fall back to the configured language's content, but will use the TypoScript configuration for the requested language.
When i change a record, no update is detected. What's wrong?
Are your records inside of your site root? EXT:solr record monitor processes records that belong to your site, which means they need to be below your site root.
If you want to index records that are outside your sideroot, you need to configure the page id's of the sysfolder as additionalPageIds:
There are two datatypes for text stringS and textS. When should i choose which datatype?
String data types like stringS store the raw string. No processing, like stemming, splitting etc. is applied. The processing is useful when you want to search in the field and support more then exact matches. When you just want to display the content you should choose a stringS type, when you want to search in the field you should choose textS.
|
I am adding content to a dynamic field but when i search for the content i can not find the document. What's wrong?
Beside the indexing part you need to configure the query part. Make sure that all relevant fields are configured as query fields:
I don't find the expected document on the first position. What can i do?
:) That's a good question. In the end, solr is a search and the sorting depends on the score, not as in a database on one or two simple criterion.
In the end solr provides a lot of settings that influence the score calculation and you need to tune the results to you needs. The following settings are helpful to tune your results.
Check your data
The quality of you data is important. Maybe a document is on the first position because, the search term is really relevant for it? Maybe it is an option to change the content?
Adjust the query field boost factors
For each query field there is a boost value after the ^ sign. To increase the factor of a single field for the whole query, you can increase the number in the query fields.
For use cases like "news are always more important then pages" or "Newer documents should be at the beginning" you can use boostFunctions (query.boostFunction) or boostQueries (query.boostQuery)
The search term only exists as a synonym
You can use the backend module synonyms (Synonyms) to maintain synonyms and configure solr to retrieve documents by a term that is not naturally inside the document.
Ask DKD support
Beside that, there are more options to tune. The DKD support can help you, to analyze and tune your search results. Call +49 (0)69 - 247 52 18-0.
Non ASCII characters like german umlauts do not work when i search, how do I fix that?
To allow search with umlauts Tomcat needs to be configured to use UTF-8 encoded urls. Go to apache-tomcat/conf/server.xml and change the URIEncoding parameter:
How can I change Solr's schema and add custom fields?
Please do not change the shipped solr schema. There are a lot of dynamic fields (Appendix - Dynamic Fields) that can be used to index any kind of datatype.
I am using varnish before my site. How can i index pages properly?
SOLR Indexer might have some issues, when the page to index is behind a Varnish Proxy. We have collected two ways of solving this issue
Bypassing varnish:
Bypass when X-Tx-Solr-Iq is present
The SOLR indexer request send the header X-Tx-Solr-Iq.
To have bypass the Varnish caching, put this into your sub vcl_recv part of the configuration
|
if (req.http.X-Tx-Solr-Iq) {
return(pipe);
}
Copied!
Using Cache-Control:
Put this into your sub vcl_fetch part of the configuration
|
if (req.http.Cache-Control ~ "no-cache") {
set beresp.ttl = 0s;
# Make sure ESI includes are processed!
esi;
set beresp.http.X-Cacheable = "NO:force-reload";
# Make sure that We remove all cache headers, so the Browser does not cache it for us!
remove beresp.http.Cache-Control;
remove beresp.http.Expires;
remove beresp.http.Last-Modified;
remove beresp.http.ETag;
remove beresp.http.Pragma;
return (deliver);
}
Copied!
I want to build the Dockerfile_full image on my mac with a local volume, how can i do that?
|
The following example shows how to build the Dockerfile image and start a container with a mapped local volume (only for the data).
This was tested with "Docker for Mac" (not Docker Toolbox). Before executing the example, make sure, that you have added " /solrdata" as allowed volume in the docker configuration.
# build the image
docker build -t typo3-solr -f Docker/SolrServer/Dockerfile .
# create volume directory locally
mkdir -p ~/solrdata
# add solr group to volume directory
sudo chown :8983 ~/solrdata
# run docker container from image with volume
docker run -d -p 127.0.0.1:8282:8983 -v ~/solrdata:/var/solr/data/data typo3-solr
Copied!
Can i index a https (SSL) site?
Yes. You need a ssl certificate (can be self signed) and change the following setting:
How can i use a configuration from AdditionalConfiguration.php when i deploy my application on several instances?
The configuration of the connection is done with typoscript. When you want to use a configuration from TYPO3_CONF_VARS or from the system environment,
you can apply an stdWrap on the configuration that reads from these configurations.
The following example shows how a host can be configured in the AdditionalConfiguration.php and used in your typoscript to connect to solr:
The following line is added to AdditionalConfiguration.php
To use this configuration for the host, you can use a TEXT element in the configuration and use override.data to use the
value from the AdditionalConfiguration.php
plugin.tx_solr.solr {
host = TEXT
host {
value = localhost
override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|host
}
}
Copied!
|
I want to index extension records, what do i need to do?
EXT:solr provides a flexible indexing for TYPO3 pages and records. You can add a custom indexing configuration for your own records with a valid TCA configuration.
The extension ships several examples in the Folder "Configuration/TypoScript/Examples", read them and try to undestand them.
EXT:solr can not know the business logic of an extension to generate a link to a detail view. You need to use typolink to build an url that points to a valid, existing detail page.
When you index records, e.g. news it these records are indexed in solr and point to a news details page. That's the reason why it makes sence to exclude the news detail page from the normal page indexing. Otherwise the indexing of this page will produce an error message, because only a url with a valid news uid produces a valid output.
|
Are in EXT:solr some cli commands available?
Yes, currently(v. 6.1) only one for initializing solr connections.
But check for new ones with bin/typo3 list command.
|
I want to overwrite the type field, why is this not possible?
The type field is a system field that EXT:solr uses to keep the system in sync. Overwritting this field might result in inconsistency.
However, if you need something like a custom type you can also write the information to a dynamic solr field and use that one as a type.
The following example shows, how to fill the field "mytype_stringS" and build a facet on this field:
I want to store HTML in solr, how can i retrieve that?
In general it is not recommend to allow html in the solr field. Especially when you index content that can be changed by the user.
However, if you want to allow html in a solr field, you need to add the field as trusted field and the content will not be escaped during the retrieval from solr.
The following example shows how to avoid html in the content field:
Note: When you allow html in the content please make sure that the usage of crop ViewHelpers or a limit of the field length does not break your markup.
I want to use two instances of the search plugin on the same page, how can i do that?
If you want to use two search plugins on the same page you can add two instances and assign a different "Plugin Namespace" in the flexform. If you want to avoid, that both plugins react on the global "q" parameter, you can disable this also in the flexform. Each instance is using the querystring from <pluginNamespace>[q] then.
How can i configure switchable templates for the results plugin?
The following example shows, how you can configure a custom switchable entry template for the Results plugin:
plugin.tx_solr {
solr {
host = TEXT
host {
value = {$plugin.tx_solr.solr.host}
override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|sites|###rootPageId###|solrhost
}
port = TEXT
port {
value = {$plugin.tx_solr.solr.port}
override.data = global:TYPO3_CONF_VARS|EXTCONF|solr|sites|###rootPageId###|solrport
}
}
}
Copied!
I want to use faceting.facets.[facetName].singleOptionMode why was it removed?
This setting belongs to the rendering and not to the facet itself. You can implement the same behaviour just with the given ViewHelpers.
The behaviour is the same, when you just call the ViewHelper s:uri.facet.setFacetItem instead of s:uri.facet.addFacetItem, which semantically just overwrites the current value.
We've added an example partial "OptionsSinglemode" that shows this behaviour. The example TypoScript template "Search - (Example) Options with singlemode (only one option at a time)" shows how to use this partial in combination with the setting "keepAllOptionsOnSelection".
I want to build a tab facet where all options remain, even with an option count of 0. How can i do that?
This can be done with the combination of several settings:
The example above changes the minimumCount to 0, the default value i 1. Setting it to zero allows to have options without any results.
The setting "keepAllFacetsOnSelection" let all facets remain and with keepAllOptionsOnSelection the options in the type facet remain.
How can i add a searchbox on every page?
In most projects you want to add a searchbox on every content page. To support this, the default EXT:solr typoscript template provides the typoscript template path "plugin.tx_solr_PiSearch_Search" that contains a configured typoscript code to render the searchbox. When you want to add that to your project in the most cases you would need to refer to a search result page.
The following example shows how you can build a typoscript lib object that configures the target page for this plugin instance:
By adding the snippet to a generic tempate you could render the searchbox on every page.
How can I index protected pages (htaccess protection)?
Protected pages can be accessed by passing the htpasswd username and password to the indexing queue.
You can set the credentials by the following configuration:
As credentials are stored as plain text, go for sure that your web server does not serve your TypoScript files publicly (protect the directory or by file endings).
If you don't want to store plain text passwords, you can configure your web server to allow access from a specific domain (see below).
If you have multiple domains to index, the webserver requires the credentials for each domain accessed by the solr indexer. The extension passes the credentials only once, so you will run into errors on a multi domain environment.
Solution: Instead of passing the credentials as shown above, configure your webserver directory protection to allow access from the solr IP:
AuthType Basic
AuthUserFile /path/to/.htpasswd
<RequireAny>
Require ip XXX.XX.XX.XX (the IP of the solr server)
Require valid-user
</RequireAny>
Copied!
Be aware, that this will allow all accesses by given IP.
How can I use different host / port configurations in Solr v10 (e.g. for local environments)?
While you could use TypoScript conditions to change the configuration for different project evironments in the past, you can now use environment variables in the config.yaml like shown below.
How can i register a custom statistic writer processor?
If the TypoScript option plugin.tx_solr.statistics = 1 is activated, a StatisticsWriterProcessor is used by EXT:solr, which writes the queries into the database.
If there are more requirements for the statistics or if you want to write the data into another system, you can register your own StatisticWriterProcessor in the ext_localconf.php of the extension:
We are happy to release EXT:solr 7.0.0. This release brings several smaller and some bigger changes
New in this release
FLUID Templating
One and a half years ago we started to implement FLUID templating for EXT:solr. This project was initially started as the addon solrfluid. Solrfuid was only available for our partners.
With EXT:solr 7.0.0 the new templating is the default templating in EXT:solr. A lot of code was added and several old stuff was removed. Since some things are conceptional different in FLUID and you also have a lot of possibilities we also dropped some parts, that can be build with fluid itself or do not make sence to do them before rendering the result in the view.
Most of the things just work like before but in the following parts we made conceptional changes (for good reasons):
No css or javascript will be added to the page automatically with the page renderer! Because the integrator wants to have control on that and TYPO3 allows to add this with TypoScript we propose to add these things via typoscript. EXT:solr offers a lot of example typoscript templates e.g. to add the default css or to add the javascript for a range facet.
The following typoscript settings have been removed because they can be implemented with FLUID:
You can add them in your project partials. If you need it just for one facet, please overwrite the render partial with facet.partialName and render the attributes different there
You can add them in your project partials. If you need it just for one facet, please overwrite the render partial with facet.partialName and render the attributes different there
This can be implemented with FLUID logic. Please check the example "Search - (Example) Options with on/off toggle" that implements that (by using the partial Facets/OptionsToggle.html)
Please use custom ViewHelpers or the cObject ViewHelper for that.
Important: The support of fluid templating would not have been possible without the financial support of all partners! If you want to support us with the implementation of features like this, please think about to join the EB 2017 or 2018. Special thanks also to Frans Saris and beech.it for working on solrfluid together!
In EXT:solr 7.0.0 the backend modules are structured into multiple backend modules. This makes the user experience in the TYPO3 backend more consistent and allows you, to give different permissions on each module.
When you login into the backend, you now have the following modules available:
Info: Gives information of your Solr system, index fields and search usage.
Core Optimization: This module can be used to maintain the synonyms and stopwords in the Apache Solr server.
Index Queue: Gives an overview on indexed records and can be used to requeue records for indexing.
Index Administration: This module can be used for administrator tasks on your solr system (clear index, index queue or reload a core)
The Query class is one of the biggest classes in EXT:solr and grown over time. This class has now been splitted into several classes.
Along with that a concept of "ParameterBuilder" has been introduced. A ParameterBuilder is responsible to build a parameter part of the query.
E.g. the Grouping ParameterBuilder is responsible to build all parameters of the solr query for the grouping.
Move FilterEncoder and FacetBuilder to Facet Package
In Solrfluid there was one folder for each facet, that contains the facet class and a parser that parsers the solr response into the facet object.
The opposite part(parse the url, build the solr query) was previously done in EXT:solr, with a FilterEncoder that was registered in the FacetRendererFactory.
Now because solrfluid and solr have been merged, this logic can also be streamlined. Every facet is now structured in a FacetPackage.
A FacetPackage describes:
Which parser should be used to parse the solr response
Which url decoder should be used to parse the EXT:solr query data
Which query builder should be used to build the faceting query part
You can also implement custom facet types by registering an on FacetPackage with the FacetRegistry.
Migration:
When you have implemented an own FacetParser for solrfluid, you should add a FacetPackage, that references a UrlDecoder and QueryBuilder.
If you have used a custom FacetParser without registring a custom facet type in EXT:solr (ApacheSolrForTypo3SolrFacetFacetRendererFactory::registerFacetType) you can just reference DefaultUrlDecorder and DefaultFacerQueryBuilder in your FacetPackage.
Before solrfluid was merged there were several parts in EXT:solr where the data was read using GeneralUtility::_GET. The drawback of this approach is that the structure of the urls is hard to change and it is not possible to have custom namespaces for each instance of a plugin.
With solrfuid a SearchRequest object was introduced. This object holds all data from the user request. Now this object is used, whenever data from the user action is read. This allows us to make the request namespace changeable. You can now add your custom plugin namespace to a search plugin instance.
As an ongoing task, we started with the migration of database queries to doctrine. Since the database is used in many parts of the extension there are still many parts open.
If you want to work on that, your help is very welcome.
By now EXT:solr had the dependency on an existing domain record. This can be a problem, when you domain is dynamic or
you need to be able to generate it.
Now you can configure a domain by the rootPageId in the TYPO3_CONF_VARS, the domain record is still used, when nothing is configured here.
There might be an approach to support this in TYPO3 Version 9 by the core and we will adopt this then.
During the implementation of this the logic to retrieve the SiteHash and get the SolrConfiguration was moved to the SiteRepository,
this requires an update of the scheduler instances because the scheduler saves a serialized task. Please run the shipped migration to
update scheduler tasks created with 6.1.x.
The following code has been removed since it is not used anymore:
Classes:
ScriptViewHelper
StyleViewHelper
AbstractSolrBackendViewHelper
StringUtility
Methods:
Util::camelize
Util::camelCaseToLowerCaseUnderscored
Util::underscoredToUpperCamelCase
Util::pageExists
Deprecated Code
Methods:
Query::setQueryFieldsFromString use setQueryFields(QueryFields::fromString('foo')) with QueryFields instead, will be removed in 8.0
Query::getQueryFieldsAsString use getQueryFields()->toString() now if needed, will be removed in 8.0
Query::setQueryField use getQueryFields()->set() now, will be removed in 8.0
Query::escape Use EscapeService::escape now, when needed
Query::addReturnField use getReturnFields()->add() now, will be removed in 8.0
Query::removeReturnField use getReturnFields()->remove() now, will be removed in 8.0
Query::getFieldList use getReturnFields()->getValues() now, will be removed in 8.0
Query::setFieldList use setReturnFields() now, will be removed in 8.0
Query::escapeMarkers not needed anymore, use your own implementation when needed
Query::setNumberOfGroups use getGrouping()->setNumberOfGroups() instead, will be removed in 8.0
Query::getNumberOfGroups use getGrouping()->getNumberOfGroups() instead, will be removed in 8.0
Query::addGroupField use getGrouping()->addField() instead, will be removed in 8.0
Query::getGroupFields use getGrouping()->getFields() instead, will be removed in 8.0
Query::addGroupSorting use getGrouping()->addSorting() instead, will be removed in 8.0
Query::getGroupSortings use getGrouping()->getSortings() instead, will be removed in 8.0
Query::addGroupQuery use getGrouping()->addQuery() instead, will be removed in 8.0
Query::getGroupQueries use getGrouping()->getQueries() instead, will be removed in 8.0
Query::setNumberOfResultsPerGroup use getGrouping()->setResultsPerGroup() instead, will be removed in 8.0
Query::getNumberOfResultsPerGroup use getGrouping()->getResultsPerGroup() instead, will be removed in 8.0
Query::setFacetFields use getFaceting()->setFields() instead, will be removed in 8.0
Query::addFacetField use getFaceting()->addField() instead, will be removed in 8.0
Query::removeFilter use getFilters()->removeByFieldName() instead, will be removed in 8.0
Query::removeFilterByKey use getFilters()->removeByName() instead, will be removed in 8.0
Query::removeFilterByValue use getFilters()->removeByValue() instead, will be removed in 8.0
Query::addFilter use getFilters()->add() instead, will be removed in 8.0
Method Arguments:
Query::setGrouping now expects the first argument to be a Grouping object, compatibility for the old argument (bool) will be dropped in 8.0
Query::setHighlighting now expects the first argument to be a Highlighting object, compatibility for the old arguments (bool, int) will be dropped in 8.0
Query::setFaceting now expects the first argument to be a Faceting object, compatibility for the old arguments (bool) will be dropped in 8.0
Hooks:
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifySearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['processSearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors for this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Andreas Lappe
Felix Eckhofer
Frans Saris
Georg Ringer
Helmut Hummel
Jonas Ulrich
Marco Bresch
Markus Friedrich
Michael Skrynski
Rafael Kähm
Rémy DANIEL
Sascha Egerer
Sebastian Hofer
Timo Hund
Also a big thanks to our partners that have joined the EB2017 program:
.hausformat
AGENTUR FRONTAG AG
amarantus - media design & conding Mario Drengner & Enrico Nemack GbR
Amedick & Sommer Neue Medien GmbH
Andrea Pausch
Animate Agentur für interaktive Medien GmbH
artig GmbH & Co. KG
b:dreizehn GmbH
BIBUS AG Group
Bitmotion GmbH
cab services ag
Causal Sarl
CHIARI GmbH
Citkomm services GmbH
clickstorm GmbH
Connecta AG
Creative360
cron IT GmbH
CYBERhouse Agentur für interaktive Kommukation GmbH
cyperfection GmbH
data-graphis GmbH
Deutsche Welthungerhilfe e.V.
Deutscher Ärzteverlag
Deutscher Volkshochschul-Verband
Die Medialen GmbH
die_schnittsteller gmbh
Dörfer engineering services
E-Magineurs
EYE Communications AG
Fachhochschule für öffentliche Verwaltung NRW Zentralverwaltung Gelsenkirchen
familie redlich AG
Fork Unstable Media GmbH
hauptsache.net GmbH
Havas Düsseldorf GmbH
Hirsch & Wölfl GmbH
Hochschule Furtwangen - IMZ Online Services
Hochschule Konstanz
Institut der deutschen Wirtschaft Köln Medien GmbH
iresults gmbh
ITK Rheinland
itl Institut für technische Literatur AG
jweiland.net
Kassenärztliche Vereinigung Rheinland-Pfalz
Kerstin Nägler Web & Social Media Beratung
Landesinstitut für Schule und Medien Berlin-Brandenburg
Leibniz Universität IT Services
Libéo
Lime Flavour GbR
LINGNER CONSULTING NEW MEDIA GMBH
LOUIS INTERNET
Maximilian Walter
MEDIA:ESSENZ
mehrwert intermediäre kommunikation GmbH
Mercedes-AMG GmbH
mlm media process management GmbH
n@work Internet Informationssystems GmbH
Netcreators
netz-haut GmbH
neuwerk interactive
Nintendo of Europe GmbH
Onedrop Solutions GmbH
Open New Media GmbH
Paints Multimedia GmbG
pixelcreation GmbH
plan2net
Pluswerk AG
polargold GmbH
punkt.de GmbH
Raiffeisen OnLine GmbH
ruhmesmeile GmbH
Rundfunk und Telekom Regulierung GmbH
Schweizer Alpen-Club SAC
sitegeist media solutions GmbH
Star Finanz-Software Entwicklung und Vertriebs GmbH
Stefan Galinski Internetdienstleistungen
Stratis - Toulon
Studio Mitte Digital Media GmbH
Studio 9 GmbH
Systime A/S
SYZYGY Deutschland GmbH
takomat Agentur GbR
THE BRETTINGHAMS GmbH
TOUMORO
Triplesense Reply GmbH
Typoheads GmbH
unternehmen online GmbH & Co. KG
Universität Bremen
VERDURE Medienteam GmbH
WACON Internet GmbH
webedit AG
Webstore GmbH
Webtech AG
wegewerk GmbH
Wohnungsbau- und Verwaltungsgesellschaft mbH Greifswald
XIMA MEDIA GmbH
zdreicom GmbH
zimmer7 GmbH
Thanks to everyone who helped in creating this release!
Outlook
In the next release we want to focus on the user experience in the backend and in the frontend. As preparation we collected several tasks.
The goal of some of them (e.g. bootstrap templating, checkbox facets, filterable options partial) is to make more things possible out of the box and make the extension more user friendly:
We are happy to release EXT:solr 8.0.0. The focus of EXT:solr 8.0.0 was, to improve the user experience in the frontend and backend.
New in this release
In the following paragraphs we want to summarize the new features that will be shipped with EXT:solr 8.0.0
New suggest
We've replaced the old jQuery UI based autosuggest with a new suggest (https://github.com/devbridge/jQuery-Autocomplete). The advanced suggest can not only show the suggestions, it can also show a configurable amount of top search results.
When the user clicks on the result, he can directly jump to the result page without opening the search results page.
Thanks:
Frans Saris and http://www.beech.it for sharing the codebase of the initial patch!
Apache Solr offers a JSON API for faceting since several versions. Starting with the options facet we've added the support to use this JSON faceting API in EXT:solr.
The support of the JSON API, in general, allows us to build new features on top of that API, that was impossible before. With the first implementation we've added the following features:
By now an option was simply the value and the count, that reflects the number of documents that belong to that option. At EXT:solr 8.0.0 we've added a TypoScript option that is called "metrics", that allows us to collect and show several metrics from documents that belong to a facet option. Examples of metrics are e.g "sum of downloads", "average price",... These metrics will be available in Option model in the FLUID template and can also be used to sort the facet options.
The following example shows an configured options facet with a configured metric:
Thanks to Jens Jacobsen and UEBERBIT for sponsoring Jens work on that feature at our code sprint.
Since we'replaced the whole internal communication from EXT:solr to Apache Solr when options facets are used we are very happy to get your feedback and bug reports when you use the options facets with EXT:solr
When you have option facets with a lot of options, it would be nice to group those options by a prefix. An example is that you group all options by the starting letter to organize them in tabs:
With EXT:solr 8 we ship the following components that allow grouping your facet options to arrange them as you need them in your template:
LabelFilterViewHelper: Can be used to filter options based on a prefix of the label.
LabelPrefixesViewHelper: Can be used to access all available prefixes of the facet options.
TypoScript example template "(Example) Options grouped by prefix" that configures a grouped facet on the author field
In the previous section, the facets get grouped by prefix to organize a large number of options. Another way that you also often see on the web is to allow to filter the options with an additional input box above the facet.
The implementation of that feature is possible just with a partial and a few JavaScript components. To simplify the integration of that feature in a project we ship
Example FLUID partial that uses the filter for options
Example JavaScript that implements the filter functionality
Example TypoScript "Search - (Example) Options filterable by option value" that uses the partials and javascript for a facet
The old templating was created with custom CSS that was shipped with the extension. Since we want to decrease the effort that is required to create a mobile search and many integrators use bootstrap.css we decided to ship bootstrap templates by default. If you want to use another framework or your own custom CSS you are still able to do that with custom templates.
Nevertheless, the mobile search in a TYPO3 introduction installation with bootstrap is much better than before and your effort to adopt it should be reduced.
In EXT:solr 7.x and below a ping request was done before each search. In EXT:solr 8.0.0 we just catch a failed search and handle the unavailability. This saves up to 30% time because we just need one HTTP request to Apache Solr instead of 2.
In the previous versions, we've introduced own backend modules that can also be used by regular TYPO3 users to perform several tasks. With EXT:solr 8.0.0 the index inspector will be moved from the common info module to our info module:
Besides the move, we also added the functionality to ReQueue a single document from the index inspector when you have permissions on the index queue module.
When you index a lot of documents you might want to create facets based on patterns that occur in the content.
The cObject SOLR_CLASSIFICATION allows you to do a lightweight classification based on regex patterns that you configure in the index configuration.
The following example shows how SOLR_CLASSIFICATION can be used to map patterns on classes that are indexed into a Solr field that could be used for faceting:
plugin.tx_solr.index.queue.pages.businessarea_stringM = SOLR_CLASSIFICATION
plugin.tx_solr.index.queue.pages.businessarea_stringM {
field = __solr_content
classes {
automotive {
patterns = car,jeep,SUV
class = automotive
}
pharma {
patterns = pharma,doc,medicine
class = pharma
}
}
}
Copied!
With the configuration above Solr documents get the value "automotive" assigned in the Solr field "businessarea_stringM" when the content contains the term "car", "jeep" or "SUV".
Thanks: Thanks to http://www.bibus.ch who sponsored the implementation of this feature.
With plugin.tx_solr.search.query.(phrase/bigramPhrase/trigramPhrase).fields you can control what is passed to Solr with the ps,ps2 and ps3 value.
With these phrase fields, you can boost documents where phrases occur in close proximity. This can be very handy when you want to tune your search in terms of relevancy.
With plugin.tx_solr.search.query.tieParameter you can now configure the tie value that is passed to Apache Solr.
This value allows you to configure the impact of low scoring fields to the overall score. 0.0 means, that only high score fields will matter, 0.99 means that all fields have the same impact
TYPO3 8 introduced Doctrine DBAL for database queries and the old API will be removed in TYPO3 9. Since we've used a lot of repositories with custom SQL queries, we had to rewrite a lot of queries.
In EXT:solr we've used the chance to restructure the SQL related code and move them to repositories whenever this was possible.
With EXT:solr 8 every usage of the old database API is removed and we are prepared in that way to be ready for TYPO3 9.
Fluent API for Queries with the QueryBuilder
Many parts of the code of EXT:solr deal with queries for Apache Solr that's no surprise :). The corresponding parts in the code especially the Query class had grown over time and reached a huge complexity.
This has several drawbacks:
It is hard to integrate new features (e.g the tiebreaker or bigram features)
TYPO3 specific logic and common Apache Solr logic is mixed and makes it hard to switch to frameworks like e.g. Solarium
The Query class does multiple things: Build the query, initialize the query from the configuration,... This could be split into multiple components.
To get better in that regards our goal is to split the Query into:
Query: Aggregate that is responsible to build the Solr query string based on the options
QueryBuilder: Builder class that is responsible to build an initialized Query object e.g. based on TypoScript configuration and user input.
With the current state the QueryBuilder does the following to build a Query from the user input:
Allow devs to compose own queries that use or ignore several aspects of EXT:solr
Simplify the switch or integration of a generic Solr API that is independent of TYPO3 (e.g. Solarium)
On the way to TYPO3 9
With EXT:solr 8.0.0 we will not officially support TYPO3 9 since it is not an LTS release! Nevertheless, we want to stay close to the TYPO3 core and allow the usage in 9 already.
By now we mainly fix Doctrine and Composer related issues and support the dropped "pageslanguageoverlay" table.
So to sum up... EXT:solr 8.0.0 will mainly support TYPO3 8 LTS and we will support TYPO3 9.x a good as we can without losing the backward compatibility to TYPO3 8 LTS.
In the long run we want to be able to use other PHP frameworks for Apache Solr e.g. solarium(http://www.solarium-project.org/). To make this possible, we
need to split the pure Solr query related logic from the TYPO3Solr specific query logic (e.g. accessFilter,...). To get a step closer into this direction, we've extracted
the logic that is required to build a TYPO3 specific Solr query into the QueryBuilder. The pure Solr related query logic remains in the Query class.
Impact:
Whenever you create or modify queries you should use the QueryBuilder class for that. In one of the next releases we will support to create solarium queries with this QueryBuilder.
Beside the query refactoring, that required to remove and change several methods, the following code has been removed:
Hooks:
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['modifySearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['processSearchResponse'] has been marked as deprecated and will be dropped in 8.0 please use a SearchResultSetProcessor registered in $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['solr']['afterSearch'] as replacement.
Deprecated Code
The following methods have been marked as deprecated and will be removed in EXT:solr 9.0.0
ApacheSolrForTypo3SolrSearch::getResultDocumentsRaw - Use the SearchResultsSet::getSearchResults now
ApacheSolrForTypo3SolrSearch::getResultDocumentsEscaped - Use the SearchResultsSet::getSearchResults now
ApacheSolrForTypo3SolrSearch::getFacetCounts - Use the SearchResultSet::getFacets now
ApacheSolrForTypo3SolrSearch::getFacetFieldOptions - Use the SearchResultSet::getFacets now
ApacheSolrForTypo3SolrSearch::getFacetQueryOptions - Use the SearchResultSet::getFacets now
ApacheSolrForTypo3SolrSearch::getFacetRangeOptions - Use the SearchResultSet::getFacets now
ApacheSolrForTypo3SolrSearch::getSpellcheckingSuggestions - Use SearchResultSet::getSpellcheckingSuggestions
ApacheSolrForTypo3SolrQuery is deprecated, use ApacheSolrForTypo3SolrDomainSearchQueryQuery now
ApacheSolrForTypo3SolrSuggestQuery is deprecated, use ApacheSolrForTypo3SolrDomainSearchQuerySuggestQuery now
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors for this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Andreas Lappe
Andri Steiner
Benni Mack
Daniel Diesenreither
Daniel Mann
Daniel Ruf
Georg Ringer
Hannes Lau
Jeffrey Nellissen
Jens Jacobsen
Marco Bresch
Marcus Schwemer
Markus Friedrich
Markus Kobligk
Markus Sommer
Nicole Cordes
Patrick Schriner
Golmann
Rafael Kähm
Sascha Egerer
Simon Schmidt
Thomas Löffler
Timo Hund
Tomas Norre Mikkelsen
Also a big thanks to our partners that have joined the EB2018 program:
Albervanderveen
Amedick & Sommer
AUSY SA
bgm Websolutions GmbH
Citkomm services GmbH
Consulting Piezunka und Schamoni - Information Technologies GmbH
This patch extends the hook PageModuleSummary displaying information about the plugins flexform settings in the page module. Now the plugin name will be displayed and linked to the edit form, similar to the default behavior.
The implementation of solrconsole required several API changes for the implementation of the commands. These changes have been added with the following pull requests:
When terms get removed because they are stopwords this might have an unwanted impact when the mm condition is evaluated. Setting mm.autoRelax to true fixes this.
The type could not be passed before, now you can pass the type to the SearchFormViewHelper as "suggestPageType" argument.
The filters have been passed as global url arguments "filter" as json encoded array, what was hard to debug and inconsistent to the other parameters. We changed the parameter to the name "additionalFilters" and each filter is passed as an array item.
Impact: If you just use the default ViewHelpers and templates you need to change nothing. When you add custom filters to the suggest you now need to pass each filter as an array item of the argument "additionalFilters"
With 8.1.0 we do not officially support TYPO3 9. You can install it on TYPO3 9.3.99 for development and we tried to fix the most important issues during the development but there are already
a few known issues:
The following methods have been marked as deprecated and will be removed in EXT:solr 9.0.0:
SearchResultSetService::getHasSearched() please use SearchResultSet::getHasSearched along with that the global template variable "hasSearched" will be removed with 9.0.0 as well.
Search::getHasSearched() please use SearchResultSet::getHasSearched instead along with that the global template variable "hasSearched" will be removed with 9.0.0 as well.
Util::isLocalizedRecord() please use TCAService::isLocalizedRecord instead
Queue::initialize() please use Queue::initializeBySiteAndIndexConfiguration instead
Queue::initializeIndexingConfigurations() please use Queue::initializeBySiteAndIndexConfiguration instead
SortingHelper::getSortFields() please use the SearchResultSet to get the parsed sorting
SortingHelper::getSortOptions() please use the SearchResultSet to get the parsed sorting
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Andreas Lappe
Benni Mack
Florian Schöppe
Frans Saris
Jens Jacobsen
Marc Bastian Heinrichs
Markus Friedrich
Michael Telgkamp
Olivier Dobberkau
Rafael Kähm
Rémy DANIEL
Thomas Löffler
Timo Hund
Thomas Hohn
Also a big thanks to our partners that have joined the EB2018 program:
4eyes GmbH
Albervanderveen
Agentur Frontal AG
AlrweNWR Internet BV
Amedick & Sommer
AUSY SA
Bibus AG
Bitmotion GmbH
bgm Websolutions GmbH
bplusd interactive GmbH
Centre de gestion de la Fonction Publique Territoriale du Nord (Siège)
Citkomm services GmbH
Consulting Piezunka und Schamoni - Information Technologies GmbH
We are happy to release EXT:solr 9.0.0. The focus of EXT:solr 9.0.0 was, to support the latest version of Apache Solr (7.6.0) and to drop the usage of the solrphpclient and use the solarium php API instead.
Important: This version is installable with TYPO3 9 LTS, but does not support all features of TYPO3 9 yet. Especially the site handling needs further development in EXT:solr to fully support it with TYPO3 9 LTS. Beside the open tasks in EXT:solr there are also parts left in the TYPO3 core (e.g. when using language fallbacks). In the next release of EXT:solr we want to improve the integration with the site management in TYPO3. Since the development budget is limited at one side and we have other project requests at the other side we could spend less time on the development of EXT:solr by the end of the year. If you want to support us please consider to sponsor us in 2019.
New in this release
Support of Apache Solr 7.6
EXT:solr 9.0.0 ships a ready to use docker container with Apache Solr 7.6. This makes new features of Apache Solr available to EXT:solr.
Replaced solrphpclient with solarium php api
For the communication between EXT:solr we've used the solrphpclient library. This library was not maintained anymore and had several custom modifications. Therefore we made the decision to move to the solarium php api.
This brings us the following advantages:
Use a common, robust, maintained library
Join the forces with other PHP projects to improve solarium and benefit from that
The migration to solarium required several changes in EXT:solr and all add-on's and we will provide compabtility releases for them as well.
With the move to solarium we donated some parts to the solarium API (e.g. the solr core handling). This allows us to remove some redundant logic in EXT:solr in the future.
Thanks:
Thanks to the whole solarium team (https://github.com/orgs/solariumphp/people) and Markus Kalkbrenner for the work on solarium and the support during the integration into EXT:solr
By now we use the Queries and Httpclient of solarium, but not the domain classes because this requires additional changes in EXT:solr and solarium.
In the future we want to get rid of redundant code and use the API where we can and it makes sence and support solarium with the features that we need for EXT:solr.
TYPO3 9 compatibility
The current release is installable and useable with TYPO3 9 LTS but not all features are supported.
Currently it is supported to:
Use EXT:solr with sites that do have a domain record or domain configuration from EXT:solr
The following parts require additional work and are not supported:
SiteHandling
Extensionscanner proofed
Since a backward compatibility to TYPO3 8.7 LTS make it harder to support the previous mentioned topics. We will drop the support for TYPO3 8 LTS
in the next version and improve the support of those TYPO3 9 LTS features.
Nevertheless a lot of work was allready done for the basic support of TYPO3 9 LTS in the following pull requests:
Support to differ between read and write connections
By now each site had one solr connection for reading and writing. In most of the cases this good enough when you want to index and search in the same core.
Some setups require a more flexible approach:
E.g. when you want to clean a core and re-index that data your index is not complete for some time on the live site
When you want to do a master/slave setup for performance reasons(e.g. by using a slave node on the web server) this was not possible by now
With a separation of read and write connections this is now possible. With these building blocks you could e.g.
Index into a shadow core (that is the write core) and swap read and write core when your re-index is done
Install a slave server on your frontend server and index into a dedicated master node that act's as a solr master server
Allow to configure additionalExcludeTags for option facets
When you want to exclude facets from the counts of another facets, Apache Solr uses tags and excludeTags to realize that.
With the setting `additionalExcludeTags` you can add custom exclude tags for a facet and `addFieldAsTag` allows you, to force the creation of a tag for a certain facet.
Thanks to Marc Bastian Heinrichs for creating a patch for that and to in2code for paying for the finalization and documentation.
We ship Apache Solr 7.5.0, you need to install that Version with our configSet.
The argument "hasSearched" was removed from the searchAction and is no longer passed. You can now retrieve this
information by calling "SearchResultSet::getHasSearch" or "{resultSet.hasSearched}" in the FLUID template.
When you access this argument in your FLUID Template, you need to change that as well.
EXT:solr 9 differs between read and write connections now. As fallback the old configuration is still supported and used for reading and writing.
Nevertheless you need to re-initialize the solr connections that the data in the registry is rewritten. If you want to make use of the new configuration
you can configure the connections like that:
The following code parts have been removed as announced in previous versions of EXT:solr:
SearchResultSetService::getHasSearched Please use SearchResultSet::getHasSearched now
SortingHelper::getSortFields
SortingHelper::getSortOptions
Queue::initialize
Queue::initializeIndexingConfigurations
Search::hasSearched
Search::getResultDocumentsRaw
Search::getResultDocumentsEscaped
Search::getFacetCounts
Search::getFacetFieldOptions
Search::getFacetQueryOptions
Search::getFacetRangeOptions
Search::getSpellcheckingSuggestions
Util::isLocalizedRecord
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Aljoscha Weber
Benni Mack
Felix Nagel
Florian Wessels
Helmut Hummel
Jens Jacobsen
Marc Bastian Heinrichs
Patrick Gaumond
Rafael Kähm
Sasche Egerer
Thomas Hohn
Timo Hund
Also a big thanks to our partners that have joined the EB2019 program:
Amedick & Sommer Neue Medien GmbH
BIBUS AG Group
Bitmotion GmbH
CS2 AG
Gernot Leitgab
Getdesigned GmbH
Hirsch & Wölfl GmbH
ITK Rheinland
Kassenärztliche Vereinigung Bayerns (KZVB)
TOUMORO
Ueberbit Gmbh
XIMA MEDIA GmbH
b13 GmbH
bgm business websolutions GmbH & Co KG
datamints GmbH
medien.de mde GmbH
mehrwert intermediale kommunikation GmbH
mellowmessage GmbH
plan2net GmbH
punkt.de GmbH
Special thanks to our premium EB 2019 partners:
jweiland.net
sitegeist media solutions GmbH
In addition i want to thank Markus Kalkbrenner and the whole solarium team for the support.
Thanks to everyone who helped in creating this release!
Outlook
In the next release we will drop the support of TYPO3 8 and focus on the integration into TYPO39. Depending on the funding we would like to support
the integration into the TYPO3 site management and want to allow to configure you Solr site with the TYPO3 site management module.
With the move to the solarium php api, we take the first step of the integration. In the next releases we want to use more parts of the solarium API and also contribute to that API to share the improvements with other PHP projects.
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
We are happy to release EXT:solr 10.0.0.
The focus of EXT:solr 10.0.0 was, to support the site handling that was introduced in TYPO3 9 LTS .
Important: This version is installable with TYPO3 9 LTS only.
Since the development budget is limited at one side and we have other project requests at the other side we could spend less time on the development of EXT:solr.
If you want to support us please consider to sponsor us.
New in this release
TYPO3 9 LTS site handling support
The major new feature of EXT:solr 10 is the support of the TYPO3 site handling. You can now configure the Apache Solr connections along with your TYPO3 site.
To get a typical solr site running you now need only a few settings and can use the site handling user interface for that.
Apache Solr 8.2 support
EXT:solr 10 ships Apache Solr 8.2.
Note: The data volume of the Apache Solr Docker container was changed from `/opt/solr/server/solr/data` to `/var/solr/data/data` this might require changes in your infrastructure configuration.
Extensionscanner cleanup
Most (not all) of the extension scanner violations have been fixed. We hope to get the extension scanner violations cleaned in the next releases.
Drop TYPO3 8 compatibility
To simplify the development we've dropped the compatibility for TYPO3 8 LTS. If you need to use TYPO3 8 please use the 9.0.x branch.
Add default field for image and price
To allow external applications to index common information for product's we've added a field for price and image. Along with that we've changed the suggest to render the content of the "image" field instead of "previewImage_stringS", this might require changes in your index configuration.
Migration from EXT:solr 9 to EXT:solr 10
Important things on sites with activated Site Handling
By default EXT:solr 10 expects the configuration of solr connections and cores in the site handling module, along with your TYPO3 site.
The configuration of the solr connections with your site are available immediatly, so now initialization of connections is required anymore.
The old fashioned setup (domain records and solr connections in TypoScript) are now the legacy mode. If you want or need to use that still, you can enable the legacy mode,
with your extension configuration by setting `allowLegacySiteMode = 1`.
The following things will become standard, and should be preferred and activated/configured as close as possible and in some cases immediately,
otherwise the things wil not work or break the whole setup:
TypoScript
plugin.tx_solr.solr
This TypoScript configurations for Constants and for Setup are at least partially obsolete and are ignored on Site Handling activated sites.
All Apache Solr connections must be stored in Site Handling "config.yaml" file for each language.
Important: The goal of the configuration with your TYPO3 site and the site module UI was to simplify the setup, especially for new users. Therefore, not all options are still possible as before, but the most of them are still possible by editing the yaml file.
There are the following known and wanted limitations:
It is not possible to configure a username and a password for the solr
server with the UI. You still have the possibility to add that configuration
in the yaml file, in that case make sure that this file is not readable from
the web!. Another approach is to configure the usage from the environment
variables. To configure a username the setting solr_username_read or
solr_password_read can be used in the yaml file.
It is not possible to configure a different solr hostname with the UI.
If you need that you can still configured that in the yaml file, by using the
fallback mechanism.
The fallback mechanism work like that:
Each setting has the following structure solr_{$setting}_{$scope}"`. The scope can be read or write. Every setting can be overwritten for the scope write, if nothing is configured it will fallback to the read setting. Every setting can be defined on the language level, if it is not configured on the language level it fallsback to the global setting.
The example above shows that you are able to define the setting
solr_host_read on the language level. Since this is a more advanced
configuration and the user interface should be kept simple, this can only be
configured in the yaml.
Removed Code
The following code parts have been removed as announced in previous versions of EXT:solr:
TBD
Contributors
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Achim Fritz
Georg Ringer
Helmut Hummel
Marc Bastian Heinrichs
Marco Pfeiffer
Markus Kobligk
Netcoop
Nicole Cordes
Rafael Kähm
Rémy DANIEL
Sascha Egerer
Stefan Frömken
Stephan Jorek
Timo Hund
Yann Weyer
Gerald Aistleitner
Also a big thanks to our partners that have joined the EB2019 program:
21TORR GmbH
3m5, Media GmbH
Absolut Research GmbH
AgenturWebfox GmbH
Amedick & Sommer Neue Medien GmbH
arndtteunissen GmbH
Arrabiata Solutions GmbH
artif GmbH & Co. KG
Atol Conseils & Développements
b13 GmbH
bgm business websolutions GmbH & Co KG
Bitmotion GmbH
BIBUS AG Group
Bitmotion GmbH
Columbus Interactive GmbH
Consulting Piezunka und Schamoni - Information Technologies GmbH
cosmoblonde GmbH
CS2 AG
datamints GmbH
Diesel Technic AG
Die Medialen GmbH
Direction des Systèmes d’Information - Département du Morbihan
dörler engineering services
E-Magineurs
Fachhochschule für öffentliche Verwaltung NRW Zentralverwaltung
fixpunkt werbeagentur gmbh
Flowd GmbH
Frequentis Comsoft GmbH
GAYA - La Nouvelle Agence
Gernot Leitgab
Getdesigned GmbH
.hausformat GmbH
Haute école de travail social et de la santé - EESP
Hirsch & Wölfl GmbH
Hochschule Furtwangen
Hypo Tirol Bank AG
Intera Gesellschaft für Software-Entwicklung mbH
interactive tools GmbH - Agentur für digitale Medien
internezzo ag
iresults gmbh
ITK Rheinland
LOUIS INTERNET GmbH
Kassenärztliche Vereinigung Bayerns (KZVB)
KONVERTO AG
kraftwerk Agentur für neue Kommunikation GmbH
Landesinstitut für Schule und Medien Berlin-Brandenburg
Libéo
LINGNER CONSULTING NEW MEDIA GMBH
MaxServ B.V.
McLicense GmbH
MeinEinkauf AG
NEW.EGO GmbH
medien.de mde GmbH
mehrwert intermediale kommunikation GmbH
mellowmessage GmbH
mentronic . Digitale Kommunikation
MOSAIQ GmbH
pietzpluswild GmbH
plan2net GmbH
plan.net - agence conseil en stratégies digitales
Proud Nerds
+Pluswerk AG
punkt.de GmbH
Redkiwi
ressourcenmangel dresden GmbH
rrdata
RKW Rationalisierungs- und Innovationszentrum der Deutschen Wirtschaft e.V.
Site’nGo
SIWA Online GmbH
Stadt Wien - Wiener Wohnen Kundenservice GmbH
Stadtverwaltung Villingen-Schwenningen
Stefan Galinski Internetdienstleistungen
Studio Mitte Digital Media GmbH
TOUMORO
Ueberbit Gmbh
WACON Internet GmbH
webconsulting business services gmbh
webschuppen GmbH
Webstobe GmbH
webit! Gesellschaft für neue Medien mbH
wegewerk GmbH
werkraum Digitalmanufaktur GmbH
XIMA MEDIA GmbH
Special thanks to our premium EB 2019 partners:
jweiland.net
sitegeist media solutions GmbH
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
Compatibility with Apache Solr 9.4.1 is checked and EXT:solr now accepts the following Apache Solr versions:
- 9.2.0
- 9.2.1
- 9.3.0
- 9.4.0
- 9.4.1
To avoid breaking changes EXT:solr 11.0.9 doesn't require Solr 9.4.1, but it's strongly recommended to use the latest supported version.
Note that due to CVE-2023-50290 you should at least check your configuration or update to at least Apache Solr 9.3.0.
Small improvements and bugfixes
[TASK:BP:11.0] Prepend wrong fe language on empty cache by @goldi42
[BUGFIX:BP:11.0] Fix connection initialization by @dkd-friedrich in #10
[BUGFIX] Handle float values in options facet parser by @dkd-kaehm in #12
[BUGFIX:11.0] Exception with tx_solr_statistics after latest TYPO3 security update by @dkd-kaehm in #13
[TASK] Remove unused field cookie in tx_solr_statistics by @dkd-kaehm in #22
[TASK] Allow Apache Solr 9.4 by @dkd-friedrich in #25
[BUGFIX] Fix result highlighting fragment size by @dkd-friedrich in #26
Apache Solr for TYPO3 11.0.8
We are happy to release EXT:solr 11.0.8 maintenance release, this release is a non-public ELTS release.
Support of Apache Solr 9.2.0
Apache Solr 8.5.1 is outdated, thus we now provide support of Apache Solr 9.2. Technically an update is not required, but recommended to avoid possible
security issues in no longer supported Apache Solr versions. If you can't update your Apache Solr server right now, EXT:solr 11.0.8 should still work with 8.5,
but you'll see warnings in the reports module.
Note: With Apache Solr 9 the following components are no longer available and you have to adapt the configuration if needed. No longer available components are:
1) Data Import Handler (DIH)
DIH is an independent project now; it is no longer a part of Solr
2) VelocityResponseWriter
VelocityResponseWriter is an independent project now; it is no longer a part of Solr. This encompasses all previously included /browse and wt=velocity examples.
Small improvements and bugfixes
[TASK:Security:P:ELTS_9.5] Update jQuery and its plugin libs by @dkd-kaehm in #3
[BUGFIX:BP:11.0] Respect indexingPriority in QueueItemRepository by @dkd-friedrich in #5
[TASK] Integrate in packagist.com : conflict with non ELTS EXTsolr by @dkd-kaehm in #4
[BUGFIX:BP:11.0] Sanitize frequent and last searches output by @dkd-friedrich in #6
[BUGFIX] Fix TypeError in StatisticsWriterProcessor by @dkd-friedrich in #6
[TASK] Remove unneeded GitHub actions by @dkd-friedrich in #6
[TASK] Remove unneeded TYPO3 version from matrix by @dkd-friedrich in #6
[BUGFIX] Try to solve conflicts issue on composer on CI by @dkd-kaehm in #7
[TASK] Prepare configsets for 11.0 ELTS by @dkd-friedrich in #8
[BUGFIX:BP:11.0] Fix expected variant results by @dkd-friedrich in #8
[TASK] Update version matrix by @dkd-friedrich in #8
Apache Solr for TYPO3 11.0.7
TYPO3 9 LTS reached the ELTS stage: free community support for TYPO3 9 LTS ended on 30 sept. 2021.
We'll join the TYPO3s ELTS regiment and provide EXT:solr support for TYPO3 9 ELTS upwardly via our EB program.
Therefore the EXT:solr release-11.0.x will not be maintained in TYPO3-Solr/ext-solr repository any more. The maintenance and builds will be moved to other place.
The new EXT:solr 11.0.8+ for TYPO3 9 ELTS versions will be provided via dkds EB program.
Apache Solr for TYPO3 11.0.6
This is a bugfix-only release that contains only bugfixes
This is a bugfix-only release that contains:
[BUGFIX:BP:11-0] Respect TCA setting of 'tstamp' field (#3037)
[DOCS] Use *_PORT variable for setting the port fca6f68 (#2759)
[BUGFIX] Correct Content-Type header for suggest response e843b44 (#2783)
[TASK] Change configuration files to TYPO3 file extensions 6d513e6 (#2813)
[BUGFIX] garbage collector on translations 9adcc40 (#2797)
[BUGFIX] Quote field within score calculation 3969340 (#2824)
[BUGFIX] disabled Solr Sites e7bc3ab (#2795)
[BUGFIX] Enable unicode when fetching pages eb33376 (#2810)
[TASK] Disable cache time information for ajax request f54213f (#2834)
[TASK] Adjust composer TYPO3 version constrains for EXT:Solr 11.0.4+ 50df86a (#2844)
[FEATURE] Allow stdWrap on sorting label 5f2cee2 (#2339)
[BUGFIX] Fix handling of case sensitive variant ids (#2865)
[FEATURE] Store number of existing variants 9c88401 (#2870)
[BUGFIX] Function call with non existing variable 0a69d45 (#2842 / #2520)
[BUGFIX:BACKPORT:11] Fix missing variant field value 8e0c648 (#2878)
[BUGFIX] Exception on Cached state of TranslateViewHelper 1765751 (#2830)
[BUGFIX] Check if $recordUid is non-numeric before substitution a9cf555 (#2836)
[TASK] Remove usages of Prophet by all occurrences within TYPO3 API 3bbf25a, 45b1703, 4f2b37a (#2862)
[TASK] Remove TYPO3 long time ago deprecated cache class 79cafe9 (#2782)
[BUGFIX] Change filter for workspace 5408889 (#2847)
[BUGFIX] Use Iconfactory to retrieve record icons fa77962 (#2900)
[BUGFIX] Language overlay for records is not retrieved since solr Version 11.x (#2788)
[BUGFIX] Temporary free mode fix d5e936b
[BUGFIX] Content id in language aspect c84ce1b
[BUGFIX] Language aspect for indexer 9af09f3
[BUGFIX] remove escaping on suggestion prefix f70829e (#2917)
[FEATURE] Exclude sub entries of page/storage recursively 4151a25 (#2934)
[BUGFIX] Make relevance sorting option markable as active bc813c8 (#2922)
Apache Solr for TYPO3 11.0.3
This is a bugfix-only release that contains only a few bugfixes
[TASK] Use minor version of solr docker image (#2740)
[BUGFIX] Make sure HtmlContentExtractor::cleanContent() is UTF-8 safe (#2514)
[BUGFIX] Database exception in RecordMonitor for records no-"enable" columns (#2512)
[BUGFIX] Indexing of records fails with solr 10.x (#2521)
[BUGFIX] Hard codes plugin namespace (#2732)
[BUGFIX] Restricted pages are not being indexed in Typo3 10 (#2695)
[BUGFIX] Prevent duplicate urls for page 0 (#2718)
[BUGFIX] Fix assignment for page uid variable (#2664)
[BUGFIX] Use num_found in static db table (#2668)
[BUGFIX] Build core base path right, when path is slash only (#2692)
[BUGFIX] Fix missing renderType attribute in flexform for search plugin (#2669)
[BUGFIX] Add option to override 'port' in frontend indexing URL (#2618)
[BUGFIX] Reset uriBuilder before building a new uri (#2658)
[DOCS] Multiple improvements to the docs
Apache Solr for TYPO3 11.0.2
This is a bugfix-only release that contains only a few bugfixes
[TASK] Add warning in the docs that a fqdn is required for the sitehandling
[BUGFIX] Re-enable Integration Tests for TYPO3 v10
[BUGFIX] Fix unit tests with new controller context check
[BUGFIX] Fix tests and add groups for tests
[BUGFIX] Remove mocks in TYPO3 v10 Integration tests
[BUGFIX] Remove unneeded constant
[BUGFIX] Fix travis.yml to use correct stable versions
[BUGFIX] Ensure to hand in PSR-7 Request to TSFE->getPageAndRootlineWithDomain
[BUGFIX] Remove unneeded is_siteroot flag in nested storage folder
[BUGFIX] Always return array on non-mounted sites
[BUGFIX] Fix multiple rootpages in nested sites
[BUGFIX] Prevent SiteNotFoundException in reports module
Apache Solr for TYPO3 11.0.1
This is a bugfix-only release that contains only a few bugfixes
[BUGFIX] Fix documentation and Versionmatrix
[BUGFIX] Fix failing build on docker hub
Apache Solr for TYPO3 11.0.0
We are happy to release EXT:solr 11.0.0.
The focus of this release was the support of TYPO3 10 LTS.
Important: This version is installable with TYPO3 9 and 10 LTS. For TYPO3 9 LTS at least version 9.5.16 is required.
EXT:solr 11 requires the usage of the TYPO3 site handling for the configuration of solr.
The `legacyMode` that allows the usage of domain records and configuration of solr cores in TypoScript was dropped with EXT:solr 11.
New in this release
Support of TYPO3 10 LTS
With EXT:solr 11 we provide the support of TYPO3 10 LTS. If you update to EXT:solr 11, make sure, that you are using the TYPO3 site management to manage your Apache Solr endpoints.
Thanks to: Achim Fritz & b13 for the support on that topic
Support of Apache Solr 8.5.1
With EXT:solr 11 we support Apache Solr 8.5.1, the latest release of Apache Solr.
To see what was changed in Apache Solr 8.5.x please read the release notes of Apache Solr:
We are happy to release EXT:solr 11.1.0.
The focus of this release has been on URL and SEO optimizations.
Important: This version is installable with TYPO3 10 LTS only and contains some breaking changes, see details below.
New in this release
Route enhancers
Introduce the TYPO3s route enhancer functionality for facets.
This feature allows to mask facets inside the query string or as part of the path segment.
Associative keys for tx_solr[filter] facet URL parameters
Introduce a new style how the facet array represented inside of the url with a combination of key and value in order to be able to keep a specific order.
This feature allows to change the url parameters from index based to associative keys for facets.
Using associative keys, the value of a facet will be replaced with 1. A value of 1 means, that the facet is active. A value of 0 means, the value is inactive.
Additionaly a new sort option for url parameters is available. The sort of parameters is mandatory for associative keys.
By default Solr behaves as before and will be changed in future releases.
To improve the search behaviour we introduce folding filters, e.g. allowing to skip accents in search terms. The following languages are
now using the ASCII folding filter:
dutch
english
finish
french
german
hungarian
irish
italian
polish
portuguese
serbian (for fields that don't include the Serbian Normalization Filter)
spanish
turkish
For the Scandinavian languages, Norwegian, Swedish and Danish, a similiar approach is used, but we're using the more specialized Scandinavian Normalization
and Scandinavian Folding Filters.
Folding process usally takes place at a late stage, so your configurations shouldn't be affected. But for the Scandinavian languages the Scandinavian Normalization
Filter processes the terms earlier, so your protected words for the Snowball Porter Filter, e.g. danish/protwords.txt, might be affected, please be sure to use the
right spelling (see https://solr.apache.org/guide/8_8/language-analysis.html#scandinavian-normalization-filter).
EXT:solrs components like range facets can not be properly handled by cHash stack, because the amount of possible range-combinations is infinite, therefore they must be excluded from cHash calculation.
This change makes it possible to exclude all EXT:solr parameters from cache hash. To prevent misconfigurations, the new extension configuration setting "pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters. This setting makes it impossible to chose invalid/unhandled EXT:solr plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)
Please follow the following migration instructions
Plugin namespaces:
Needed only if other as default (tx_solr) plugin namespace is used in instance. Add the used namespace[s] to $GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] or via backend
"Settings" -> "Extension Configuration" -> "solr" -> "A list of white listed plugin namespaces"
Global q parameter:
Needed only if global "q" parameter without plugin namespace is used and wants to be included in cache hash calculation. Set the setting $GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] = '1'
or enable it via backend "Settings" -> "Extension Configuration" -> "solr" -> "Include/Exclude global q parameter in/from cacheHash"
Makes it possible to manage resources per core instead of previously used per schema approach.
Now is it possible to maintain the stopwords and synonyms for each core/site separately,
and avoid mixing the synonyms between sites using same core.
!!! All earlier 11.2.x releases stay vulnerable and will not be re-published
11.2.8 is the only release of this branch that fixes the vulnerabilities listed below.
Releases 11.2.0 to 11.2.7 remain affected and will not be re-published with a fix,
so there is no patched 11.2.7 or earlier to move to — upgrade to 11.2.8.
!!! Solarium raised to 6.2.8, PHP 7.2 no longer supported
solarium/solarium is raised from 6.0.4 to 6.2.8, and the minimum PHP version from 7.2 to 7.3.
This branch needs it to enforce the Unified Highlighter fix below in PHP at all: earlier Solarium
versions ship no HighlightingInterface, no setOffsetSource() and no setFragsizeIsMinimum().
Composer resolves this on update; installations that pin solarium/solarium themselves must allow
6.2.8, and PHP 7.2 is no longer supported.
!!! Security: close FVH FieldExistsQuery HTTP 500 oracle (CVE-2026-56096)
A stand-alone field:* query was forwarded through the FastVector highlighter (FVH); Lucene rewrote
it to FieldExistsQuery and FVH crashed with HTTP 500, turning the response into a field-existence
oracle on the indexed schema.
The highlighter is switched to the Unified Highlighter with hl.offsetSource=ANALYSIS,
hl.bs.type=WORD and hl.fragsizeIsMinimum=false, closing the oracle while keeping the same soft
fragment-size behaviour.
!!! Recommendation: align existing Solr volumes with the new configset
The ext_solr_11_2_0_elts configset now sets the Unified Highlighter as default on both the
/select and /browse request handlers. Solr volumes created from older configsets default to
the legacy highlighter and remain vulnerable to the FieldExistsQuery HTTP 500 oracle when queried
directly (bypassing EXT:solr). Run the bundled migration script against the existing configset to
align the defaults; the script is idempotent and writes a solrconfig.xml.Backup-SST-235567 backup
next to the modified file:
EXT:solr itself enforces the Unified Highlighter unconditionally in PHP, so this configset alignment
is a defence-in-depth measure for clients that query Solr directly.
!!! New: TypoScript settings for query-syntax handling
Two new TypoScript settings govern how user input on tx_solr[q] is parsed:
plugin.tx_solr.search.query.userFields — whitelist of fields a Solr field-selector (field:value) may target.
By default derived from query.queryFields; selectors against other fields are now treated as literal terms and silently miss.
Sites that rely on selectors against non-qf fields must extend the whitelist via a scalar override or the add / remove sub-keys.
plugin.tx_solr.search.query.allowSolrOperatorSyntax — toggle for operator-syntax passthrough.
Default 1 keeps the documented + - && || ! * ? UX functional; set to 0 for strict mode (additionally escapes | & ;).
Selector, range and grouping characters (: [ ] ( ) { } ^ " ~ \ /) are always escaped regardless.
!!! Not applicable: CVE-2026-56095 (insecure deserialization in the cObj indexer pipeline)
This CVE (SST #2026011610000017), fixed on other branches by migrating the SOLR_MULTIVALUE /
SOLR_RELATION / SOLR_CLASSIFICATION content objects from serialize()/unserialize() to
json_encode()/json_decode(), does not apply to 11.2.x and was deliberately not ported here.
The underlying vulnerability is a regression introduced by a later commit that made the indexer's
unserialize() call unconditional on every cObj output. That commit was never applied to this branch:
AbstractIndexer::isSerializedValue() here still gates unserialize() behind either a registered
detectSerializedValue hook (opt-in, and no known extension in the wild registers one) or one of the
three built-in multi-value cObjs — whose own serialize() output only ever nests attacker-influenced
bytes as a plain string inside an array, which plain unserialize() does not recursively re-parse
into an object. Without the regression, there is no reachable PHP object-injection sink to fix here.
Applying the JSON migration anyway would have been a needless breaking change (third-party content
objects returning serialize($array) would stop working) for a version this old with no corresponding
vulnerability, so it was left out of this release.
!!! Security: additionalFilters can no longer preempt the siteHash filter
Request-provided tx_solr[additionalFilters] could register a named siteHash filter that AbstractQueryBuilder::useFilter() refused to overwrite,
so the system siteHash filter added later by AccessComponent was dropped.
In a shared-Solr-core multi-site installation this let an anonymous visitor of one site read public documents of another site sharing the same core (CVE-2026-56094).
EXT:solr now strips the reserved filter names siteHash and access from request-provided additionalFilters before they reach the query,
and applies the system siteHash filter with remove-then-set semantics so request input can no longer preempt it.
Filters that integrators set server-side — TypoScript plugin.tx_solr.search.query.filter. or plugin/FlexForm — are unaffected.
Impact for integrators: a frontend request can no longer override siteHash (or access) through tx_solr[additionalFilters].
Cross-site search must be configured server-side via plugin.tx_solr.search.query.allowedSites as documented in tx_solr.search.
!!! Security: detailAction enforces siteHash and access-group filters (CVE-2026-56093)
SearchResultSetService::getDocumentById() built the by-id lookup query used by the cacheable
detail action directly, without running any search component, so neither the siteHash nor the
frontend user access-group filter that AccessComponent applies on the normal search path ever
applied to it. An anonymous visitor who obtained a document's id — publicly exposed via
data-document-id in the default templates — could fetch an access-restricted or cross-site
document's full title, content and metadata through the detail view.
getDocumentById() now applies AccessComponent to the lookup query before executing it, so a
restricted or foreign-site document is excluded the same way it already is from search results.
detailAction responds with a plain 404 status when the lookup then finds nothing, indistinguishable
from a genuinely unknown documentId.
During an indexer sub-request, UserGroupDetector forged empty fe_group/extendToSubpages
values onto every pages record it saw, via a getPageOverlay hook that TYPO3's
RootlineUtility persisted into the shared, persistent rootline cache for translated page
views. A subsequent anonymous visitor of a page that inherits its access restriction from an
ancestor via extendToSubpages could read that cache entry and reach an otherwise
access-restricted page tree.
The forging hook is removed; the indexer's own access bypass (needed so it can read restricted
records at all) continues to work through its other, non-cached hooks.
!!! Last release for TYPO3 10.4 ELTS
This is the last planned release on this branch. EXT:solr no longer supports TYPO3 10.4 ELTS
after 11.2.8.
All Changes
[SECURITY] Fix CVE-2026-56096 — close FVH FieldExistsQuery HTTP 500 oracle by @dkd-kaehm in #4777
[SECURITY] Fix CVE-2026-56092 — Stop rootline cache poisoning via forged fe_group/extendToSubpages by @dkd-kaehm in #4777
[SECURITY] Fix CVE-2026-56093 — Enforce siteHash and access filters in detailAction lookup by @dkd-kaehm in #4777
[SECURITY] Fix CVE-2026-56094 — Prevent request additionalFilters from preempting siteHash filter by @dkd-kaehm in #4777
[FEATURE] Add HEALTHCHECK to Dockerfile by @dkd-kaehm in #4777
[BUGFIX] Ignore an emptied filter section in the plugin FlexForm by @dkd-kaehm in #4790
Apache Solr for TYPO3 11.2.7 ELTS
This is a non-public a security release for TYPO3 10.4 ELTS.
!!! Upgrade to Apache Solr 9.10.1
Apache Solr 9.10.1 fixes several security issues, please upgrade your Apache Solr instance!
CVE-2025-54988: Apache Solr extraction module vulnerable to XXE attacks via XFA content in PDFs
CVE-2026-22444: Apache Solr: Insufficient file-access checking in standalone core-creation requests
CVE-2026-22022: Apache Solr: Unauthorized bypass of certain "predefined permission" rules in the RuleBasedAuthorizationPlugin
Apache Solr for TYPO3 11.2.6 ELTS
This is a non-public maintenance release for TYPO3 10.4 ELTS, containing:
[FEATURE] Add arm64 platforms to docker-images and push to registry.dkd.de by Rafael Kähm (9a56f004a)
[TASK] Allow Apache Solr 9.8.1 by Rafael Kähm (d6d6de2c7)
[TASK] Set proper retention-days on actions/upload-artifact by Rafael Kähm (96b119381)
Revert "[TASK] Integrate TYPO3 10.4 ELTS" by Rafael Kähm (20a596125)
Apache Solr for TYPO3 11.2.5 ELTS
This is a non-public security release for TYPO3 10.4 ELTS, containing:
!!![SECURITY] Update to Apache Solr 9.8.0 : CVE-2025-24814
Updates EXT:solr to Apache Solr 9.8.0.
Apache Solr 9.8.0 disables the possibility to load the jar files with lib directive by default,
which was used to load jar files within the EXT:solr configsets. Apache Solr 10.0.0 will drop that functionality.
All Apache Solr libs, modules or plugins must be configured within the main server configuration files.
See: https://issues.apache.org/jira/browse/SOLR-16781
Impact:
Docker
You can wipe the volume and start the container with v. 11.2.5+ image, but that method will wipe the index as well.
See the script EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh
Other server setups
You have 2 possibilities to fix that issue in your Apache Solr Server:
(PREFERRED) Migrate the EXT:solr's Apache Solr configuration
Or do that in other ways to set the solr.config.lib.enabled=true to sys-props of Apache Solr Server.
!!![FIX] Docker execution order issue for as-sudo tweaks
This change renames the file
from /docker-entrypoint-initdb.d/as-sudo-tweaks.sh
to /docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh
and moves the folder
from /docker-entrypoint-initdb.d/as-sudo/
to /docker-entrypoint-initdb.d-as-sudo/
to fix the execution order issue when setting the correct file permissions
when starting the docker container, leading to a Operation not permitted errors.
/docker-entrypoint-initdb.d/0_as-sudo-tweaks.sh instead of
/docker-entrypoint-initdb.d/as-sudo-tweaks.sh
/docker-entrypoint-initdb.d-as-sudo instead of
/docker-entrypoint-initdb.d/as-sudo/
Upgrade to Apache Solr 9.7.0
This release requires Apache Solr v9.7.0+.
Along with the compatibility to Solr 9.7 the dependency to SOLR_ENABLE_STREAM_BODY is removed.
Minor changes & bugfixes
[TASK] Update GitHub actions by @dkd-friedrich in #42
[DOCS] Improve Solr core creation via API and other deployment parts by @dkd-kaehm & @dkd-friedrich in #42
[TASK] Use relative path to typo3lib in Apache Solr config by @dkd-kaehm & @dkd-friedrich in #42
[BUGFIX] Docker twaks as-sudo do not preserve the Docker image ENV by @dkd-kaehm & @dkd-friedrich in #42
[BUGFIX] Docker tests suite does not contain all logs by @dkd-kaehm & @dkd-friedrich in #42
[BUGFIX] docker image tests do not fail if core can not start by @dkd-kaehm & @dkd-friedrich in #42
Apache Solr for TYPO3 11.2.4 ELTS
Support of Apache Solr 9.5.0
Compatibility with Apache Solr 9.5.0 is checked and EXT:solr now accepts the following Apache Solr versions:
- 9.3.0
- 9.4.0
- 9.4.1
- 9.5.0
Small improvements and bugfixes
[BUGFIX] Handle float values in options facet parser by @dkd-kaehm in #11
[BUGFIX:11.2] Exception with tx_solr_statistics after latest TYPO3 security update by @dkd-kaehm in #14
!!![TASK] Update to Apache Solr 9.2 for TYPO3 10.4 ELTS by @dkd-kaehm in #19
[TASK] Remove unused field cookie in tx_solr_statistics by @dkd-kaehm in #21
[BUGFIX] Fix autosuggest with non-ascii terms by @dkd-friedrich in #28
[TASK] Prepare release-11.2.x ELTS by @dkd-kaehm in #17
[BUGFIX] Fix branch alias by @dkd-friedrich in #30
[TASK] Allow custom ELTS repository by @dkd-friedrich in #30
[BUGFIX:P:11.2] prevent undefined array key warning if filter is empty by Achim Fritz in #32
[TASK] Allow Apache Solr 9.5 by @dkd-friedrich in #31
Apache Solr for TYPO3 11.2.3 - Last non ELTS release
This is a maintenance release for TYPO3 10.4 and the last non ELTS release, as TYPO3 10 LTS reaches the ELTS phase on April 30, 2023.
EXT:solr release-11.2.x will not be maintained in TYPO3-Solr/ext-solr repository any more. The maintenance and builds will be moved to a private
repository and ELTS versions, EXT:solr 11.2.4+ for TYPO3 10 ELTS versions, can be obtained through the dkd EB program.
This release contains:
[BUGFIX:P:11.2] make CE search form in backend editable again by @dkd-kaehm in #3640
[DOC] Fix wrong type for boostQuery in the docs and example by @rr-it and @dkd-kaehm in a997a2f4
Apache Solr for TYPO3 11.2.2
This release contains:
[BUGFIX] Type-hinting for SiteUtility::getConnectionProperty() by @dkd-kaehm in #3395
[TASK:Security:P:11.2] Update jQuery and its plugin libs by @dkd-kaehm in #3430
[FEATURE] Add signal before search in resultsAction by @stat1x in #3391
[BUGFIX:BP:11.2] Don't use minimum-stability dev on TYPO3 stable in build/CI by @dkd-kaehm in #3466
[TASK] Docker version check on docker image build by @dkd-kaehm in #3524
[BUGFIX:P:11.2] Use ConfigurationManager to get typscript in plugin FlexForm by @dkd-kaehm in #3497
[TASK] Use PCOV for coverage instead of xDebug :: Upgrade to phpunit 8 by @dkd-kaehm in #3528
[BUGFIX:BP:11.2] Make API eID script compatible with TYPO3 v11.5 by @dkd-friedrich in #3550
[BUGFIX:BP:11.2] Use plugin namespace as label for flexforms by @dkd-friedrich in #3553
[BUGFIX:BP:11.2] Respect indexingPriority in QueueItemRepository by @dkd-friedrich in #3557
[BUGFIX:BP:11.2] use siteLanguage TypoScript object to get language id by @dkd-friedrich in #3555
[TASK:11.2] equalize CI/Github-Actions for release-11.0|2|5 and main by @dkd-kaehm in #3467
[BUGFIX:BP:11.2] Sanitize frequent and last searches output by @dkd-friedrich in #3592
[BUGFIX:BP:11.2] changed from always picking the 0 array value to pic… by @dkd-friedrich in #3594
[BUGFIX] Enforce visibility context in Tsfe by @saitho in #3050
[TASK:BP:11.2] Disable sql handler by @dkd-friedrich in #3604
Apache Solr for TYPO3 11.2.1
This is a maintenance release for TYPO3 10.4, containing:
[BUGFIX] Prepend wrong fe language on empty cache (#3375)
[TASK] Use TYPO3 IpAnonymizationUtility (#3262)
[BUGFIX:BP:11.2] Shortcircuit work in SolrRoutingMiddleware (#3202)
[TASK] Fix scrutinizer on release-11.2.x
[BUGFIX:BP:11.2] Do not handle page updates on new page with uid 0 (#3338)
[TASK] Remove not used strptime() adaption for windows.
[BUGFIX] Respect indexing configuration for new and updated subpages (#3276)
[BUGFIX] Ensure keywords string does not exceed database field length (#3321)
[TASK:BP:11.2] Adapt column arrangement within sites config (#3295)
[DOCS:BP:11.2] add missing doc for ..pageIndexed
[BUGFIX] Fix PSR-4 Namespaces and Paths (#3285)
[BUGFIX] Silence DebugWriter for PageIndexerRequest (#3030)
[BUGFIX] AbstractSolrTask::setRootPageId(): Argument #1 must be of type int, string given (#3267)
[BUGFIX:11.2] Update GarbageCollector.php (#3249)
[DOCS] Align with new TYPO3 documentation standards (#3242)
[DOCS] Align README.md with other extensions (#3218)
[BUGFIX] Missing dot in configuration in numberOfResultsPerGroup method (#3098
[TASK] Added info about using page content in fields
[TASK] Added info about the virtual field __solr_contents
[BUGFIX:BP:11.2] Fix write connection (#2916)
Apache Solr for TYPO3 11.2.0
We are happy to release EXT:solr 11.2.0.
The focus of this release has been on supporting the latest Apache Solr version 8.11.1 and on optimizing the data update monitoring.
New in this release
Apache Solr 8.11.1 support
With EXT:solr 11.2.0 we support Apache Solr 8.11.1, the latest release of Apache Solr.
To ensure the Solr index is up-to-date an extensive monitoring is done, in huge sites this may slow down the TYPO3 backend, as many records and
pages have to be checked and updated. With EXT:solr 11.2 you can configure how EXT:solr will monitor and handle data updates, by default EXT:solr
acts as in all former versions, but you can now configure a delayed update handling or even turn the monitoring off.
Small improvements and bugfixes
Beside the major changes we did several small improvements and bugfixes:
Like always this release would not have been possible without the help from our
awesome community. Here are the contributors to this release.
(patches, comments, bug reports, reviews, ... in alphabetical order)
Georg Ringer
@itzonban
Lars Tode
Mario Lubenka
Markus Friedrich
Marc Bastian Heinrichs
Michael Wagner
Rafael Kähm
Also a big thank you to our partners who have already concluded one of our new development participation packages for Apache Solr EB for TYPO3 10 LTS (Maintenance, ELTS):
ACO Ahlmann SE & Co. KG
AmedickSommer Neue Medien GmbH
Causal Sarl
Cobytes
Columbus Interactive GmbH
Connetation Web Engineering GmbH
cyperfection GmbH
FONDA GmbH
Hochschule Niederrhein
i-fabrik GmbH
i-kiu motion, graphic, backend gmbh
in2code
Intersim AG
jweiland
Landeskriminalamtes Thüringen
Leitgab Gernot
medien.de mde GmbH
MOSAIQ GmbH Glenn Kusardi
NEW.EGO GmbH
novotegra
Pädagogische Hochschule Karlsruhe
ProPotsdam GmbH
proudnerds.com
Provitex GmbH
PSV NEO GmbH
Québec.ca gouv.qc.ca
Shopseam media group gmbh
Shopwegewerk GmbH
SOS Software Service GmbH
Studio 9 GmbH
techniconcept.ch
tirol.gv.at Land Tirol, p.A. DVT-Daten-Verarbeitung-Tirol GmbH
TOUMORØ
visuellverstehen GmbH
WACON Internet GmbH
WE DO communication GmbH GWA
we.byte GmbH
webschuppen GmbH
WIND Internet BV
How to Get Involved
There are many ways to get involved with Apache Solr for TYPO3:
Dynamic fields allow you to add custom fields to Solr documents. That said, you never need to modify Solr's schema (which could cause problems or at least unnecessary additional work when updating the Solr extension).
The following sections describe how to use dynamic fields with your Solr for TYPO3 installation.
Usage of dynamic fields
You can use dynamic fields by following a special naming convention for document fields.
E.g. to create a dynamic field that is a string the field name should end with _stringS. So if you want to
create a field for storing a title you would name it title_stringS. We suggest you use lower camel case for the field name followed by an underscore followed by the dynamic field type "extension".
