Local file

It is also possible to provide the contents of your robots.txt in a local robots.txt file in the web root of your TYPO3 installation. If configured, the extension tries to inject XML sitemaps during the request process the same way as in the usage of static routes.

As with static routes, the injection of XML sitemaps must be explicitly enabled in the appropriate site configuration.

Important

Your web server must be properly configured to route requests to an existing robots.txt to TYPO3's entrypoint at index.php. Otherwise, the underlying middleware will never be called, thus being unable to inject XML sitemaps during the request process.

Read more in the web server configuration section below.

Web server configuration

In order to route requests to an existing robots.txt file to TYPO3's main entrypoint, your web server configuration must be modified. This depends on which server type you're actually using.

Apache

Add the following to the .htaccess in the web root of your TYPO3 installation:

RewriteEngine on
RewriteRule ^robots.txt /index.php [L]

Nginx

Add the following to the configuration file of your Nginx server:

location = /robots.txt {
    rewrite ^ /index.php;
}