Crawler configuration
The following configuration applies only to the default crawlers shipped by this extension.
See also
If you want to learn more about configurable crawlers, read more in the developer corner.
Note
Both default crawlers utilize the same crawler options that are described in the
option reference
of the underlying eliashaeussler/ library.
In addition, the following crawler options are available:
perform_subrequests
-
- Type
- boolean
- Default
- 0
New in version 3.3.0
Enable sub request handler for cache warmup requests. This will significantly increase performance of the whole cache warmup progress, since no HTTP requests will be performed anymore. Instead, the bootstrapped frontend application is re-used for each cache warmup request.
Note
Only URLs matching the site's base URL are handled by the sub request handler. All other URLs (e.g. pages linking to external URLs) will be handled by Guzzle's default handler.