Additional configuration is added for crawling external links:
# User-Agent string is filled with information about the crawling site
httpAgentName = TYPO3 Linkvalidator
method = HEAD
range = 0-4048
For a description of the fields, see the linkvalidator documentation:
It is recommended to fill out
httpAgentEmail so that the User-Agent
string is filled for crawling external URLs.
For headers, method and range it is recommended to stick with the default values.
It is possible to configure information for the ‘User-Agent’ string as is customary
The settings ‘headers’, ‘method’ and ‘range’ are advanced settings. They can be used to
optimize the crawling.