Crawling strategies

Before URLs are crawled by a crawler, they can be prepared by a specific strategy. This, for example, allows to prioritize specific URLs or provide additional information to URLs.

interface CrawlingStrategy
Fully qualified name
\EliasHaeussler\CacheWarmup\Crawler\Strategy\CrawlingStrategy

Interface for crawling strategy to prepare URLs before crawling.

prepareUrls ( $urls)

Prepare given URLs for crawling.

param array $urls

List of URLs to be prepared for crawling.

returntype

list<EliasHaeussler\CacheWarmup\Sitemap\Url>

static getName ( )

Get name of crawling strategy for use as identifier.

returntype

string

Shipped crawling strategies

The extension ships with the following crawling strategies:

  • sort-by-changefreq: Sorts given URLs by their changefreq node value.
  • sort-by-lastmod: Sorts given URLs by their lastmod node value.
  • sort-by-priority: Sorts given URLs by their priority node value.

Implement a custom strategy

  1. Create a new crawling strategy

    The new strategy must implement the \EliasHaeussler\CacheWarmup\Crawler\Strategy\CrawlingStrategy interface. Make sure to properly implement the \EliasHaeussler\CacheWarmup\Crawler\Strategy\CrawlingStrategy::getName method to identify the crawling strategy.

  2. Configure the new crawling strategy

    Add the new strategy to the extension configuration. Use the strategy's name as configuration value.

  3. Flush system caches

    Finally, flush all system caches to ensure the correct crawling strategy is used for further cache warmup requests.