Cache warmup of pages located in XML sitemaps 

• Console commands
• Backend toolbar
• Using the API
• Custom crawlers

Extension key

warming

Package name

eliashaeussler/typo3-warming

Version

0.5

Language

en

Author

Elias Häußler & contributors

License

This extension documentation is published under the CC BY-NC-SA 4.0 (Creative Commons) license.

An extension for TYPO3 CMS that warms up Frontend caches based on an XML sitemap. Cache warmup can be triggered via TYPO3 backend or using a console command. It supports multiple languages and custom crawler implementations.

Table of Contents

Introduction 

Installation 

composer require eliashaeussler/typo3-warming

Configuration 

The extension can be configured in many ways. Especially the behavior during the cache warmup can be configured extensively. For example, it is possible to limit the maximum number of crawled pages or even use your own crawler implementations.

Learn what configuration options are available on the following pages:

Extension configuration 

The extension currently provides the following configuration options:

Site configuration 

The cache warmup is based of the configured sites in the TYPO3 installation. Therefore, in order to control the cache warmup behavior, the site configuration can be used. The following configuration options are available for this purpose:

Permissions 

Administrators are able to run cache warmup for all available sites and pages. All other users are by default not allowed to run those tasks. Thus, they cannot see both the cache warmup toolbar item and context menu items.

However, you can use User TSconfig to allow cache warmup for specific users/usergroups and sites/pages. Add the following configuration to the options.cacheWarmup User TSconfig:

Usage 

This section describes how to properly use the extension in various ways. Basically the cache warming can be executed in two different modes: Either for whole sites based on their XML sitemap or for single pages based on their page ID.

Backend toolbar 

As soon as the extension is installed, a new toolbar item in your TYPO3 backend should appear. You can click on the toolbar item to get a list of all sites. If a site does not provide an XML sitemap, it cannot be used to warm up caches.

Page tree 

Next to the item in the backend toolbar, one can also trigger cache warmup using the context menu of pages inside the page tree.

Console commands 

The extension provides the following console commands:

vendor/bin/typo3 warming:cachewarmup

typo3/sysext/core/bin/typo3 warming:cachewarmup

vendor/bin/typo3 warming:showuseragent

typo3/sysext/core/bin/typo3 warming:showuseragent

Using the API 

Besides the usage via the TYPO3 backend and the console commands, there is also a public PHP API. It can be used to execute the cache warmup directly in PHP code.

class CacheWarmupService

Fully qualified name

\EliasHaeussler\Typo3Warming\Service\CacheWarmupService

Service to run cache warmup for sites and pages.

warmupSites ( $sites, $request)

Run cache warmup for given list of sites.

param array $sites

List of sites to be warmed up.

param EliasHaeussler\\Typo3Warming\\Request\\WarmupRequest $request

Additional cache warmup request parameters.

returntype

EliasHaeussler\CacheWarmup\Crawler\CrawlerInterface

warmupPages ( $pageIds, $request)

Run cache warmup for given list of pages.

param array $pageIds

List of pages to be warmed up.

param EliasHaeussler\\Typo3Warming\\Request\\WarmupRequest $request

Additional cache warmup request parameters.

returntype

EliasHaeussler\CacheWarmup\Crawler\CrawlerInterface

generateUri ( $pageId, $languageId = null)

Generate uri for given page and optional language.

param int $pageId

ID of the page for which the uri is to be generated.

param int $languageId

Optional language ID to respect when generating the uri.

returntype

Psr\Http\Message\UriInterface

getCrawler ( )

Return crawler being used for cache warmup.

returntype

Psr\Http\Message\UriInterface

setCrawler ( $crawler)

Set crawler to use for cache warmup.

param string|EliasHaeussler\\CacheWarmup\\Crawler\\CrawlerInterface $crawler

The crawler to use for cache warmup.

Returns

The service object (fluent setter).

use EliasHaeussler\Typo3Warming;
use TYPO3\CMS\Core;

$cacheWarmupService = Core\Utility\GeneralUtility::makeInstance(Typo3Warming\Service\CacheWarmupService::class);
$request = new Typo3Warming\Request\WarmupRequest();

// Get all sites
$siteFinder = Core\Utility\GeneralUtility::makeInstance(Core\Site\SiteFinder::class);
$sites = $siteFinder->getAllSites();

// Run cache warmup for all sites
$crawler = $cacheWarmupService->warmupSites($sites, $request);

// Run cache warmup for single pages only
$crawler = $cacheWarmupService->warmupPages([1, 2, 3], $request);

// Evaluate crawling states
$failedUrls = $crawler->getFailedUrls();
$successfulUrls = $crawler->getSuccessfulUrls();

Developer corner 

Many of the cache warmup processes are configurable or can be extended with custom implementations. This section provides an insight into which development options are available and how these can be integrated into the cache warmup process.

Crawlers 

The extension uses crawlers to visit all URLs configured for cache warmup. While visiting a URL, the appropriate page cache gets warmed. Learn more about which crawlers are available by default and how to implement a custom crawler on this page.

interface CrawlerInterface

Fully qualified name

\EliasHaeussler\CacheWarmup\Crawler\CrawlerInterface

Interface for crawlers used to crawl and warm up URLs.

crawl ( $urls)

Crawl a given list of URLs.

param array $urls

List of URLs to be crawled.

getSuccessfulUrls ( )

Return all successfully crawled URLs.

Returns

A list of eliashaeussler-cachewarmup-crawlingstate instances

getFailedUrls ( )

Return all failing crawled URLs.

Returns

A list of eliashaeussler-cachewarmup-crawlingstate instances

Sitemap providers 

To localize the sitemap of a site, so-called "sitemap providers" are available. These are executed one after the other (depending on priority) to localize XML sitemaps according to certain criteria.

interface ProviderInterface

Fully qualified name

\EliasHaeussler\Typo3Warming\Sitemap\Provider\ProviderInterface

Interface for sitemap providers used to locate the path to an XML sitemap of a given site.

get ( $site, $siteLanguage = null)

Locate the XML sitemap path of the given site.

param TYPO3\\CMS\\Core\\Site\\Entity\\Site $site

The site whose XML sitemap path should be located.

param TYPO3\\CMS\\Core\\Site\\Entity\\SiteLanguage $siteLanguage

An optional site language to include while locating the XML sitemap path.

Returns

An instance of eliashaeussler-typo3warming-sitemap-siteawaresitemap or null.

getPriority ( )

Get the provider's priority. The higher the returned value, the earlier the provider will be executed in the sitemap location process.

returntype

int

Caching 

Once a sitemap is located by a sitemap provider, the path to the XML sitemap is cached. This speeds up following warmup requests. Caching happens with the core cache which defaults to a filesystem cached located at var/cache/code/core/tx_warming.php.

class CacheManager

Fully qualified name

\EliasHaeussler\Typo3Warming\Cache\CacheManager

Manager to read and write the core cache tx_warming.

get ( $site = null, $siteLanguage = null)

Get all located sitemaps or the located sitemap of a given site and/or site language.

param TYPO3\\CMS\\Core\\Site\\Entity\\Site $site

The sitemap's site object or NULL to lookup all sitemaps.

param TYPO3\\CMS\\Core\\Site\\Entity\\SiteLanguage $siteLanguage

An optional site language

Returns

Either an array of all located sitemaps or the located sitemap of a given site.

set ( $sitemap)

Add the located sitemap to the tx_warming cache.

param EliasHaeussler\\Typo3Warming\\Sitemap\\SiteAwareSitemap $sitemap

The located sitemap to be cached.

Configuration API 

In order to access the extension configuration, a slim PHP API exists. Each configuration option is accessible by an appropriate class method.

class Configuration

Fully qualified name

\EliasHaeussler\Typo3Warming\Configuration\Configuration

API to access all available extension configuration options.

getLimit ( )

Get the configured crawler limit.

returntype

int

getCrawler ( )

Get the configured crawler class.

returntype

class-string<EliasHaeusslerCacheWarmupCrawlerCrawlerInterface>

getCrawlerOptions ( )

Get the configured crawler options.

returntype

array

getVerboseCrawler ( )

Get the configured verbose crawler class.

returntype

class-string<EliasHaeusslerCacheWarmupCrawlerVerboseCrawlerInterface>

getVerboseCrawlerOptions ( )

Get the configured verbose crawler options.

returntype

array

getUserAgent ( )

Get the calculated user-agent.

returntype

string

getAll ( )

Get all extension configuration options of this extension.

returntype

array

AccessUtility 

The AccessUtility can be used to determine permissions of the current user to warm up caches of specific sites or pages.

class AccessUtility

Fully qualified name

\EliasHaeussler\Typo3Warming\Utility\AccessUtility

Utility class to determine cache warmup permissions.

canWarmupCacheOfPage ( $pageId, $languageId = null)

Check if the current user can warm up caches of the given page.

param int $pageId

ID of the page to be checked.

param int $languageId

Optional language ID to be included in the check.

canWarmupCacheOfSite ( $site, $languageId = null)

Check if the current user can warm up caches of the given site.

param TYPO3\\CMS\\Core\\Site\\Entity\\Site $site

The site to be checked.

param int $languageId

Optional language ID to be included in the check.

Contributing 

Thanks for considering contributing to this extension! Since it is an open source product, its successful further development depends largely on improving and optimizing it together.

The development of this extension follows the official TYPO3 coding standards. To ensure the stability and cleanliness of the code, various code quality tools are used and most components are covered with test cases.

Contribution workflow 

Note

This extension follows Semantic Versioning.

Preparation 

Clone the repository first:

git clone https://github.com/eliashaeussler/typo3-warming.git
cd typo3-warming

Copied!

Now install all Composer dependencies:

composer install

Copied!

Next, install all Node dependencies:

yarn --cwd Resources/Private/Frontend

Copied!

Check code quality 

TYPO3 

# Run all linters
composer lint

# Run Composer linter only
composer lint:composer

# Run PHP linter only
composer lint:php

# Run TypoScript linter only
composer lint:typoscript

# Run PHP static code analysis
composer sca

Copied!

Frontend 

# Run all linters
yarn --cwd Resources/Private/Frontend lint
yarn --cwd Resources/Private/Frontend lint:fix

# Run SCSS linter only
yarn --cwd Resources/Private/Frontend lint:scss
yarn --cwd Resources/Private/Frontend lint:scss:fix

# Run TypeScript linter only
yarn --cwd Resources/Private/Frontend lint:ts
yarn --cwd Resources/Private/Frontend lint:ts:fix

Copied!

Run tests 

# Run tests
composer test

# Run tests with code coverage
composer test:ci

Copied!

The code coverage reports will be stored in .Build/log/coverage.

Build documentation 

# Rebuild and open documentation
composer docs

# Build documentation (from cache)
composer docs:build

# Open rendered documentation
composer docs:open

Copied!

The built docs will be stored in .Build/docs.

Pull Request 

When you have finished developing your contribution, simply submit a pull request on GitHub: https://github.com/eliashaeussler/typo3-warming/pulls

Sitemap 

Index