Usage

At the moment, the extension provides two possible ways to inject located XML sitemaps in a requested robots.txt file. This section describes how to properly use each of them and what steps are necessary to enable all provided features.

Example

The following example describes what processes happen when a robots.txt file is requested and XML sitemaps get injected.

  1. Base URL

    The base URL of your site is https://www.example.com/.

  2. XML sitemaps

    Currently, the following XML sitemaps are available for your site:

    • https://www.example.com/sitemap.xml
    • https://www.example.com/sitemap-news.xml
  3. robots.txt

    Given the following content of your robots.txt route:

    User-agent: *
    Allow: /
    Disallow: /forbidden/
    Copied!
  4. Sitemap injection

    If you have injection of XML sitemaps enabled in your site configuration and each configured XML sitemap actually exists, the robots.txt will be enhanced like follows:

    User-agent: *
    Allow: /
    Disallow: /forbidden/
    
    Sitemap: https://www.example.com/sitemap.xml
    Sitemap: https://www.example.com/sitemap-news.xml
    
    Copied!