Migration: From Sphinx to PHP-based rendering

The main difference compared to the Sphinx rendering is that the PHP-based rendering requires a file called Documentation/guides.xml for configuration. The Sphinx rendering required a file called Documentation/Settings.cfg. In the transition period the GitHub action of the official rendering process detects if a file called Documentation/guides.xml is present and then automatically switches to the PHP-based rendering.

Create the settings file Documentation/guides.xml

The Docker container for the PHP-based rendering additionally consists out of a migration tool. This tool can be used to automatically create a Documentation/guides.xml based on the information contained in your Documentation/Settings.cfg.

Docker (or a drop-in replacement like Podman) needs to be installed on your operating system for the tool to work:

docker run --rm --pull always \
  -v $(pwd):/project \
  -it ghcr.io/typo3-documentation/render-guides:latest \
  migrate Documentation
Copied!
podman run --rm --pull always \
  -v $(pwd):/project \
  -it ghcr.io/typo3-documentation/render-guides:latest \
  migrate Documentation
Copied!

The last parameter (Documentation) is the name of the directory, where your Settings.cfg is currently placed in.

After the migration is performed, you will see some output in the terminal about which settings were converted, if some old settings were discarded, or errors occurred. When you see this:

Output of the command
Note: Some of your settings could not be converted:
    * html_theme_options
    * project_discussions
    * use_opensearch
Copied!

everything went well. They can be ignored since these files are usually files that not need to be converted. You can now delete Settings.cfg. If you ever should need it again it is still in the Git history. Also delete file genindex.rst in your Documentation/ directory (if available).

While you could already jump in and render your documentation directly (see local rendering), we suggest to clean up the following other things first (see).

Further steps to adapt to the PHP-based rendering

You should perform the following tasks to conclude the migration to the PHP-based rendering tool:

  1. Manual modifications of the guides.yml

    You have to manually change the following: in the <extension> tag add the attribute

    Changes in your guides.yml
    interlink-shortcode="my-vendor/my-extension"
    Copied!

    You can find the package name in your composer.json file under "name". For example, my-vendor/my-extension. We recommend to reformat the code, for example in PhpStorm, using the Mac shortcut cmd (⌘) + option/alt (⌥) + L, or Ctrl + Alt + L on Windows/Linux and to use for every attribute one line.

  2. Improve your documentation to render without warning

    Rendering your documentation should not yield any warnings or errors.

    If you get error messages, often they refer to wrong indentation, missing interlinks, orphaned files or outdated ReST identifiers.

    If you are unable to address a warning/error with changes in your documentation feel free to ask in Slack channel #typo3-documentation (see Help & Support).

    If you believe you found a specific bug in the PHP-based rendering, please open an issue on GitHub.

  3. Remove outdated files

    After the guides.xml file has been created, you can remove the old Settings.cfg file.

    You can also delete the genindex.rst file which was previously used to generate an index.

  4. Adapt Includes.rst.txt

    The main documentation directory can contain a file Includes.rst.txt to include any fixed text, which will be placed on every page of your rendered documentation.

    Previously, it was also used to define a list of utilized directives/roles.

    You can either remove that file, or add your fixed text to it. If you remove the file, remember to also remove all references pointing to that file, like:

    Documentation/Index.rst
    ..  include:: /Includes.rst.txt
    Copied!

    Most official documentation uses this as the stub of the file:

    Documentation/Includes.rst.txt
    ..  You can put central messages to display on all pages here
    Copied!
  5. Remove the entry genindex from Index.rst (Index/Glossary)

    If you previously had a genindex.rst file, this optional index (or glossary) was rendered as a page through an entry in the file Index.rst like this:

    Documentation/Index.rst
     .. toctree::
        :hidden:
    
        Sitemap
    -   genindex
    Copied!

    Remove the entry genindex from the list.

  6. Avoid code snippets with .rst extension

    All files with the extension .rst will be interpreted by the PHP-based rendering, and every file that is just a code snippet placed in an external file should be renamed to use a .rst.txt extension instead.

Render your Documentation files locally

Use our Docker container to render your documentation locally. Read more about it in local rendering.

The following list is not a requirement to utilize the PHP-based rendering, but follows "best practice" to make the most of your documentation project. One of the biggest benefits is, that this simplifies finding errors in your reST syntax. The Makefile helps you to launch the local rendering described in here. You create the Makefile in order to register the command shortcuts that you would use to render your documentation locally and also provide an entry point for further shortcuts (like testing).

Add a Makefile to your project

A Makefile is a simple command line runner configuration file, which requires the utility GNU make to be available on your Unix-based operating system (Linux, macOS, WSL on Windows, for example).

This allows you to render your documentation by using

make docs
Copied!

instead of typing a long docker run... or podman run... command.

When rendering locally you should ideally see something like this

Successfully placed 7 rendered HTML, SINGLEPAGE, and INTERLINK files into /project/Documentation-GENERATED-temp
Copied!

Possible errors using make docs

We provide four example errors to guide you through the fixes.

Nested PHP domain components (php:class, php:interface, php:enum etc) are not supported
[2024-03-25T13:26:11.600367+00:00] app.WARNING: Nested PHP domain components
(php:class, php:interface, php:enum etc) are not supported.
Found php:\Vendor\MyExtension\Interfaces\RequireJsModuleInterface inside
\Vendor\MyExtension\Interfaces\AnotherImportantInterface {"rst-file":"Developer/Index.rst"} []
Copied!

The file Index.rst in Documentation/Developer/ has a wrong indentation. A class must not belong to another class. Here is the wrong rst code.

..  php:class:: AnotherImportantInterface

    Used for ...

    ..  php:class:: RequireJsModuleInterface

    Widgets implementing this interface will add the provided RequireJS modules.
    Those modules will be loaded in dashboard view if the widget is added at least once.
Copied!

Additionally regarding the name it has to be interfaces and not classes. .. php:class:: ExampleInterface has to be changed to .. php:interface:: ExampleInterface.

Reference sitehandling-addinglanguages could not be resolved in LocalizedContent/Index {"rst-file":"LocalizedContent/Index"} []

The next step is to visit the site which was rendered with the Sphinx rendering, in our example https://docs.typo3.org/m/typo3/guide-frontendlocalization/main/en-us/LocalizedContent/Index.html. There we can search for sitehandling-addinglanguages in the restructured text code by clicking the button "</> View Source". We have found this:

..  tip::
    For more information on how to add languages and configure their
    behaviour in the site configuration, see
    :ref:`Adding Languages <sitehandling-addinglanguages>`.
Copied!

The link is leading to https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/SiteHandling/AddLanguages.html#sitehandling-addinglanguages. We have to click the symbol next to the heading and copy the correct link which is the one for restructured text

:ref:`Adding Languages <t3coreapi:sitehandling-addingLanguages>`
Copied!

You have to replace the correct link in, for example, Documentation/LocalizedContent/Index.rst to fix the error. Note: Since we already migrated the Frontend Localization Guide (https://docs.typo3.org/m/typo3/guide-frontendlocalization/main/en-us/LocalizedContent/Index.html) to use the PHP-based rendering you cannot find the state that we have shown above anymore.

Makefile example

For inspiration, check out the Makefile of the main PHP-based rendering repository:

https://github.com/TYPO3-Documentation/render-guides/blob/main/Makefile

A small example Makefile:

Makefile
.PHONY: help
help: ## Displays this list of targets with descriptions
	@echo "The following commands are available:\n"
	@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[32m%-30s\033[0m %s\n", $$1, $$2}'


.PHONY: docs
docs: ## Generate projects docs (from "Documentation" directory)
	mkdir -p Documentation-GENERATED-temp
	docker run --rm --pull always -v "$(shell pwd)":/project -t ghcr.io/typo3-documentation/render-guides:latest --config=Documentation


.PHONY: test-docs
test-docs: ## Test the documentation rendering
	mkdir -p Documentation-GENERATED-temp
	docker run --rm --pull always -v "$(shell pwd)":/project -t ghcr.io/typo3-documentation/render-guides:latest --config=Documentation --no-progress --minimal-test
Copied!

Activate automatic testing in your project

It is recommended to use an automatic workflow on GitHub Or GitLab to ensure the extension's documentation renders without warnings.

An example workflow on GitHub would be established via this file in .github/workflows/documentation.yml:

.github/workflows/documentation.yml
name: Test documentation

on: [ push, pull_request ]

jobs:
  tests:
    name: Render documentation
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Test if the documentation will render without warnings
        run: |
          mkdir -p Documentation-GENERATED-temp \
          && docker run --rm --pull always -v $(pwd):/project \
             ghcr.io/typo3-documentation/render-guides:latest --config=Documentation --no-progress --minimal-test
Copied!

This creates a workflow entry Test documentation, so that on every push to your repository, and every pull request, the rendering is executed. A commit will then not be possible, if the rendering fails. The workflow run will be marked with an error, and shown on pull requests.

To be discussed: Index generation (glossary)

The Sphinx rendering allowed to utilize a syntax like the following to add indexes to your documentation:

Documentation/Index.rst
..  index::
    XLIFF; Files
    File; XLIFF
Copied!

The automatically generated file genindex.html would show a two-column layout of all indexes, with the pages that they were used on.

The PHP-based rendering does not (yet) support this indexing.

The current recommendation is to only remove the genindex.rst file from your documentation directory, but keep all the placed .. index directives. If at some point the automatic index generation is re-introduced, your old indexes will be able to show up again.