Why contribute?

Here are some ideas why contribution is great:

• Do you earn your income through the TYPO3 ecosystem? Contribute to keep it alive and going, securing your work.
• Do you want to become a more experience developer? Or gather UX/UI expertise in a large, broadly used system?
• Do you work in a company that has less sophisticated code testing and review capacities, and you want to learn from the TYPO3, understand it, and use your knowledge to work more effectively?
• Do you want to keep up to date with latest development, be a part of it to shape it suit your needs? Have a democratic impact on a large project affecting many people?
• If you know about fresh changes, it will make it much easier to perform updates for your projects! If you spot breaking changes affecting your project, you can much better give feedback on it and shape the final implementation.
• Get in touch with awesome, kind and dedicated like-minded people like you. Be part of a community and feel connected, and see people appreciating your help.

How this guide is structured

The guide is structured in a way to give you all necessary information in the order that you need it. This means you can read it from the beginning and use it as a hands on guide, performing the steps as you go along.

The further you go along, the more advanced the topics will become.

But, you can also use it as a reference guide and jump straight to a section you are looking for. In this case, use the search box to search for what you are looking for or browse through the menu.

For example:

• Create a Patch walks you through making a change in the TYPO3 core (e.g. fix a bug or create a new feature).

The chapters are structured as follows:

• The chapters in INTRODUCTION give you an introduction. They are not necessary for contribution but will give you some background information which may make some things easier.
• The chapters in SETUP show how to setup a TYPO3 installation, Git and your accounts for contribution. This is a necessary prerequisite for most actions (e.g. creating a patch), but must be done only once. We recommend to start with this.
• The chapters in HOWTOS walk you through one task. In most cases, it is assumed that you already setup your environment.
• ADDITIONAL INFORMATION contains a cheat sheet git cheat sheet, Troubleshooting and the Appendix which is a reference of some topics in more depth than what was described in the main section. These pages near the end of the manual assume that you are already familiar with contributing and serve as reference pages.

Conventions used in this guide

• When referring to elements in the gui, we use this formatting: Add new SSH key.
• Keystrokes are referred to by CTRL + c

Other ways to contribute

Besides developing for the TYPO3 core, there are many other ways to contribute and help the TYPO3 community.

You can find general information here:

• General information about the various Teams & Committees
• General information on how you can Contribute and Get Involved

Contribute to this Guide

If you find a bug in this manual, please be so kind as to check the online version on https://docs.typo3.org/typo3cms/ContributionWorkflowGuide/. From there you can hit the Edit on GitHub button in the top right corner and submit a pull request via GitHub. Alternatively you can just report an issue on Github.

Maintaining high quality documentation requires time and effort and the TYPO3 Documentation Team always appreciates support. If you want to support us, please join the slack channel #typo3-documentation.

Have a look at Contribute to the TYPO3 documentation for more information about how to contribute to the TYPO3 documentation.

And finally, as a last resort, you can get in touch with the Documentation Team by mail.

Slack

The main communication platform for TYPO3 is the Slack chat system. In order to register for Slack, you first need a typo3.org account and then request a Slack account. This is explained here: Slack.

You can ask specific questions about contributing in the #typo3-cms-coredev channel, general TYPO3 questions should be asked in the #typo3-cms channel.

For more information on Slack, see Slack in the Appendix.

Events

Coding is great, but coding with others may be even better. There is a wide variety of TYPO3 events where you can meet people, gain knowledge and have fun doing it. Some of these events are especially suited for core contributors, though you will probably benefit from any TYPO3 event in some way or another.

• For a general overview of TYPO3 events, read the blogpost: What You Gain by Taking Part in TYPO3 Events
• For an up-to-date list of events, see Events on typo3.org

Look for events like "Review Friday", "Code Sprints" and "Developer Days" or one of the many "BarCamps".

Next, we will describe some events that are relevant for Core contribution. They are not specifically targeted at providing support, but the event may be open to new Core contributors and may give you the opportunity to ask questions and get help. In any case, read the event description and if in doubt, contact the person listed in the event description.

Common Workflow

So this it what your usual coding workflow may look like.

1. You clone or pull from a central Git repository and change some lines of code
2. You push right back to that central repository

This workflow has some drawbacks though, because there is only a very low level of peer control over the code you are pushing. Chances are relatively high that your change might break something elsewhere, does not comply to TYPO3s coding guidelines, doesn't work on the respective PHP versions for the TYPO3 version you are working on.... in short: it's dangerous.

TYPO3 Contribution Workflow

As you can see, our workflow is a bit different, simply because we do not allow to push right back to our Git repository. We use a 6-eye principle to ensure proper quality assurance and Gerrit helps us here - but let's go over the workflow step by step.

1. You clone or pull as usual and do changes to the codebase.
2. When trying to push back to the Git repository, you will not be allowed to do so. In fact, the only entity that may push back to our Git repository is Gerrit itself, so the same rules apply for everybody.
3. Instead, you push your changes to Gerrit and they will show up as a pending code review.
4. From here, any contributor can review the change.
5. Reviewing consists of two parts, the Code Review where a contributor checks if the code style is according to our coding guidelines or in general makes sense by reading and the Verification which means if the code does what it is supposed to do (like fixing an issue) and checks whether the unit, functional and acceptance tests run through.
6. If a review has enough positive votes (at least two people voting verified+reviewed), an active contributor (aka member of the Core team, called "Merger") is able to merge that review into the existing codebase. The reason we need a member of the Core team to finally merge the change is that we run a "You break it, you fix it" policy. So, the Core team member that merged a change immediately becomes responsible for that change in case the original author decides to no longer work on it. And he / she will be responsible for backporting it to selected older TYPO3 versions.

Register for Slack account

1. Signup for typo3.org account (if not done already)

   In order to join the slack workspace, you must already have your typo3.org account.

2. Go to Slack for TYPO3 on my.typo3.org.

   On that page, you will also find information about recommended channels.

3. Login with your typo3.org account, then go to the "My Profile" tab and select "Use TYPO3 Slack".

   

4. Follow the next steps

   including clicking on the link in the welcome email with subject "Stefan Busemann has invited you to join a Slack workspace" from slack.

Join #typo3-cms-coredev channel

1. Open Channels Browser

   Once logged in https://typo3.slack.com, go to Channels Browser by clicking on Channels:

   

2. Search for coredev-channel and select it:

   

3. Click "Join Channel"

   

More information

See Appendix: Slack for more information on Slack.

git clone

Create a directory for your TYPO3 core installation and change into it, e.g.:

mkdir /var/www/t3coredev
cd /var/www/t3coredev

Clone the TYPO3 CMS core repository:

git clone git@github.com:typo3/typo3 .

git clone https://github.com/typo3/typo3.git .

Of course you can also use your custom user-specific workspace like /home/kaspar/TYPO3-Contribution/ to store the files, as you do not necessarily need to run a webserver to later setup your installation.

Set Username and Email

-- required (unless this is already setup globally, see git config --global -l)

You need to instruct git to work with your name and email address. Make sure the email address and user name are the same as those you used when setting up your TYPO3 account:

git config user.name "Your Name"
git config user.email "your-email@example.org"

Set autosetuprebase

-- required

In order to avoid weird merges in your local repository when pulling in new commits from typo3.org, we encourage everybody to set the autosetuprebase Git option, such that your local commits are always rebased on top of the official code:

git config branch.autosetuprebase remote

Install Your Commit Hooks

There are two git hooks available for TYPO3 development:

• commit-msg Hook: required
• pre-commit Hook: optional, the pre-commit hook runs on Linux and MacOS. To use the pre-commit hook on Windows you can use a tool like the Git BASH.

To set them up, you can use the existing Composer command or copy the hooks manually.

composer gerrit:setup

# ensure folder exists
mkdir -p .git/hooks

# copy commit-msg hook
cp Build/git-hooks/commit-msg .git/hooks/commit-msg
# optional: copy pre-commit hook
cp Build/git-hooks/unix+mac/pre-commit .git/hooks/

# make executable
chmod +x .git/hooks/commit-msg
chmod +x .git/hooks/pre-commit

Setting up Your Remote

-- required

You must instruct Git to push to Gerrit instead of the original repository. It acts as a kind of facade in front of Git:

git config remote.origin.pushurl ssh://<YOUR_TYPO3_USERNAME>@review.typo3.org:29418/Packages/TYPO3.CMS.git

This will instruct Git to push using the refs/for namespace when you do git push:

git config remote.origin.push +refs/heads/main:refs/for/main

Reminder: The TYPO3 source repository is hosted on GitLab. That Git repository is only mirrored to the TYPO3 GitHub repository. Gerrit is coupled to GitLab.

Setting up a Commit Message Template

-- optional

If you follow these instructions, whenever you create a new commit, Git will use the template to create the commit message, which you can then modify in your editor. So use this to make it easier for you to fill out the required information.

First, create a file, for example ~/.gitmessage.txt.

[BUGFIX|TASK|FEATURE]

Resolves: #
Releases: main, 13.4

Make Git use this file as a template for the commit message:

git config commit.template ~/.gitmessage.txt

For additional information about how to write a proper commit message see Commit Message rules for TYPO3 CMS.

Show Configuration

-- optional

Show current configuration:

git config -l

The result should look like this:

...
remote.origin.url=git@github.com:typo3/typo3
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.main.remote=origin
branch.main.merge=refs/heads/main
branch.autosetuprebase=remote
remote.origin.pushurl=ssh://<YOUR_TYPO3_USERNAME>@review.typo3.org:29418/Packages/TYPO3.CMS.git
commit.template=/path/to/.gitmessage.txt
...

Or, compare the .git/config file inside the repository:

[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[remote "origin"]
    url = git@github.com:typo3/typo3
    fetch = +refs/heads/*:refs/remotes/origin/*
    pushurl = ssh://<TYPO3_USER_NAME>@review.typo3.org:29418/Packages/TYPO3.CMS.git
[branch "main"]
    remote = origin
    merge = refs/heads/main
[branch]
    autosetuprebase = remote
[commit]
    template = /path/to/.gitmessage.txt

Other resources

• Troubleshooting
• See git cheat sheet for more git commands.
• We have compiled a list of more information for you in the Appendix section.

composer install

Run composer install in the same directory you cloned the TYPO3 CMS core repository.

It is recommended to use runTests.sh for this (see Core testing in depth). The "direct command" is an alternative, but it requires your local system to have proper PHP and Composer versions ready to use. You only need to run one of these!

# cd <cloned project>
export COMPOSER_ROOT_VERSION=13.0.0

Build/Scripts/runTests.sh -s composerInstall

composer install

Setup

1. Activate the styleguide extension in the Extension Manager

   Tip

   Once the styleguide extension is activated, it is possible to create (or delete) the pages via command line:

   shell command

   # create pages
   bin/typo3 styleguide:generate -c
   # or (with DDEV)
   ddev typo3 styleguide:generate -c
   
   # show help
   bin/typo3 styleguide:generate -h

   Copied!
2. In the Modules bar click on System > Styleguide

3. Click on the Styleguide menu item (former TYPO3 versions present a custom module menu item)

   The Styleguide overview appears.

4. Click on TCA / Records / Frontend

5. Create pages

   Click the buttons:

   • Create styleguide page tree with data
   • Create styleguide frontend

The pages will now be created. Wait a moment until a flash message appears.

Coding Guidelines

There is an .editorconfig file in the TYPO3 core repository. See https://editorconfig.org/ for more information.

Please note that the rules in .editorconfig are very minimal. So, additionally, setup your editor / IDE to use the recommendation as described in the Coding Guidelines of TYPO3.

See PhpStorm setup CGL for information about setting up PhpStorm to comply with the Coding Guidelines.

Introduction to Forge

The issue tracker Forge is currently based on Redmine. It is used to report and handle open issues (including bugs and feature requests).

Note that Forge is not only used for the Community to report bugs and features, but also the Core development team creates issues for each and every change to TYPO3.

First, get yourself an account, see Setting up Your Accounts.

When you want to report a bug or suggest a new feature, go to the "Issues" section for the TYPO3 Core.

Searching for Existing Issues

Before you go ahead and report a bug, it is recommended that you check Forger to see if the same issue or something similar has already been reported.

Forger's search functionality makes it easy to find existing issues. Filters are located on the the left hand side of the navigation menu. You can use this feature to help refine searches.

Of course you are also able to use the search functionality of Forge itself, which offers some more specific filter options.

You help the people maintaining the issue tracker a lot by first making sure your problem is not covered already. The less duplicate issues our team needs to triage, the more time we have for actually addressing bugs and features.

Identify the Issue

Before you report a bug or suggest a new feature, make sure that the issue you report will be helpful by following these guidelines:

Remove side effects

Work on a TYPO3 instance which is as clean as possible so you can rule out extensions messing with the TYPO3 Core. If you need to set up an extension to illustrate the problem, make sure it is as free of side effects as possible. Ideally, try to reproduce your problem or feature by using existing Core extensions (like Use EXT:styleguide).

Narrow down the problem

Try different browsers, this will help the team (and you) a lot to provide a proper description of the problem.

Be up to date

First of all you should make sure that the bug does exist on the latest TYPO3 version; we always recommend you to upgrade your TYPO3 environment to the latest release of the LTS version, see https://get.typo3.org/.

Or even better: use the latest version from git (main).

Be explicit

It can take a lot of time to reproduce or understand issues. Please try to be as brief as possible, but as detailed as needed. Supplying code examples to reproduce an issue, or showing screenshots helps a lot!

Remain involved

If we have questions about your issue, and need feedback from you, please ensure you get notified to replies to your issue and try to respond. Often, further details are needed.

Talk to the core team

If you are a developer yourself and are able to address an issue (no matter if bug or feature) yourself, but need some guidance on how to approach it, don't hesitate to talk to us on Slack in the #typo3-cms-coredev channel. (Remember to register first.) This channel is not a support channel for end-users. Those needs are covered by https://typo3.org/help.

Create an issue

Get your typo3.org account, head over to Forge and log in (if you aren't already). You can find the TYPO3 core issue tracker here: https://forge.typo3.org/projects/typo3cms-core/issues.

If you click "New issue" you will see a form with a couple of fields that are important. Let's go over these really quick.

(optional) Files

You can additionally uploads files if they help to understand and reproduce the problem.

Some hints for files:

• Do not copy-paste huge stack dumps into the description. Cut out the relevant parts for the description and add the complete stack dump as extra file.
• Images can be a huge help in understanding the problem. Do not insert complete screen dumps but clip the image to the relevant parts. If it helps, add boxes or arrows to highlight important things in the image. Use Redmine formatting for inline images
• If a video is even better in understanding a bug, try to create a short (!) video that clarifies your issue. For example, use an animated gif screen capture tool to create an animated gif.

Best practices for writing a good bug report

It is not necessary to add all available information to your bug report. It is important to provide the specific information that is necessary to be able to understand, reproduce and fix the bug.

Use your common sense and your experience to guide you: What would you need if you wanted to find the bug in the code and fix it yourself? What do you need if you want to reproduce it to debug it or test if the patch solves the problem?

A good bug report should contain all or any of these elements:

1. Prerequisites:

   Here you can add:

   • A brief description of your environment. Depending on the nature of the bug it might include your operating system (Windows, Linux, macOS, ...), the full version of TYPO3 and PHP, the webserver used, the database used (MySQL, MariaDB, ...) and its version. In any case, the full version of TYPO3 (e.g. 12.4.22) is very helpful. If your bug is reproducible on several versions (e.g. 13.4.0 and 12.4.22), that is helpful as well.
   • A description of the TYPO3 setup that you are using or that is necessary to trigger the bug. Your issue came out using TYPO3 with multiple languages? Or when you have more than two frontend groups? You have to tell us, otherwise we could not be capable of reproducing your issue.

   We don't need a full description of your environment, or the full TypoScript configuration, but just the parts that are relevant to trigger the bug.

2. Steps to reproduce the problem

   This is a short easy-to-follow guide that allows us to understand how to trigger the bug following it. Using a numerated list of steps is just fine here; you can also add screenshots. If possible, use TYPO3 Core extensions like Use EXT:styleguide to make an issue reproducible. The easier our support helpers are able to follow your instructions, the more likely it is we can also find the problematic area in the TYPO3 code.

3. Actual results

   This is the heart of your problem: what happened after you followed the steps? Please add also here if your problem is repeatable or comes out randomly.

4. Expected results

   What you expected to happen instead.

5. Additional notes

   Additional information like special conditions or other details not reported on the previous points.

Please consider that these guidelines are very generic. Not always all these parts are necessary, but having the necessary information could help a lot to reproduce and fix the bug.

This not only applies to bugs, but also feature requests. For feature requests, state why a change is needed/helpful, what problems it solves. TYPO3 is a very modular and highly configurable system - not every feature needs to be solved in the TYPO3 core, but can be an extension (maybe yours!) instead.

Hints for formatting in Redmine

Redmine offers quite a few text formatting options: use them to make your report readable. Remember, good formatting makes reading the bug report easier and increases the probability that people will be able to reproduce the problem and help with fixing, testing and merging patches. During the life cycle of a bug report and patch, several people will be reading your report. High readability and clarity makes things easier for everyone and saves time.

!filename.png!

"Anchor text":url

"Forger search example":https://forger.typo3.com/search?query=really+long+query+string+with+filters&filters%5Btypo3_version%5D%5B8%5D=true&filters%5Bcategory%5D%5BLink_Handling_%26_Routing%5D=true

Continued workflow

So, you've filed an issue following the steps above, and submitted it.

What now? Read further in Issue Workflow (Forge).

Additional Resources

• Redmine Info
• Symfony : Tips for reporting a bug

Step by Step Walkthrough

You should have a cloned Git repository with a working TYPO3 installation as described in setup (or via the Quick start guide). Especially the Git setup is required.

1. Create an Issue on Forge

   More information: Report an Issue

   Every patch must have a matching issue on Forge, so create an issue now or submit a patch for an existing issue.

2. Make your changes to the code, add documentation, tests

   This part is pretty straightforward. But be warned, there are still a few dark places deep inside the TYPO3 core dating back to the medieval times of PHP4. Yes, TYPO3 has been around for quite some time now. And there is ancient code we didn't have to touch yet because it just works.

   Make sure to look at How to deprecate classes, methods, arguments and hooks in the TYPO3 core in the Appendix for information about how to deprecate things if you need to make changes to the public API.

   For new features, breaking changes and deprecations, it is necessary to add information to the changelog.

   If you change SCSS, JavaScript or TypeScript files, you can build locally.

   Add Unit Tests or Functional Tests for new functionality, refine existing tests if necessary. Tests are important because they ensure that TYPO3 will behave consistently now and in the future.

   See Testing the core in TYPO3 Explained for more information about writing and running tests.

   Once you have finalized your patch, check out the Common code review checks / checklist for a list of what kind of review checks people may perform on your contribution. Stay ahead of the game and address those yourself first.

3. Commit your changes

   Please make sure that you read the Commit Message rules for TYPO3 CMS in the Appendix. Your code will not be merged if it does not follow the commit message rules.

   Important

   The section Commit Message rules for TYPO3 CMS is a must-read. Read it. Follow it.

   For a bugfix, your commit message may look something like this:

   commit message

   [BUGFIX] Subject line of max 52 chars
   
   Some descriptions with line length of max. 72 characters
   
   Resolves: #12346
   Releases: main, 13.4

   Copied!

   Only create one commit. Do not create a branch. Work on main.

   shell command

   git commit -a

   Copied!

   The commit-msg hook will do some sanity checks and add a line starting with Change-Id:.

   If you have activated the pre-commit hook it will loudly complain if something does not conform to the coding guidelines.

   In that case, use the runTests.sh script to to fix CGL issues.

   After you have created your commit, you can still make changes by amending to your commit:

   shell command

   git commit -a --amend

   Copied!

   Tip

   Keep in mind that you can commit with --amend as often as you want, just make sure you keep the Change-Id: line intact.

4. Push to Gerrit

   To submit the patch to Gerrit, issue the following command:

   shell command

   git push origin HEAD:refs/for/main

   Copied!

   If you have setup the default as described in Setting up Your Remote it is sufficient to use:

   shell command

   git push

   Copied!

   In case you want to push a "Work in progress", check out: Workflow - work in progress.

   If Gerrit accepts your push, it responds with the following messages:

   result

   remote: SUCCESS
   remote:
   remote:   https://review.typo3.org/c/Packages/TYPO3.CMS/+/<gerrit-id> [...] ... [NEW]
   remote:
   To ssh://review.typo3.org:29418/Packages/TYPO3.CMS.git
   * [new reference]         main -> refs/for/main

   Copied!

   If you see an error, check out the Git Troubleshooting section.

   You can visit the link to https://review.typo3.org to see your patch in Gerrit.

   The Continuous Integration service, running on GitLab Pipelines, will automatically be executed in the background and perform checks and tests on your patch. Failing tests will be linked to that instance, where jobs can also be retried (given sufficient permissions when being logged in to GitLab).

   Advanced users / core team only: See cheat sheet: other branches for pushing to other branches.

5. Optional: Advertise review on Slack channel

Once your push to Gerrit goes through, you will receive a URL for your new

change. If you are on Slack you can now advertise your new change in the #typo3-cms-coredev channel. You can get a preformatted line of your change to post in the channel by clicking the copy button next to the title in Gerrit_:

This is not something, you will do for every review. As a first contributor it is recommended to mention that you are new to the process.

Now, it's time to sit back and await feedback on your changes. The review team process dozens of requests each day, so expect a succinct response that is short and to the point. You will get notified by email, if there is activity on your patch in Gerrit (e.g. votes, comments, new patchsets, merge etc.).

Check out the section Review a patch for more about this process, in which you can also be involved!

It is not unusual for a patch to get comments requesting changes. If that happens, please respond in a timely fashion and improve your review. If things are unclear, ask in the #typo3-cms-coredev channel on https://typo3.slack.com.

Helpful links

• Gerrit Review server
• Forge Issue tracker
• Forger Search for reviews and issues
• Slack chat system

Next Steps

You will find some more information about the review process in the chapter Handle and Improve a Patch (Gerrit). The following pages are especially relevant for new contributors:

• Tips for new contributors
• Introduction to Gerrit describes the review tool Gerrit.
• Find a review on Gerrit is helpful if you don't know how to find your patch on Gerrit.
• Gerrit works with up- and downvoting patches. A patch must get a specific number of upvotes before it can be merged. Code Review gives an introduction to how this works.
• When you make additional changes to your patch, make sure you do not add another commit. Append to your original commit instead as described in Upload a new Patch Set.
• Before starting to work on a new, unrelated patch you need to run the Cleanup tasks.

Quickstart to contribute documentation

To work on the Core documentation of TYPO3, you need to work with the main TYPO3 mono-repository. You can not contribute Documentation patches on single read-only repositories like https://github.com/typo3-cms/felogin through the GitHub interface!

You can, however, use the GitHub interface and contribute to https://github.com/TYPO3/typo3/tree/main/typo3/sysext/felogin. Submitting a GitHub PR will result in a GitHub Action workflow that closes your PR, transfers it to forge, transfers it to gerrit, and link them to each other. That workflow can be prone to errors though, especially if a branch other than main is involved.

So, if possible you could better follow the Quickstart guide to set up a Core contribution installation. A lot can be left out, as you do not necessarily even need a TYPO3 instance running when you only want to contribute documentation.

The minimal steps to contribute documentation "the right way" (and to allow you to properly participate in our review workflow) is this:

1. Prerequisites

   From the Quickstart Prerequisites you need:

   1. Operating System
   2. GIT client
   3. SSH client + keys
   4. Optional Bonus: Docker (to render documentation), a suitable Text-Editor

2. Set-up accounts

   You need all of the Quickstart Accounts (My TYPO3, GitHub, Gerrit, Forge)

3. Set-up GIT

   Set up and clone the TYPO3 mono-repository as described in Quickstart GIT.

   Note: the steps Quick Start: Set up DDEV and Quick Start: Set up TYPO3 are not needed for Documentation-only use.

4. Start documenting

   Now you can start editing files in, for example, typo3/sysext/felogin/Documentation/Index.rst and when you are done, you can render the documentation (see Render any system documentation locally) to verify the look of your changes.

5. Create issue

   Once you feel comfortable and happy with your patch, you go to create an issue on Forge. Choose the category Documentation and enter an appropriate description like:

   Tracker: Task

   Subject: Add example for EXT:felogin RedirectLoginHandler

   Description: (Describe what kind of documentation changes you made. Mention to which TYPO3 version it applies)

6. Submit patch

   Now follow the steps outlined in Quick Start: Create a patch, and refer to the Documentation files you edited, instead of the PHP files given as examples there. This will then submit your patch to our Gerrit review instance.

Changelog

Some patches require a .rst (reStructuredText) Changelog file describing the change. Not all patches need an entry in the Changelog. Check the list below. Also see the current TYPO3 Core Changelog for some examples.

Every file may optionally contain tags, but it must contain at least a NotScanned, PartiallyScanned or FullyScanned tag for the extension scanner. See Extension scanner in TYPO3 Explained for more information.

cd typo3/sysext/core
docker run --rm --pull always -v $(pwd):/project -it ghcr.io/typo3-documentation/render-guides:latest --config=Documentation
xdg-open "Documentation-GENERATED-temp/Index.html"

cd typo3/sysext/core
docker run --rm --pull always -v $(pwd):/project -it ghcr.io/typo3-documentation/render-guides:latest --config=Documentation
xdg-open "Documentation-GENERATED-temp/Index.html"

cd typo3/sysext/felogin
docker run --rm --pull always -v $(pwd):/project -it ghcr.io/typo3-documentation/render-guides:latest --config=Documentation
xdg-open "Documentation-GENERATED-temp/Index.html"

cd typo3/sysext/felogin
docker run --rm --pull always -v $(pwd):/project -it ghcr.io/typo3-documentation/render-guides:latest --config=Documentation
xdg-open "Documentation-GENERATED-temp/Index.html"

Build/Scripts/validateRstFiles.php

Policy for Changing the Main Documentation

Once a new TYPO3 release comes out, the main documentation (e.g. TYPO3 Explained, TCA Reference etc.) must be updated.

The procedure is documented in Apply changelog entries to the docs.

Document System Extensions

Documentation for system extensions is maintained within a Documentation directory in the respective system extension directory, e.g. typo3/sysext/form/Documentation.

Not all system extensions have their own documentation. Some documentation (e.g. for the system extension core) is maintained within the TYPO3 Explained.

If in doubt, ask in the #typo3-cms-coredev channel on Slack.

For starting a system extension from scratch, please see Use the init command to create the Documentation folder.

For an overview of the rendered documentation for system extensions, see System Extensions.

When you have made changes to the documentation, you can render locally with docker to test your changes as described in Render the Changelog locally.

More Information

• See Documenting Changes for more information on the Changelog
• See Extension scanner in TYPO3 Explained

What is runTests.sh?

This is a shell script which is included in the TYPO3 Core git repository. It used to supply commands for running tests, but has since evolved to include other commands as well for checking, fixing, building etc. It will be used for a number of tasks and will be a vital part during your Core contribution workflow.

The script uses Docker (also supporting podman) and several Docker images to run the tasks - using the correct versions (of PHP, Node/NPM etc.) for the current branch of your TYPO3 repository.

Show help

It is recommended to use the -h option (help) to show the help output and especially look at the beginning of the output for a description and the listing of dependencies. The listing in section -s shows available commands, and the bottom lists several examples you can copy+paste to try.

Show help:

Build/Scripts/runTests.sh -h

Prerequisites

See the listed options/commands in the help output.

Options

-h

show help

-s

choose the command to run (if omitted, unit tests are run)

-p

PHP version (if omitted, the default version is used)

-b

choose podman (default) or docker for image execution

-d

Database type (sqlite, mariadb, mysql, postgres)

-i

Database version in conjunction with -d, for example -d mariadb -i 11.4 for MariaDB 11.4

-x

Enable xdebug usage

-u

Update docker image versions

For more options, see the help (-h).

We will quickly go over each type of command next.

Output example:

Options:

-s <...>
    Specifies which test suite to run
        - acceptance: main application acceptance tests
        - acceptanceInstall: installation acceptance tests, only with -d mariadb|postgres|sqlite
        - build: execute frontend build (TypeScript, Sass, Contrib, Assets)
        - cgl: test and fix all core php files
...

Commands

We categorize the commands and show some examples here. It is recommended to use -h as described previously to view the help output and examples. The trailing * is used as wildcard to indicate that there are further commands which start with the same string.

Commands for building:

• -s composerInstall: install composer dependencies from composer.lock
• -s build: build CSS+JS assets
• ...

Checks and fixes, run static analyzer, lint etc:

• -s cglGit : check (and fix!) the CGL issues in the files which are in the most recent git commit
• -s cgl (cgl*)
• -s lintPhp
• -s lintScss (lint*)
• -s phpstan (phpstan*)
• -s checkComposer (check*)
• ...

Commands for running tests:

• -s unit (unit*)
• -s functional (functional*)
• -s acceptance (acceptance*)
• ...

Cleanup, clear cache:

• -s clean (clean*)
• ...

Dispatchers:

• -s npm -- [npm command]
• -s composer -- [composer command]

Additional setup

Be sure to exclude the typo3temp and .cache directory from indexing in your IDE (e.g. PhpStorm) before starting the acceptance tests.

Also, if you are using DDEV for example with Mutagen performing filesystem synchronization, it is vital that you configure the typo3temp and .cache directory to be excluded in your file .ddev/mutagen/mutagen.yml like this:

sync:
  defaults:
    mode: "two-way-resolved"
    stageMode: "neighboring"
    ignore:
      paths:
        # ...
        - "typo3temp/"
        - "public/typo3temp"
        - ".cache"
        # ...

Due to the testing framework and involved components like PHPstan, PHPUnit and Composer writing many, many small files this can make your tests several HUNDREDS percent slower, especially on macOS.

Examples

All examples expect to be executed from a git cloned working directory of TYPO3 CMS main branch (as described in TYPO3 Core Contribution Guide).

Build/Scripts/runTests.sh -s composerInstall

Build/Scripts/runTests.sh -s cglGit

Build/Scripts/runTests.sh -s cglGit -n

Build/Scripts/runTests.sh

./Build/Scripts/runTests.sh -x

Build/Scripts/runTests.sh -x <directory or file>

Build/Scripts/runTests.sh -x typo3/sysext/core/Tests/Unit/LinkHandling/

./Build/Scripts/runTests.sh -s functional

./Build/Scripts/runTests.sh -s functional -d postgres

./Build/Scripts/runTests.sh -s acceptance

Troubleshooting

Before diagnosing problems in the script, make sure to check if Docker is running smoothly on your system.

A quick test is to run the docker hello-world image:

docker run hello-world

You should see something like this message:

Hello from Docker!
This message shows that your installation appears to be working correctly.

See Get Started, Part 1: Orientation and setup

Also, see docker troubleshooting pages for more in-depth information, such as Docker for Mac: Logs and troubleshooting

There is no built-in debugging to the runTests.sh script. It can happen that (docker or other) command execution needs to be debugged.

For this you need to get your hands dirty and inspect the file. Most commands are performed like this:

lintScss)
    COMMAND="cd Build; npm ci || exit 1; node_modules/grunt/bin/grunt stylelint"
    ${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name lint-css-${SUFFIX} -e HOME=${CORE_ROOT}/.cache ${IMAGE_NODEJS} /bin/sh -c "${COMMAND}"
    SUITE_EXIT_CODE=$?
    ;;

to debug this, you can add an "echo" statement to reveal $COMMAND and also put an echo before the ${CONTAINER_BIN} statement, like this:

lintScss)
    COMMAND="cd Build; npm ci || exit 1; node_modules/grunt/bin/grunt stylelint"
    CONTAINER_COMMAND="${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name lint-css-${SUFFIX} -e HOME=${CORE_ROOT}/.cache ${IMAGE_NODEJS} /bin/sh -c ${COMMAND}"
    echo "EXECUTE:"
    echo $COMMAND
    echo $CONTAINER_COMMAND
    SUITE_EXIT_CODE=$?
    ;;

Then when you execute the script, instead of the command being executed, you can see what is used. Copy+paste that execution to your local shell and see if you can spot specific errors. You may want to add options like -v to the $COMMAND to see, why that command may fail.

Results

All results will be displayed on the screen. The script should exit with standard exit codes:

• 0 means all is ok
• != 0 means error

Reports of the acceptance tests will be stored in typo3temp/var/tests/AcceptanceReports with screenshots from the remotely controlled browser.

Direct commands without Docker

If you have problems with docker, you can run some of the lowlevel scripts and commands directly. However it does depend on your system, whether they run successfully (due to PHP, composer and other dependencies). The docker / runTests.sh method gives us the possibility to have a controlled environment where the tests run in. That also means, the databases that the functional tests require are already created automatically. That is not the case, if you run the tests locally on your current system.

That being said, running the tests directly is not being officially supported, but you can try this out yourself.

You can look in the source of Build/Scripts/runTests.sh to see which commands the runTests.sh script calls and run these directly (also see the "Troubleshooting" section above). Not everything will work, because there may be dependencies, that are only available in the docker container.

Also, you can run some of the scripts in Build/Scripts directly, which are otherwise just dispatched from within a docker container.

Examples:

bin/phpunit -c vendor/typo3/testing-framework/Resources/Core/Build/UnitTests.xml

bin/phpunit -c vendor/typo3/testing-framework/Resources/Core/Build/UnitTests.xml <directory or file>

bin/phpunit -c vendor/typo3/testing-framework/Resources/Core/Build/UnitTests.xml typo3/sysext/core/Tests/Unit/LinkHandling/

bin/phpunit -c vendor/typo3/testing-framework/Resources/Core/Build/FunctionalTests.xml

Build/Scripts/cglFixMyCommit.sh

Build/Scripts/cglFixMyCommit.bat

More information

• More details about the test system, test strategies, execution and set up can be found in TYPO3 explained.

Introduction

This chapter is about executing TYPO3 Core tests locally and is intended to give you a better understanding of testing within TYPO3's Core. A full Core git checkout comes with everything needed to run tests in TYPO3.

Core development is most likely bound to the Core main branch - backporting patches to older branches is usually handled by Core maintainers ("Mergers") and often doesn't affect other Core contributors.

The main entry point to perform testing and build-related actions is the helper Build/Scripts/runTests.sh. It works best when executed on a Linux based host but can be run under macOS and Windows with some filesystem performance drawbacks on macOS. It utilizes Docker containers, and more details can be found in Using runTests.sh.

Additionally, it is possible to execute tests on a local system without using Docker. Depending on which test suite is executed, developers may need to configure their environments to run the desired test. We however learned not too many people actually do that as it can become tricky. This chapter does not talk about test execution outside of Build/Scripts/runTests.sh.

System dependencies

Many developers are familiar with Docker. As outlined in the history chapter, test execution needs a well defined, isolated, stable and reliable environment to run tests and also remove the need to manage niche dependencies on your local environment for tests such as "execute functional test 'X' using PostgreSQL with xdebug".

Git and docker (or podman) are required. For standalone test execution, a local installation of PHP is not required. You can even composer install a Core by calling Build/Scripts/runTests.sh -s composerInstall in a container.

If you're using a Mac, install or update Docker to the most recent version using the packaging system of your choice.

If you are using Ubuntu Linux 18.04 or higher, everything should be ok after calling sudo apt-get install git docker once. For other Linux distributions including older releases of Ubuntu, users should have a look at the Docker homepage to see how to update to a recent version. It usually involves adding some other package repository and updating / installing using it. Make sure your local user is a member of the docker group, else the script will fail with something like /var/run/docker.sock: connect: permission denied.

Windows can rely on WSL to have a decent docker version, too.

Quick start

From now on, it is assumed that git and docker (or podman) are available with the most up-to-date release running on the host system. Executing the basic Core unit test suite boils down to:

# Initial core clone
git clone git@github.com:typo3/typo3.git && cd typo3
# Install Composer dependencies
Build/Scripts/runTests.sh -s composerInstall
# Run unit tests
Build/Scripts/runTests.sh

That's it. You just executed the entire unit test suite. Now that we have examined the initial Core clone and a Composer install process, we will look at the different ways we can utilize the runTests.sh for other scenarios.

Overview

So what just happened? We cloned a Core, Composer installed dependencies and executed Core unit tests. Let's have a look at some more details: runTests.sh is a shell script that figures out which test suite with which options a user wants to execute, does some error handling for broken combinations and then uses local docker commands to run specific containers with specific options. Using these containers, the actions are performed, and the containers are stopped after execution.

A Core developer doing this for the first time may notice that several container images will be pulled before continuing. These are the dependent images needed to execute certain jobs. For instance, a container providing the specific PHP-version may be fetched. The same containers are used for the TYPO3 CI GitLab Pipeline, even utilizing the same runTests.sh script. What's impressive is that you can locally run the same tests like a fully-fledged CI server..

The GitLab CI Pipeline is maintained through the Ansible infrastructure found on https://git.typo3.org/typo3/CI/testing-infrastructure/-/tree/main/ansible?ref_type=heads, and the Pipeline itself is set up through https://github.com/TYPO3/typo3/tree/main/Build/gitlab-ci/.

Compared to your local execution it's just that the combinations of tests and splitting to different jobs which is slightly different, for instance GitLab CI paralelly performs multiple tests with more complex version matrixes (PHP and Databases).

If a patch is pushed to GitLab and it complains about something being broken, it is possible to replay and fix the failing suite locally, then push an updated patch and hopefully enable the tests to pass.

A runTests.sh run

Let's pick a runTests.sh example and have a closer look:

lolli@apoc /var/www/local/cms/Web $ Build/Scripts/runTests.sh -s functional typo3/sysext/core/Tests/Functional/Authentication/
PHPUnit 11.2.5 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.19
Configuration: /Users/garvin/TYPO3/typo3-core-bugreproduce-base/typo3-core/Build/phpunit/FunctionalTests.xml

................................................................. 65 / 67 ( 97%)
..                                                                67 / 67 (100%)

Time: 00:12.077, Memory: 103.00 MB

OK (67 tests, 176 assertions)

###########################################################################
Result of functional
Container runtime: docker
PHP: 8.2
DBMS: sqlite
SUCCESS
###########################################################################

lolli@apoc /var/www/local/cms/Web $ echo $?
0
lolli@apoc /var/www/local/cms/Web $

The command asks runTests.sh to execute the "functional" test suite -s functional and to not execute all available tests but only those within typo3/sysext/core/Tests/Functional/Authentication/. The script first starts the containers it needs: Redis, memcached (previously also MariaDB by default, which is now using SQLite instead, due to less dependencies). All in one network. It then starts a PHP 8.2 container and calls phpunit from there to execute the tests. phpunit executes only one test in this case, that one is green. The containers and networks are then removed again. Note the exit code of runTests.sh (echo $?) is identical to the exit code of the phpunit call: If phpunit reports green, runTests.sh returns 0, and if phpunit is red, the exit code would be non zero.

Examples

First and foremost, the most important call is -h - the help output. The output below is cut, but the script returns a useful overview of options. The help output is also returned if given options are not valid:

lolli@apoc /var/www/local/cms/Web $ Build/Scripts/runTests.sh -h
TYPO3 Core test runner. Execute acceptance, unit, functional and other test suites in
a container based test environment. Handles execution of single test files, sending
xdebug information to a local IDE and more.
...

Some further examples: The most important tests suites are unit tests, functional tests and acceptance tests, but there is more:

# Execute the unit test suite with PHP 8.3
Build/Scripts/runTests.sh -s unit -p 8.3

# Execute some backend acceptance tests
Build/Scripts/runTests.sh -s acceptance typo3/sysext/core/Tests/Acceptance/Backend/Topbar/

# Execute some functional tests with PHP 8.2 and postgres DBMS
Build/Scripts/runTests.sh -s functional -p 8.2 -d postgres typo3/sysext/core/Tests/Functional/Package/

# Execute the cgl fixer
Build/Scripts/runTests.sh -s cglGit

# Verbose runTests.sh output. Shows main steps and Composer commands for debugging
Build/Scripts/runTests.sh -v

As shown there are various combinations available. Just go ahead, read the help output and play around. There are tons of further test suites to try.

Also note that you can use the -b option to switch between docker and podman container execution, with podman being the default (when available).

One interesting detail should be mentioned: runTests.sh uses several containers from https://github.com/orgs/TYPO3/packages for PHP and JavaScript environments. Use the command Build/Scripts/runTests.sh -u to fetch the latest versions of these containers.

Debugging

To speed up test execution, the PHP extension xdebug is not usually loaded. However, to allow debugging tests and system under tests, it is possible to activate xdebug and send debug output to a local IDE. We'll use PhpStorm for this example.

Let's verify our PhpStorm debug settings first. Go to File > Settings > Languages & Frameworks > PHP > Debug. Make sure "Can accept external connections" is enabled, remember the port if it is not the default port (9000) and also raise "Max. simultaneous connections" to two or three. Note remote debugging may impose a security risk since everyone on the network can send debug streams to your host.

Accept changes and enable "Start listening for PHP connections". If you changed settings, turn them off and on once to read new settings.

Now set a break point in an assignment. Note break points do not work "everywhere", for instance not on empty lines and not on array assignments. The best way is to use a straight command. We'll use a simple test file for now, add a breakpoint and then execute this test. If all goes well, PhpStorm stops at this line and opens the debug window.

Build/Scripts/runTests.sh -x -s functional -p 8.1 -d postgres typo3/sysext/core/Tests/Functional/Package/RuntimeActivatedPackagesTest.php

The important flag here is -x! This is available for unit and functional testing. It enables xdebug in the PHP container and sends all debug information to port 9000 of the host system. If a local PhpStorm is listening on a non-default port, a different port can be specified with -y.

If PhpStorm does not break as expected, some adjustments in this area may be required. First, make sure "local" debugging works. Set a breakpoint in a local project and see if it works. If it works locally, the container based debugging should also work. Next, make sure a proper break point has been set. Additionally, it may be useful to activate "Break at first line in PHP scripts" in your PhpStorm settings. runTests.sh mounts the local path to the same location within the container, so path mapping is not needed. PhpStorm also comes with a guide how to set up debugging.

Building

Luckily, runTests.sh also helps us to build JavaScript and CSS assets:

Build/Scripts/runTests.sh -s build

Again, this utilizes all the needed containers for the proper NodeJS environment, so you have zero local dependencies on properly building.

You can also run a watch task thanks to the full integration of npm command execution:

Build/Scripts/runTests.sh -s npm -- run watch:build

Introduction

The TYPO3 Core development has quite an impressive history on automatic testing. This chapter outlines some of the important steps the system went through over the years. It may be just an interesting read but may also explain why things are as they are now.

Next to this chapter, typo3.com blog picks up the testing topic once in a while. If a developer reads this who still needs to convince management persons that testing saves time and money on a project, the series starting with Serious Software Testing may give some ideas.

2009

The first Core unit test has been committed in early 2009. The Core was still using SVN as version control system at this point. More than ten years ago. The tests have later been released with TYPO3 version 4.3 in fall 2009. The system back then relied on the TYPO3 extension phpunit. This TER extension bundled the native phpunit package and added a TYPO3 backend module on top. It found all extensions that delivered unit tests, allowed to execute them and showed the result in the module. This was the first time "green bar feeling" came up: All tests green.

2012

This was after TYPO3 v4.5 times - a version that carried us for a long time. Several TYPO3 Core contributors meanwhile added some hundreds of unit tests in various Core extensions. There was an issue, though: Not too many persons developing the TYPO3 Core cared about unit tests and executed them before providing or merging patches. As a result, tests were frequently failing and only a small group of persons took care and fixed them once in a while. Unit tests and system under test are symbiotic: If one is changed, the other one needs changes, too. If that does not happen, unit tests fail.

However, the young project Travis CI came online and allowed free test environments for open source projects. The TYPO3 Core quickly started using that, a first .travis.yml has been added early 2012 and all merged patches executed the test suite. Persons merging patches got feedback on failed builds and were able to act upon: Either fix the build or revert the patch. The Core Team established an "always green" rule for Core development.

The Travis CI setup at this point basically created a working instance around the checked out the Core to run tests: It additionally cloned a helper repository, cloned the phpunit extension, did set up a database and other stuff, then executed the Core unit tests to result with "good" or "bad".

Until 2018, this first .travis.yml file went through more than 100 changes.

2013

With frequent test execution via Travis CI more and more developers working on the Core were forced to run tests locally to debug tests or add new ones. We slowly got an idea in which situations unit tests are helpful and when they are not.

One flaw in our unit test system became more pressing, though: The phpunit extension that we relied on has been designed to run tests in the backend context of TYPO3 as a module. The unit tests were executed with all the state the backend created to run modules. This was troublesome since lots of unit tests now directly or indirectly relied on this state, too. And worse, this state changed depending on the developers local test system - other extensions that hooked into the system could lead to failing unit tests. With this system, tests tend to execute fine locally but then broke on Travis CI or on some other persons development system. Test execution has at this point already been done via CLI by most developers, and the unit test bootstrap basically created a full TYPO3 backend context similar to the GUI based phpunit extension.

Moreover, we had many tests that somehow changed global state and then influenced other tests. This part lead to the situation that a test worked if executed as single test but failed if executed together with all others - or vice versa.

We ultimately learned at this time that managing system state is an essential part of the Core framework. And so we started refactoring: The Core bootstrap has been hacked into manageable pieces that could be called by the unit test bootstrap in small steps. The tests started to become "real" unit tests that test only one small piece of code at a time.

With improving the unit tests and their bootstrap it also became clear that we needed a second type of tests that do the opposite of unit testing: We wanted to test not only a single isolated fraction of code, we also wanted to test the collaboration of bigger framework parts that includes many classes and database operations. In short: functional testing.

With our learning's from unit tests however it was clear that functional test execution needed to be executed in a well defined and isolated environment to be reliable: We could not just execute them in the context of the local developers system. Moreover, we had to isolate tests from each other: PHP is designed to work on a per-request basis. A CLI or web request comes in, the system bootstraps, does the job, then dies. The next request does a new bootstrap from scratch. This simplifies things a lot for developers since they don't need to take care of request overlapping state and don't need to take care too much about consumed memory. And if a single request dies in the middle of the execution, the next one still may happily work and successfully do its job. This characteristic of a scripting language can be a huge advantage over other server-side languages. And TYPO3 uses this a lot: If a request is finished in TYPO3 context, the system is "tainted" and can't be used for a second request again.

To handle this situation we came up with a functional bootstrap that creates standalone TYPO3 instances per test file. The system sets up a new TYPO3 instance for each test file in an own folder in typo3temp, links over source code from the main system, creates configuration files in this instance, creates a dedicated database and initializes it with all tables needed by extensions loaded in this instance. To then handle the isolation of single requests, each functional test runs as a new forked process that does not carry any information from another test run. Doing all this was expensive, but at least the functional test were so stable that we had very little trouble with tests that execute fine on one system but fail on another one. Additionally, running functional tests could never destroy the main instance.

2014

Next to tons of detail changes, two main steps happened in 2014.

First, the unit test isolation has been finished. The initiative "standalone unit test" changed the unit test bootstrap to execute only a very basic part of the system. Instance specific configuration files like config/system/settings.php were no longer read, no database connection established, the global backend user and language objects were no longer set up and so on. In the end, not much more than the class auto loading is initialized. To reach this, many tests had to improve their mocking of dependencies and had to specify the exact state they needed. With this being done, side effects between tests reduced a lot and a dedicated unit test runner executing tests in random order was added to find situations where test isolation was still not perfect. Nowadays unit testing is pretty stable on all machines that execute them due to these works. With nearly ten thousand tests in place it is rather seldom that a test fails on one machine and is successful on another. And if that happens, the root cause is often a detail down below in PHP itself that has not been perfectly aligned during test bootstrap - for instance a missing locale or some detail php.ini setting.

Second, the test execution was changed to use a Composer based setup instead of cloning things on its own. This was at TYPO3 v6.2 times when Composer was first introduced in TYPO3 world - testing was one of the first usages. In this process we were able to ditch the TYPO3 specific extension based flavor of phpunit and switched to the native version instead. This turned out to be a wise decision since TYPO3 Core testing now no longer relied on development of a third party TER extension but could use the native testing stack directly and for instance pick up new versions quickly.

2015

Functional testing gained a lot of traction: The DataHandler and various related classes in the TYPO3 Core are the most crucial and at the same time complex part of the framework. All the language, multi-site, workspace and inline handling is nifty and it's hard to change code in this area without breaking something. This is still an issue and improving this situation is a mid- to long-term goal. So we decided to use functional tests to specify what the DataHandler does in which situations. There are hundreds of tests that play through complex scenarios, example: "Add some fixture pages and content, call DataHandler to create a localized version in a workspace, call DataHandler to merge that workspace content into live, verify database content is as expected, set up a basic frontend, call frontend as see if expected content is rendered." Nowadays, if changing DataHandler code, functional tests can tell precisely if a change in this area is ok or not. As a result, we don't see many regressions in this area anymore.

Adding so many functional tests has a drawback, though: The needed isolation and expensive functional test setup is rather slow. Executing the functional test suite means creating tens of thousands of database tables. While unit testing is quick (a decent machine handles our ten thousand unit tests in thirty seconds), executing a thousand functional tests can take an hour or more. This can be improved by setting up a database in a memory driven ram disk and some other tricks, but still, functional test execution is clearly not a super charged turbo.

Additionally, we had to increase the test isolation even more: There are test scenarios that execute both backend and frontend functionality. This is hard in TYPO3: A backend request is a backend request and it can't be used as a frontend request at the same time. Extension developers may know this: In TYPO3 it's hard to do frontend requests from within the backend or from CLI - extensions like solr or direct_mail struggle at this point, too and need to find some solution working around this. In functional testing, a test scenario that does a frontend request thus forks processes twice: First, the backend part is executed as standalone process as explained above, which then forks another process to execute the frontend request. As a result, only hard-boiled Core developers tend to work on such functional tests: They are slow, hard to debug and complex to set up.

2016

In early 2016, Core developers added another type of testing: Acceptance tests. Those tests use a browser to actually click around in the backend to verify various parts of the system. For instance, TYPO3 Core had a history of breaking the installation procedure once in a while: Most Core developers set up a local development system once and then never or only seldom see the installation procedure again. If code is changed that breaks the installer, this may go through not noticed. Acceptance testing put an end to this: There are tests to install a fresh TYPO3, log in to the backend, install the introduction extension and then verify the frontend works. The installer never hard broke again since that. Other acceptance tests nowadays use the styleguide extension to click through some complex backend scenarios, verify the backend throws no javascript errors and so on.

We however quickly learned that acceptance testing is fragile: Unit and functional testing has been stabilized very well meanwhile - they do not break at arbitrary places. Acceptance testing however is more complex: A web server is needed, some system to pilot the browser is needed, single clicks may run into timeouts if the system is loaded, pages are sometimes not fully loaded before the next click is performed. Additionally the TYPO3 backend still relies on iframes for all main modules, which again does not simplify things. It took the Core development two further years to stabilize this well enough so acceptance tests could be executed often without throwing false positives at various places. In the end acceptance testing is another great leap forward to ensure major parts of the TYPO3 Core do work as expected.

Another thing became more and more pressing in 2016: The automatic testing via Travis CI started to show drawbacks. We continued adding lots of tests and test suites over the years and executing everything after each code merge took an increasing amount of time. Even with all sorts of tricks, Travis CI was busy for more than half an hour to go through the suite, merging more than two patches per hour thus added to a queue. There were Core code sprints were Travis reported green or red on a just merged patch only half a day later. We tried to pay the service for more processing power, but payed plans do not work with Travis CI for open source repositories (maybe they changed that restriction meanwhile). We also knew that the amount of tests will increase and thus lead to even longer run times. Additionally, Travis CI was configured to only test patches that were actually merged into the git main branches. So we always only knew after a merge if the test suite stays green. But we wanted to know if the test suite is green before merging a patch. Enabling Travis CI to test each and every patch set that is pushed to the review system was out of question due to the long run times, though.

So we looked for alternatives. Luckily, the TYPO3 GmbH was founded in 2016 and got an open source license by atlassian for their main products. Atlassian has an own continuous integration solution called bamboo. This CI allows adding "remote agents" that pick up single jobs to run them. It's possible to scale by just adding more agents. We thus split the time consuming test tasks into single parts and execute them in parallel on many agents at the same time. This also allowed us to execute the test suite before merging patches: If pushing a patch set to the review system, the bamboo based testing immediately picks up the new patch version and runs the entire suite, a result is reported a couple of minutes later. So, this is all about throwing enough hardware at the testing issue: The TYPO3 GmbH has a deal with the Leibniz Rechenzentrum who grant us hardware on one of their clusters to perform the tests.

2017

To the end of TYPO3 Core v8 development the bootstrap, helper and set up code to execute Core tests has been extracted from the Core to an own repository, the typo3/testing-framework. This allowed re-using this package within extensions to execute own tests. It however took that repository another major Core version to mature well enough to easily do that. Writing and executing tests for TYPO3 extensions is possible for a long time already, but extension authors were mostly on their own in finding a suitable solution to do that. This chapter may put an end to this confusion.

2018

Since 2016, the TYPO3 Core test setup went through further changes and improvements: Various test details were added that checked the integrity of the system. TYPO3 v8 switched to doctrine so we started executing the functional tests on meanwhile four different database systems, a nightly test setup has been established that checks even more system permutations and software dependencies and much more.

As another important step, the Core developers worked on the functional test isolation again in TYPO3 v9: As explained above, the functional tests forked processes twice if frontend testing was involved. With TYPO3 v9 however, the TYPO3 Core bootstrap has been heavily improved, with having a special eye on system state encapsulation: Next to the incredible PSR-15 works in this area, two further API's have been established: Context and Environment. Remember each functional tests case runs in an own instance within typo3temp? TYPO3 Core always had the PHP constant PATH_site that contained the path to the document root. With having test cases in different locations, this constant would have to change. But it can't, it is a constant and PHP luckily does not allow redefining constants. The environment API of TYPO3 Core v9 however is an object that is initialized during Core bootstrap. Next to some other details, it also contains the path to the document root. Adding this class allowed us to ditch the usage of PATH_site in the entire Core. This removed the main blocker to execute many functional test suites in one PHP process. After solving another series of hidden state of the framework, the functional test setup could finally be changed to not fork new processes for each and every test anymore. So now, we can proof that one TYPO3 backend instance can handle many backend requests in one process - we are sure our framework state is encapsulated well enough to allow such things. This change in the TYPO3 Core and dropping the process isolation for functional backend tests significantly simplified working with functional tests now and debugging is much easier and improved Core code at the same time. This pattern repeated often over the years: The test suites show quite well which parts of the Core need attention. Working in these areas in turn improves the Core for everyone and allows usages that have not been possible before.

In late 2018 another thing has been established: The runTests.sh script allows Core developers to easily execute tests within a container based environment that takes care of all the nasty system dependency problems. The test setup for some test suites is far from trivial: Acceptance tests need a web server, chrome and selenium, functional tests need different database systems that at best run in RAM, and so forth. Not too many Core developers went through all that to actually run and develop tests locally. The script now hides away all that complexity and creates a well defined and simple to use environment to run the tests, the only dependencies are recent docker and docker-compose versions.

2019

The above milestones show that efforts in the Core testing area have positive effects on Core and extension code and allow system usages that have not been possible before.

There are some further hard nuts we have to crack, though: For example, while the process isolation for functional backend tests has been dropped in 2018, the tests still fork processes to execute frontend scenarios. This is still ugly. It shows that calling a TYPO3 frontend from within the backend context or from CLI is still not easily possible. As a goal, a developer in such a situation would usually want to do this: Preserve the current framework state, create a PSR-7 request for the frontend, fire it, get a PSR-7 response object back, reset the framework state and then further work with the response object. Lots of details to allow this are in place since TYPO3 v9 already, with only some missing details: For instance, there is that nasty constant TYPO3_MODE that is set to "FE" in a frontend call and "BE" in a backend call. So yeah, this constant is not constant. It is one of the main blockers that prevents us from dropping the backend/frontend functional test isolation. So, this constant must fall, and this will be one of the things that will be hopefully resolved with TYPO3 v10. As soon as this last process isolation is dropped from the functional test setup, extension authors will know that executing a frontend request from within the backend must be easily possible. We're looking forward to that - it will be one of the last steps to finally manage framework state in a good way and maybe we can rewrite this documentation section soon.

2020

The pending milestone of 2019 has been achieved in late 2020: The core functional tests no longer spawn PHP processes to execute frontend requests. A PSR-7 sub request is initiated with core v11 instead.

This is quite an achievement: It is the proof that core framework state is encapsulated well enough to finally execute a frontend request from within a backend request or CLI. As one major pre-condition, the broken constant TYPO3_MODE is finally gone (deprecated and unused in core). Further core versions can drop that constant and extensions will have to drop their usage, too. So with v12, TYPO3_MODE will be gone, and TYPO3 can create cool features from this. So again, the core testing paved the way for new opportunities and TYPO3 usages.

2024

TYPO3 currently uses 4 dedicated "bare bone" Servers to perform CI tasks. This hardware performs 11.513 unit tests in  15 seconds. A typical pre-merge pipeline runtime is at  5 minutes with 2 permutations of acceptance tests, 3 permutations of  7500 functional tests, 3 permutations of unit tests plus statical code analysis, linting, build checks.

The 4 servers are provisioned using Ansible: https://git.typo3.org/typo3/CI/testing-infrastructure/-/tree/main/ansible?ref_type=heads and Pipelines configured in https://github.com/TYPO3/typo3/tree/main/Build/gitlab-ci/.

Debugging With PhpStorm and Xdebug

In order to configure PhpStorm with Xdebug you need to do three things:

1. Install xdebug
2. Configure xdebug settings in your php.ini
3. Use the appropriate plugin in your browser
4. Configure PhpStorm

xdebug.remote_enable = 1
xdebug.remote_host = localhost
xdebug.max_nesting_level = 1000

More Information

• Configuring Xdebug with PhpStorm (Jetbrains)

Tips for new contributors

You are a new contributor. You found an issue in the Core you wanted to see fixed, you found a solution, you are satisfied with it, you setup your environment, you figured out how to create a patch and push to Gerrit, you managed to create a forge issue and you finally pushed your patch.

Congratulations! That is great!

Now your patch is hanging around in the review queue on Gerrit and either nothing happens, it is voted down, people tell you your patch is not good enough or suggestions are made to improve it and you don't know how.

Don't despair young padawan! The community around TYPO3 tries to act friendly and helpful, we're not hurting anyone on purpose. We're open to anyone and especially like to see new contributors and try to help getting them onboarded and up to speed. Please consider the following things before becoming frustrated or giving up:

• Most of the time spent on TYPO3 CMS core development is done for free. This gives contributors and reviewers freedom on what they work on. Unfortunately, this also means that sometimes a patch is not given the dedication it should receive.
• The review queue is often longer than the review power available. (See the dashboard on Forger for the number of open reviews.) Sometimes patches are overlooked just because there is other hot stuff being worked on.
• Parts of the team usually work on something bigger. It may happen that some patches in the hot areas are merged very quickly while others do not get any attention. This is somehow natural and cannot be avoided since at any given time some reviewers usually focus on certain areas to get them right.
• Sometimes the whole team focuses on different stuff. For example, usually around "big" releases, the team focuses on bug fixing, so feature patches may get stuck for some time.
• Sometimes large parts of the active contributors are inactive at the same time. There are for example main holiday sessions with low overall activity.
• Sometimes getting little or no feedback on patches also has other reasons. Maybe the issue description is insufficient or not understood, maybe there is no documented way on how to reproduce a certain issue, or the bug or feature is bogus, or the issue is so complex and hard to solve that no one really wants to get their hands dirty on it.

Tips for reviewers and active contributors

First, everyone is allowed to vote positive and negative on pending patches in the Gerrit review queue for the TYPO3 CMS core product. We appreciate good reviews on both code review and testing and this entry point was used by quite some people already who were later nominated to be component or framework mergers.

Frequently giving good reviews and improving patches is noticed and rewarded and the team recognizes new helping hands. Another good entry point is joining code sprints, look out for some if you have ambitions to be involved in core development.

Acting as a reviewer and especially as an active contributor gives some additional responsibility in dealing with the project and working with other people in the review system. While technical discussions are sometimes mastered with different opinions clashing each other, it is always important to be respectful and fair to each other. There is no reason to act rude just because some other reviewer has a different view to a specific point.

That being said, now some common guidelines and practical hints on reviewing habits.

Overview of the UI

This chapter will explain the most important parts of the Gerrit UI to you. For most developers, this interface should be mostly self-explanatory. Gerrit also comes with its own help system.

This is a screenshot of an open review on Gerrit. We will go through the parts of the UI next:

1. The search box lets you write complex search queries with little coding knowledge. For detailed information on how to use the search, refer to the official documentation on https://review.typo3.org/Documentation/user-search.html. Most people use Forger, because it provides more sophisticated ways to find a review. Look at Find a review on Gerrit for more information.
2. The commit message formatted like we explained in "The commit message".
3. The Reply Button which you will use to vote on code quality and testing as well as to send comments. We will be covering this button in more detail later.
4. The Download Button. You will be using this control to cherry pick a change.
5. Other changes related to this change. This can be either due to a similar topic or because the commit summary is related to the change we are currently reviewing.
6. The current review status. Here you can see who has voted (and how they voted) on a change. The votes only apply to one given patchset. If a new patchset is uploaded, it must be revoted. Note the +1 by TYPO3com, which means that the automatic testsuite ran successfully.
7. All changed files in this current change. You can click every file to take a look at what exactly has changed. There is a select box on top of the file list that allows you to do a diff between patch sets to quickly see what has changed.
8. The history of this change. Here you can see comments, new patch sets, votes on a change etc.

Forger

In any case, you can use Forger to search for the review.

Examples:

• select Reviews : All to view and filter for reviews
• Sprints Reviews to see a board of all open patches by type (Bugs, Tasks, Features)

• Sprints TYPO3-Core to see a board of all open core issues by type (Next patchlevel, Regressions)

Email notification

If you are in any way involved in the review (e.g. you are author, reviewer or you made changes), you will get a notification about it to your email address. The notification contains a link.

Gerrit: Your Changes

For your own patches, go to Gerrit and select Your: Changes.

Gerrit: Open Changes

Select Changes : Open to get the latest changes or changes with recent modifications.

Gerrit: Search

Or use the search box on Gerrit

GitHub: Links to Gerrit

Each commit to the TYPO3 Git repository comes with some metadata linking issue and patch reviews, for example:

[TASK] Provide PSR-7 Request in PolicyMutatedEvent

For additional context does the PolicyMutatedEvent
now provide the current PSR-7 Request.

Resolves: #104141
Releases: main, 13.4
Change-Id: I1817366e77f20f6c43eef0ee209fbb419e7237e2
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/84913
Tested-by: Lorem Ipsum <example@example.com>
Reviewed-by: Lorem Ipsum <example@example.com>
Tested-by: core-ci <typo3@example.com>

The line Resolves contains the Forge issue id. The line Reviewed-on contains the link to the gerrit patch.

Forge

Once a patch has been pushed for an issue, the corresponding issue on Forge will contain comments with the topic "Updated by Gerrit Code Review ..." and a link to the review on Gerrit.

Cleanup git repository

Before Cherry-picking a patch or starting a new patch:

Build/Scripts/runTests.sh -s clean && \
    git fetch --all && \
    git reset --hard origin/main && \
    git pull --rebase

Cleanup TYPO3 installation

After downloading a new patch or making changes to an existing patch, you may need to cleanup your TYPO3 installation.

It depends on the files that have changed, what you need to execute. In any case or if in doubt, you can safely perform all steps.

ddev typo3 cache:flush && \
    ddev typo3 cache:warmup

Build/Scripts/runTests.sh -s composerInstall

Changes in .css / .js files:

Delete browser cache or hard refresh (e.g. CTRL + F5)

Also you may need to create the JS/CSS assets:

./Build/Scripts/runTests.sh -s build

Changes in DB schema (ext_tables.sql):

ddev typo3 extension:setup

Maintenance: Analyze Database Structure, Apply selected changes.

Update core-testing docker images

./Build/Scripts/runTests.sh -u

./Build/Scripts/runTests.sh -u && \
    Build/Scripts/runTests.sh -s clean && \
    git fetch --all && \
    git reset --hard origin/main && \
    git pull --rebase && \
    ./Build/Scripts/runTests.sh -s composerInstall && \
    ddev typo3 cache:flush && \
    ddev typo3 cache:warmup && \
    ddev typo3 extension:setup

Code Review

A basic code review is possible by using the Gerrit web interface.

For some tips on what to review, check our Common code review checks / checklist.

1. Select the latest patchset

   This should already be selected by default, but if you changed the patchset, you should select Go to the latest patch set (above the list of changed files).

2. To leave a comment, click on one of the files

   

   Commenting in the files directly is optional, but strongly recommended. If you just want to leave a general comment which needs no context, you can skip to step 8 (Reply).

3. Optionally change the view

   You will see a diff of this file against the current codebase.

   You can change the view in the top right (Diff view) e.g. side-by-side or unified (or change this in your settings).

4. Leave one or more comments

   Click on the line number in the file and a comment box will open.

   

5. Click Save.

   Important

   You comment(s) will not yet be visible for others until you go back and click Reply.

6. Go back

   
7. (Optional) Add comments to more files.

8. Now press Reply.

   Using the Reply button, you can post your comments (and optionally add an additional note).

   

   Of course you should also vote for the change (Be graceful with -1 votes though).

Test a patch

For testing the patch you need to import the change into your local repository.

Look at Cherry-pick a patch for information on how to do this.

Test the patch in your local TYPO3 installation and verify the reported bug is fixed and no other bugs are introduced with the change.

Depending on the outcome of your tests, place your positive/negative vote in Gerrit, using the Reply button.

If you want to help the author and provide an improved patch, continue with the section Upload a new Patch Set.

Otherwise throw the changes away, to bring your repository back to a clean state:

git reset --hard origin/main

Vote

In order to comment or vote on a change you can click on the Reply and enter your comment. Here, you can also apply your votes.

Remember, everyone with just a TYPO3 user account is able to vote, you do not need to be a team member or Core merger.

• +1 : you approve of the patch
• -1 : you do not approve, in this case give your reason as a comment

Click on Send and your comments will be saved. At the same time all other contributors who either watched this change or have already voted on this change will get notified.

How to handle [WIP] patches

It is possible to push patches as "WIP" (work in progress) patches. This is a way to show people a quick-shot version of something that is not finished yet, but goes into the right direction. Usually, WIP patches are not actively reviewed by others and the original author should take responsibility to finish this patch later.

As a contributor, you usually can not expect someone else to pick up your WIP patch and finish it, except you stated that goal clearly: "Hey, I've done a quick shot with this patch to show a possible solution for this issue, but the patch is not finished yet. Foo and bar is missing and we still need a concept for foobar. I'll probably not work actively work on it anytime soon, but maybe the current state is helpful already".

Better: "Hey, I worked in this area and came up with this WIP patch for now. I wanted to show into which direction this patch is leading, but we currently have some open questions. However, it would be great if you can give me feedback to the general approach at this early state already to decide if it is worth following this solution to its end."

Having too many WIP patches in the review queue is not really helpful. Consider to fork the project in GitHub or somewhere else and push to gerrit again if your patches evolved.

Stale reviews / Cleaning up

Since TYPO3 iterates on many, many patches and issues each day, the list of Gerrit reviews (as well as Forge issues) may feel intimidating.

Due to this it is vital to "clean up" patches from time to time:

• If you are no longer planning to work on a patch, maybe better abandon it, or ask other involved people to carry on.
• If your patch is not getting feedback for a long time, ask in the #typo3-cms-coredev channel of the TYPO3 slack workspace if people may want to review or give feedback on. Or try to find people working in a similar area of your patch and see if you can join forces.
• From time to time, check on patches you have voted on, to see if you can push things forward to either get merged or abandoned.
• Sometimes just check all your own open patches and see if you might catch interest in picking it up again.
• Please either update older patches in "Merge conflict" mode or state your intent to abandon the patch.

What is rebase?

Simply speaking, git reapplies commits on top of another base tip (as explained in the Pro Git book).

When you start your patch, it will be based on a specific commit. Meanwhile, other commits are merged and the TYPO3 codebase changes. Your patch will still be based on an older version of the source.

remote | local | commits
------------------------

 +                 x : latest commit
 |
 |
 +                 d
 |
 +                 c
 |       +         b : your commit
 |       |
 +--------         a : parent of your commit
 |
 +

When you rebase, your change, will be reapplied to a different commit (typically the latest one) and your patch will be based on the latest version.

There are other applications for rebase (e.g. squash commits), but only this is the one relevant for the TYPO3 Contribution Workflow, so this is what we mean when we talk about rebase.

Do not let information on the Web confuse you: Usually it is assumed that someone is working on a different branch (feature branch). In our commit model, we always work on one branch (main) and we only create one commit for a change.

When should you rebase?

When you are working on a patch, you can regularly rebase in order to get the latest changes.

You can also do it before you run tests and push your change.

In fact, it is good practice to do this!

How do you rebase?

The following assumes, there are not yet any merge conflicts. If there are merge conflicts, you must resolve them as you rebase / merge / cherry-pick. See the section Resolve Merge conflicts.

git pull --rebase origin main

git fetch
git rebase origin/main

What are merge conflicts?

Usually, git does a good job of merging several changes in one file.

However, there are occasions where git does not know what to do, because one change conflicts with another, meaning there are more than one possible way to merge.

This is actually a good thing.

In case of a merge conflict you will usually have these possibilities:

1. Use change 1
2. Use change 2
3. Use a combination which you manually combine.

111
222

111
aaa
222

111
bbb
222

111
aaa
bbb
222

111
bbb
aaa
222

111
<<<<<<< HEAD
bbb
=======
aaa
>>>>>>> aaa
222

How to see merge conflicts in Gerrit or Forger

If your patch has merge conflicts. Gerrit as well as Forger will show you:

Gerrit: Load your patch page in the browser.

Forger: Open one of the ReviewSprint views on Forger.

In that case, you will need to resolve the conflicts in some way.

See next section How to resolve conflicts? for more information about resolving merge conflicts.

How to resolve conflicts?

Some git commands will show you that there are conflicts e.g. git cherry-pick, git rebase, git pull --rebase etc. It will have inserted markers in the files and ask you to resolve them and then continue.

You must resolve the conflict for all these files. There are several ways to do this.

Most editors or IDEs assist you in doing this. Check the information for your IDE.

Resolving the conflict, involves a 3-step process:

1. Identify files with conflicts: the git commands usually show which files have conflicts, also your Editor or IDE should show you (PhpStorm does).
2. Resolve the conflicts
3. Resume: Depending on the command that was executed (e.g. rebase, merge, cherry-pick), you will usually need to add the files and call something like rebase --continue or cherry-pick --continue

CONFLICT (content): Merge conflict in typo3/sysext/install/Classes/Updates/ExtensionManagerTables.php

$ git cherry-pick d48c3626d828de880342
$ > error: could not apply d48c362... bbb
$ > hint: after resolving the conflicts, mark the corrected paths
$ > hint: with 'git add <paths>' or 'git rm <paths>'
$ > hint: and commit the result with 'git commit'

$ git status
$ > On branch main
$ > You are currently cherry-picking commit 805a207.
$ > (fix conflicts and run "git cherry-pick --continue")
$ > (use "git cherry-pick --abort" to cancel the cherry-pick operation)
$ >
$ > Unmerged paths:
$ >  (use "git add <file>..." to mark resolution)
$ >
$ >  both modified:   file_with_conflicts.txt
$ >
$ > no changes added to commit (use "git add" and/or "git commit -a")

111
<<<<<<< HEAD
bbb
=======
aaa
>>>>>>> aaa
222

# optionally, reset state
Build/Scripts/runTests.sh -s clean

Build/Scripts/runTests.sh -s build

git add <path1>
git add <path2>

git rebase --continue

git cherry-pick --continue

git commit --amend
git push

Status

1. New

   This is the initial state of an issue. It may get reverted to this status, when possible feedback loops are done, but the issue itself could not yet be "Accepted" due to missing strategic or capacity decisions.

2. No status change, but "things happen"

   Your issue may remain in status "New" or "Accepted", but further feedback might be added, or relations to other issues could be added, or other metadata (like tags) get adjusted. If you see this happening, some of our support staff has "seen" your issue. If your issue is older, and you see this happening, this might be a good time to get involved again and see if your issue still applies.

3. Needs Feedback:

   Your issue has been set from Status "New" to "Needs Feedback".

   This means, we have further questions to process or discuss your issue. Once you provide the feedback, the person involved with it will usually get back to you again.

4. Accepted

   Your issue has been "accepted" to be reproducible and understandable, and will hopefully find a volunteer to fix it.

   Please bear in mind that this status does not mean someone immediately begins to work on it. It is just a status that developers can use to search for accepted issues to work on. And it is a good pool of issues to address for example in remote or on-site Code sprints (see https://typo3.org/community/meet/regular-open-sprints).

5. In Progress

   Someone has actively taken on your issue to work on.

6. Under Review

   This is the status you are looking for.

   This means a Patch has been made that should address your problem. The link to such a Patch in our review system will be attached to the comment.

   Now it is very important for that patch to:

   • actually fix your issue
   • be accepted by Core mergers to be part of the TYPO3 code

   Anyone can create/contribute a patch to any issue, one does not need to be assigned to it. One is also free to contribute to existing Patches of an issue.

   And most importantly: You are VERY welcome to review the patch yourself. If you can comment on the patch tracking system that a patch solves your problem, and even vote for it, this will highly increase the chances of your patch getting merged.

   Sometimes discussion about solving an issue is then carried on to the Gerrit review system, which is linked in the issue comment. You may want to regularly check on that patch state and maybe give feedback there.

   Please note that the patch system is meant for highly technical feedback. If your feedback is more general, please provide that only in the issue tracker.

   An issue that is "Under Review" may get merged very quickly, but it may also even take very long to get merged, or even rejected/abandoned.

7. Resolved

   Making a note here: Huge success! Your issue report lead to an actual code improvement of TYPO3, and the patch resulting from your issue has been merged. Thank you!

   Sometimes it might not be what you expected at first, because requirements and needs may change, and need to be adapted for the broad community.

   It can also occur that the issue you reported for a specific TYPO3 version is only fixed for a more recent version, and not the one you reported with. Sadly we cannot backport every single bugfix, and need to make internal decissions about which changes get put into older versions. Patches never get merged to versions below the LTS (Long-term support) threshold.

   The upside: The patch that has been made may apply to your own instance. You can always download patches and apply them locally, see Applying Core patches for details.

8. Closed

   This may be a frustrating status for your issue to be put in. We need to close issues that:

   • are no longer reproducible
   • are outdated
   • are not fixable/adressable within reasonable resource limits
   • do not align with the TYPO3 project goal
   • receive no feedback for a longer time
   • relate to abandoned patches

   If your issue has been closed but you believe this is a mistake, our support staff are just humans after all, please bear with us. You can still give feedback to closed issues, you can reach out through other community means, or you can also create follow-up issues.

9. Outside help / Volunteering opportunity

   The more watchers an issue has, the more likely it is regarded as being of importance.

   It can help if you regularly search the issue tracker yourself for similar problems than yours, and maybe set relations to new issues, or comment in issues that you haven't submitted yourself, but where you can contribute.

   The task of maintaining the issue tracker is a large one; several dozen issues are worked on each day. If you can help to add relations to related tickets, or provide feedback or additional information in "foreign" tickets, this can help us out.

   The ideal help of course is if you are able to solve issues by providing solutions or patches to issues.

Target Versions

Target versions are set by the Core team only. Until we can enforce this technically, feel free to remove target versions set by issue reporters.

Candidate for patchlevel:

All issues that qualify for patchlevel versions should go into this pool. Naturally, this can only be bug fixes or tasks.

A board does not make sense for this, since the issue status can only be New or Needs Feedback on these issues. Find the pool here.

Dedicated Patchlevel Versions:

This is the “real” next patchlevel release that is planned.

Whenever a contributor takes up a task he/she will have to make sure the ticket is set from “Candidate for patchlevel” to this dedicated version.

Taking up such an issue comes with the commitment to get this issue done by the time of release. Other members of the core team should support this cause by reviewing as usual.

Candidate for Major Version

Features and tasks will go into this pool. These can be worked on as usual.

Voting +1

Voting +2

Voting -2

A -2 vote by a Core Merger blocks a patch from merging. In contrast to -1, a -2 is a veto. Use with care. With a -2, you are taking responsibility of the patch and basically stating that it will not be done until you actively remove your negative vote.

To date, we have had no real problem with someone giving -2 and then not acting responsibly. It would be great if it stays this way.

The merging process

1. Prerequisites

   A patch can only be merged if it has received a + 2 vote from a Core Merger. A +2 can be given by the second Core Merger to review the patch positively.

   Merging gets blocked by a single -2 vote.

2. Submit the patch

   You can merge by clicking the Submit button. The Core Merger to merge the patch also has to do the Backporting if the submit-message requests for a backport.

   

3. The patch is successfully merged

   When the merging is successful, the Change looks like this:

   

4. Backport if necessary

   If required in the commit message, do the backport next.

5. How to revert a submitted change

   There is a Revert button available now. Please stick to the revert workflow if reverting a change should become necessary.

Use Gerrit for the cherry-pick

First try to use the Gerrit cherry-pick feature for automatic backporting.

1. Cherry pick into a branch

   

2. Choose branch and adjust commit message

   In the following modal, you can select the branch to backport. Existing branches are shown by autocomplete.

   Remove from the commit message everything below the Change-ID because the information about former reviewers is not needed for the cherry pick. Make sure that you don't alter the Change-ID but remove every line (also empty ones) below it. After doing so hit the Cherry Pick button.

   

3. Review backport patch

   This creates a new Change on Gerrit that needs to be reviewed and merged once more.

   

4. Merge the backport

   If the backport is straight-forward and not complex, you can vote +2 and continue to merge the backport. Otherwise it should be reviewed by at least one other Core Merger.

   

Manual backport

If the automatic backport fails, you need to manually cherry-pick the patch to the target branch. (e.g. cherry-pick the main patch onto your local (up to date) 13.4 branch) You will most likely need to adjust the code for the older branch.

Edit the commit message to comply to the guidelines again. (e.g. remove the Reviewed- and Tested- lines added by Gerrit)

Push the review back to Gerrit.

On Gerrit the original patch will show the cherry-pick as a related patch.

In which TYPO3 release was a patch merged into?

See Information: Where was a patch included? for information on how to use the Included in menu button to see, in which TYPO3 releases a patch was merged into.

git clone

Clone TYPO3 CMS Git repository into current directory:

git clone git@github.com:typo3/typo3 .

Setup

For detailed setup instructions, please see: Git Setup

Migrations

# Make sure that git pull loads from the "main" branch
git branch --set-upstream-to origin/main

# Rename your current "master" branch locally to "main" to match TYPO3's naming scheme
git branch -m master main

# set remote url for "origin"
git remote set-url origin https://github.com/typo3/typo3.git
# set push URL to gerrit (this has not changed)
git config remote.origin.pushurl "ssh://<your-username>@review.typo3.org:29418/Packages/TYPO3.CMS.git"

Workflow - common commands

For details see Create a Patch

Reset repo to last remote commit in (remote) main branch:

git reset --hard origin/main && git pull origin main

Stage and commit all changes:

git commit -a

Is the same as:

git add .
git commit

Stage and commit all changes to already existing commit:

git commit -a --amend

Push changes to remote main branch on gerrit (default method):

git push

This assumes, you have correctly configured your remote as described in Setting up Your Remote. If not, you must explicitly push using the refs/for namespace:

git push origin HEAD:refs/for/main

Workflow - work in progress

In case you want to push a "Work in progress", use the following instead:

git push origin HEAD:refs/for/main%wip

You can also configure Gerrit to always mark your pushes as WIP. In order to do this head over to https://review.typo3.org/settings/ and configure "Set new changes" to "work in progress" by default".

See: https://gerrit-review.googlesource.com/Documentation/user-upload.html#wip

Workflow -other branches

Show all branches:

git branch -a

Checkout 13.4 branch:

git checkout 13.4

Checkout 12.4 branch:

git checkout 12.4

Long story short: In most cases, push to main. The rest is being taken care of by core team members!

Push 13.4 branch:

git push origin HEAD:refs/for/13.4

Push 12.4 branch:

git push origin HEAD:refs/for/12.4

Workflow - commit msg

Details: Commit Message rules for TYPO3 CMS

Example commit message for a bugfix:

[BUGFIX] Subject line

Description

Resolves: #12345
Releases: main, 10.4

Other keywords:

[BUGFIX]
[FEATURE]
[DOCS]
[TASK]
[!!!][FEATURE]
[WIP][TASK]

• subject < 52 chars (if possible, otherwise <= 72)
• other lines <= 72 chars
• hyperlinks with > 72 chars are allowed when required (Inserting links / long lines)

Workflow - push with Gerrit message

Due to the nature of the commit workflow with gerrit, and a commit always getting amended, you have no individual commit messages for one Gerrit patch set.

To workaround that, you can add small comments to individual patch sets pushed to Gerrit like this:

git push origin HEAD:refs/for/main%m=Fixed_issue_in_JavaScript

Since that notation is a bit bland (no spaces, no linebreaks, no special characters, it can be helpful to utilize a small Bash script:

#!/bin/bash
read -p "Please enter pseudo-commit message: " answer

answer=$(echo "$answer" | tr -c '[:alnum:]' '_')
git push origin HEAD:refs/for/main%m=$answer

Replace "main" with possibly other branches (13.4, 12.4, ...) that you may want to push to.

Workflow - Undoing / fixing things

Throw away all changes since last commit:

git reset HEAD --hard

Unstage a file (remove file from index, but keep in working dir):

git reset <path>

Change author for last commit:

git commit --amend --author "Some Name <some@email>"

Squash last 2 commits:

git rebase -i HEAD~2

In the editor, replace 'pick' with 'squash' in the line describing the latest commit

This is very handy, in case you accidentally created a new commit instead of adding to an existing commit (with git commit --amend). This way, you can merge the last 2 commits and the commit messages.

Information: Where was a patch included?

Sometimes you see an old gerrit issue that was committed and pushed against main, but wonder in which TYPO3 version it was included. Often, you can already deduce from a Releases: main, 12.4 line that it was committed when 13.4 was main. But if you only see Releases: main you might begin to look up the date of a patch and relate it to release dates, or begin to inspect the git repository manually.

But: Hold on! Just check out the Gerrit interface and on the top right you see the menu, from where you also cherry-pick a patch or download a patch. There's a menu entry Included in which will reveal all TYPO3 releases (via GIT tags), a patch was included in.

References

See also these not TYPO3 specific cheat sheets for git if you are not very familiar with git:

• cheat sheet for git (in several languages)
• "git - the simple guide" by Roger Dudler
• "Oh, shit, git!" by Katie Sylor-Miller is basically a cheat sheet for Git, but focuses mostly on fixing things that went wrong.

To learn more about the internal working of Git, check out these resources:

• Git Internals in the Pro Git book

Git Troubleshooting

Before you're able to push your commits you have to set up your account. Make sure, you setup Git correctly.

Also, you may want to check your configuration

git push origin HEAD:refs/for/main

Permission denied (publickey).
fatal: The remote end hung up unexpectedly

git push origin HEAD:refs/for/main -v

Pushing to ssh://<username>@review.typo3.org:29418/REPOSITORY_NAME.git
Permission denied (publickey).
fatal: The remote end hung up unexpectedly

ssh -p 29418 <username>@review.typo3.org

****    Welcome to Gerrit Code Review    ****

Hi <Name>, you have successfully connected over SSH.

Unfortunately, interactive shells are disabled.
To clone a hosted Git repository, use:

git clone ssh://<username>@review.typo3.org:29418/REPOSITORY_NAME.git

Connection to review.typo3.org closed.

ssh -p 29418 -i <path-to-private-key> <username>@review.typo3.org

Host review.typo3.org
  User <username>
  IdentityFile ~/.ssh/<keyfile>
  Port 29418

ssh -v -p 29418 -i <path-to-private-key> <username>@review.typo3.org 2>&1 | grep "no mutual signature algorithm"

debug1: send_pubkey_test: no mutual signature algorithm

Host review.typo3.org
  ...
  PubkeyAcceptedKeyTypes +ssh-rsa

git push -v origin HEAD:refs/for/main

Pushing to ssh://<username>@review.typo3.org:29418/REPOSITORY_NAME.git
remote: ERROR:  https://review.typo3.org/#/settings/contact
remote:
To ssh://<username>@review.typo3.org:29418/REPOSITORY_NAME.git
! [remote rejected] HEAD -> refs/for/main (invalid committer)
error: failed to push some refs to 'ssh://<username>@review.typo3.org:29418/REPOSITORY_NAME.git'

git config user.email

git config --global user.email your-email@example.org

git commit --amend

! [remote rejected] HEAD -> refs/for/X/X (missing Change-Id in commit message)

git commit --amend

[remote rejected] main -> main (prohibited by gerrit)

git push origin HEAD:refs/for/<release-branch>

git push origin HEAD:refs/for/main

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,0314A92F87D7FEDF

git commit --amend

Mergeconflict in typo3/sysext/backend/Resources/Public/Css/backend.css

# Perform re-build using the helper "runTests.sh"

# Make sure dependencies are up to date
./Build/Scripts/runTests.sh -s composerInstall

# Clean possibly previously build files
./Build/Scripts/runTests.sh -s clean

# Execute the build
./Build/Scripts/runTests.sh -s build

# Now add all conflicting files as resolved:
git add typo3/sysext/backend/Resources/Public/Css/backend.css

# Continue the cherry-pick process
git cherry-pick --continue

Enabling Extended Gerrit Support in Tower

In Tower's preferences window, activate the Gerrit Code Review option. This will enable the following adaptions in Tower's UI.

Restart Tower after you enabled the setting.

Pushing to Gerrit

With the Gerrit option enabled in Tower's preferences, performing "Push" operations to Gerrit becomes easier:

Custom Menu Options

When right-clicking a local branch in the sidebar, a new Push <branch> to Gerrit… menu option is available. If you hold down the ⌥ key, the item becomes a "quick" action that will be performed without a confirmation dialog.

Custom Toolbar Button

You can add a custom Gerrit Push button to the toolbar. To do this, choose Customize Toolbar from the "View" main menu and drag the corresponding button onto the toolbar.

You'll find that the following dialog is optimized for pushing to Gerrit. It will automatically format the Push Refspec according to Gerrit's requirements - so you can simply enter the name of the code review branch you want to push to.

Installing native SSH tools

Using the native tools has the benefit of having an ssh-agent as a Windows service.

OpenSSH Client tools is an optional Windows feature and can be installed via the Windows settings. Search for "Manage Optional Features". Check for "OpenSSH Client" and install via "Add a feature" button if not already present.

Installing Git For Windows

Download Git For Windows from https://git-scm.com/download/win.

Run the installer Git-x.y.z-nn-bit.exe.

The following steps outline some of the dialogs shown in the process to help you making the right decisions.

Accept the GPL2 license.

Windows Explorer integration is very handy, especially the "Git Bash here" option.

Choose your editor.

Adjust the default Git branch naming to your likings.

Follow the recommendation. This is especially necessary for some IDEs.

Decide whether you want to go with the bundled OpenSSH binaries or use "external" ones, for instance those shipped with Windows. For users of PuTTY the second option might be feasible too, but this usually involves more setup with IDEs and other tools than going with OpenSSH binaries.

Select what best fits your needs.

Select the option "Checkout as-is, commit commit as-is". This ensures Git does not mangle your files magically. Better make sure your editor is set to use LF while contributing to TYPO3. (While the other options seem convenient, keep in mind that maybe you have to use CRLF for some reason at some point. This will need extra work to achieve.)

Select the terminal of your choice. MinTTY is a good choice, because it's mostly what you would expect from a terminal.

Choosing "Rebase" is a safe default setting to start with. It makes sure that your commits are placed on top of whatever you pull in from the remote. If conflicts occur you have to solve them in your commit, contrary to a merge commit. This is a good option for TYPO3 development, but you may want to adjust this for other projects. It can be configured per repository, this is just the global default.

Check if GCM is helpful for you. With TYPO3 repositories you always use SSH to push things, so this won't matter.

Try those features. The caching is indeed neat.

Pseudo console support eases use of docker exec commands from within Git Bash. The "file system monitor" is up to you.

Done! You are good to go.

Continue with the general Git setup instructions.

Creating an SSH Public Key on Windows

Microsoft has decent documentation on this:

https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement#user-key-generation

SourceTree General Setup

Basic Settings

First of all set the basic settings for your Git configuration by clicking on the settings button in SourceTree and add your name, email and SSH Key. Make sure the option "SSH Client" is set to PuTTY/Plink.

Clone

To clone a new repository click on "Clone/New".

Enter the URL to your repository and the path to your local folder. For easier access to your repositories leave the bookmark check box checked.

Click on Clone. Checking out the full TYPO3 Core repository might take some time, please be patient.

Automatic links to Forge and Gerrit

The commit messages contain a reference to the corresponding Forge ticket and a Gerrit review link. To get direct links from SourceTree choose the Core repository bookmark on the left and then click on the Settings button in the top bar. Choose tab "Advanced" and click "Add" at the section for "Commit text links":

Choose "Replacement type:" "Other" and add

• Regex pattern: (\#)+((?:[0-9]*)?)
• Link to URL: https://forge.typo3.org/issues/$2

Repeat the steps for the Gerrit links and add

• Regex pattern: (https:\/\/review\.typo3\.org\/.*)
• Link to URL: $1

After adding these two settings your commit history now shows links:

Reviewing and testing patches

Update the repository

Before applying a patch you should update the repository to its latest state. Press the pull button from the main button bar:

Then click OK:

Cherry-picking

First visit the page of the patch in Gerrit. From the Download menu, choose the copy button after the 'Cherry Pick' line.

The next step has to be done via the command line. Open git bash by pressing "Terminal" and paste the copied line with the 'Ins' button:

The patch is now applied and you can start testing.

Creating patches

curl -o .git/hooks/commit-msg "https://typo3.org/fileadmin/resources/git/commit-msg.txt" && chmod +x .git/hooks/commit-msg

git config url."ssh://<username>@review.typo3.org:29418/Packages/TYPO3.CMS.git"
git config remote.origin.push refs/heads/*:refs/for/*
git config branch.autosetuprebase remote

Create a branch

It's easier to undo all the changes in a patch if you create a branch for it. Click on the branch button and enter a name for your new branch, then click "Create Branch".

Now start coding and commit your changes (By pressing the "Commit" button). Make sure your commit message is writtenmaccording to the >>>rules for the commit message<<< and click OK.

Your changes are now stored locally in a separate branch.

Send the patch to Gerrit

Click the "Push" button to open the push dialog.

Check the checkbox in front of your feature branch. As remote branch add "refs/for/main" - which will create a new patchset for main. If you want to create patches for older branches use 'refs/for/<branchName>', for example 'refs/for/TYPO3_6-2'. Be aware that you have to fix a bug in the main branch first before it can go to older branches.

Now click ok. You should get a Gerrit link to your new change in the resulting output.

Cleaning up

To get back to the main branch just click on it at the "Branches" section. If you want to delete your feature branch, right click on it and choose "Delete".

SSH

Permission denied (publickey).
fatal: Could not read from remote repository.

Prerequisites

You will need a Webserver, PHP and database on your system. Look at the page System requirements and check out the specific system requirements for the current main branch on the Download TYPO3 page.

Just as a minimal example, this is a list of packages you might want to install on a Debian flavored Linux system (e.g. Ubuntu).

Setup

sudo apt-get install apache2
sudo a2enmod deflate rewrite headers mime expires

sudo apt-get install libapache2-mod-<version> php<version>-cli ...

memory_limit = 512M
max_execution_time = 240
max_input_vars = 1500

sudo service apache2 restart

sudo apt-get install mysql-server

sudo apt-get install imagemagick

<VirtualHost *:80>
  ServerAdmin youremail@yourdomain
  ServerName t3coredev
  DocumentRoot /var/www/t3coredev
  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

sudo a2ensite t3coredev
sudo service apache2 reload

127.0.0.1 localhost t3coredev

Next step

Proceed with Setting up TYPO3 manually.

PHP

Make sure you are using the correct PHP code style. It is based on the TYPO3 Coding Standards, (which follows PER-CS1.0 / PSR-12 and transition into PER-CS2.0 at the time of this writing), and covered in the TYPO3 Coding Guidelines.

See PhpStorm setup: CGL for information on how to configure the correct coding style.

The Appendix contains information on scripts to check / fix coding guideline issues: CGL check and fix

JavaScript

The following rules should be applied for JavaScript: Airbnb JavaScript Style Guide

TypeScript

The following rules should be applied for TypeScript: Excel Micro TypeScript Style Guide.

Xliff files

Language files are usually stored in a Folder Resources/Private/Language in files with the ending .xlf. While no tabs are allowed to indent in PHP files, you should edit Xliff files using tabs. Please also check Xliff / language files for Xliff-specific things to pay attention to.

commit-msg Hook

• File: .git/hooks/commit-msg
• Source file: Build/git-hooks/commit-msg

This hook is mandatory. It must be used for core contribution!

First of all, this hook will be executed whenever you do a commit on your local machine.

The most important task of the hook is to generate a unique Change-Id for your commit. This Change-Id is mandatory for Gerrit to identify your change. There is no way around this and it is not up for discussion :) This Change-Id will only be generated if it is not yet present in your commit message - please do not try to come up with a Change-Id on your own, the result will be chaos.

Apart from that the hook will check your commit message for logical errors like missing keywords, Resolves lines etc. For detailed information on the format of a commit message, click here. This also describes cases in which you might exceed the line length of 72 characters (hyperlinks).

If the commit-msg hook finds errors in your commit-msg, you can try again, by amending to the commit:

git commit --amend

pre-commit Hook

• File: .git/hooks/pre-commit
• Source file: Build/git-hooks/unix+mac/pre-commit
• This hook is optional. This hook is not available for Windows., however it can be executed on Windows machines by using a tool like the Git BASH.

The pre-commit hook checks all added PHP files staged for the commit for Coding Guideline issues and will report any problems it finds.

To fix the issues, see CGL check and fix.

After fixing the files you must amend your commit:

git commit -a --amend

Post-checkout hook for composer install

If you want to run composer install for each checkout (e.g. when switching branches, tags, etc), then you can use the post-checkout commit hook.

Create a file named .git/hooks/post-checkout with the following contents:

#!/usr/bin/env python3
#-*- coding: utf-8 -*-

"""A post-checkout git hook to execute `composer install`

:Author: Philipp Gampe
"""

import subprocess
import string
import os
import sys

PATH_TO_COMPOSER = '/home/phil/bin/composer.phar'
HOOK_DIR = os.path.dirname(os.path.realpath(__file__))
GIT_ROOT_DIR = os.path.dirname(os.path.dirname(HOOK_DIR))

def main():
    if sys.argv[3] == '1':
        os.chdir(GIT_ROOT_DIR)
        return subprocess.run([PATH_TO_COMPOSER, 'install'], timeout=30)

if __name__ == '__main__':
    result = main()
    if result:
        exit(result.returncode)
    else:
        exit(0)

Summary line (first line)

A summary line starts with a keyword and a brief summary of what the change does. Make sure to describe how the behavior is now, not how it used to be - in the end, telling someone what was broken doesn't help anyone, you want to tell what is working now :)

• Prefix the line with a keyword: [BUGFIX], [FEATURE], [TASK] or [DOCS].

Possible keywords are:

[BUGFIX]

A fix for a bug.

[FEATURE]

A new feature (also small additions). Most likely it will be an added feature, but it could also be removed. Features may exclusively be targeted for the "main" branch of TYPO3 CMS, because no new features are allowed in older branches. Exceptions to this have to be discussed on a case-to-case basis with the designated release managers. Features have to be documented in the changelog.

[DOCS]

This tag is used for changes regarding the documentation.

[TASK]

Anything not covered by the above categories. E.g. Refactoring of a component

Additionally other flags should be added under certain circumstances:

[!!!]

Breaking change. After this patch, something works different than before and the user / admin / extension developer will have to change something. Has to be documented accordingly and should only be targeted for the main branch. See Editing the changelog.

[SECURITY]

Visualizes that a change fixes a security issue. This tag is used by the Security Team.

• Keep the whole line below 52 characters if possible, but below 72 in any case

• It is very important that the message is written in imperative mood; that means that it must be written as if you are giving a command or an instruction, since a commit is a set of instructions for how to go from a previous state to the new state and the commit message should describe this process. This convention matches up with generated commit messages by commands like git merge and git revert. If in doubt, the golden rule to follow is very simple: Review your subject lines, and apply the following words in front of it:

if applied, this commit will **"your subject line here"**

For example (you will find some more examples later):

If applied, this commit will **Invalid session token on creating content element in admin panel** Does not make any sense.

If applied, this commit will **"Fix backend edit URL in admin panel"** Reads nicely and explains what happened

• After the keyword, make sure to start the summary with a capital letter.
• Avoid using EXT:somesystemextension in the commit subject: looks ugly and is redundant when you look at what code changed.
• In case of reverting a previous commit, basically you should just prepend [TASK] to the message automatically generated by a git revert. For example, in case of reverting a feature you would use: [TASK] Revert "[FEATURE] Transform Lead to Gold"

Description (Message body)

Here you can go into detail about the how and why of the change. It should be brief, but yet descriptive so people reviewing your change get an idea what they need to look out for

• Describe the fix/change introduced by the Change Request. (The problem is already described in the Forge ticket.)
• Keep it simple and don't repeat information that is already part of the issue tracker. Especially avoid "How to reproduce" part. At most, try to explain the change itself, if it is not already clear by reading the diff. Do not repeat the code change itself in the body text.
• Wrap the lines after 72 characters manually

[BUGFIX] Link some long links

In [1] is is documented, that something is not wrong. But
in reality [2] properly indicates, this is indeed right.

So we follow the advice of [3][4] and make things better.

[1] https://example.com/a/very/very/long-link/because/it/is/really-needed-I-mean-it-its-long-but-okay
[2] https://example.com/this/is/short/but/also/nice
[3] https://example.com/snafu
[4] https://example.com/mostlyharmless

Resolves: #12346
Related: #12340
Releases: main, 12.4
Change-Id: I<some string generated by the git commit-msg hook>

Reverting patches

If there's the need to revert a patch, please add this information to the commit message:

1. Add a Resolves-line for the ticket that is giving the reason for the revert.
2. Add a Reverts-line for the ticket that belongs to the original patch.

You will find more information about the life cycle of a patch here Revert patches.

Commit Template

You can use a custom template for automatically generating a commit message with the basics.

This is covered in the Git setup instructions, see Setting up a Commit Message Template.

Bad summary lines examples vs. good examples

Please note that the following examples are taken from real commits.

Example 1

[FOLLOWUP][BUGFIX] Remove uglify of jquery-ui/sortable.js

should have been:

[BUGFIX] Remove uglify of jquery-ui/sortable.js

Example 2

[BUGFIX] EXT:filelist Cancelling the file exists already modal works now

should have been:

[BUGFIX] Allow cancelling modal that appears when file exists

Example 3

Revert "[FEATURE] EXT:form - introduce YAML "imports""

should have been:

[TASK] Revert "[FEATURE] introduce YAML "imports""

Example 4

[TASK] Revert "Add support for PSR-15 HTTP middlewares"

should have been:

[TASK] Revert "[FEATURE] Add support for PSR-15 HTTP middlewares"

Example 5

[TASK] use horizontal ellipsis instead of 3 dots

should have been:

[TASK] Use horizontal ellipsis instead of 3 dots

Example 6

[BUGFIX] Element Browser should only render default language pages

should have been:

[BUGFIX] Limit element browser to only render default language pages

Example 7

[BUGFIX] D3.js uses basic authentication credentials cached in browser

should have been:

[BUGFIX] Use only basic authentication credentials cached in browser in D3.js

Example 8

[DOCS] 1/1 9.1 Documentation

should have been:

[DOCS] Add documentation for version 9.1 (1/1)

Example 9

[FEATURE] Option to globally enable redirect hit count

should have been:

[FEATURE] Add option to globally enable redirect hit count

Example 10

[TASK] Improved extension configuration API

should have been:

[TASK] Improve extension configuration API

Example 11

[BUGFIX] NewContentElementWizardController to NewContentElementController

should have been:

[BUGFIX] Remove recently introduced NewContentElementWizardController

Example 12

[BUGFIX] Invalidate session token on creating content element in admin panel

Should have been:

[BUGFIX] Fix backend edit URL in admin panel

About Composer

Composer is a dependency manager for PHP.

So what it basically does is find packages you have defined to be part of your application (in our case TYPO3). But what if these packages rely on other packages as well? This is where Composer jumps in and takes care of keeping all these packages in sync.

Since we use quite some packages (because why would we invent things ourselves that are already there?) Composer is an extremely useful tool for us.

Install Composer

Follow the installation instructions from https://getcomposer.org. Afterwards, you should have a working executable composer available.

Verify composer is working:

$ composer --version

Composer Commands

Once you have installed Composer, this is the command you should run after you clone the Git source and after every git pull request or switching branches:

composer install

But, just follow the setup instructions, it will walk you through the commands in the correct order!

Custom TYPO3 Composer Commands

Some additional Composer commands have been added for Core development.

Just run:

composer

to list them. You will see something like:

gerrit:setup                           Enable all the git hooks needed to make contribution easy
gerrit:setup:commitMessageHook:enable  Enable the commit message hook needed for gerrit
gerrit:setup:preCommitHook:disable     Disable pre commit hook to run some checks locally
gerrit:setup:preCommitHook:enable      Enable pre commit hook to run some checks locally

Connection to the changelog reStructuredText files

All changelog type reStructuredText (reST) files since Core version 9 have to be tagged with one of the three tags FullyScanned, PartiallyScanned or NotScanned. In particular, the FullyScanned tag is used by the extension scanner to mark instances as "not affected by this change", as such they should be added with care and only if the scanner configuration matches all changes mentioned in the reST file.

If only parts of the reST file are covered, PartiallyScanned has to be added and the reST file should mention which parts are and are not covered. If the scanner does not cover a reST file at all then NotScanned can be added.

..  index:: Frontend, FullyScanned, ext:frontend

If a reST file is renamed the file may be covered in a matcher configuration which then needs to be adapted as well. The reST files are not bound to specific directories in the matcher configuration so moving a reST file to a different location within the Changelog directory has no effect.

Extension scanner PHP configuration

The match configurations can be found below the following path: EXT:install/Configuration/ExtensionScanner. Choose the file that is related to your change and add the removed functionality to the extension scanner match configuration. As a key we use the fully qualified name.

'TYPO3\CMS\Frontend\Http\UrlProcessorInterface' => [
    'restFiles' => [
        'Deprecation-96641-UnusedHookRelatedUrlProcessorInterface.rst',
    ],
],

'TYPO3\CMS\Core\Authentication\BackendUserAuthentication->checkAuthMode' => [
    'maximumNumberOfArguments' => 3,
    'restFiles' => [
        'Breaking-97265-SimplifiedAccessModeSystem.rst',
    ],
],

About the PHP part of the extension scanner

The PHP part of the extension scanner is based on the library nikic/php-parser. This library creates an abstract syntax tree from any given compilable PHP file. It comes with a traverser for recursive iteration over the tree that implements a visitor pattern, own visitors can be added. A single "matcher" is a visitor added to the traverser. A default visitor resolves all shortened namespaced class usages to their fully qualified name, which is a great help for our matchers.

This basically means: The whole AST is traversed exactly once for each PHP file and all matchers are called for each node. Matches can then decide if they match a configured deprecation or breaking scenario.

All matchers are covered by unit tests and a fixture that shows what exactly is matched. Studying the fixture can be a good way to understand the matcher.

Matchers are systematically named: for method calls there is a usually one variant for dynamic and one for static calls. If for example a static method changed its argument signature by removing an argument then the according matcher class is \TYPO3\CMS\Install\ExtensionScanner\Php\MethodArgumentDroppedStaticMatcher.

Single matcher configurations are pretty obvious, new ones should be added at the end. When adding matcher configurations it should be verified the match it is not already covered by some other matcher (possibly in another reST file).

Deprecate a class

• Add a @deprecated annotation to the class doc comment
• Deprecate the constructor - see next section about deprecating a method

Deprecate method

• Add a @deprecated annotation to the method doc comment
• Add a trigger_error call to the method, describing the migration path and triggering an error of type E_USER_DEPRECATED

trigger_error(
  'Content with <link> syntax was found, update your content to use the t3:// syntax, and migrate your content via the upgrade wizard in the install tool',
  E_USER_DEPRECATED
);

Deprecate method arguments

If you want to deprecate method arguments, check for argument existence and trigger an error the same way you would for a method. If you want to deprecate and remove an unused argument, use func_get_args() or func_num_args() (see example below):

public function TS_AtagToAbs($value)
{
    if (func_num_args() > 1) {
        trigger_error('Second argument of TS_AtagToAbs() is not in use and is removed, however the argument in the callers code can be removed without side-effects.', E_USER_DEPRECATED);
    }
    // ...
}

Make class properties protected

To reach full encapsulation of classes, the public access to properties should be removed. The property access should be done by public methods only.

Public properties that should be protected can be migrated to protected or private and wrapped with getters/setters as needed.

During a phase of deprecation, entries into the deprecation log are triggered, if an extension accesses a previously public property. The code still keeps working until the next major release, when the deprecation and public access fallback are removed from the code.

To deprecate usage of a (still) public property that should be changed to protected in the future:

1. Add the \TYPO3\CMS\Core\Compatibility\PublicPropertyDeprecationTrait trait to the class.
2. Add the property $deprecatedPublicProperties to the class and list all properties that should no longer be accessed from outside of the class.
3. Make the property private/protected.

+use TYPO3\CMS\Core\Compatibility\PublicPropertyDeprecationTrait;
+
class RecordHistory
{
+       use PublicPropertyDeprecationTrait;
+
+       /**
+         * @var string[]
+         */
+        private $deprecatedPublicProperties = [
+            'changeLog' => 'Using changeLog is deprecated and will not be possible anymore in TYPO3 v11.0. Use getChangeLog() instead.',
+       ];
+
        /**
         * @var array
         */
-       public $changeLog = [];
+       protected $changeLog = [];

Make class methods protected

This works in a similar way as Make class properties protected: The trait \TYPO3\CMS\Core\Compatibility\PublicMethodDeprecationTrait allows to make public methods protected or private without breaking extensions.

This can be used in case the public methods may still be called by extensions. This will then trigger a deprecation.

In order to make a method private or protected:

1. Add the \TYPO3\CMS\Core\Compatibility\PublicMethodDeprecationTrait trait to the class.
2. Add the property $deprecatedPublicMethods to the class and list all methods that should no longer be accessed from outside of the class.
3. Make the method private/protected.

+use TYPO3\CMS\Core\Compatibility\PublicMethodDeprecationTrait;
+
class PageRepository
{
+       use PublicMethodDeprecationTrait;
+
+       /**
+         * @var string[]
+         */
+        private $deprecatedPublicMethods = [
+            'init' => 'init() is now called implicitly on object creation, and is not necessary anymore to be called explicitly. Calling init() will throw an error in TYPO3 v10.0.',
+       ];
+

-       public function init($show_hidden)
+       protected function init($show_hidden)

Deprecate a hook

As hooks provide extension points throughout the core, deprecating and removing them should be carefully considered, but sometimes the placement of a hook doesn't make sense anymore as the functionality around it changed or other hooks / mechanisms replaced it's purpose. If you have to deprecate a hook, call trigger_error before the hook call:

if (isset($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_parsehtml_proc.php']['modifyParams_LinksDb_PostProc'])
    && is_array($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_parsehtml_proc.php']['modifyParams_LinksDb_PostProc'])
) {
    trigger_error(
        'The hook "t3lib/class.t3lib_parsehtml_proc.php->modifyParams_LinksDb_PostProc" will be removed in TYPO3 v10, use LinkService syntax to modify links to be stored in the database.',
        E_USER_DEPRECATED
    );
    $parameters = [
        'currentBlock' => $v,
        'linkInformation' => $linkInformation,
        'url' => $linkInformation['href'],
        'attributes' => $tagAttributes
    ];
    foreach ($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_parsehtml_proc.php']['modifyParams_LinksDb_PostProc'] as $className) {
        $processor = GeneralUtility::makeInstance($className);
        $blockSplit[$k] = $processor->modifyParamsLinksDb($parameters, $this);
    }
}

Deprecate methods still called by the TYPO3 Core

If you want to deprecate a method that still has to be used by the core itself to provide the functionality for the time being, you can achieve that by adding a new method parameter that will prevent the deprecation warning message being triggered and add that parameter to the callees in the core.

Example:

/**
 * Transformation handler: 'ts_links' / direction: "rte"
 * Converting TYPO3-specific <link> tags to <a> tags
 *
 * This functionality is only used to convert legacy <link> tags to the new linking syntax using <a> tags,  and will
 * not be converted back to <link> tags anymore.
 *
 * @param string $value Content input
 * @param bool $internallyCalledFromCore internal option for calls where the Core is still using this function, to suppress method deprecations
 * @return string Content output
 * @deprecated will be removed in TYPO3 v10, only ->TS_AtagToAbs() should be called directly, <link> syntax is deprecated
 */
public function TS_links_rte($value, $internallyCalledFromCore = null)
{
    if ($internallyCalledFromCore === null) {
        trigger_error('This method will be removed in TYPO3 v10, use TS_AtagToAbs() directly and do not use <link> syntax anymore', E_USER_DEPRECATED);
    }
    $hasLinkTags = false;
    $value = $this->TS_AtagToAbs($value);
    // ...
}

More information

Changelogs

• 9.0 Dealing with properties that become protected
• 9.0 Trait to migrate public access to protected by deprecation
• 9.4 Trait to detect public deprecated methods: PublicMethodDeprecationTrait

Conventions on this page

If you need to select something from the menu in PhpStorm, menu items are displayed like this: File > Settings > ....

General setup

File > Settings > Languages & Frameworks > PHP

(ctrl + alt + s opens File > Settings)

• PHP Language Level: choose appropriate version
• CLI Interpreter: choose appropriate version

Coding Guidelines

Make sure your IDE is setup properly to comply with the Coding Guidelines for TYPO3 (CGL).