.. include:: /Includes.rst.txt .. index:: Testing; Project .. _testing-projects: =============== Project testing =============== Introduction ============ Testing entire projects is somehow different from Core and extension testing. As a developer or maintainer of a specific TYPO3 instance, you probably do not want to test extension details too much - those should have been tested on an extension level already. And you probably also do not want to check too many TYPO3 backend details but look for acceptance testing of your local development, stage and live frontend website instead. Project testing is thus probably wired into your specific CI and deployment environment. Maybe you want to automatically fire your acceptance tests as soon as some code has been merged to your projects develop branch and pushed to a staging system? Documenting all the different decisions that may have been taken by agencies and other project developers is way too much for this little document. We thus document only one example how project testing could work: We have some "site" repository based on `ddev `_ and add basic acceptance testing to it, executed locally and by GitHub Actions. This is thought as an inspiration you may want to adapt for your project. Project site-introduction ========================= The `site-introduction `_ TYPO3 project is a straight ddev based setup that aims to simplify handling the `introduction `_ extension. It delivers everything needed to have a working introduction based project, to manage content and export it for new introduction extension releases. Since we're lazy and like well defined but simply working environments, this project is based on ddev. The repository is a simple project setup that defines a working TYPO3 instance. And we want to make sure we do not break main parts if we fiddle with it. Just like any other projects wants. The quick start for an own site based on this repository boils down to these commands, with more details mentioned in `README.md `_: .. code-block:: shell lolli@apoc /var/www/local $ git clone git@github.com:TYPO3-Documentation/site-introduction.git lolli@apoc /var/www/local $ cd site-introduction lolli@apoc /var/www/local/site-introduction $ ddev start lolli@apoc /var/www/local/site-introduction $ ddev import-db --src=./data/db.sql lolli@apoc /var/www/local/site-introduction $ ddev import-files --src=./assets lolli@apoc /var/www/local/site-introduction $ ddev composer install This will start various containers: A database, a phpmyadmin instance, and a web server. If all goes well, the instance is reachable on :samp:`https://introduction.ddev.site`. .. index:: Testing; Acceptance Local acceptance testing ======================== There has been one `main patch `_ adding acceptance testing to the site-introduction repository. The goal is to run some acceptance tests against the current website that has been set up using ddev and execute this via GitHub Actions on each run. The solution is to add the basic selenium-chrome container as additional ddev container, add codeception as require-dev dependency, add some codeception actor, a test and a basic codeception.yml file. Tests are then executed within the container to the locally running ddev setup. Let's have a look at some more details: ddev allows to add further containers to the setup. We did that for the selenium-chrome container that pilots the acceptance tests as :file:`.ddev/docker-compose.chrome.yaml`: .. code-block:: yaml version: '3.6' services: selenium: container_name: ddev-${DDEV_SITENAME}-chrome image: selenium/standalone-chrome:3.12 environment: - VIRTUAL_HOST=$DDEV_HOSTNAME - HTTP_EXPOSE=4444 external_links: - ddev-router:$DDEV_HOSTNAME With this in place and calling `ddev start`, another container with name `ddev-introduction-chrome` is added to the other containers, running in the same docker network. More information about setups like these can be found in the `ddev documentation `_. To execute acceptance tests in this installation you have to activate this file, usually it is now appended with the suffix `.inactive` and therefore not used when DDEV starts. To activate acceptance tests the file :file:`.ddev/docker-compose.chrome.yaml.inactive` has to be renamed to :file:`.ddev/docker-compose.chrome.yaml`. By default acceptance tests are disabled because they slow down other tests significantly. Next, after adding codeception as require-dev dependency in :file:`composer.json`, we need a basic :file:`Tests/codeception.yml` file: .. code-block:: yaml namespace: Bk2k\SiteIntroduction\Tests\Acceptance\Support suites: acceptance: actor: AcceptanceTester path: . modules: enabled: - Asserts - WebDriver: url: https://introduction.ddev.site browser: chrome host: ddev-introduction-chrome wait: 1 window_size: 1280x1024 extensions: enabled: - Codeception\Extension\RunFailed - Codeception\Extension\Recorder paths: tests: Acceptance output: ../var/log/_output data: . support: Acceptance/Support settings: shuffle: false lint: true colors: true This tells codeception there is a selenium instance at `ddev-introduction-chrome` with chrome, the website is reachable as :samp:`https://introduction.ddev.site`, it enables some codeception plugins and specifies a couple of logging details. The `codeception documentation `_ goes into details about these. Now we need a simple first test which is added as :file:`Tests/Acceptance/Frontend/FrontendPagesCest.php`: .. code-block:: php :caption: EXT:site_introduction/Tests/Acceptance/Frontend/FrontendPagesCest.php amOnPage('/'); $I->see('Open source, enterprise CMS delivering content-rich digital experiences on any channel, any device, in any language'); $I->click('Customize'); $I->see('Incredible flexible'); } } It just calls the homepage of our instance, clicks one of the links and verifies some text is shown. Straight, but enough to see if the basic instance does work. Ah, and we need a "Tester" in the `Support directory `_. That's it. We can now execute the acceptance test suite by executing a command in the ddev PHP container: .. code-block:: shell lolli@apoc /var/www/local/site-introduction $ ddev exec bin/codecept run acceptance -d -c Tests/codeception.yml Codeception PHP Testing Framework v2.5.6 Powered by PHPUnit 7.5.20 by Sebastian Bergmann and contributors. Running with seed: Bk2k\SiteIntroduction\Tests\Acceptance\Support.acceptance Tests (1) ------------------------------------------------------------------------------------------------- Modules: Asserts, WebDriver --------------------------------------------------------------------------------------------------------------------------------------------------------------------- ⏺ Recording ⏺ step-by-step screenshots will be saved to /var/www/html/Tests/../var/log/_output/ Directory Format: record_5be441bbdc8ed_{filename}_{testname} ---- FrontendPagesCest: First page is rendered Signature: Bk2k\SiteIntroduction\Tests\Acceptance\Frontend\FrontendPagesCest:firstPageIsRendered Test: Acceptance/Frontend/FrontendPagesCest.php:firstPageIsRendered Scenario -- I am on page "/" [GET] https://introduction.ddev.site/ I see "Open source, enterprise CMS delivering content-rich digital experiences on any channel, any device, in any language" I click "Customize" I see "Incredible flexible" PASSED --------------------------------------------------------------------------------------------------------------------------------------------------------------------- ⏺ Records saved into: file:///var/www/html/Tests/../var/log/_output/records.html Time: 8.46 seconds, Memory: 8.00MB OK (1 test, 2 assertions) lolli@apoc /var/www/local/site-introduction $ Done: Local test execution of a projects acceptance test! .. index:: Testing; GitHub Actions GitHub Actions ============== With local testing in place, we now want tests to run automatically when something is merged into the repository, and when people create pull requests for our project, we want to make sure that our carefully crafted test setup actually works. We're going to use Github's Actions CI service to get that done. It's free for open source projects. To tell the CI what to do, create a new workflow file in `.github/workflows/tests.yml `__ .. code-block:: yaml name: tests on: push: pull_request: workflow_dispatch: jobs: testsuite: name: all tests runs-on: ubuntu-20.04 steps: - name: Checkout uses: actions/checkout@v2 - name: Start DDEV uses: jonaseberle/github-action-setup-ddev@v1 - name: Import database run: ddev import-db --src=./data/db.sql - name: Import files run: ddev import-files --src=./assets - name: Install Composer packages run: ddev composer install - name: Allow public access of var folder run: sudo chmod 0777 ./var - name: Run acceptance tests run: ddev exec bin/codecept run acceptance -d -c Tests/codeception.yml It's possible to see executed test runs `online `_. Green :) Summary ======= This chapter is a show case how project testing can be done. Our example projects makes it easy for us since the ddev setup allows us to fully kickstart the entire instance and then run tests on it. Your project setup may be probably different, you may want to run tests against some other web endpoint, you may want to trigger that from within your CI and deployment phase and so on. These setups are out of scope of this document, but maybe the chapter is a good starting point and you can derive your own solution from it.