Version 5 supported

Testing with Silverstripe CMS

Introduction

The Silverstripe CMS core contains various features designed to simplify the process of creating and managing automated tests.

Silverstripe CMS uses PHPUnit for unit tests, and the framework contains features to simplify the process of creating and managing tests.

If you're more familiar with unit testing, but want a refresher of some of the concepts and terminology, you can browse the Testing Glossary. To get started now, follow the installation instructions below.

You should also read over the PHPUnit manual. It provides a lot of fundamental concepts that we build on in this documentation.

Running tests

In order to run tests, you need to install Silverstripe CMS using Composer, which will pull in the required development dependencies to run tests.

Tests are run from the commandline, in your webroot folder:

  • vendor/bin/phpunit: Runs all tests (as defined by phpunit.xml)
  • vendor/bin/phpunit vendor/silverstripe/framework/tests/: Run all tests of a specific module
  • vendor/bin/phpunit vendor/silverstripe/framework/tests/filesystem: Run specific tests within a specific module
  • vendor/bin/phpunit vendor/silverstripe/framework/tests/filesystem/FolderTest.php: Run a specific test
  • vendor/bin/phpunit vendor/silverstripe/framework/tests '' flush=1: Run tests with optional request parameters (note the empty second argument)

Check the PHPUnit manual for all available command line arguments.

On Linux or OSX, you can avoid typing the full path on every invocation by adding vendor/bin to your $PATH definition in the shell profile (usually ~/.profile): PATH=./vendor/bin:$PATH

Caching

Just like on web requests, Silverstripe CMS caches metadata about the execution context. This cache can get stale, e.g. when you change YAML configuration or add certain types of PHP code. In order to flush the cache, use the flush=1 CLI parameter:

vendor/bin/phpunit vendor/silverstripe/framework/tests '' flush=1

Generating a coverage report

PHPUnit can generate a code coverage report (docs) which shows you how much of your logic is executed by your tests. This is very useful to determine gaps in tests.

vendor/bin/phpunit --coverage-html <output-folder> <optional-tests-folder>

To view the report, open the index.html in <output-folder> in a web browser.

Typically, only your own custom PHP code in your project should be regarded when producing these reports. To exclude some thirdparty/ directories add the following to the phpunit.xml configuration file.

<filter>
    <blacklist>
        <directory suffix=".php">vendor/silverstripe/framework/dev/</directory>
        <directory suffix=".php">vendor/silverstripe/framework/thirdparty/</directory>
        <directory suffix=".php">vendor/silverstripe/cms/thirdparty/</directory>

        <!-- Add your custom rules here -->
        <directory suffix=".php">app/thirdparty/</directory>
    </blacklist>
</filter>

Configuration

The phpunit executable can be configured by command line arguments or through an XML file. File-based configuration has the advantage of enforcing certain rules across test executions (e.g. excluding files from code coverage reports), and of course this information can be version controlled and shared with other team members.

Silverstripe CMS comes with a default phpunit.xml.dist that you can use as a starting point. Copy the file into a new phpunit.xml and customize to your needs - PHPUnit will auto-detect its existence, and prioritize it over the default file.

There's nothing stopping you from creating multiple XML files (see the --configuration flag in PHPUnit documentation). For example, you could have a phpunit-unit-tests.xml and phpunit-functional-tests.xml file (see below).

Database permissions

Silverstripe CMS tests create their own temporary database on every execution. Because of this the database user in your config file should have the appropriate permissions to create new databases on your server, otherwise tests will not run.

Writing tests

Tests are written by creating subclasses of SapphireTest. You should put tests for your site in the app/tests directory. If you are writing tests for a module, put them in the tests/ directory of your module (in vendor/).

Generally speaking, there should be one test class for each application class. The name of the test class should be the application class, with "Test" as a suffix. For instance, we have all the tests for SiteTree in vendor/silverstripe/framework/tests/SiteTreeTest.php

You will generally write two different kinds of test classes.

  • Unit Test: Test the behaviour of one of your DataObjects.
  • Functional Test: Test the behaviour of one of your controllers.

Tutorials and recipes for creating tests using the Silverstripe CMS:

Edit on Github