Continuous Integration with Travis CI
You did it. You have covered your code with tests and now you can trust you code. Actually, you can't. One day, you will forget to run the tests, deploy to production and get an angry call from your boss later.
This needn't happen if you use contrinuous integration. In this tutorial you will learn how to setup Nette Tester with Travis CI.
How Travis CI works
Travis CI („Travis“) is a very popular service for hosted continuous integration. The service is free of charge and offers automated execution of your tests after pushing to your repository. It is also fully integrated with GitHub.
In order to configure Travis, you need to add a file named
.travis.yml to the root of your repository. The file
consists of sections described later in this tutorial. None of the sections is mandatory, however if we leave out the
script section, Travis will attempt to run
The sections are executed in following order:
.travis.yml allows space indentation only. You can use validator to check your file.
For more see official documentation.
Setting up .travis.yml
First of all, you need to tell Travis which language environment to select for your project. You can do so using
language: php option. You can specify for which PHP versions will the tests be executed. Not introducing patch version tells Travis to use the latest available.
language: php php: - 7.1 - 7.2 - 7.3 - 7.4 - nightly # development version
You can instruct Travis to do multiple runs with different sets of environment variables values. To do so, add
key. Each bullet is understood as a different environment and tests are run seperately. We are going to use
TESTER_PHP_BIN later in the file to run tester with
-p php-cgi option too.
env: - TESTER_PHP_BIN="php" - TESTER_PHP_BIN="php-cgi"
Combination of five php versions and two environment variations generates total of 10 runs.
This tutorial assumes that you use Composer for managing your project's dependencies and that you require nette/tester in dev section. For more information see tutorial on Composer.
In order to install your dependencies, use
install section. Each bullet means single command. Composer installs
your dev dependencies by default. You should use
--no-interaction so composer doesn't ask questions Travis can't
answer. You should also use
--prefer-source. It prevents your tests from randomly failing if you run out of limited
amount of GitHub API requests.
If you want to use the latest build, update Composer in
before_install: - composer self-update install: - composer install --no-interaction --prefer-source
Depending on the configuration above, a build matrix is generated. The matrix contains all combinations of environment
settings. A single combination is called a job and is run separately. You can modify the matrix in
If you want to exclude a job, use
exclude key. In our case, we don't want to use php-cgi binary for older PHP
matrix: exclude: - php: 7.0 env: TESTER_PHP_BIN="php-cgi" - php: 7.1 env: TESTER_PHP_BIN="php-cgi"
Travis shows the build as passing only if every single job pass. However, you can define jobs that are allowed to fail without
causing the whole build to shown as failed. To do so, declare
allow_failures. For our sake, we allow nightly PHP
matrix: allow_failures: - php: nightly
Notice that when allowing failures, we only specified php version, not environment variables. This means all jobs
with nightly version will be allowed to fail, nevermind the environment variables values. It works with
Running the Tests
Tests are run in
script section, where you only need to execute Tester. Let's assume your tests are in
tests/ folder and you provide your own
php.ini in the same folder. Additionaly, tell Tester to display
information about skipped tests with
-s option and to use value of earlier declared
PHP binary with
script: - ./vendor/bin/tester -p $TESTER_PHP_BIN -s -c ./tests/php.ini ./tests
If Test Fails
after_failure is executed if a test fails. Tester stores actual value of variables in case of assertion
fail. We will use this section to print the actual values.
after_failure: # Prints *.actual files content - for i in $(find ./tests -name \*.actual); do echo "--- $i"; cat $i; echo; echo; done
Setting up Other Services:
Travis comes with multiple popular services (e.g. MySQL) pre-installed. However, if you need to use for example Redis storage,
you can tell Travis in
services: - redis-server
MySQL runs on
127.0.0.1 and you can log in using
root as username. Password is
not required. You can import your database in
before_script section. Let's say your database set-up script is in
before_script: - mysql -u root -e 'CREATE DATABASE testbase;' - mysql -u root testbase < tests/testbase.sql
.travis.yml should look something like this by now:
language: php php: - 7.1 - 7.2 - 7.3 - 7.4 - nightly env: - TESTER_PHP_BIN="php" - TESTER_PHP_BIN="php-cgi" matrix: allow_failures: - php: nightly exclude: - php: 7.0 env: TESTER_PHP_BIN="php-cgi" - php: 7.1 env: TESTER_PHP_BIN="php-cgi" services: - redis-server before_install: - composer self-update install: - composer install --no-interaction --prefer-source before_script: - mysql -u root -e 'CREATE DATABASE testbase;' - mysql -u root testbase < tests/testbase.sql script: - ./vendor/bin/tester -p $TESTER_PHP_BIN -c ./tests/php.ini -s ./tests/ after_failure: # Prints *.actual files content - for i in $(find ./tests -name \*.actual); do echo "--- $i"; cat $i; echo; echo; done
As mentioned above, Travis is integrated with GitHub. However, you need to specify which repositories are to be tested. This is done using Webhook which notifies Travis about changes in your repository.
First, go to Travis CI and sign in with your GitHub account. After synchronizing your account, you`ll see all repositories you have access to. Flip switch to ON for all repositories you'd like to enable.
Travis will now add your repository to queue after every pushed commit or created pull-request. After a short while, your repository will be tested.
Travis can generate a status image for you. You can embed this
icon into your
README.md file for example.
Some commits don't need testing. You can add
[skip ci] somewhere in your commit message and Travis will ingore the