-
Notifications
You must be signed in to change notification settings - Fork 9
Continuous Integration
JASS's development integrates automatically with five tools. The first four, Travis CI, AppVeyor CI, CodeCov, and CodeDocs, execute on every push to the master branch. The fifth tool, Coverity, runs on a push to the coverity_scan branch, necessary due to Coverity's limit of one scan per day.
This page details the intricacies of how the five tools work together, and their configurations.
Travis CI provides the CPU power to run unit tests on Linux and OSX as well as being responsible for uploading results to CodeCov and Coverity. It is configured through the .travis.yml file in the repository root. Described here is an overview of JASS's unique .travis.yml. To understand fully how Travis CI is configured please refer to their documentation.
When a push to Github occurs, Github triggers a webhock notifying Travis of the updated code. Travis then downloads the repository, analyses the config and spawns VMs according to the config. Each VM then executes the following steps:
- OPTIONAL Install
apt addons - OPTIONAL Install
cache components before_installinstallbefore_scriptscript- OPTIONAL
before_cache(for cleaning up cache) -
after_successorafter_failure - OPTIONAL
before_deploy - OPTIONAL
deploy - OPTIONAL
after_deploy after_script
Normal builds occur on the master branch. They are specified in the build matrix which indicates that their are four builds. The build matrix cannot execute scripts, so instead sets environment variables which alter how the later scripts run.
- Linux build for unit tests
- OSX build for unit tests
- Linux build for code coverage
- Linux build which runs unit tests under valgrind
Each build spawns its own VM and runs the before_install, install, script, and after_success sequences.
The before_install is only used by Coverity Builds.
install on Linux calls out to a script which installs GNU G++5 and valgrind. This script also creates symlinks so that Coverity and CMake can find the updated compilers. On OSX install isn't executed.
script performs the build and executes the unit tests. There are two variants, one which executes under valgrind, and the other which executes normally. The code coverage VM also used the normal script but is modified by environmental variables.
after_success is executed only on the code coverage VM. It uploads the gcov results to CodeCov for analyses.
A lot of the oddity in the Travis config results from keeping the same config across the coverity_scan branch and the master branch. As a result of using the same config, four VMs are spawned but only one is used.
before_install checks which VM we are running under. If it is any but the standard linux build. We feign success and shut down. This check was placed here as it's the earliest point in which arbitrary code can be executed.
install calls out to a script which installs GNU G++5 and valgrind. This script also creates symlinks so that Coverity and CMake can find the updated compilers.
None of the further stages perform any actions. The remainder of the VM's work is configured and performed automatically through the addons section. This downloads a script which builds our project and uploads it to the Coverity website authenticating with a secure token.
AppVeyor is another CI service but executes the unit tests under Windows. It is configured through the appveyor.yml file in the repository root.
AppVeyor's config centers around building VisualStudio project files. As we are using CMake and building from the command line, under the build_script section we first have to import the needed tools. After which we can build and run.
CodeCov visualises Code Coverage of the unit tests. It provides graphs, trends over time, and allows you to browse the source code showing executed and unexecuted lines. It is configured through the .codecov.yml file in the repository root.
Some of the useful options available include notifying when coverage drops below a certain point. As well as which partial coverage results are displayed.
CodeDocs hosts code documentation. It integrates directly with Github, fetching the code on every push to 'master' and performing a Doxygen build. It uses the .codedocs config file which is a subset of the Doxygen config.
Coverity Scan is a static analyses tool. It is available through paid licenses, or for free use for open source projects subject to daily use count restrictions. There are two ways to interface with the tool, downloading a package of scripts from the website or integrated into Travis CI using the addon section in their config file. We chose the second route.
Because of the daily use restrictions, a Coverity build is only performed when a push to the coverity_scan branch occurs. It is recommended that you develop in master, and merge into coverity_scan when required. The configs were explicitely written to make this possible.