- Table of contents
- Unit Tests and Integration Tests
Unit Tests and Integration Tests¶
Unit tests and integration tests are tests that run during the mrb build process. Conceptually, a unit test runs on the smallest possible unit of code (typically, a single c++ class or source file, not necessarily an art object like an art module). Conceptually, an integration test tests multiple units of code. in practice, a unit test runs out of the build area, whereas an integration test runs out of the install area.
Invoking Tests¶
Either type of test can be invoked by typing "mrb test
" or "make test
" in the mrb build area.
Directory structure¶
The standard directory structure in the mrb srcs
area is to have a test
subdirectory in the top level of each package (so that tests are in a completely separate directory tree than production code). The directory structure below the test directory is arbitrary. Add the following directives in the top level CMakeLists.txt
file.
include(CetTest) add_subdirectory(test)
Also put
add_subdirectory
directives in lower level CMakeLists.txt
files to reach the entire test directory tree.
Configuring Tests¶
Either type of test is configured by adding a directive in CMakeLists.txt
that looks like this.
cet_test( mytest ...)
Here
mytest
is the name of the test, which can be the name of a stand alone executable, or the name of a script, or simply an invented name. Full documentation of the cet_test
cmake directive can be found in $CETBUILDTOOLS_DIR/Modules/CetTest.cmake
. Note that cetbuildtools
ups product is only set up in an initailized build area (after typing "mrbsetenv
"). Additional details about configuring unit tests and integration tests can be found in the following sections.
Unit Tests¶
A unit test is a stand alone test executable that links against the package's shared libraries. The test executable decides whether the test is successful and returns the result as an exit status (0=success). The configuration of a simple unit test can look like this in CMakeLists.txt
.
cet_test( mytest LIBRARIES mylib )
Here
mytest
is the name of a c++ source file (i.e. mytest.cc
) containing a c++ main program. The LIBRARIES
keyword defines which libraries the test executable needs to link against, which would typically include (one of) the package's shared libraries (i.e. libmylib.so
). The test executable, called mytest
, is built and placed in the build area under $MRB_BUILDDIR/<package_name>/bin
, from which location it can be run outside the build system, if necessary (for example, in a debugger).
Assertions in unit tests.¶
It is perfectly acceptable to use assertions inside unit tests as a way of indicating failure. In this case, you should make sure that assertions are enabled in CMakeLists.txt
using this directive.
cet_enable_asserts()
In most cases, this directive should be in all top level test CMakeLists.txt files, and is automatically inherited to subdirectories.
Boost unit tests.¶
The boost product provides a framework for unit tests consisting of c++ macros and libraries. To enable boost unit tests, use keyword USE_BOOST_UNIT
in your cet_test
directive.
cet_test( mytest USE_BOOST_UNIT LIBRARIES mylib )
Integration Tests¶
The basic idea of an integration test is to get cmake
to run an installed executable (i.e. lar) with specified command line options, using installed shared libraries, and to report a result. There are two ways to do this: with or without a script.
Integration tests without a script¶
With this type of integration test, you can run a lar
job, and the only indication of whether the test succeeded is the exit status of lar
. The cmake configuration in CMakeLists.txt
for this type of integration test looks something like this.
cet_test( mytest HANDBUILT TEST_EXEC lar TEST_ARGS -c mytest.fcl )
In this case, cmake will create a test area in
$MRB_BUILDDIR/<package name>/test/<src path>/mytest.d
). The test command will be run from there (i.e. lar -c mytest.fcl), and any files generated by the test job will be in the test area after running the test. The package name is the name of the top level package (e.g. uboonecode
), and the source path will match the path in the mrb srcs
area.
Integration tests with a script¶
Sometimes you want a more sophisticated determination of success than simply whether lar
crashed or not. For example, you might want to examine an output file or do tests involving generated histograms, etc.. In these cases, you should write a test script to run lar
and determine whether the test was successful.
Your test script (mytest.sh
) should look like this.
#! /bin/bash lar -c mytest.fcl > mytest.out 2> mytest.err stat=$? echo "lar exited with status ${stat}." if [ $stat -ne 0 ]; then exit $stat fi # Do additional tests. . . . exit 0 # Success.
And the
CMakeLists.txt
configuration should look like this.cet_test( mytest.sh PREBUILT )
In this case, the test directory in the build area will be called
$MRB_BUILDDIR/<package name>/test/<src path>/mytest.sh.d
). In this example, mytest.out
and mytest.err
will be found there.
Test dependencies.¶
In general unit and integration tests are allowed to depend on any external or larsoft product, provided that the shared libraries need to link and/or run are available in $LD_LIBRARY_PATH
.
Integration tests will often depend on microboone-specific resources from uboonecode
(e.g. microboone geometry or microboone fcl files). An integration test that depends on uboonecode
can not be included in any general larsoft product. Any integration test that depends on uboonecode
in any way will fail during the larsoft build, because the larsoft build does not set up uboonecode
. For this reason, most integration tests should go in the uboonecode
package.
Tests with data files.¶
For tests that require additional data files use keyword DATAFILES in the cet_test
directive.
cet_test( mytest ... DATAFILES file1.dat file2.dat ... )
The specified data files will be copied to the test area before running the test.