Request change in numbering and ordering of unit tests
Art opened the SNOW ticket RITM0741181 with this request.
Can I ask that the individual tests in the report for a particular CI run (e.g. (e.g. at http://dbweb5.fnal.gov:8080/ArtCI/app/build_detail/phase_details?build_id=art_ci/3165&platform=Linux%202.6.32-754.2.1.el6.x86_64&phase=unit_test&buildtype=slf6%20e19:debug%20develop) are:
Have the CMake-assigned test number (if present) shown for each test.
Partitioned according to pass / fail (as now); and
Sorted within a partition according to completion order from The CMake log (preferable) or at least the start order (CMake test number)?
Allow the faster diagnosis of certain kinds of failure;
Make it easier to correlate information between the CI log and (e.g.) the CMake test log.
An example of CMake test log is: http://dbweb6.fnal.gov:8080/ArtCI/app/storage/docs/2018/10/24/unit_test__T4rkXew.log
[...] Start 725: ES_w02 1/847 Test #665: test_tiered_input_01a_runs_t ....................... Passed 3.45 sec [...]
In this snippet the index after
Start is the
CMake-assigned test number, this index is known as soon the unit test starts.
In the second raw
1 is the index related to the
completion order from The CMake log,
847 is the total number of unit tests and
#665 is the
CMake-assigned test number. Here the index related to the
completion order from The CMake log is known only when the unit test is completed.