Project

General

Profile

Bug #15696

Some CMake tests using PASS_REGULAR_EXPRESSION should also fail for non-zero exit code.

Added by Christopher Green over 2 years ago. Updated over 2 years ago.

Status:
Accepted
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
03/01/2017
Due date:
% Done:

0%

Estimated time:
16.00 h
Scope:
Internal
Experiment:
-
SSI Package:
-
Co-Assignees:
Duration:

Description

It is the case with some tests in the art suite (e.g. ProductMix_r1c2) that specific output is expected in addition to a non-zero exit code. Unfortunately, using PASS_REGULAR_EXPRESSION causes CTest to ignore the exit code of the test program. This may lead to tests passing when in fact they should have failed.

One solution for art invocations would be, where the required output is from message logger output, to configure the message logger to mirror output to a second, file destination, and make verifying the output of that file a second test, depending on the first.

The general solution is to write a wrapper script which handles the checking of the output and is configurable to AND or OR the exit code with the result of the output check as appropriate. This should probably be integrated into CetTest.cmake.


Related issues

Related to cet-is - Bug #15695: Module respondToOpenOutputFiles / respondToCloseOutputFiles not being called.Assigned2017-03-01

History

#1 Updated by Christopher Green over 2 years ago

  • Related to Bug #15695: Module respondToOpenOutputFiles / respondToCloseOutputFiles not being called. added

#2 Updated by Kyle Knoepfel over 2 years ago

  • Status changed from New to Accepted
  • Estimated time set to 16.00 h

Time estimate assumes that a script will be written for each test that needs it.



Also available in: Atom PDF