Project

General

Profile

Feature #23381

Harmonize CI test scripts

Added by Marco Mambelli 8 months ago. Updated 17 days ago.

Status:
New
Priority:
Normal
Category:
-
Target version:
Start date:
04/23/2020
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Stakeholders:
Duration:

Description

Provide some utility function in utils.sh to harmonize the various CI script (shellcheck, run_pylint, unittest, ...).
  • invocation options should be the same
  • ability to run on current or multiple branches
  • similar return codes (script failed, warnings found, errors found, ok, ...)
  • similar output summary (table, email)

This will need to be in a separate branch and coordinated w/ Fermilab's CI

Some links about formats:

List report example

filename info warning errors
name1 1 0 2

XML report example

Pass (one test, that linting was run)
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="jest tests" tests="1" failures="0" time="0.346">
  <testsuite name="undefined" errors="0" failures="0" skipped="0" timestamp="2019-06-07T15:44:08" time="0.092" tests="1">
    <testcase classname=" OAS Linter" name=" OAS Linter" time="0.092">
    </testcase>
  </testsuite>
</testsuites>

Fail (multiple tests, one for each warning from the linter)
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="jest tests" tests="4" failures="4" time="0.39">
  <testsuite name="undefined" errors="0" failures="4" skipped="0" timestamp="2019-06-07T15:50:20" time="0.148" tests="4">
    <testcase classname=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AddressRegion/oneOf/0 (schema-properties-type)" name=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AddressRegion/oneOf/0 (schema-properties-type)" time="0.148">
      <failure>schema-properties-type if.then test failed:undefined != object</failure>
    </testcase>
    <testcase classname=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AddressRegion/oneOf/1 (schema-properties-type)" name=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AddressRegion/oneOf/1 (schema-properties-type)" time="0.148">
      <failure>schema-properties-type if.then test failed:undefined != object</failure>
    </testcase>
    <testcase classname=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AuctionListingAddress/oneOf/0 (schema-properties-type)" name=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AuctionListingAddress/oneOf/0 (schema-properties-type)" time="0.148">
      <failure>schema-properties-type if.then test failed:undefined != object</failure>
    </testcase>
    <testcase classname=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AuctionListingAddress/oneOf/1 (schema-properties-type)" name=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AuctionListingAddress/oneOf/1 (schema-properties-type)" time="0.148">
      <failure>schema-properties-type if.then test failed:undefined != object</failure>
    </testcase>
  </testsuite>
</testsuites>


Subtasks

Feature #24345: Provide a runner to run the different tests with consistent CLIResolvedMarco Mambelli


Related issues

Blocked by GlideinWMS - Feature #18042: Change exit code of pylint and unit test scriptsNew10/26/2017

History

#1 Updated by Marco Mambelli 5 months ago

  • Target version changed from v3_6_2 to v3_6_3

#2 Updated by Marco Mambelli about 1 month ago

  • Subject changed from Harmonize CI scripts to Harmonize CI test scripts

#3 Updated by Marco Mambelli about 1 month ago

  • Start date changed from 10/04/2019 to 04/23/2020
  • Due date set to 04/23/2020

due to changes in a related task: #24345

#4 Updated by Marco Mambelli about 1 month ago

  • Related to Feature #18042: Change exit code of pylint and unit test scripts added

#5 Updated by Marco Mambelli about 1 month ago

  • Related to deleted (Feature #18042: Change exit code of pylint and unit test scripts)

#6 Updated by Marco Mambelli about 1 month ago

  • Blocked by Feature #18042: Change exit code of pylint and unit test scripts added

#7 Updated by Marco Mambelli 17 days ago

  • Target version changed from v3_6_3 to v3_6_4


Also available in: Atom PDF