GlideinWMS

Provide some utility function in utils.sh to harmonize the various CI script (shellcheck, run_pylint, unittest, ...).

• invocation options should be the same
• ability to run on current or multiple branches
• similar return codes (script failed, warnings found, errors found, ok, ...)
• similar output summary (table, email)

This will need to be in a separate branch and coordinated w/ Fermilab's CI

Some links about formats:

• https://testanything.org/
  • https://testanything.org/tap-specification.html
  • https://testanything.org/producers.html
  • https://github.com/python-tap/pytest-tap
  • https://github.com/sstephenson/bats
• https://help.catchsoftware.com/display/ET/JUnit+Format
  • https://blog.ilias.xyz/unit-test-output-formats-a-state-of-affairs-5734b4c8e577
• https://circleci.com/docs/2.0/collect-test-data/
• https://jestjs.io/docs/en/cli (--reporters jest-junit)
• https://www.baeldung.com/checkstyle-java
• https://checkstyle.org/checks.html

List report example

filename info warning errors
name1 1 0 2

XML report example

Pass (one test, that linting was run)
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="jest tests" tests="1" failures="0" time="0.346">
  <testsuite name="undefined" errors="0" failures="0" skipped="0" timestamp="2019-06-07T15:44:08" time="0.092" tests="1">
    <testcase classname=" OAS Linter" name=" OAS Linter" time="0.092">
    </testcase>
  </testsuite>
</testsuites>

Fail (multiple tests, one for each warning from the linter)
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="jest tests" tests="4" failures="4" time="0.39">
  <testsuite name="undefined" errors="0" failures="4" skipped="0" timestamp="2019-06-07T15:50:20" time="0.148" tests="4">
    <testcase classname=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AddressRegion/oneOf/0 (schema-properties-type)" name=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AddressRegion/oneOf/0 (schema-properties-type)" time="0.148">
      <failure>schema-properties-type if.then test failed:undefined != object</failure>
    </testcase>
    <testcase classname=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AddressRegion/oneOf/1 (schema-properties-type)" name=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AddressRegion/oneOf/1 (schema-properties-type)" time="0.148">
      <failure>schema-properties-type if.then test failed:undefined != object</failure>
    </testcase>
    <testcase classname=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AuctionListingAddress/oneOf/0 (schema-properties-type)" name=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AuctionListingAddress/oneOf/0 (schema-properties-type)" time="0.148">
      <failure>schema-properties-type if.then test failed:undefined != object</failure>
    </testcase>
    <testcase classname=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AuctionListingAddress/oneOf/1 (schema-properties-type)" name=" schema-properties-type if.then test failed:undefined != object - #/components/schemas/AuctionListingAddress/oneOf/1 (schema-properties-type)" time="0.148">
      <failure>schema-properties-type if.then test failed:undefined != object</failure>
    </testcase>
  </testsuite>
</testsuites>