Project

General

Profile

ReleaseTestingMatrix 3 6 2 rc3

  • Summary: contains a summary of the tests
  • Automatic tests: is for automated tests, e.g. the ones in Jenkins
  • Manual tests: it is for manual install, on Fermicloud or outside resources
    • it is an itemized list with elements with a specific structure
    • Please be specific!
    • each test generally involves 2 hosts (Frontend and Factory)
    • title (first line) - is a summary: whether was an install (fresh install) or upgrade, the OS, a description, who performed the test (by ...)
    • hosts descriptions - Factory/Frontend, GWMS version, install/upgrade (from GWMS version), OS, hostname
    • 3 lists with Details (more test details), Issues (things to be fixed if any) and additional Notes (if any)

All the above should be done for SL7 and at least 1 also for SL6.
Releases should be tested as a fresh install and as an upgrade. At least once each for the Frontend and once each for the Factory.

Each developer must announce by email to the team, which tests are going to be done by him/her and perform the tests and update the testing document within 2 days form the email.

If an issue is found and the release blocked, strike through the summary line of the test in the list and the version number corresponding to that test in the summary table. Open also a ticket to solve the issue before the release.

For the record, in each release, we should add below the matrix template, a list with the issues found.

------------------Copy below this line-------------

Summary

Summary of the automatic and manual tests

TEST ID FRONTEND version FACTORY version
1 release release
2 previous release
3 release previous
4 release subsequent(if proceed)
5 subsequent(if proceed) release

Automatic tests

Manual tests

Follows a list of the manual tests performed for this release:

  1. install on SL7, OSG 3.5 smoke test by Dennis Box 3/13/20
    • Factory v3.6.2.rc2, SL7, OSG3.5, fermicloud170.fnal.gov
    • Frontend v3.6.2.rc2,SL7, OSG3.5, fermicloud354.fnal.gov
    • Details
      • No issues or unusual errors encountered
  2. install on SL7, OSG 3.4 smoke test by Dennis Box 3/16/20
    • Factory v3.6.2.rc3, SL7, OSG3.5, fermicloud384.fnal.gov
    • Frontend v3.6.2.rc3,SL7, OSG3.5, fermicloud391.fnal.gov
    • Details
      • No issues or unusual errors encountered
  3. install on SL6, OSG 3.4 smoke test by Dennis Box 3/16/20
    • Factory v3.6.1, SL6, OSG3.5, fermicloud398.fnal.gov
    • Frontend v3.6.1 ,SL6, OSG3.5, fermicloud399.fnal.gov
    • Details
      • osg 3.4 SL6 does not have 3.6.2.rcX installed?
  4. install on SL7, OSG 3.5 smoke test by Dennis Box 3/16/20
    • Factory v3.6.2.rc3, SL7, OSG3.5, fermicloud337.fnal.gov
    • Frontend v3.6.1, SL7, OSG3.5, fermicloud339.fnal.gov
    • Details
      • No issues or unusual errors encountered
  5. install on SL7, OSG 3.5 smoke test by Dennis Box 3/16/20
    • Factory v3.6.1, SL7, OSG3.5, fermicloud340.fnal.gov
    • Frontend v3.6.2.rc3, SL7, OSG3.5, fermicloud348.fnal.gov
    • Details
      • Tests ran to completion, glideins submitted, userjobs connected and ran
      • Unusual errors observed in collectorLog(s)
        var/log/condor/Collector38Log:03/16/20 14:01:07 CCB: ERROR: line 3 is invalid in /var/lib/condor/spool/131.225.154.238-9618.ccb_reconnect.03/16/20 14:01:07 CCB: ERROR: line 11 is invalid in /var/lib/condor/spool/131.225.154.238-9618.ccb_reconnect.03/16/20 14:01:07 CCB: loaded 2 reconnect records from /var/lib/condor/spool/131.225.154.238-9618.ccb_reconnect.
        
  6. install on SL7, OSG 3.5 smoke test by Marco Mambelli 3/17/20
    • Factory v3.6.2.rc3, SL7, OSG3.5, fermicloud321.fnal.gov
    • Frontend v3.6.2.rc3, SL7, OSG3.5, fermicloud322.fnal.gov
    • Details
      • Tests ran to completion, glideins submitted, userjobs connected and ran
      • Did not see collectorLog(s) errors observed by Dennis
  7. install on SL7, OSG 3.4 ITB test by Marco Mascheroni 3/24/20
    • Factory v3.6.2.rc3, SL7, OSG3.4, vocms0203.cern.ch and vocms0205.cern.ch
    • Frontend v3.6.2.rc3, SL7, OSG3.4, vocms0801.cern.ch and vocms0802.cern.ch
    • Details
      • Run CRAB jobs that run to completion. Jobs that run at CERN were successfully. Saw some failures at PIC (https://cmsweb.cern.ch/scheddmon/068/cms1921/200325_142314:mmascher_crab_ALL_testITBClassadYES2/job_out.3.0.txt). Maybe a site issue (Valid site-local-config not found at /cvmfs/cms.cern.ch/SITECONF/local/JobConfig/site-local-config.xml)?
      • Observed factory monitoring page not working when no frontend is running. Tried to look for a fix, but eventually judged the time investment to fix such a corner case is too high (considering the monitoring page is not supported anymore in theory)
  1. Install on SL7, smoke test by Bruno
    • Factory, v3.6.2 install, SL7, fermicloud120
    • Frontend, v3.6.2 install, SL7, fermicloud086
    • Details
      • Tested Python condor_chirp
    • Issues:
      • No issues
  2. Install on SL7, smoke test by Bruno
    • Factory, v3.6.2 install, SL7, fermicloud120
    • Frontend, v3.6.1 install, SL7, fermicloud390
    • Details
      • Tested Python condor_chirp
    • Issues:
      • No issues
  3. Install on SL7, smoke test by Bruno
    • Factory, v3.6.1 install, SL7, fermicloud077
    • Frontend, v3.6.2 install, SL7, fermicloud086
    • Details
      • Tested Python condor_chirp
    • Issues:
      • No issues
  1. Install on SL6, smoke test by Developer
    • Factory, v3.4.6 install, SL6, fermicloudAAA
    • Frontend, v3.4.6 install, SL6, fermicloudBBB
    • Details
      • More details about the test, what was done, ...
    • Issues:
      • Add here problems that need to be addressed. Open a ticket and write the number here
    • Notes
      • Add here general note
  2. Install/upgrade on SL6, smoke test by Developer
    • Factory ...