Project

General

Profile

ReleaseTestingMatrixTemplate

This is a template to guide and simplify the writing of the release tests.
There is a summary of the tests performed.
GlideinwmsTesting contains a checklist for the tests to perform.

Copy the next section in a document named after the release or release candidate, e.g. ReleaseTestingMatrix_3_5_1_rc1, and start filling in the info about the tests.
  • Summary: contains a summary of the tests
  • Automatic tests: is for automated tests, e.g. the ones in Jenkins
  • Manual tests: it is for manual install, on Fermicloud or outside resources
    • it is an itemized list with elements with a specific structure
    • Please be specific!
    • each test generally involves 2 hosts (Frontend and Factory)
    • title (first line) - is a summary: whether was an install (fresh install) or upgrade, the OS, a description, who performed the test (by ...)
    • hosts descriptions - Factory/Frontend, GWMS version, install/upgrade (from GWMS version), OS, hostname
    • 3 lists with Details (more test details), Issues (things to be fixed if any) and additional Notes (if any)

All the above should be done for SL7 and at least 1 also for SL6.
Releases should be tested as a fresh install and as an upgrade. At least once each for the Frontend and once each for the Factory.

Each developer must announce by email to the team, which tests are going to be done by him/her and perform the tests and update the testing document within 2 days form the email.

If an issue is found and the release blocked, strike through the summary line of the test in the list and the version number corresponding to that test in the summary table. Open also a ticket to solve the issue before the release.

For the record, in each release, we should add below the matrix template, a list with the issues found.

------------------Copy below this line-------------

Summary

Summary of the automatic and manual tests

TEST ID FRONTEND version FACTORY version
1 release release
2 previous release
3 release previous
4 release subsequent(if proceed)
5 subsequent(if proceed) release

Automatic tests

Manual tests

Follows a list of the manual tests performed for this release:

  1. Install on SL6, smoke test by Developer
    • Factory, v3.4.6 install, SL6, fermicloudAAA
    • Frontend, v3.4.6 install, SL6, fermicloudBBB
    • Details
      • More details about the test, what was done, ...
    • Issues:
      • Add here problems that need to be addressed. Open a ticket and write the number here
    • Notes
      • Add here general note
  2. Install/upgrade on SL6, smoke test by Developer
    • Factory ...