ReleaseTestingMatrix 3 6 2¶
This is a template to guide and simplify the writing of the release tests.
There is a summary of the tests performed.
GlideinwmsTesting contains a checklist for the tests to perform.
- Summary: contains a summary of the tests
- Automatic tests: is for automated tests, e.g. the ones in Jenkins
- Manual tests: it is for manual install, on Fermicloud or outside resources
- it is an itemized list with elements with a specific structure
- Please be specific!
- each test generally involves 2 hosts (Frontend and Factory)
- title (first line) - is a summary: whether was an install (fresh install) or upgrade, the OS, a description, who performed the test (by ...)
- hosts descriptions - Factory/Frontend, GWMS version, install/upgrade (from GWMS version), OS, hostname
- 3 lists with Details (more test details), Issues (things to be fixed if any) and additional Notes (if any)
All the above should be done for SL7 and at least 1 also for SL6.
Releases should be tested as a fresh install and as an upgrade. At least once each for the Frontend and once each for the Factory.
Each developer must announce by email to the team, which tests are going to be done by him/her and perform the tests and update the testing document within 2 days form the email.
If an issue is found and the release blocked,
strike through the summary line of the test in the list and the version number corresponding to that test in the summary table. Open also a ticket to solve the issue before the release.
For the record, in each release, we should add below the matrix template, a list with the issues found.
------------------Copy below this line-------------
Summary of the automatic and manual tests
|TEST ID||FRONTEND version||FACTORY version|
Follows a list of the manual tests performed for this release:
- Install on SL6, smoke test by Developer
- Factory, v3.4.6 install, SL6, fermicloudAAA
- Frontend, v3.4.6 install, SL6, fermicloudBBB
- More details about the test, what was done, ...
- Add here problems that need to be addressed. Open a ticket and write the number here
- Add here general note
- Install/upgrade on SL6, smoke test by Developer
- Factory ...