Automated Validation » History » Version 18
Julia Yarba, 02/08/2019 05:55 PM
1 | 7 | Julia Yarba | h1. GENIE AVS CI - Automated Validation |
---|---|---|---|
2 | 1 | Gabriel Perdue | |
3 | 3 | Julia Yarba | h2. General Information |
4 | 3 | Julia Yarba | |
5 | 3 | Julia Yarba | GENIE AVS CI is GENIE Automated Validation Suite combined with the "Continuous Integration (CI) services":https://cdcvs.fnal.gov/redmine/projects/ci. |
6 | 3 | Julia Yarba | |
7 | 3 | Julia Yarba | The procedure can be subdivided into 2 categories: |
8 | 3 | Julia Yarba | * GENIE CI build(s) "on commit" |
9 | 3 | Julia Yarba | ** Triggered every time when new code is committed (to GitHib) to either Generator, Reweighs, or Comparisons package (given 20min wait time, to allow consolidation of multiple commits within a short time frame) |
10 | 3 | Julia Yarba | ** Includes the followjng steps: |
11 | 3 | Julia Yarba | *** Checkout from GitHub |
12 | 3 | Julia Yarba | *** Build of all 3 packages |
13 | 3 | Julia Yarba | *** Execution of Unit Tests |
14 | 8 | Julia Yarba | *** "Operational" test which includes |
15 | 8 | Julia Yarba | a) generation of minimalistic cross section splines for the CCQE process |
16 | 9 | Julia Yarba | b) generation of 5 CCQE events using splines generated in a) |
17 | 8 | Julia Yarba | both are done for the Default tune |
18 | 3 | Julia Yarba | * GENIE Validation workflow is triggered weekly (on Friday's) |
19 | 3 | Julia Yarba | ** GENIE build, Unit Tests, and "operational" test, as described above |
20 | 7 | Julia Yarba | ** Benchmark of GENIE generated predictions vs a variety of experimental datasets; the list of benchmarks can be found "here":https://cdcvs.fnal.gov/redmine/projects/genie_ci/wiki/grid_phase |
21 | 3 | Julia Yarba | |
22 | 5 | Julia Yarba | Results are reported to the dedicated "GENIEMC Slack channel":https://geniemc.slack.com/messages/C50RB57JS/convo/GA63C15T7-1539263877.000100/ (password may be needed) |
23 | 12 | Julia Yarba | Each report includes link(s) to the "GENIE CI dashboard":http://genie-ci-history.fnal.gov/GenieCI/app/view_builds/index for further details (log files, plot books, etc.) |
24 | 5 | Julia Yarba | |
25 | 13 | Julia Yarba | Various technical details can also be found in the presentation at the GENIE meetings; see later in this document. |
26 | 3 | Julia Yarba | |
27 | 17 | Julia Yarba | h2. Some Tips on Interpreting GENIE CI Dashboard |
28 | 14 | Julia Yarba | |
29 | 14 | Julia Yarba | Each raw on the "GENIE CI dashboard":http://genie-ci-history.fnal.gov/GenieCI/app/view_builds/index corresponds to a separate workflow. |
30 | 14 | Julia Yarba | Obviously, the workflows are shorter for the "on commit" builds and longer for the full blown GENIE AVS rounds. |
31 | 14 | Julia Yarba | |
32 | 14 | Julia Yarba | On the dashboard this is reflected wit the addition of the following: |
33 | 14 | Julia Yarba | * "setup_validation" section - preparation of the storage area in dCache, generation of job configuration files, etc. |
34 | 14 | Julia Yarba | * one or several "ci_validation_<TuneID> sections (e.g. ci_validation_G18_02a_00_000, etc.) |
35 | 14 | Julia Yarba | |
36 | 16 | Julia Yarba | Each ci_validation_<TuneID> section is a folder on its own. |
37 | 14 | Julia Yarba | If one clicks on it, one will be redirected to another monitoring board that corresponds to the validation of a given GENIE tune. |
38 | 14 | Julia Yarba | Such validation procedure consists of multiple steps and includes generation of neutrino-nucleon, then neutrino-nucleus splines, generation of GENIE event files for various MC-data benchmarks, |
39 | 14 | Julia Yarba | and the MC-data benchmarks themselves. |
40 | 14 | Julia Yarba | They are named as dag_<NN>, where NN is the serial number of a particular step in the validation chain. |
41 | 15 | Julia Yarba | The notation of "dag" is associated with the term DAGman (it is a meta-scheduler for HTCondor that menages dependencies between jobs at a high level). |
42 | 14 | Julia Yarba | |
43 | 14 | Julia Yarba | Description of what each dag_<NN> actually means/does is linked to the "?" icon in the box on the left side of the web page, located below the "Progress Legend" chart; feel free to click on the icon. |
44 | 14 | Julia Yarba | |
45 | 3 | Julia Yarba | h2. Presentations on GENIE AVS CI |
46 | 3 | Julia Yarba | |
47 | 6 | Julia Yarba | * "GENIE AVS CO Overview at the STWG meeting on 6/13/2018":https://cdcvs.fnal.gov/redmine/attachments/52114/GENIE-AVS-CI-Status-June2018.pdf |
48 | 3 | Julia Yarba | NOTE: A few details may be slightly out of date as a result of updates implemented since June 2018 |
49 | 3 | Julia Yarba | |
50 | 10 | Julia Yarba | h2. How to Launch GENIE AVS CI |
51 | 3 | Julia Yarba | |
52 | 3 | Julia Yarba | GENIE "on commit" and/or weekly AVS CI are triggered automatically. |
53 | 3 | Julia Yarba | |
54 | 18 | Julia Yarba | However, the procedure can be triggered manually at any given moment, if needs be. |
55 | 3 | Julia Yarba | |
56 | 3 | Julia Yarba | Any authorized GENIE developer can trigger one or another workflow, following "these instructions": https://cdcvs.fnal.gov/redmine/projects/genie_ci/wiki |
57 | 1 | Gabriel Perdue | |
58 | 2 | Julia Yarba | h2. Legacy Information |
59 | 2 | Julia Yarba | |
60 | 2 | Julia Yarba | * Meetings |
61 | 2 | Julia Yarba | *** [[AVS - April 2, 2015]] |