Project

General

Profile

Notes for NuTools release managers » History » Version 25

Lynn Garren, 02/03/2020 12:55 PM

1 1 Lynn Garren
h1. Notes for NuTools release managers
2 1 Lynn Garren
3 4 Lynn Garren
{{>TOC}}
4 4 Lynn Garren
5 6 Lynn Garren
h2(count). Updating one package
6 1 Lynn Garren
7 1 Lynn Garren
* If you only need a new version of a single package, such as nugen, then simply checkout that package, edit, tag and build.
8 1 Lynn Garren
9 5 Lynn Garren
* If you want a complete nulite distribution with the new package, then you would also checkout nutools and follow the instructions for updating the nu suite.
10 2 Lynn Garren
11 2 Lynn Garren
h2(count). The nu suite
12 2 Lynn Garren
13 2 Lynn Garren
The nu suite consists of nutools, nuevdb, nug4, nugen, nurandom, and nusimdata.
14 2 Lynn Garren
15 2 Lynn Garren
h2(count).  Tools
16 2 Lynn Garren
17 19 Lynn Garren
* mrb v3_04_04 or later 
18 19 Lynn Garren
** This version adds the nu_suite gitCheckout option and fixes some bugs in mrb uv.
19 2 Lynn Garren
* buildFW (used by the Jenkins build)
20 7 Lynn Garren
* copyFromJenkins
21 7 Lynn Garren
* copyToSciSoft
22 8 Lynn Garren
* pullProducts
23 7 Lynn Garren
* pullPackage
24 25 Lynn Garren
* larreltools v1_08_00 or later
25 1 Lynn Garren
** This package was created for LArSoft, but contains common tools.
26 1 Lynn Garren
** dogit
27 25 Lynn Garren
** startNuLiteRel
28 1 Lynn Garren
** updateVersion
29 5 Lynn Garren
30 9 Lynn Garren
h2(count). Updating the nutools suite for LArSoft
31 9 Lynn Garren
32 12 Lynn Garren
These instructions presume that the work is done on scisoftbuild01.fnal.gov.  These instructions also presume that the head of develop will be used.  If the tag will be made on a branch, refer to the NOvA instructions.
33 10 Lynn Garren
34 14 Lynn Garren
h3(count).  update "third party" products
35 14 Lynn Garren
36 15 Lynn Garren
Before building the nu suite, new releases of "third party" products need to be available.  Our definition of "third party products" is any product that does not use cetbuildtools or cetmodules.
37 15 Lynn Garren
38 15 Lynn Garren
This example updates the nu suite to use art 3.04.00.  python3 is the default. This means that the py3 qualifier is no longer used.  Instead, the py2 qualifier is used to specify a build with python v2_7_15a.  Because of this change, root has been updated to v6_18_04c.  Many products in the nulite distribution will need new releases with updated table files.  
39 15 Lynn Garren
40 21 Lynn Garren
* genie v3_00_06e
41 21 Lynn Garren
* dk2nudata v1_08_00f
42 21 Lynn Garren
* dk2nugenie v1_08_00h
43 14 Lynn Garren
44 10 Lynn Garren
h3(count). setup the working environment
45 10 Lynn Garren
46 9 Lynn Garren
<pre>
47 9 Lynn Garren
source /products/setup
48 9 Lynn Garren
setup mrb
49 9 Lynn Garren
setup larreltools
50 9 Lynn Garren
export PATH=$LARRELTOOLS_DIR/expert:${PATH}
51 9 Lynn Garren
export MRB_PROJECT=nu
52 9 Lynn Garren
</pre>
53 9 Lynn Garren
54 10 Lynn Garren
h3(count). make a working directory and checkout the code
55 10 Lynn Garren
56 10 Lynn Garren
<pre>
57 25 Lynn Garren
startNuLiteRel `pwd` v3_06_00 e19
58 25 Lynn Garren
startNuLiteRel `pwd` v3_06_00 c7
59 13 Lynn Garren
</pre>
60 13 Lynn Garren
61 13 Lynn Garren
62 12 Lynn Garren
h3(count). update, build, and test the code
63 12 Lynn Garren
64 12 Lynn Garren
Create a temporary release branch.  This workflow mimics, but does not use, gitflow.  This allows us more flexibility when dealing with multiple repositories.
65 12 Lynn Garren
<pre>
66 12 Lynn Garren
source v3_06_00/e19p/localProducts_nu_v3_06_00_e19_prof/setup
67 12 Lynn Garren
cd $MRB_SOURCE
68 12 Lynn Garren
tagLAr start $MRB_PROJECT_VERSION
69 12 Lynn Garren
</pre>
70 12 Lynn Garren
Now, we update the code.  This example updates the head of the nutools suite to use art 3.04.
71 13 Lynn Garren
<pre>
72 13 Lynn Garren
mrb uv canvas_root_io v1_05_00
73 16 Lynn Garren
mrb uv art v3_04_00
74 13 Lynn Garren
mrb uv art_root_io v1_02_00
75 16 Lynn Garren
mrb uv gallery v1_14_00
76 17 Lynn Garren
mrb uv toyExperiment v0_05_00
77 1 Lynn Garren
mrb uv ifdh_art v2_10_01
78 22 Lynn Garren
</pre>
79 24 Lynn Garren
In this case, we use mrb cq (changeQualifier) from py3 to py2 and also update the version_set.
80 22 Lynn Garren
<pre>
81 22 Lynn Garren
mrb uv version_set s94
82 22 Lynn Garren
mrb cq py3 py2
83 13 Lynn Garren
</pre>
84 23 Lynn Garren
Make sure the required products are available.  We install the critic distribution in /products on scisoftbuild01.  If there are local builds of other products, make sure they are installed in any private product directory.  Add symbolic links to the product in $MRB_INSTALL.
85 23 Lynn Garren
<pre>
86 23 Lynn Garren
cd $MRB_INSTALL
87 23 Lynn Garren
ln -s /home/garren/scratch/products/genie
88 23 Lynn Garren
etc.
89 23 Lynn Garren
</pre>
90 23 Lynn Garren
Build and test.  Iterate as necessary until everything builds cleanly.
91 23 Lynn Garren
<pre>
92 23 Lynn Garren
cd $MRB_BUILDDIR
93 23 Lynn Garren
mrb z (when necessary)
94 23 Lynn Garren
mrbsetenv
95 23 Lynn Garren
mrb t -jN
96 23 Lynn Garren
</pre>
97 23 Lynn Garren
98 23 Lynn Garren
h3(count). Tagging the release
99 23 Lynn Garren
100 23 Lynn Garren
First update the version.  In this case, the art release and accompanying qualifiers have changed.  The convention is to update the minor version of the packages in the nu suite.
101 23 Lynn Garren
<pre>
102 23 Lynn Garren
updateVersion minor
103 23 Lynn Garren
</pre>
104 23 Lynn Garren
It's wise to have a look at each package and make sure the changes are what you expect.
105 23 Lynn Garren
106 12 Lynn Garren
107 1 Lynn Garren
h2(count). Updating the nutools suite for NOvA
108 12 Lynn Garren
109 12 Lynn Garren
All NOvA releases are tagged and built off a special branch.
110 9 Lynn Garren
111 11 Lynn Garren
h3(count). setup the working environment
112 9 Lynn Garren
113 9 Lynn Garren
<pre>
114 9 Lynn Garren
source /cvmfs/nova.opensciencegrid.org/externals/setup
115 9 Lynn Garren
export PRODUCTS=$PRODUCTS:/cvmfs/larsoft.opensciencegrid.org/products
116 9 Lynn Garren
setup mrb
117 9 Lynn Garren
setup larreltools
118 9 Lynn Garren
export PATH=$LARRELTOOLS_DIR/expert:${PATH}
119 9 Lynn Garren
export MRB_PROJECT=nu
120 9 Lynn Garren
</pre>
121 10 Lynn Garren
122 10 Lynn Garren
h3(count). make a working directory and checkout the code
123 10 Lynn Garren
124 10 Lynn Garren
<pre>
125 10 Lynn Garren
makeNuLiteWorkDir `pwd` v3_01_04 e17
126 10 Lynn Garren
makeNuLiteWorkDir `pwd` v3_01_04 c2
127 10 Lynn Garren
</pre>
128 10 Lynn Garren
129 9 Lynn Garren
130 9 Lynn Garren
131 9 Lynn Garren
132 5 Lynn Garren
h2(count). just saving for reference
133 5 Lynn Garren
134 5 Lynn Garren
* If you want a complete nulite distribution with the new package, then you would also checkout nutools and follow the instructions for updating the nu suite.
135 5 Lynn Garren
** Edit the package version and the nudist version in nutools/bundle/CMakeLists.txt.
136 5 Lynn Garren
** The nudist version must be the same as the nutools version, but with an alphabet character added to the end.
137 5 Lynn Garren
*** For instance, v3_01_03 would become v3_01_03a, v3_01_03b would become v3_01_03c, etc.
138 5 Lynn Garren
** Build nutools and use the generated nutools/bundle/nu* files. (See below for more information.)