How to tag and build a LArSoft patch release » History » Version 11
Lynn Garren, 02/27/2019 02:11 PM
1 | 1 | Lynn Garren | h1. How to tag and build a LArSoft patch release |
---|---|---|---|
2 | 1 | Lynn Garren | |
3 | 3 | Lynn Garren | {{>TOC}} |
4 | 3 | Lynn Garren | |
5 | 10 | Lynn Garren | [[Policy for development from a tagged release]] |
6 | 1 | Lynn Garren | Experiments will build patch releases themselves with help from the LArSoft release manager. |
7 | 2 | Lynn Garren | There will be a very limited set of people with privileges to do this. |
8 | 7 | Lynn Garren | We ask that the experiment release managers to whom these privileges have been granted do not grant privileges to anyone else. Instead request permissions from LArSoft. |
9 | 6 | Lynn Garren | Permissions will be removed when someone ceases to have this responsibility. |
10 | 2 | Lynn Garren | |
11 | 10 | Lynn Garren | * Permissions include: |
12 | 10 | Lynn Garren | ** larsoft redmine manager role |
13 | 10 | Lynn Garren | ** ability to login to larsoft cvmfs |
14 | 10 | Lynn Garren | ** ability to upload to SciSoft with the larsoft role |
15 | 10 | Lynn Garren | |
16 | 11 | Lynn Garren | h2(count). Request a release |
17 | 11 | Lynn Garren | |
18 | 11 | Lynn Garren | * Before starting any patch release, fill out a redmine issue requesting the release with all pertinent information. |
19 | 10 | Lynn Garren | |
20 | 10 | Lynn Garren | |
21 | 2 | Lynn Garren | h2(count). Tools |
22 | 2 | Lynn Garren | |
23 | 2 | Lynn Garren | * Scripts used to build the patch release will be found in the larreltools product. |
24 | 2 | Lynn Garren | * larreltools is not part of the standard distribution, but will be available from SciSoft and in cvmfs. |
25 | 4 | Lynn Garren | |
26 | 2 | Lynn Garren | h2(count). Initial setup |
27 | 1 | Lynn Garren | |
28 | 10 | Lynn Garren | *This step is done once for any patch release step and will be done by the LArSoft release manager.* |
29 | 9 | Lynn Garren | |
30 | 2 | Lynn Garren | h3(count). Make branches for the release |
31 | 2 | Lynn Garren | |
32 | 2 | Lynn Garren | * If a release is named vaa_bb_cc, branches will be created for the larsoft/larsoftobj suite with the name vaa_bb_cc_br. |
33 | 2 | Lynn Garren | <pre> |
34 | 2 | Lynn Garren | (setup the latest larreltools) |
35 | 2 | Lynn Garren | setup mrb |
36 | 2 | Lynn Garren | export MRB_PROJECT=larsoft |
37 | 2 | Lynn Garren | mrb newDev -v vaa_bb_cc -q eN:prof |
38 | 2 | Lynn Garren | source local*/setup |
39 | 2 | Lynn Garren | cd $MRB_SOURCE |
40 | 2 | Lynn Garren | mrb g -t LARSOFT_SUITE_vAA_BB_CC larsoft_suite |
41 | 2 | Lynn Garren | mrb g -t LARSOFT_SUITE_vAA_BB_CC larsoftobj_suite |
42 | 2 | Lynn Garren | dogit branch (check that you are on the head of the tag) |
43 | 2 | Lynn Garren | dogit "co -b vaa_bb_cc_br" |
44 | 2 | Lynn Garren | dogit "push -u origin vaa_bb_cc_br" |
45 | 1 | Lynn Garren | </pre> |
46 | 8 | Lynn Garren | ** Note that some older patch releases may have used a slightly different branch naming scheme. |
47 | 2 | Lynn Garren | |
48 | 4 | Lynn Garren | h3(count). Housecleaning |
49 | 2 | Lynn Garren | |
50 | 2 | Lynn Garren | * cd $MRB_SOURCE/larsoft/releaseDB |
51 | 2 | Lynn Garren | * Make sure that larutils, larbatch, and mrb are removed from the cfg scripts. |
52 | 2 | Lynn Garren | ** streamline this by creating a small "larsoft extras" distribution that can be optionally included. |
53 | 2 | Lynn Garren | * commit and push the change |
54 | 2 | Lynn Garren | |
55 | 4 | Lynn Garren | h2(count). Making a patch release |
56 | 2 | Lynn Garren | |
57 | 2 | Lynn Garren | * These notes presume that you do not have an existing directory |
58 | 2 | Lynn Garren | |
59 | 4 | Lynn Garren | h3(count). MicroBooNE setup |
60 | 2 | Lynn Garren | |
61 | 2 | Lynn Garren | * setup the latest release of larreltools |
62 | 2 | Lynn Garren | * startUboonePatchRel <working_dir> <tag> <larsoft branch> [<uboone branch>] |
63 | 2 | Lynn Garren | ** This script will create a directory named "tag" under the "working_dir". |
64 | 2 | Lynn Garren | ** This script will clone the larsoft/larsoftobj on the named branch. |
65 | 2 | Lynn Garren | ** The same branch name will be used when cloning the uboone_suite unless a second branch name is supplied. |
66 | 2 | Lynn Garren | ** The directory structure will include a single source code directory with subdirectories for e17 and c2 debug and prof builds. |
67 | 2 | Lynn Garren | ** Use "dogit branch" to double check that you are on the expected branch. |
68 | 2 | Lynn Garren | |
69 | 4 | Lynn Garren | h3(count). Setup for other experiments |
70 | 2 | Lynn Garren | |
71 | 2 | Lynn Garren | * Appropriate scripts will be supplied when necessary. |
72 | 2 | Lynn Garren | |
73 | 4 | Lynn Garren | h3(count). Local development |
74 | 2 | Lynn Garren | |
75 | 2 | Lynn Garren | * Before integrating any feature branches, make sure that the release builds as is and that the unit tests work. |
76 | 2 | Lynn Garren | ** Make sure to build and test both e17 and c2. Usually the prof build is sufficient for testing. prof is preferred over debug, which sometimes masks problems. |
77 | 2 | Lynn Garren | <pre> |
78 | 2 | Lynn Garren | source local*/setup |
79 | 2 | Lynn Garren | cd $MRB_BUILDDIR |
80 | 2 | Lynn Garren | mrb t -jN |
81 | 2 | Lynn Garren | </pre> |
82 | 2 | Lynn Garren | * Next make changes and integrate feature branches |
83 | 2 | Lynn Garren | ** Because feature branches may conflict or not have been thoroughly tested, we recommend merging one feature branch set at a time. Build and test after each set. |
84 | 2 | Lynn Garren | |
85 | 4 | Lynn Garren | h3(count). Update the package versions |
86 | 2 | Lynn Garren | |
87 | 2 | Lynn Garren | * Do this once all changes have been made. |
88 | 2 | Lynn Garren | * ONLY update the version if there is a change. |
89 | 2 | Lynn Garren | * cd to each directory separately |
90 | 2 | Lynn Garren | * git diff LARSOFT_SUITE_vaa_bb_cc[_dd] |
91 | 2 | Lynn Garren | ** You are comparing against the previous patch release in this set. |
92 | 2 | Lynn Garren | ** If this is the first patch release in the set, compare against the original release. |
93 | 2 | Lynn Garren | ** If there are no changes, move this package out of the way. |
94 | 2 | Lynn Garren | <pre> |
95 | 2 | Lynn Garren | cd $MRB_SOURCE |
96 | 2 | Lynn Garren | mkdir ../notag |
97 | 2 | Lynn Garren | mv xxx ../notag |
98 | 2 | Lynn Garren | mrb uc |
99 | 2 | Lynn Garren | </pre> |
100 | 2 | Lynn Garren | ** Move all experiment code out of the way. (_mv ub* ../notag/_ for MicroBooNE) |
101 | 2 | Lynn Garren | ** Use "mrb uv" to update the version by one in the patch field. |
102 | 2 | Lynn Garren | *** There should be a script for this... |
103 | 2 | Lynn Garren | * You must update larsoftobj/bundle/CMakeLists.txt and larsoft/releaseDB/CMakeLists.txt by hand with the new versions. |
104 | 2 | Lynn Garren | ** commit and push these changes |
105 | 2 | Lynn Garren | * DO NOT commit the changes to ups/product_deps |
106 | 2 | Lynn Garren | |
107 | 2 | Lynn Garren | |
108 | 4 | Lynn Garren | h3(count). Tagging the release |
109 | 2 | Lynn Garren | |
110 | 2 | Lynn Garren | * Do a final build and test with just the packages you expect to tag. |
111 | 2 | Lynn Garren | ** This step will also create the necessary build cfg files. |
112 | 2 | Lynn Garren | ** If there are changes in both larsoft and larsoftobj, double check for consistency. |
113 | 2 | Lynn Garren | <pre> |
114 | 2 | Lynn Garren | cd $MRB_BUILDDIR/larsoft |
115 | 2 | Lynn Garren | diff releaseDB ../larsoftobj/bundle | grep diff |
116 | 2 | Lynn Garren | diff releaseDB ../larsoftobj/bundle | grep Only |
117 | 2 | Lynn Garren | </pre> |
118 | 2 | Lynn Garren | ** If all is good, copy the cfg scripts to SciSoft |
119 | 2 | Lynn Garren | <pre> |
120 | 2 | Lynn Garren | cd $MRB_BUILDDIR/larsoft/releaseDB |
121 | 2 | Lynn Garren | copyToSciSoft lar* |
122 | 2 | Lynn Garren | </pre> |
123 | 2 | Lynn Garren | * Now tag |
124 | 2 | Lynn Garren | ** tagPatchRel <existing branch> <new master tag> |
125 | 2 | Lynn Garren | ** tagPatchRel will commit ups/product_deps with an appropriate message, make the tag, and push both tag and branch to origin. |
126 | 2 | Lynn Garren | ** Use "dogit status" to doublecheck. |
127 | 2 | Lynn Garren | ** *IMPORTANT*: patch release tags are never merged with the master branch |
128 | 2 | Lynn Garren | |
129 | 4 | Lynn Garren | h3(count). Official build |
130 | 2 | Lynn Garren | |
131 | 2 | Lynn Garren | * build on Jenkins |
132 | 2 | Lynn Garren | * Use special patch release build jobs. |
133 | 2 | Lynn Garren | ** This avoids possible conflict with normal release builds. |
134 | 1 | Lynn Garren | * use copyFromJenkins to download the tarballs and release manifests from Jenkins |
135 | 1 | Lynn Garren | * use copyToSciSoft to upload the tarballs and release manifests to SciSoft |
136 | 1 | Lynn Garren | ** This step requires extra privileges. |
137 | 5 | Lynn Garren | |
138 | 5 | Lynn Garren | h3(count). Release Notes and Cross Package Tag |
139 | 5 | Lynn Garren | |
140 | 2 | Lynn Garren | * Once the build is complete for ALL variants (SLF and macOS), make the cross package tag. |
141 | 5 | Lynn Garren | ** We wait because sometimes a problem is not found until building and we need to move a tag. |
142 | 2 | Lynn Garren | ** This step requires redmine manager persmission for larsoft |
143 | 2 | Lynn Garren | <pre> |
144 | 2 | Lynn Garren | cp-lar-tag <larsoft tag> <larsoftobj tag> |
145 | 2 | Lynn Garren | </pre> |
146 | 2 | Lynn Garren | *** It would be nice to fix cp-lar-tag so it deduces the larsoftobj tag |
147 | 2 | Lynn Garren | * Make the release notes |
148 | 2 | Lynn Garren | ** make-release-notes <working_directory> <tag> <previous tag> |
149 | 2 | Lynn Garren | *** This script will create a subdirectory named tag under the working_directory |
150 | 2 | Lynn Garren | ** cd into the newly created tag directory |
151 | 2 | Lynn Garren | ** cat ReleaseNotes-vaa_bb_cc_dd |
152 | 2 | Lynn Garren | *** The first line of this output is the entry in the [[LArSoft_release_list|release list]] |
153 | 2 | Lynn Garren | *** The remainder of the output is used to create the release note itself. |
154 | 1 | Lynn Garren | **** copy and edit the top part if appropriate. |
155 | 1 | Lynn Garren | **** Do not edit anything under the "Change List" section. |
156 | 1 | Lynn Garren | **** It is important to use cat so that there are no unexpected line breaks. |
157 | 5 | Lynn Garren | |
158 | 5 | Lynn Garren | h3(count). Upload to cvmfs |
159 | 5 | Lynn Garren | |
160 | 5 | Lynn Garren | * This step requires login privileges. |
161 | 5 | Lynn Garren | * ssh cvmfslarsoft@oasiscfs.fnal.gov |
162 | 5 | Lynn Garren | * The README provides an overview of the necessary steps. |
163 | 5 | Lynn Garren | ** start a transaction |
164 | 5 | Lynn Garren | ** install using installBundle.sh |
165 | 5 | Lynn Garren | ** publish |
166 | 5 | Lynn Garren | * installBundle.sh larsoft vaa_bb_cc_zz sNN-eY |
167 | 5 | Lynn Garren | ** Note that if you call installBundle.sh with sNN-e17, it will install both e17 and c2 variants. |
168 | 5 | Lynn Garren | ** The script will attempt to install the release for all supported platforms and clean up after itself. |
169 | 5 | Lynn Garren | ** If some expected distributions are not available, then manifests will be left in the .working subdirectory. Check and remove them if necessary. |