Project

General

Profile

What Lynn Does » History » Version 1

Rob Kutschke, 10/10/2019 09:52 AM

1 1 Rob Kutschke
2 1 Rob Kutschke
h1. What Lynn does
3 1 Rob Kutschke
4 1 Rob Kutschke
{{>TOC}}
5 1 Rob Kutschke
6 1 Rob Kutschke
h2. Build "third party" ups products as needed for art and larsoft
7 1 Rob Kutschke
8 1 Rob Kutschke
* We have a [[build-framework:|defined procedure]] that uses a number of scriptlets found in the ssibuildshims product.
9 1 Rob Kutschke
* Each product has a tiny redmine repository for the build and bootstrap scripts.
10 1 Rob Kutschke
** Scripts include autobuild.sh, bootstrap.sh, build_xxx.sh, and usually ups/xxx.table, where xxx is the product name.
11 1 Rob Kutschke
** We sometimes need patch files.
12 1 Rob Kutschke
** Other scripts are occasionally included, such as bootstrap_datasets.sh for geant4.
13 1 Rob Kutschke
** We tag these repositories with the product version.
14 1 Rob Kutschke
* To ensure reproducible builds, we only build from tagged releases.  In rare cases, we use a commit hash.
15 1 Rob Kutschke
* We require that the source code be captured in a source code tarball for reproducible builds.
16 1 Rob Kutschke
* We bootstrap and run local test builds for the supported compiler options.  This is a rapid turnaround development step.
17 1 Rob Kutschke
** Whenever possible, we run unit tests as part of the build procedure and require that these tests pass.
18 1 Rob Kutschke
* Once everything looks good, the source code tarball is uploaded to SciSoft.
19 1 Rob Kutschke
* Also provide help to people attempting to make their own ups products.
20 1 Rob Kutschke
21 1 Rob Kutschke
h2.  Build art releases after they are tagged.
22 1 Rob Kutschke
23 1 Rob Kutschke
* Kyle sends around an email to the SciSoft Team when a release is tagged.
24 1 Rob Kutschke
* Update information in build-framework/CMakeLists.txt
25 1 Rob Kutschke
** There are multiple branches for different product configurations
26 1 Rob Kutschke
* Run cmake to get the new build configuration files.
27 1 Rob Kutschke
* Check the new config files.
28 1 Rob Kutschke
** Always compare against the previous release.
29 1 Rob Kutschke
* If they are OK, use copyToSciSoft to install ONLY the appropriate build configuration files on SciSoft.  There are html files for each release that also need to be installed.
30 1 Rob Kutschke
** canvas
31 1 Rob Kutschke
** art
32 1 Rob Kutschke
** gallery
33 1 Rob Kutschke
** critic
34 1 Rob Kutschke
* Start a Jenkins build
35 1 Rob Kutschke
** https://buildmaster.fnal.gov/buildmaster/view/Art/job/critic-all/
36 1 Rob Kutschke
** The build needs two iterations.  First for python 2 (the LABEL field is empty) and then a second pass with LABEL set to py3.
37 1 Rob Kutschke
* Sometimes there are problems with the build.
38 1 Rob Kutschke
** Investigate and fix.
39 1 Rob Kutschke
** A local test build is sometimes helpful, depending on the problem.
40 1 Rob Kutschke
* Once the build has completed successfully, download the tarballs locally and then upload them to SciSoft.
41 1 Rob Kutschke
** Work in an empty directory
42 1 Rob Kutschke
** copyFromJenkins -N -q c2 -q c7 -q e17 -q e19 critic-all
43 1 Rob Kutschke
** copyToSciSoft *
44 1 Rob Kutschke
45 1 Rob Kutschke
h2. Update other products to use the appropriate release of art
46 1 Rob Kutschke
47 1 Rob Kutschke
* artdaq_core
48 1 Rob Kutschke
** Owned by the artdaq group
49 1 Rob Kutschke
** This product is designed to build against several different releases of art, as reflected in the generated table file.
50 1 Rob Kutschke
** If the head of develop matches the latest tag, then use git flow to tag against the head of develop.  Otherwise make a release branch against the latest tag and edit product deps.
51 1 Rob Kutschke
** We only edit ups/product_deps to make a new release.
52 1 Rob Kutschke
* ifdh_art
53 1 Rob Kutschke
** This is the ifdh service developed by Marc Mengel at the request of the art team.
54 1 Rob Kutschke
** ifdh_art can build against several different releases of art.
55 1 Rob Kutschke
** ifdh_art is updated when libwda or ifdhc is updated or if it needs to recognize a new release of art.
56 1 Rob Kutschke
* nutools
57 1 Rob Kutschke
** This product is owned by NOvA
58 1 Rob Kutschke
** However, larsoft uses it extensively and sometimes makes contributions to the code.
59 1 Rob Kutschke
** Any changes beyond those required by a new release of art, genie, or geant4 must be approved.
60 1 Rob Kutschke
61 1 Rob Kutschke
h2. Build mu distributions
62 1 Rob Kutschke
63 1 Rob Kutschke
* The mu distribution is basically an art distribution with a few additional products.
64 1 Rob Kutschke
* The distribution is built when requested.
65 1 Rob Kutschke
* Found on a branch of build-framework
66 1 Rob Kutschke
67 1 Rob Kutschke
68 1 Rob Kutschke
h2. Build nutools releases and the nu or nulite distributions.
69 1 Rob Kutschke
70 1 Rob Kutschke
* Both NOvA and LArSoft can drive the need for a new release of nutools.
71 1 Rob Kutschke
** NOvA uses the full nu distribution.
72 1 Rob Kutschke
** LArSoft now uses the nulite distribution which ignores packages that are NOvA specific.
73 1 Rob Kutschke
* Discussion will start soon on a plan to split nutools into modular components.
74 1 Rob Kutschke
** Presently nutools contains, a genie interface, a geant4 interface, a random number interface, and other bits.  Updates are needed on different timescales and we would like to decouple them.
75 1 Rob Kutschke
76 1 Rob Kutschke
h2. LArSoft release management
77 1 Rob Kutschke
78 1 Rob Kutschke
* Advise and collect information for each weekly release.
79 1 Rob Kutschke
** I am often the first point of contact for code changes.
80 1 Rob Kutschke
** Sometimes push back about proposed changes.
81 1 Rob Kutschke
* Send notices that the release is about to start.
82 1 Rob Kutschke
* Make a test build with larsoft and all the experiment code.
83 1 Rob Kutschke
** From long experience, if there is more than one set of feature branches to merge, each set must be tested sequentially.
84 1 Rob Kutschke
** While this could be done via CI, it is currently more efficient to test on woof.  Especially since overlapping feature branches must sometimes be tested.
85 1 Rob Kutschke
** We do encourage anyone submitting a feature branch to run the CI themselves.
86 1 Rob Kutschke
* We work on a temporary release branch.
87 1 Rob Kutschke
* Once the tests are complete and successful, update the release versions for those packages which will be changing and then tag.
88 1 Rob Kutschke
* The tag is merged with master, but not with develop. 
89 1 Rob Kutschke
** Merging with develop before the tagged release is available on cvmfs causes all kinds of problems. 
90 1 Rob Kutschke
* Build on Jenkins.  Sometimes we find problems during this step.  We move the tag if necessary when resolving problems.
91 1 Rob Kutschke
* Once the build is complete, we finalize the release.
92 1 Rob Kutschke
** pull the tarballs from Jenkins and upload them to SciSoft
93 1 Rob Kutschke
** install the release on cvmfs
94 1 Rob Kutschke
** For macOS building, we install the c2 build on /grid/fermiapp/products/larsoft
95 1 Rob Kutschke
** make a cross package tag
96 1 Rob Kutschke
** make release notes
97 1 Rob Kutschke
** final merge with develop
98 1 Rob Kutschke
** send an announcement
99 1 Rob Kutschke
* The procedure is described in detail at [[larsoft:How_to_tag_and_build_a_LArSoft_vx_yy_zz_release]]
100 1 Rob Kutschke
** Although it is tempting to take shortcuts, the procedure is designed to allow the release manager to recover from various problems that may arise.  Taking shortcuts removes those safeguards.
101 1 Rob Kutschke
102 1 Rob Kutschke
h2.  support various infrastructure products
103 1 Rob Kutschke
104 1 Rob Kutschke
* cetbuildtools and mrb
105 1 Rob Kutschke
** both products were originally developed by me, but have significant contributions from others
106 1 Rob Kutschke
** We have provided cetmodules for use within spack and are contemplating a move across the board from cetbuildtools to cetmodules.  Details need to be considered.
107 1 Rob Kutschke
* larreltools
108 1 Rob Kutschke
** Replaces laradmin
109 1 Rob Kutschke
** Contains scripts used to build regular and patch releases of larsoft
110 1 Rob Kutschke
* larutils
111 1 Rob Kutschke
** The larsoft build scripts used on Jenkins reside in larutils
112 1 Rob Kutschke
* artutilscripts
113 1 Rob Kutschke
** The build scripts used when building art, nutools, etc. on Jenkins reside in artutilscripts. 
114 1 Rob Kutschke
** This is also where buildFW, pullProducts, copyFromJenkins, copyToSciSoft, etc. are found.
115 1 Rob Kutschke
* laradmin
116 1 Rob Kutschke
** This is now only of archival interest
117 1 Rob Kutschke
** It contains the record of various larsoft migrations.
118 1 Rob Kutschke
119 1 Rob Kutschke
h2. SciSoft
120 1 Rob Kutschke
121 1 Rob Kutschke
* approve requests for read-write access
122 1 Rob Kutschke
** The possibility exists for someone to accidentally remove or overwrite products.
123 1 Rob Kutschke
** For this reason, we use nfs4 groups to limit read-write access to necessary directories.
124 1 Rob Kutschke
** Some members of the SciSoft team have read-write access to all of SciSoft.
125 1 Rob Kutschke
* copy files to pnfs for archiving
126 1 Rob Kutschke
** There is a procedure, but it is slow and not automatic.
127 1 Rob Kutschke
128 1 Rob Kutschke
h2. spack
129 1 Rob Kutschke
130 1 Rob Kutschke
* I attempt to help with the spack development, but this takes a back seat to the other efforts.
131 1 Rob Kutschke
132 1 Rob Kutschke
h2. heppdt
133 1 Rob Kutschke
134 1 Rob Kutschke
* This is a legacy product that was spun off as a C++ replacement for part of the original stdhep.
135 1 Rob Kutschke
* It is still in use by mu2e.
136 1 Rob Kutschke
137 1 Rob Kutschke
h2. Mailing lists
138 1 Rob Kutschke
139 1 Rob Kutschke
* owner/manager of several mailing lists
140 1 Rob Kutschke
** many are unused and can be retired
141 1 Rob Kutschke
142 1 Rob Kutschke
h2. CLHEP maintenance and support
143 1 Rob Kutschke
144 1 Rob Kutschke
* I have been the main point person for CLHEP bug fixes.
145 1 Rob Kutschke
* Support will be passed to CERN.
146 1 Rob Kutschke
147 1 Rob Kutschke
h2. system management
148 1 Rob Kutschke
149 1 Rob Kutschke
* Still provide backup support for oink.  Rare that I do anything.
150 1 Rob Kutschke
* detsim.fnal.gov
151 1 Rob Kutschke
** Now supported by CCD