LArSoft

h1. How to tag and build a LArSoft patch release

{{>TOC}}

[[Policy for development from a tagged release]]

Experiments will build patch releases themselves with help from the LArSoft release manager.

There will be a very limited set of people with privileges to do this.  

We ask that the experiment release managers to whom these privileges have been granted do not grant privileges to anyone else.  Instead request permissions from LArSoft.

Permissions will be removed when someone ceases to have this responsibility.

* Permissions include:

** larsoft redmine manager role

** ability to login to larsoft cvmfs

** ability to upload to SciSoft with the larsoft role

** developer status for larreltools

[[LArSoft responsiblities for patch releases]]

h2(count). Request a release

* Before starting any patch release, fill out a redmine issue requesting the release with all pertinent information.

h2(count). Tools

* Scripts used to build the patch release will be found in the larreltools product.  

* *larreltools* is not part of the standard distribution, but will be available from SciSoft and in cvmfs.

** Always use the latest release, which will be declared current.

** More [[larreltools:|information]]

h2(count). Initial setup

*This step is done once for any patch release set and will be done by the LArSoft release manager.*

h2(count). Making a patch release

* These notes presume that you do not have an existing directory

h3(count). MicroBooNE setup

* setup the latest release of larreltools

* startUboonePatchRel  <working_dir> <tag> <larsoft branch> [<uboone branch>]

** This script will create a directory named "tag" under the "working_dir".

** This script will clone the larsoft/larsoftobj on the named branch.

** The same branch name will be used when cloning the uboone_suite unless a second branch name is supplied.

** The directory structure will include a single source code directory with subdirectories for e17 and c2 debug and prof builds.

** Use "dogit branch" to double check that you are on the expected branch.

h3(count). Setup for other experiments

* Appropriate scripts will be supplied when necessary.

** We expect to provide a more generic script.

h3(count). Local development

* Before integrating any feature branches, make sure that the release builds as is and that the unit tests work.

** Make sure to build and test both e17 and c2.   Usually the prof build is sufficient for testing.  prof is preferred over debug, which sometimes masks problems.

<pre>

source local*/setup

cd $MRB_BUILDDIR

mrb t -jN

</pre>

* Next make changes and integrate feature branches

** Because feature branches may conflict or not have been thoroughly tested, we recommend merging one feature branch set at a time.  Build and test after each set.

h3(count). Update the package versions

* Do this once all changes have been made.

* ONLY update the version if there is a change.

* cd to each directory separately

* git diff LARSOFT_SUITE_vaa_bb_cc[_dd]

** You are comparing against the previous patch release in this set.

** If this is the first patch release in the set, compare against the original release.

** If there are no changes, move this package out of the way.

<pre>

cd $MRB_SOURCE

mkdir ../notag

mv xxx ../notag

mrb uc

</pre>

** Move all experiment code out of the way.  (_mv ub* ../notag/_ for MicroBooNE)

* Use the *updatePatchVersion* script from larreltools.

** This script takes no arguments.

** This script will call "mrb uv" to update each patch release by one.

** If there was no previous patch release, the version will be vaa_bb_cc_01

* You must update larsoftobj/bundle/CMakeLists.txt and larsoft/releaseDB/CMakeLists.txt by hand with the new versions.

** commit and push these changes

* DO NOT commit the changes to ups/product_deps

h3(count).  Tagging the release

* Do a final build and test with just the packages you expect to tag.  

** This step will also create the necessary build cfg files.

** If there are changes in both larsoft and larsoftobj, double check for consistency.

<pre>

cd $MRB_BUILDDIR/larsoft

diff releaseDB ../larsoftobj/bundle | grep diff

diff releaseDB ../larsoftobj/bundle | grep Only

</pre>

** If all is good, copy the cfg scripts to SciSoft

<pre>

cd $MRB_BUILDDIR/larsoft/releaseDB

copyToSciSoft lar*

</pre>

* Now tag

** tagPatchRel <existing branch> <new master tag>

** tagPatchRel will commit ups/product_deps with an appropriate message, make the tag, and push both tag and branch to origin.

** Use "dogit status" to doublecheck.

** *IMPORTANT*:  patch release tags are never merged with the master branch

h3(count).  Official build

* build on Jenkins

* Use special patch release build jobs.

** This avoids possible conflict with normal release builds.

** "larpatch-slf":https://buildmaster.fnal.gov/buildmaster/view/LArSoft/job/larpatch-slf/

** "larpatch-mac":https://buildmaster.fnal.gov/buildmaster/view/LArSoft/job/larpatch-mac/

* use copyFromJenkins to download the tarballs and release manifests from Jenkins

* use copyToSciSoft to upload the tarballs and release manifests to SciSoft

** This step requires extra privileges.

h3(count).  Release Notes and Cross Package Tag

* Once the build is complete for ALL variants (SLF and macOS), make the cross package tag.

** We wait because sometimes a problem is not found until building and we need to move a tag.

** This step requires redmine manager persmission for larsoft

<pre>

cp-patch-tag <larsoft tag> <larsoftobj tag>

</pre>

*** It would be nice to fix cp-patch-tag so it deduces the larsoftobj tag

* Make the release notes

** make-release-notes <working_directory> <tag> <previous tag>

*** This script will create a subdirectory named tag under the working_directory

** cd into the newly created tag directory

** cat ReleaseNotes-vaa_bb_cc_dd

*** The first line of this output is the entry in the [[LArSoft_release_list|release list]]

*** The remainder of the output is used to create the release note itself.

**** copy and edit the top part if appropriate.

**** Do not edit anything under the "Change List" section.

**** It is important to use cat so that there are no unexpected line breaks.

h3(count). Upload to cvmfs

* This step requires login privileges.

* ssh cvmfslarsoft@oasiscfs.fnal.gov

* The README provides an overview of the necessary steps.

** start a transaction

** install using installBundle.sh

** publish

* installBundle.sh larsoft vaa_bb_cc_zz sNN-eY

** Note that if you call installBundle.sh with sNN-e17, it will install both e17 and c2 variants.

** The script will attempt to install the release for all supported platforms and clean up after itself.

** If some expected distributions are not available, then manifests will be left in the .working subdirectory. Check and remove them if necessary.