LArSoft

h1. How to tag and build a LArSoft patch release

{{>TOC}}

[[Policy for development from a tagged release]]

Experiments will build patch releases themselves with help from the LArSoft release manager.

There will be a very limited set of people with privileges to do this.  

We ask that the experiment release managers to whom these privileges have been granted do not grant privileges to anyone else.  Instead request permissions from LArSoft.

Permissions will be removed when someone ceases to have this responsibility.

* Permissions include:

** larsoft redmine manager role

** ability to login to larsoft cvmfs

** ability to upload to SciSoft with the larsoft role

h2(count). Request a release

* Before starting any patch release, fill out a redmine issue requesting the release with all pertinent information.

h2(count). Tools

* Scripts used to build the patch release will be found in the larreltools product.  

* larreltools is not part of the standard distribution, but will be available from SciSoft and in cvmfs.

h2(count). Initial setup

*This step is done once for any patch release step and will be done by the LArSoft release manager.*

h3(count). Make branches for the release

* If a release is named vaa_bb_cc, branches will be created for the larsoft/larsoftobj suite with the name vaa_bb_cc_br.

<pre>

(setup the latest larreltools)

setup mrb

export MRB_PROJECT=larsoft

mrb newDev -v  vaa_bb_cc -q eN:prof 

source local*/setup

cd $MRB_SOURCE

mrb g -t LARSOFT_SUITE_vAA_BB_CC larsoft_suite

mrb g -t LARSOFT_SUITE_vAA_BB_CC larsoftobj_suite

dogit branch (check that you are on the head of the tag)

dogit "co -b vaa_bb_cc_br"

dogit "push -u origin vaa_bb_cc_br"

</pre>

** Note that some older patch releases may have used a slightly different branch naming scheme.

h3(count). Housecleaning

* cd $MRB_SOURCE/larsoft/releaseDB

* Make sure that larutils, larbatch, and mrb are removed from the cfg scripts.

** streamline this by creating a small "larsoft extras" distribution that can be optionally included.

* commit and push the change

h2(count). Making a patch release

* These notes presume that you do not have an existing directory

h3(count). MicroBooNE setup

* setup the latest release of larreltools

* startUboonePatchRel  <working_dir> <tag> <larsoft branch> [<uboone branch>]

** This script will create a directory named "tag" under the "working_dir".

** This script will clone the larsoft/larsoftobj on the named branch.

** The same branch name will be used when cloning the uboone_suite unless a second branch name is supplied.

** The directory structure will include a single source code directory with subdirectories for e17 and c2 debug and prof builds.

** Use "dogit branch" to double check that you are on the expected branch.

h3(count). Setup for other experiments

* Appropriate scripts will be supplied when necessary.

h3(count). Local development

* Before integrating any feature branches, make sure that the release builds as is and that the unit tests work.

** Make sure to build and test both e17 and c2.   Usually the prof build is sufficient for testing.  prof is preferred over debug, which sometimes masks problems.

<pre>

source local*/setup

cd $MRB_BUILDDIR

mrb t -jN

</pre>

* Next make changes and integrate feature branches

** Because feature branches may conflict or not have been thoroughly tested, we recommend merging one feature branch set at a time.  Build and test after each set.

h3(count). Update the package versions

* Do this once all changes have been made.

* ONLY update the version if there is a change.

* cd to each directory separately

* git diff LARSOFT_SUITE_vaa_bb_cc[_dd]

** You are comparing against the previous patch release in this set.

** If this is the first patch release in the set, compare against the original release.

** If there are no changes, move this package out of the way.

<pre>

cd $MRB_SOURCE

mkdir ../notag

mv xxx ../notag

mrb uc

</pre>

** Move all experiment code out of the way.  (_mv ub* ../notag/_ for MicroBooNE)

** Use "mrb uv" to update the version by one in the patch field.

*** There should be a script for this...

* You must update larsoftobj/bundle/CMakeLists.txt and larsoft/releaseDB/CMakeLists.txt by hand with the new versions.

** commit and push these changes

* DO NOT commit the changes to ups/product_deps

h3(count).  Tagging the release

* Do a final build and test with just the packages you expect to tag.  

** This step will also create the necessary build cfg files.

** If there are changes in both larsoft and larsoftobj, double check for consistency.

<pre>

cd $MRB_BUILDDIR/larsoft

diff releaseDB ../larsoftobj/bundle | grep diff

diff releaseDB ../larsoftobj/bundle | grep Only

</pre>

** If all is good, copy the cfg scripts to SciSoft

<pre>

cd $MRB_BUILDDIR/larsoft/releaseDB

copyToSciSoft lar*

</pre>

* Now tag

** tagPatchRel <existing branch> <new master tag>

** tagPatchRel will commit ups/product_deps with an appropriate message, make the tag, and push both tag and branch to origin.

** Use "dogit status" to doublecheck.

** *IMPORTANT*:  patch release tags are never merged with the master branch

h3(count).  Official build

* build on Jenkins

* Use special patch release build jobs.

** This avoids possible conflict with normal release builds.

* use copyFromJenkins to download the tarballs and release manifests from Jenkins

* use copyToSciSoft to upload the tarballs and release manifests to SciSoft

** This step requires extra privileges.

h3(count).  Release Notes and Cross Package Tag

* Once the build is complete for ALL variants (SLF and macOS), make the cross package tag.

** We wait because sometimes a problem is not found until building and we need to move a tag.

** This step requires redmine manager persmission for larsoft

<pre>

cp-lar-tag <larsoft tag> <larsoftobj tag>

</pre>

*** It would be nice to fix cp-lar-tag so it deduces the larsoftobj tag

* Make the release notes

** make-release-notes <working_directory> <tag> <previous tag>

*** This script will create a subdirectory named tag under the working_directory

** cd into the newly created tag directory

** cat ReleaseNotes-vaa_bb_cc_dd

*** The first line of this output is the entry in the [[LArSoft_release_list|release list]]

*** The remainder of the output is used to create the release note itself.

**** copy and edit the top part if appropriate.

**** Do not edit anything under the "Change List" section.

**** It is important to use cat so that there are no unexpected line breaks.

h3(count). Upload to cvmfs

* This step requires login privileges.

* ssh cvmfslarsoft@oasiscfs.fnal.gov

* The README provides an overview of the necessary steps.

** start a transaction

** install using installBundle.sh

** publish

* installBundle.sh larsoft vaa_bb_cc_zz sNN-eY

** Note that if you call installBundle.sh with sNN-e17, it will install both e17 and c2 variants.

** The script will attempt to install the release for all supported platforms and clean up after itself.

** If some expected distributions are not available, then manifests will be left in the .working subdirectory. Check and remove them if necessary.