Project

General

Profile

How to tag and build a LArSoft patch release » History » Version 14

« Previous - Version 14/33 (diff) - Next » - Current version
Lynn Garren, 03/04/2019 04:29 PM


How to tag and build a LArSoft patch release

Policy for development from a tagged release
Experiments will build patch releases themselves with help from the LArSoft release manager.
There will be a very limited set of people with privileges to do this.
We ask that the experiment release managers to whom these privileges have been granted do not grant privileges to anyone else. Instead request permissions from LArSoft.
Permissions will be removed when someone ceases to have this responsibility.

  • Permissions include:
    • larsoft redmine manager role
    • ability to login to larsoft cvmfs
    • ability to upload to SciSoft with the larsoft role
    • developer status for larreltools

LArSoft responsiblities for patch releases

Request a release

  • Before starting any patch release, fill out a redmine issue requesting the release with all pertinent information.

Tools

  • Scripts used to build the patch release will be found in the larreltools product.
  • larreltools is not part of the standard distribution, but will be available from SciSoft and in cvmfs.

Initial setup

This step is done once for any patch release step and will be done by the LArSoft release manager.

Making a patch release

  • These notes presume that you do not have an existing directory

MicroBooNE setup

  • setup the latest release of larreltools
  • startUboonePatchRel <working_dir> <tag> <larsoft branch> [<uboone branch>]
    • This script will create a directory named "tag" under the "working_dir".
    • This script will clone the larsoft/larsoftobj on the named branch.
    • The same branch name will be used when cloning the uboone_suite unless a second branch name is supplied.
    • The directory structure will include a single source code directory with subdirectories for e17 and c2 debug and prof builds.
    • Use "dogit branch" to double check that you are on the expected branch.

Setup for other experiments

  • Appropriate scripts will be supplied when necessary.

Local development

  • Before integrating any feature branches, make sure that the release builds as is and that the unit tests work.
    • Make sure to build and test both e17 and c2. Usually the prof build is sufficient for testing. prof is preferred over debug, which sometimes masks problems.
      source local*/setup
      cd $MRB_BUILDDIR
      mrb t -jN
      
  • Next make changes and integrate feature branches
    • Because feature branches may conflict or not have been thoroughly tested, we recommend merging one feature branch set at a time. Build and test after each set.

Update the package versions

  • Do this once all changes have been made.
  • ONLY update the version if there is a change.
  • cd to each directory separately
  • git diff LARSOFT_SUITE_vaa_bb_cc[_dd]
    • You are comparing against the previous patch release in this set.
    • If this is the first patch release in the set, compare against the original release.
    • If there are no changes, move this package out of the way.
      cd $MRB_SOURCE
      mkdir ../notag
      mv xxx ../notag
      mrb uc
      
    • Move all experiment code out of the way. (mv ub* ../notag/ for MicroBooNE)
    • Use "mrb uv" to update the version by one in the patch field.
      • There should be a script for this...
  • You must update larsoftobj/bundle/CMakeLists.txt and larsoft/releaseDB/CMakeLists.txt by hand with the new versions.
    • commit and push these changes
  • DO NOT commit the changes to ups/product_deps

Tagging the release

  • Do a final build and test with just the packages you expect to tag.
    • This step will also create the necessary build cfg files.
    • If there are changes in both larsoft and larsoftobj, double check for consistency.
      cd $MRB_BUILDDIR/larsoft
      diff releaseDB ../larsoftobj/bundle | grep diff
      diff releaseDB ../larsoftobj/bundle | grep Only
      
    • If all is good, copy the cfg scripts to SciSoft
      cd $MRB_BUILDDIR/larsoft/releaseDB
      copyToSciSoft lar*
      
  • Now tag
    • tagPatchRel <existing branch> <new master tag>
    • tagPatchRel will commit ups/product_deps with an appropriate message, make the tag, and push both tag and branch to origin.
    • Use "dogit status" to doublecheck.
    • IMPORTANT: patch release tags are never merged with the master branch

Official build

  • build on Jenkins
  • Use special patch release build jobs.
    • This avoids possible conflict with normal release builds.
  • use copyFromJenkins to download the tarballs and release manifests from Jenkins
  • use copyToSciSoft to upload the tarballs and release manifests to SciSoft
    • This step requires extra privileges.

Release Notes and Cross Package Tag

  • Once the build is complete for ALL variants (SLF and macOS), make the cross package tag.
    • We wait because sometimes a problem is not found until building and we need to move a tag.
    • This step requires redmine manager persmission for larsoft
      cp-lar-tag <larsoft tag> <larsoftobj tag>
      
      • It would be nice to fix cp-lar-tag so it deduces the larsoftobj tag
  • Make the release notes
    • make-release-notes <working_directory> <tag> <previous tag>
      • This script will create a subdirectory named tag under the working_directory
    • cd into the newly created tag directory
    • cat ReleaseNotes-vaa_bb_cc_dd
      • The first line of this output is the entry in the release list
      • The remainder of the output is used to create the release note itself.
        • copy and edit the top part if appropriate.
        • Do not edit anything under the "Change List" section.
        • It is important to use cat so that there are no unexpected line breaks.

Upload to cvmfs

  • This step requires login privileges.
  • ssh
  • The README provides an overview of the necessary steps.
    • start a transaction
    • install using installBundle.sh
    • publish
  • installBundle.sh larsoft vaa_bb_cc_zz sNN-eY
    • Note that if you call installBundle.sh with sNN-e17, it will install both e17 and c2 variants.
    • The script will attempt to install the release for all supported platforms and clean up after itself.
    • If some expected distributions are not available, then manifests will be left in the .working subdirectory. Check and remove them if necessary.