What Lynn does » History » Version 12

« Previous - Version 12/67 (diff) - Next » - Current version
Lynn Garren, 05/07/2019 10:56 AM

What Lynn does

Build "third party" ups products as needed for art and larsoft

  • We have a defined procedure that uses a number of scriptlets found in the ssibuildshims product.
  • Each product has a tiny redmine repository for the build and bootstrap scripts.
    • Scripts include,,, and usually ups/xxx.table, where xxx is the product name.
    • We sometimes need patch files.
    • Other scripts are occasionally included, such as for geant4.
    • We tag these repositories with the product version.
  • To ensure reproducible builds, we only build from tagged releases. In rare cases, we use a commit hash.
  • We require that the source code be captured in a source code tarball for reproducible builds.
  • We bootstrap and run local test builds for the supported compiler options. This is a rapid turnaround development step.
    • Whenever possible, we run unit tests and require that these tests pass.
  • Once everything looks good, the source code tarball is uploaded to SciSoft.
  • Also provide help to people attempting to make their own ups products.

Build art releases after they are tagged.

  • Kyle sends around an email to the SciSoft Team when a release is tagged.
  • Update information in build-framework/CMakeLists.txt
    • There are multiple branches for different product configurations
  • Run cmake to get the new build configuration files.
  • Check the new config files.
    • Always compare against the previous release.
  • If they are OK, use copyToSciSoft to install ONLY the appropriate build configuration files on SciSoft. There are html files for each release that also need to be installed.
    • canvas
    • art
    • gallery
    • critic
  • Start a Jenkins build
  • Sometimes there are problems with the build.
    • Investigate and fix.
    • A local test build is sometimes helpful, depending on the problem.
  • Once the build has completed successfully, download the tarballs locally and then upload them to SciSoft.
    • Work in an empty directory
    • copyFromJenkins -N -q c2 -q c7 -q e17 -q e19 critic-all
    • copyToSciSoft *

Update other products to use the appropriate release of art

  • artdaq_core
    • Owned by the artdaq group
    • This product is designed to build against several different releases of art, as reflected in the generated table file.
    • If the head of develop matches the latest tag, then use git flow to tag against the head of develop. Otherwise make a release branch against the latest tag and edit product deps.
    • We only edit ups/product_deps to make a new release.
  • ifdh_art
    • This is the ifdh service developed by Marc Mengel at the request of the art team.
    • ifdh_art can build against several different releases of art.
    • ifdh_art is updated when libwda or ifdhc is updated or if it needs to recognize a new release of art.
  • nutools
    • This product is owned by NOvA
    • However, larsoft uses it extensively and sometimes makes contributions to the code.
    • Any changes beyond those required by a new release of art must be approved.

Build mu distributions

  • The mu distribution is basically an art distribution with a few additional products.
  • The distribution is built when requested.
  • Found on a branch of build-framework

Build nutools releases and the nu or nulite distributions.

  • Both NOvA and LArSoft can drive the need for a new release of nutools.
    • NOvA uses the full nu distribution.
    • LArSoft now uses the nulite distribution which ignores packages that are NOvA specific.
  • Discussion will start soon on a plan to split nutools into modular components.
    • Presently nutools contains, a genie interface, a geant4 interface, a random number interface, and other bits. Updates are needed on different timescales and we would like to decouple them.

LArSoft release management

  • Advise and collect information for each weekly release.
  • Send notices that the release is about to start.
  • Make a test build with larsoft and all the experiment code.
    • From long experience, if there is more than one set of feature branches to merge, each set must be tested sequentially.
    • While this could be done via CI, it is currently more efficient to test on woof. Especially since overlapping feature branches must sometimes be tested.
    • We do encourage anyone submitting a feature branch to run the CI themselves.
  • We work on a temporary release branch.
  • Once the tests are complete and successful, update the release versions for those packages which will be changing and then tag.
  • The tag is merged with master, but not with develop.
    • Merging with develop before the tagged release is available on cvmfs causes all kinds of problems.
  • Build on Jenkins. Sometimes we find problems during this step. We move the tag if necessary when resolving problems.
  • Once the build is complete, we finalize the release.
    • pull the tarballs from Jenkins and upload them to SciSoft
    • install the release on cvmfs
    • For macOS building, we install the c2 build on /grid/fermiapp/products/larsoft
    • make a cross package tag
    • make release notes
    • final merge with develop
    • send an announcement
  • The procedure is described in detail at How_to_tag_and_build_a_LArSoft_vx_yy_zz_release
    • Although it is tempting to take shortcuts, the procedure is designed to allow the release manager to recover from various problems that may arise. Taking shortcuts removes those safeguards.