Building releases using Jenkins

This article explains how to build a release using the Jenkins build machine. The instructions include how to upload releases to the ups products server and install in the Fermilab ups products area /grid/fermiapp.

Scisoft scripts

The method described in this article makes use of several scripts that can be downloaded from, namely:

The above scripts are not part of any ups product, but rather must be downloaded directly from scisoft and made executable. If you use these scripts frequently, you may want to keep copies in a personal script location that is on your execution path. The group account uboone, which owns the microboone-maintained ups products in /grid/fermiapp, has copies of these scripts on its execution path when you log in.

Tag packages

Before you start, tag any packages that will be built for release on the master branch. In the case of building microboone-specific packages, that means tagging uboonecode, ubutil, and uboonedata. Instructions on tagging branches for release can be found in this page.

Jenkins authentication

Building is controlled via the web interface in Authenticate yourself by installing cilogon certificate in your web browser (you can obtain a cilogin certificate from the website Open your web browser and navigate to the Jenkins home page If you are properly authenticated, your username should appear in the upper right corner of the web page.

Jenkins projects, OSes, build types, and qualifiers.

Jenkins builds are organized by a multidimensional matrix consisting of dimensions such as those summarized in the following table.

Project OS flavor Build Type Qualifiers
uboone-release-build slf6 debug e14
uboone-release-build slf6 prof e14
uboone-release-build slf7 debug e14
uboone-release-build slf7 prof e14
uboone-release-build-mac d14 (aka mac-10.10 aka Yosemite) debug e14
uboone-release-build-mac d14 (aka mac-10.10 aka Yosemite) prof e14
uboone-release-build-mac d15 (aka mac-10.11 aka El Capitan) debug e14
uboone-release-build-mac d15 (aka mac-10.11 aka El Capitan) prof e14
uboone-release-build-mac d16 (aka mac-10.12 aka Sierra) debug e14
uboone-release-build-mac d16 (aka mac-10.12 aka Sierra) prof e14

Each row of the above table corresponds to a different binary flavor.

Build release.

Here are step-by-step for controlling the build via the Jenkins web interface.

  1. Open the Jenkins home page (Jenkins dashboard) Make sure you are authenticated (user name appears in upper right).
  2. Click on tab "uboone".
  3. Click on a project "uboone-release-build" or "uboone-release-build-mac."
  4. Click on "Configure." Scroll down to the parameters section and make any desired updates to default values (package versions and qualifiers).
  5. Click on "Build with Parameters." You will be given another chance to update parameters. If you edited the default values of the parameters in the previous step, the listed values should already be correct. Click "Build."
  6. Wait for build to complete. Progress of the build can be monitored using the progress bar in the Build History box. While the build is progressing, status icons on the project page will blink. When build is complete, status icons will stop blinking. Make sure that each binary flavor built successfully (status icon displayed as green ball).

Upload release to scisoft.

Do this step after a successful Jenkins build. Make sure you have access to scisoft scripts copyFromJenkins and copyToSciSoft on your execution path.

  1. Go to an empty directory on any machine where you have kerberized scp access to (uboonegpvmXX login machines are fine). You will assemble in this directory all files that will be uploaded to
  2. Fetch results of Jenkins build using copyFromJenkins. This script is normally invoked with a single argument specifying the Jenkins project name.
    copyFromJenkins uboone-release-build

    copyFromJenkins uboone-release-build-mac

    This command will fetch Jenkins build "artifacts," including tarballs (*.tar.bz2 or *.tar.gz), manifests (*.txt), and html files. There should be one html file, one manifest for each binary flavor, and one tarball for each binary flavor of each ups product.
  3. Upload all files to scisoft using copyToSciSoft. Specify all files that you want to upload on the command line, which should be all of the files your current directory.
    copyToSciSoft *

Script copyToSciSoft decides where to copy files based on the file name and file type. The files returned by Jenkins (tarballs, manifests, and release html file) all follow naming conventions that are understood by copyToSciSoft@.

CopyToSciSoft is written such that it will not replace an existing file with the same name. If you make a mistake and need to replace an exiting file on, ssh in to and cd to /nasroot/SciSoft. From there you can make arbitrary edits of files being served by, including deleting them.

Install release in /grid/fermiapp

Log in to the uboone account on any uboonegpvmXX machine. You will need acces to scisoft script pullProducts. This script should already be on your execution path when you log in as uboone.

You need one invocation of pullProducts for each binary flavor that you want to install in /grid/fermiapp (total six invocations, depending on the number of binary flavors you built).

pullProducts /grid/fermiapp/products/uboone slf6 uboone-04.03.00 e7 debug
pullProducts /grid/fermiapp/products/uboone slf6 uboone-04.03.00 e7 prof
pullProducts /grid/fermiapp/products/uboone d13 uboone-04.03.00 e7 debug
pullProducts /grid/fermiapp/products/uboone d13 uboone-04.03.00 e7 prof
pullProducts /grid/fermiapp/products/uboone d14 uboone-04.03.00 e7 debug
pullProducts /grid/fermiapp/products/uboone d14 uboone-04.03.00 e7 prof

Note that the name of the distribution bundle is specified in "dot version" format as uboone-xx.yy.xx.

All supported binary flavors can be installed using script, which is available when you log in as uboone. This script is invoked with one argument, which is the name of the scisoft distribution bundle. uboone-04.03.00

Install release in CVMFS

In general, the contents of /grid/fermiapp/products/uboone are mirrored to the oasis server after /grid/fermiapp is updated. This does not happen automtically, but the process must be initiated when someone decides to do it. Refer to this article for instructions.