Building releases using Jenkins¶
This article explains how to build a release using the Jenkins build machine. The instructions include how to upload releases to the ups products server
scisoft.fnal.gov and install in the Fermilab ups products area
The method described in this article makes use of several scripts that can be downloaded from http://scisoft.fnal.gov/scisoft/bundles/tools, namely:
The above scripts are not part of any ups product, but rather must be downloaded directly from
scisoft and made executable. If you use these scripts frequently, you may want to keep copies in a personal script location that is on your execution path. The group account
uboone, which owns the microboone-maintained ups products in
/grid/fermiapp, has copies of these scripts on its execution path when you log in.
Before you start, tag any packages that will be built for release on the master branch. In the case of building microboone-specific packages, that means tagging
uboonedata. Instructions on tagging branches for release can be found in this page.
Building is controlled via the web interface in
buildmaster.fnal.gov. Authenticate yourself by installing cilogon certificate in your web browser (you can obtain a cilogin certificate from the website cilogin.org). Open your web browser and navigate to the Jenkins home page http://buildmaster.fnal.gov. If you are properly authenticated, your username should appear in the upper right corner of the web page.
Jenkins projects, OSes, build types, and qualifiers.¶
Jenkins builds are organized by a multidimensional matrix consisting of dimensions such as those summarized in the following table.
|Project||OS flavor||Build Type||Qualifiers|
|uboone-release-build-mac||d14 (aka mac-10.10 aka Yosemite)||debug||e14|
|uboone-release-build-mac||d14 (aka mac-10.10 aka Yosemite)||prof||e14|
|uboone-release-build-mac||d15 (aka mac-10.11 aka El Capitan)||debug||e14|
|uboone-release-build-mac||d15 (aka mac-10.11 aka El Capitan)||prof||e14|
|uboone-release-build-mac||d16 (aka mac-10.12 aka Sierra)||debug||e14|
|uboone-release-build-mac||d16 (aka mac-10.12 aka Sierra)||prof||e14|
Each row of the above table corresponds to a different binary flavor.
Here are step-by-step for controlling the build via the Jenkins web interface.
- Open the Jenkins home page (Jenkins dashboard) http://buildmaster.fnal.gov. Make sure you are authenticated (user name appears in upper right).
- Click on tab "
- Click on a project "
uboone-release-build" or "
- Click on "Configure." Scroll down to the parameters section and make any desired updates to default values (package versions and qualifiers).
- Click on "Build with Parameters." You will be given another chance to update parameters. If you edited the default values of the parameters in the previous step, the listed values should already be correct. Click "Build."
- Wait for build to complete. Progress of the build can be monitored using the progress bar in the Build History box. While the build is progressing, status icons on the project page will blink. When build is complete, status icons will stop blinking. Make sure that each binary flavor built successfully (status icon displayed as green ball).
Upload release to
- Go to an empty directory on any machine where you have kerberized
uboonegpvmXXlogin machines are fine). You will assemble in this directory all files that will be uploaded to
- Fetch results of Jenkins build using
copyFromJenkins. This script is normally invoked with a single argument specifying the Jenkins project name.
This command will fetch Jenkins build "artifacts," including tarballs (
*.tar.gz), manifests (
*.txt), and html files. There should be one html file, one manifest for each binary flavor, and one tarball for each binary flavor of each ups product.
- Upload all files to
copyToSciSoft. Specify all files that you want to upload on the command line, which should be all of the files your current directory.
copyToSciSoft decides where to copy files based on the file name and file type. The files returned by Jenkins (tarballs, manifests, and release html file) all follow naming conventions that are understood by copyToSciSoft@.
CopyToSciSoft is written such that it will not replace an existing file with the same name. If you make a mistake and need to replace an exiting file on
scisoft.fnal.gov, ssh in to
scisoftportal.fnal.gov and cd to
/nasroot/SciSoft. From there you can make arbitrary edits of files being served by
scisoft.fnal.gov, including deleting them.
Install release in /grid/fermiapp¶
Log in to the
uboone account on any
uboonegpvmXX machine. You will need acces to
pullProducts. This script should already be on your execution path when you log in as
You need one invocation of
pullProducts for each binary flavor that you want to install in /grid/fermiapp (total six invocations, depending on the number of binary flavors you built).
pullProducts /grid/fermiapp/products/uboone slf6 uboone-04.03.00 e7 debug pullProducts /grid/fermiapp/products/uboone slf6 uboone-04.03.00 e7 prof pullProducts /grid/fermiapp/products/uboone d13 uboone-04.03.00 e7 debug pullProducts /grid/fermiapp/products/uboone d13 uboone-04.03.00 e7 prof pullProducts /grid/fermiapp/products/uboone d14 uboone-04.03.00 e7 debug pullProducts /grid/fermiapp/products/uboone d14 uboone-04.03.00 e7 prof
Note that the name of the distribution bundle is specified in "dot version" format as
All supported binary flavors can be installed using script
install_from_scisoft.sh, which is available when you log in as
uboone. This script is invoked with one argument, which is the name of the
scisoft distribution bundle.
Install release in CVMFS¶
In general, the contents of /grid/fermiapp/products/uboone are mirrored to the oasis server after /grid/fermiapp is updated. This does not happen automtically, but the process must be initiated when someone decides to do it. Refer to this article for instructions.