Project

General

Profile

Building GENIE on your own

Really? Must you?

This should be a prescription that will work for unpriv'd users. It goes something along the lines of:

Personal UPS area

Pick a location where your own UPS will be it must be on a "app" disk (e.g. /nova/app). You'll need to refer to this again for setting things up in new sessions after it is built ... so remember where you parked

   export MYUPSDIR=/<expt>/app/users/$USER/altups   # where <expt> is "nova" or "uboone" or whatever

Initializing your personal UPS

The ALTUPSSRC is where you can get some stuff that make this easier. You'll only need to know this location during the initial setup of your UPS area

   ALTUPSSRC=/nusoft/app/alt

Create your UPS directory, and populate it with the magic files

   mkdir -p $MYUPSDIR
   cd $MYUPSDIR
   cp -vax $ALTUPSSRC/.upsfiles $MYUPSDIR 
   cp -vax $ALTUPSSRC/.updfiles $MYUPSDIR 

Initializing GENIE product in your personal UPS

Create the genie product area

   mkdir genie
   cd genie

Here are some helper scripts you'll use for GENIE.

   ln -s /nusoft/app/alt/genie/make_genie_ups_skeleton $MYUPSDIR/genie
   ln -s /nusoft/app/alt/genie/make_genie_nusoft_build $MYUPSDIR/genie

You shouldn't have to make copies unless you plan on editing them. If you're editing them then they need revision and you should let rhatcher@fnal.gov know what and why. They are also available from https://github.com/nusense/altups_scripts

Setup the build environment

The easiest way to start is to setup an existing genie release with similar properties relative to what you're attempting to build. This is what will setup the compiler and auxiliary packages (ROOT, LHAPDF, etc) for you.
At a minimum you must have a UPS area defined that has the auxiliary products you desire installed.

   # bootstrap UPS
   source /nusoft/app/alt/setup
   # or perhaps ...
   source /grid/fermiapp/products/uboone/setup_uboone.sh

   # add your UPS installation area to the list
   export PRODUCTS=${MYUPSDIR}:${PRODUCTS}

   # get list of existing genie versions
   ups list -aK+ genie

   # setup one of many existing versions (this is an example ... pick appropriately)
   setup genie v3_00_06i -q debug:e19

Create a GENIE version in your UPS area

Create the UPS skeleton, and check out GENIE.

In this example:
  • my3_00_06alt = the UPS version name being created
  • e19:debug = the UPS qualifiers
  • HEAD = check out the master branch (script might not handle branches/tags correctly)

Don't reuse the official UPS version name here, lest confusion reign supreme. Pick something like my3_00_06 or such. Use something that describes what you have (i.e. not "test") to avoid it as it being too ambigious later ... the UPS dependencies will allow you to get those setup correctly, but if you've forgotten what this variant was for (or perhaps from whence you started) then you're on your own. The minimum necessary to get a version started in your UPS area (and check out the code from the repository) is:

   ./make_genie_ups_skeleton --show-config --checkout=yes  my3_00_06alt e19:debug HEAD

Using the --help flag will show you additional options. The 3 non-flag args are version qualifier and svn-tag.

Now it should be a UPS product

Now you should be able to "setup" the UPS product. it isn't built, but all the paths can be/are defined.

   setup genie my3_00_06alt -q e19:debug

Note, you'll need to know what qualifier (-q) as well as the version name that was assigned by the skeleton making process. If you forget you can get hints via:

   ups list -aK+ genie -z $MYUPSDIR
   # or if you know the version name you gave it
   ups list -aK+ genie my3_00_06alt -z $MYUPSDIR

You can see what settting up genie also sets up using:

   ups depend genie my3_00_06alt -q debug:e19

yields a dependency tree such as:
genie my3_00_06alt -f Linux64bit+3.10-2.17 -z <myupsarea> -q debug:e19
|__root v6_18_04d -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug:e19
|  |__numpy v1_15_4b -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e19:p372:prof
|  |  |__python v3_7_2 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products
|  |  |  |__sqlite v3_26_00_00 -f Linux64bit+3.10-2.17 -z /cvmfs/dune.opensciencegrid.org/products/dune
|  |  |__lapack v3_8_0c -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e19:prof
|  |     |__gcc v8_2_0 -f Linux64bit+3.10-2.17 -z /cvmfs/dune.opensciencegrid.org/products/dune
|  |__fftw v3_3_8a -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug
|  |__gsl v2_5 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug
|  |__pythia v6_4_28p -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug:gcc820
|  |__postgresql v9_6_11b -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q p372
|  |__mysql_client v5_5_62 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q e19
|  |__xrootd v4_8_5c -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug:e19
|  |__tbb v2019_3 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug:e19
|  |__libxml2 v2_9_9 -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug
|__lhapdf v6_2_3d -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug:e19:p372
|__log4cpp v1_1_3b -f Linux64bit+3.10-2.17 -z /cvmfs/larsoft.opensciencegrid.org/products -q debug:e19

Actually building GENIE

Configure and build genie using another of the helper scripts

   # set it up if you haven't already, this defines $GENIE
   cd $GENIE  
   $MYUPSDIR/genie/genie_build_nusoft.sh -C -m

So that should get you a build. This build script just makes configuring and running make easier and consistent, there's nothing you couldn't do by hand.

Future sessions will require:

   source /nusoft/app/alt/setup  # or other UPS bootstrapping (e.g. uboone example above)

   export MYUPSDIR=/nova/app/users/$USER/altups
   export PRODUCTS=${MYUPSDIR}:${PRODUCTS}
   setup genie my3_00_06alt -q debug:e19

to get it setup again.

Future Rebuilds of GENIE and SRT

Now ... there are conflicts between GENIE's build system and the one that is present after setting up the NOvA/LArSoft SRT code. The above assumes you did not setup experiment specific code, especially anything to do with SRT, before you started. This approach is generally the simplest and least likely to cause confusion.

But if it can't be avoided, in any session where you're going to compile GENIE and have setup SRT, do the following:

   cd $GENIE
   srt_setup --unsetup
   gmake [genie-build-target]

You can also use the genie_build_nusoft.sh script to rebuild. The -c flag cleans out everything, -C configures GENIE, and -m it to do the build. Use them in appropriate combinations. If you don't do a clean (and generally you don't need to) then in principle only the things that changed get rebuilt; I've found that if you change only a header sometimes it misses that and doesn't rebuild something that it should. So pay attention to what actually is getting compiled and make into libraries.

The "srt_setup --unsetup" will make that session unable to build NOvA/LArSoft SRT code (and re-doing setup_nova or LArSoft equivalent is probably going to fail). So expect to do some multi-terminal session juggling. I usually have one window open for editing/compiling GENIE and a separate session for my test release work.