How to Build Professor on geniegpvm01 » History » Version 22
How to Build Professor on geniegpvm01.fnal.gov¶
This notes describe, as of May 2019, the procedure of building Professor tuning toolkit on the GENIE dedicated virtual machine at FNAL (geniegpvm01.fnal.gov).
If you have specific questions, please try to contact Julia Yarba: email@example.com
Please bear in mind that it uses 3rd party packages that are centrally installed in the CVMFS disk space in a form of UPS products..
NOTE: This procedure has been tried on other resources (e.g. Wilson cluster at FNAL, with gvv7.3.0 or gcc v6.4.0).
WARNING: These instructions may or may not work for building Professor2.3.0.
As of May 2019, everything worked well when building Professor2.3.0 (w/Eigen-3.3.5) on the FNAL Wilson cluster
However, an initial attempt to build Professor2.3.0 on geniegpvm01 has failed - work in progress.
STEP-1: Login to geniegpvm01.fnal.gov¶
STEP-2: Setup necessary externals, as UPS products¶
setup root v6_12_06a -q e17:prof
NOTE: Your compiler will be gcc v7_3_0
NOTE: Package pyDOE (needed later) is NOT a part of standard numpy distro !!!
STEP-3: Make sure matplotlib is installed; if not one can install using pip:¶
pip install matplotlib -t /path/to/your/local/matplotlib export PYTHONPATH=/path/to/your/local/matplotlib:$PYTHONPATH
NOTE: One can check under interactive python if matplotlib is correctly installed:
python >>> import matplotlib >>> exit() If matplotlib is still missing, one will get an error message (before exiting, that is).
STEP-4: Make sure iMinuit is installed¶
Trying to import it under interactive python; if not, install it using pip:
pip install iMinuit -t /path/to/you/local/iMinuit export PYTHONPATH=/path/to/your/local/iMinuit:$PYTHONPATH
Double check interactively if iMinuit is properly in your PYTHONPATH:
python >>> import iminuit >>> exit()
GENERAL REMARK: There's a slight chance that running pip will screw up the font in a terminal window...
STEP-5: Download Professor¶
NOTE-1: There's a more recent version 2.3.0; however, when tried 2.3.0.beta in May 2018, some technical issues were present.
Although if one wishes to repeat this exercise with 2.3.0, please do.
NOTE-2: According to Professor's development team, it may make sense to wait for Professor-2.4 series, as Professor-2.3 is "nothing special".
STEP-6: Download YODA-1.6.7¶
NOTE: The YODA1.7.0 "configure" script crashed on tev.fnal.gov on m4_ifblank.
It happened with gcc6.4.0 compiler (that is set together with e.g. art, etc.).
If trying to build e.g. in "native" tev.fnal.gov environment (gcc4-series, etc.), other issues are present.
However, on geniegpvm01 we're building with gcc v7_3_0 so it may make sense to re-check more recent versions.
STEP-7: 6. Download Eigen-3.3.4¶
See the following URL for details: http://eigen.tuxfamily.org/index.php?title=Main_Page
NOTE: tarball is called 3.3.4.tar.gz (yes, that's how it's called, no less)
STEP-8: Un-tar Eigen-3.3.4¶
NOTE: It un-tar's into "eigen-eigen-5a0156e40feb" (hahaha).
Maybe better to mv eigen-eigen-5a0156e40feb to eigen_3.3.4 ???
tar -xzf 3.3.4.tar.gz mv eigen-eigen-5a0156e40feb eigen-3.3.4
STEP-9: Build ("build" ?) Eigen (it may take a few minutes)¶
Take a look at the INSTALL notes in the directory - there're installation instructions there (the procedure uses cmake).
NOTE: The INSTALL notes don't explicitly say, but the build procedure requites cmake 2.8.5 or later (while on geniegpvm01 the default is cmake 2.6-patch4)
One may want to setup a modern version of cmake, e.g.
setup cmake v3_12_1
Note that apparently Eigen requires an "out-of-source" build, so one should first create a build directory for it, and do the actual build from there.
However, if one can still istall into eigen-3.3.4 (or whatever you'd call it), if one wishes (via -DCMAKE_INSTALL_PREFIX).
mkdir eigen-3.3.4-build cd eigen-3.3.4-build cmake -DCMAKE_INSTALL_PREFIX=/path/to/eigen-3.3.4 /path/to/eigen-3.3.4 ( e.g. cmake -DCMAKE_INSTALL_PREFIX=../eigen-3.3.4 ../eigen-3.3.4 ) make install
STEP-10: Un-tar and build Professor, add it to various paths¶
Instructions are available in Professor-2.2.2/README and online, via the following URL:
NOTE: Professor does NOT require out-of-source build
tar -xzf Professor-2.2.2.tar.gz cd Professor-2.2.2 CXXFLAGS="-I../eigen-3.3.4/include/eigen3 -O4" make all ( generic way: CXXFLAGS="-I/LOCATION_OF_EIGEN_HEADERS -O4" make all ) export PATH=/path/to/Professor-2.2.2/bin:$PATH export LD_LIBRARY_PATH=/path/to/Professor-2.2.2/lib:$LD_LIBRARY_PATH export PYTHONPATH=/path/to/Professor-2.2.2/lib/python2.7/site-packages:$PYTHONPATH
Example: assuming that one is in the Professor-2.2.2 top dir, one can do as follows:
export PATH=$PWD/bin:$PATH export LD_LIBRARY_PATH=$PWD/lib:$LD_LIBRARY_PATH export PYTHONPATH=$PWD/lib/python2.7/site-packages:$PYTHONPATH
- STEP-11: Give a try to Professor's most basic utility prof2-ncoeffs
The output should look like the following:
Polynomial order Minimum samples 0 1 1 4 2 10 3 20 4 35 5 56 6 84 7 120 8 165 9 220 10 286
STEP-12: Un-tar, configure, and build YOUA-1.6.7, with Root extension¶
Bear in mind that installation into the build directory isn't supported.
For this reason you'll need a separate install directory.
tar -xzf YODA-1.6.7.tar.gz mkdir YODA-1.6.7-install cd YODA-1.6.7 PYTHON=$PYTHONHOME/bin/python ./configure --prefix=$PWD/../YODA-1.6.7-install --enable-root make && make install
NOTE: If one tries to configure as stated in the instructions
./configure --prefix=$PWD/../YODA-1.6.7-install --enable-root
i.e. without leading PYTHON=$PYTHONHOME/bin/python, the configuration procedure will
eventually chock up on not being able to locate python.
So do exactly as recommended above (it was a verbal recommendation from Holger Sch.).
NOTE: If building on a multi-core node, you can do a parallel build. e.g.
make -j4 && make install
STEP-13: Extend paths to YODA¶
export LD_LIBRARY_PATH=/path-to/YODA-1.6.7-install/lib:$LD_LIBRARY_PATH export PATH=/path-to/YODA-1.6.7-install/bin:$PATH export PYTHONPATH=/path-to/YODA-1.6.7-install/lib/python2.7/site-packages:$PYTHONPATH
E.g. to export PYTHONPATH from from the YODA-1.6.7-install top dir, one would do
NOTE: Technically speaking NO YODA is needed to run prof2-ncoeffs, prof2-sample, prof2-ipol, or even prof2-tune.
However, prof2-tune will kind of complain of not finding YODA, thus not writing YODA nistos, but just "tunes" output.
In principle, YODA would be useful since it has yoda2root features, etc. (but fortunately it's not mandatory)
STEP-14: You may want to install pyDOE¶
If you plan to e.g. tuning with the --scan-n option, you may want to install pyDOE as well.
pip install pyDOE -t /path/to/you/local/pyDOE export PYTHONPATH=/path/to/your/local/pyDOE:$PYTHONPATH
Double check interactively if pyDOE is properly in your PYTHONPATH:
python >>> import pyDOE >>> exit()
STEP-15: Professor in 5 minutes¶
For those new to Professor and who wishes to see/learn its basics "in 5 minutes", there is a nice example ("Toy Professor") by Robert Hatcher:
The create_toy_data.py operates in YODA.
However, the Root-based script would look as follows:
#! /usr/bin/env python2.7 """ This is a script to create data for toy "problem" for professor Uses values in params.json (professor uses params.dat) """ import random import ROOT import json from numpy.random import poisson from sympy import symbols myfile = open("params.json","r"); myjson = json.load(myfile) print "myjson object: ",myjson params = myjson["params"] print "params object: ",params a = params["a"] b = params["b"] c = params["c"] x, y, z = symbols('x y z') func1 = 100*(400.-c*100.*pow(abs(x-5.0*b)/10.0,2+a)) print 'func1: ',func1 def eval3_1(xx,yy,zz): v = func1.subs(x,xx) print "func1: ",func1,'xx=',xx,' v=',v return v ROOT.TH1.SetDefaultSumw2() nbins=50 xmin=0.0 xmax=20.0 h1 = ROOT.TH1D( 'h1_foo1', 'toy data', nbins, xmin, xmax ) # for b in range(1,h1.GetNbinsX()+1): for b in range(h1.GetNbinsX()): xmid = h1.GetBinLowEdge(b+1) + 0.5*h1.GetBinWidth(b+1) n = eval3_1(xmid,0,0) if (n<=0): continue # mean for this bin ... now poisson nthrows = poisson(n) print b, n, nthrows for toss in range(nthrows): h1.Fill(xmid) ofile = ROOT.TFile( "hists.root", "RECREATE" ) h1.Write() ofile.Close() # end-of-script
STEP-16: Try a mini-example (from Geant4)¶
It's in the followng area:
So please cd to that area, and take a look.
This is a very minimal "pilot" example from Geant4 Fritiof model (FTF), and it's based on 25 points in the 3-parameter space.
There're 2 directories there, scan and data.
The data directory contains a Root file which histograms representing a group of experimental data (from IAEA/Ishibashi, ITEP771, and NA49).
In addition there're params.dat and params.json files that contain Geant4 default settings of the FTF parameters that we want to fit.
The scan directory contains 25 subdirectories, and each subdirectory contains a Root file with simulated histograms (observables) that correspond to the experimental data mentioned above, plus params files with the FTF parameters settings used in this particular simulation.
Try the following steps:
prof2-ipol scan - it'll create file ipol.dat (Eigen interpolation of the data) prof2-tune -d data ipol.dat - this is the actual fit NOTE: if one uses --scan-n NPoints option it actually greatly improves fit results, e.g. prof2-tune -d data ipol.dat --scan-n 25
The output is written to the "tunes" directory.