Project

General

Profile

Python-based EDM analysis

Python is awesome! You can start using it for your g-2 analysis on your own computer/laptop in two ways:
1) Writing Python-based ROOT macros (pyROOT), or interact with ROOT files directly in a JupyerLab environment (JupyROOT)
2) Extract data from ROOT Trees into Panda/Numpy format, and plot, fit, etc. using Python tools directly (e.g. scipy-optimize).

1) JupyROOT/pyROOT

To install JupyterLab on your laptop follow the instructions here: https://jupyterlab.readthedocs.io/en/stable/ Make sure to go via the pip/pip3 installation route (not anaconda!).

Many tutorials to get started with Jupyter are also available (e.g. https://www.dataquest.io/blog/jupyter-notebook-tutorial/)

Finally, test that you can import ROOT into your JupyteLab notebook (e.g. https://root.cern.ch/notebooks/HowTos/HowTo_ROOT-Notebooks.html). If you have ROOT installed via homebrew, you might need to import the path to ROOT explicitly i.e. (in the very first cell)

import sys
# add brew ROOT (Mac) to the system path
sys.path.append("/usr/local/Cellar/root/6.18.04/lib/root")
import ROOT as r

Alternatively, add "export JUPYROOT=/usr/local/Cellar/root/6.18.04/lib/root" to your .bash_profile and
import sys
# add brew ROOT (Mac) to the system path
sys.path.append(os.environ["JUPYROOT"]) 
import ROOT as r

Then you only need to change the path to ROOT in a single place, if you decide to update ROOT on your laptop.

Blinding

To get started you need to set-up the official blinding tools on your laptop here: https://cdcvs.fnal.gov/redmine/projects/gm2analyses/wiki/Library_installation
Then go to the bottom of that page, and follow "Using the code in Python3". There are already examples provided for a 5 parameter fit by Kim!

Here is one for the tracker fit for an EDM-based analysis (*more on EDM blinding in section 2 below)
https://github.com/glukicov/EDMTracking/blob/master/JupyterNB/SimpleBlindedFit.ipynb*

That's it! The rest is the same as using a ROOT C macro, but being in a Jupyer environment one can execute things interactively and use native Python plotting tools.

2) Pure Python analysis (ROOT and art-independent)

James has kindly provided the Trees for all the track and vertex variables in Run-1: /gm2/data/g2be/Production/Trees/Run1 . One can easily skim the Tress and extract Quality Tracks (QT) and Quality Vertices (QV) which are ~15% of all tracks. Example macro is here: /gm2/data/users/glukicov/scripts/ApplyTQ.C (see instructions inside the macro).

This quality-only data (Trees) is already available for the four Run-1 datasets (60h, 9D, HighKick, EndGame): /gm2/data/users/glukicov/Run1_QualityTrees
Each file contains two Trees: QualityTracks and QualityVertices. The QualityVertices cut is inclusive of QualityTracks, that is, all tracks in QualityVertices Tree belong to quality vertices.

For each of the datasets, there are two folder versions:
1) For each run in 60h/9D/HighKick/EndGame : 60/106/262/86 files [~80MB each with ~0.5M QT and ~0.3M QV]
2) Entire datasets split into 5/9/22/7 files [~800MB each/~6M QT and ~4M QV]
+ Entire 60h dataset in a single file (!) [4.1GB/35M QT and 18M QV]

One can load the entire 60h dataset into Python pandas to work with 35M tracks! It takes only two minutes to load the entire 60h dataset on a laptop. However, once in the memory, the subsequent operations take seconds.

See here https://github.com/glukicov/EDMTracking/blob/master/JupyterNB/PYReadTree.ipynb for a quick demo on how to load, make cuts, and plot data.

One need only two extra packages to read ROOT Trees into Python (pandas) data frames: 1) root_numpy and 2) root_pandas. Both can be easily installed with pip:

python3 -m pip install --user root_numpy

python3 -m pip install --user root_pandas

To get the EDM-style blinding to work in Python with the blinding libraries:

A. Add changes to your local files from this commit: https://github.com/glukicov/EDMTracking/commit/8c1ecff08ceb905aa449d6e18a858b5ec73980b7

B. Re-compile (in the Blinders.cc directroy):

g++ -I rlib/include -I /usr/local/opt/openssl/include Blinders.cc -std=c++11 -Wall -Wextra -Werror -pedantic-errors -fpic -c

C. Now re-create a shared object library that can be used with both ROOT and Python:

g++ -shared -o libBlinders.so Blinders.o rlib/src/Random.o -L/usr/local/opt/openssl/lib -lssl -lcrypto

D. Now construct your blinder with 4 input arguments as follows (e.g.):

edmBlinded = Blinders(FitType.Omega_a, blinding_string, boxWidth, gausWidth)

E. Test that EDM-blinding works with

python3 testBlindingPy3.py