History

Reproducing the nue Second Analysis results¶

Set up the appropriate release¶

Setup the second analysis branch

setup_nova -b maxopt -r R16-07-15-secondana.a
newrel -t R16-07-15-secondana.a mytestrel
cd mytestrel
srt_setup -a
addpkg_svn -b CAFAna R16-07-15-secondana-br

Producing the prediction objects¶

The main object containing the nue prediction is PredictionSystNueSecondAna. It requires, among other things, a set of files which includes nominal and systematically shifted predictions, located in predDir. By default, this directory is $NOVA_ANA/nu_e_ana/SecondAna/Predictions/CombinedPeriods/.

If you need to recreate these files, but no interest in looking under the hood, just run

python $SRT_PUBLIC_CONTEXT/CAFAna/nue/SecondAna/make_systematics_all.py -p

and wait ~ 1 day. Move the 157 output files to a folder of your choice. To learn more, keep reading.

Systematic prediction files¶

Systematics for SA were run shift by shift in order to minimize memory issues at the BENDecomp step. make_systematics_all.py script runs all the samples in series in the following way:

Make predictions for 2 sets of periods/epochs (calls make_nueFDprediction.C). The analysis used 2 sets: 1+2+3b, and 3c.
Make combined predictions (calls merge_predictions.C)
Make plots, tables and systs templates (calls plot_nueFDprediction_systematics.C)

Usage (python make_systematics_all.py --help)

python make_systematics_all.py -n --nsets <number of samples to process>
                               -f --first <sample to start at>
                               -m --skipmake      -a --skiphadd
                               -c --skipcombine   -p --skipplot

List of samples: (order defined in make_systematics_all.py, datasets defined in systDatasets.h). Note that the sample number required for --first above would be 0-19 instead of 1-20

Nominal
Nominal - No Tau
Calibration - Functional shift X
Calibration - Functional shift Y
Calibration - Scale shift positive
Calibration - Scale shift negative
Nominal - No rock
Birks C
Nominal - Genie Reweight
Genie - MaCCQEshape
Genie - NormCCQE
Genie - MaCOH
Genie - CCQEPauliSupViaKF
Genie - MaRES
Genie - MvRES
Genie - allpi
MEC Scale
RPACCQE
Beam - NA49
Beam - All Transport

The details of the Beam and Genie systematics evaluated is in systVariations.h. A few others that were not used in the final analysis are defined in the same file; they can be added to the list in make_systematics_all.py if needed.

Also note that there are 4 Nominal predictions, to be matched with the different samples according to the details of the simulation available. Calibration samples need to be compared with a NoTau nominal; BirksC with NoRock; Genie with Genie. This information is saved in the compareto option in make_systematics_all.py and flags in ana::DummyNueSyst

Examples¶

Create only the nominal PredictionCombinePeriods:
```
python make_systematics_all.py -n1 -a -p
```

Create the prediction files split by period in parallel, then combine, then hadd

Run e.g. in different machines. ETA: ~6 hrs

python make_systematics_all.py -f0  -n5 -c -a -p
python make_systematics_all.py -f5  -n5 -c -a -p
python make_systematics_all.py -f10 -n5 -c -a -p
python make_systematics_all.py -f15 -n5 -c -a -p

Combine by periods. ETA: 30 min

python make_systematics_all.py -m -a -p

Hadd different sigmas in a single file. ETA: 30 min
```
python make_systematics_all.py -m -c -p
```

Different combination of periods
Everything is designed to be split in 2 sets, with a combination of periods/epochs from {p1,p2,p3b,p3c} (default periods = ["p1p2p3b","p3c"] in make_systematics_all.py). If you need two different sets, just modify that line (e.g. periods = ["p1p2","p3bp3c"]). For different combinations, you'll need to call make_ macro and create your own PredictionCombinePeriods elsewhere. For example, the lines below will create 3 predictions: p1, p2 and p3.
```
cafe -bq make_nueFDprediction.C Nominal Nominal Nominal NueSA_prediction_Nominal_p1.root true 4 true &
cafe -bq make_nueFDprediction.C Nominal Nominal Nominal NueSA_prediction_Nominal_p2.root true 4 true &
cafe -bq make_nueFDprediction.C Nominal Nominal Nominal NueSA_prediction_Nominal_p3bp3c.root true 4 true &
```

Normalization systematics¶

In addition to the 16 systematics from the samples above, the analysis used 3 normalization systematics: Sig+Bkg; Sig; Bkg. These are set in PredictionSystNueSecondAna::AddNormSysts.

Systematic plots, tables and templates¶

Producing the FD cosmic and data spectra¶

sa_box_opening_macro.C makes in-time and out-of-time spectra for box opening. ETA 5 min

cafe -bq sa_box_opening_macro.C

The output file, NueSA_numi_data.root, contains a series of spectra named spec_{lid,lem,cvn}_{(pid)2d,nueE,time,slctime}_{numi,cosmic}. Of special interest for the next steps in the analysis are spec_cvn_cvn2d_numi and spec_cvn_cvn2d_cosmic. This macro will also produce 6 text files (3 pids, numi or cosmic) with the values of &kRun, &kSubrun, &kEvt, &kSlice, &kSliceMeanTime, &kLID, &kLEM, &kCVNe, &kNueEnergy for the selected events.

You can find copies of these outputs in /nova/ana/nu_e_ana/SecondAna.

Event list and event displays¶

(Copied from commit message. TODO check, run, copy output to nu_e_ana (?))

make_sa_event_list macro makes three lists of events that pass all nue selection cuts, one for each PID.

we would supply that as an input to the make_artfile_from_eventlist.sh which will create an art file for event display scanning (needs a PID dataset). It will also run sa_box_opening_decaf macro on the corresponding caf files, which will contain only events that pass any of the PIDs in the beam spill window.

Reproducing Data/MC comparisons¶

Near Detector Data/MC¶

Far Detector Data/MC¶

Reproducing the fit results¶

There are at least 3 distinct versions of the fit results:

Nue only: using the nue data and prediction, and the PDG 2015 dmsq and reactor constraint

Numu constraint: as presented in Neutrino 2016, in addition to the nue data and the reactor constraint, we use the numu result in the form of 2D chisq surfaces in ssth23 and dmsq32. This is not a joint fit!

Nue+Numu joint fit: as prepared for the nue SA paper, we use both nue and numu data and predictions, plus the reactor constraint, fitting simultaneously. Correlations among systematic uncertainties ARE included.

These will reproduce the figures in the paper:

cafe -bq joint_fit_make_experiments.C CVN combo  #saves experiments once for faster loading

cafe -bq joint_fit_contours.C CVN combo true true joint true 
cafe -bq joint_fit_contours.C CVN combo false true joint true  #joint fit, syst, FC 

cafe -bq joint_fit_contours.C CVN combo true true nueOnly true 
cafe -bq joint_fit_contours.C CVN combo false true nueOnly true  #nue only, syst, FC 

cafe -bq joint_fit_slices.C CVN combo true true false true 
cafe -bq joint_fit_slices.C CVN combo false true false true true true #joint slices, octant constraint, FC

If you want to make other blessed plots or want more info, keep reading.

Nue only (+ numu constraint) contours¶

The macro contours.C prints the S plot (th13) or wave plot (th23).

Usage:
```
cafe -bq contours.C PID decomp numuConstr fccorr th13plot
```
Options for PID: CVN, LID, LEM. Options for decomp: combo, prop.
For the wave plots, only the CVN combo versions will have filled areas. Other PID/decomp combinations will print lines only.
numuConstr toggles between the PDG dmsq constraint or the NOvA numu ssth23 x dmsq constraint.
fccorr=true will use the FC Surfaces available.

S plot: nue + PDG dmsq constraint
dCP vs ss2th13 contour. Prints 1 file, NH top, IH bottom.
```
cafe -bq contours.C CVN combo false false true
```

Wave plot: nue + reactor + PDG dmsq constraint
ssth23 vs dCP contour. Prints 2 files: NH, IH
```
cafe -bq contours.C CVN combo 
```
Wave plot: nue + reactor + PDG dmsq constraint + Feldman-Cousins corrections
ssth23 vs dCP contour. Feldman-Cousins surfaces from /nova/ana/nu_e_ana/SecondAna/FC/ForPaper/FCWave_?H.root; using 1,2,3 sigma _smooth versions.
Prints 2 files: NH, IH.
```
cafe -bq contours.C CVN combo false true
```
Wave plot: nue + reactor + numu constraint
ssth23 vs dCP contour. Numu 2D chisq surfaces provided by the numu group via email, renamed and placed in /nova/ana/nu_e_ana/SecondAna/SAoutputs_Numu.root. FC corrections are not available.
Prints 2 files: NH, IH.
```
cafe -bq contours.C CVN combo true
```

Joint fit contours¶

First get a file with the nue and numu experiments:
```
cafe -bq joint_fit_make_experiments.C CVN combo
```

Run the contours + plot
```
cafe -bq joint_fit_contours(PIDStr, decomp, bool createFile, bool corrSysts, TString options="joint", bool fccorr, bool dmsqSurf)
```
PIDStr and decomp will look for the experiment created above. createFile = true runs the fit, =false makes the plots.
Set corrSysts=false for stats only
options are joint, nueOnly, numuOnly
Set fccorr=true to correct the significances when FC are available
dmsqSurf=true adds the dmsq x th23 (numu-style) plot to the list; takes longer to run

Wave plot: nue + numu + reactor constraint, stats only, no FC

cafe -bq joint_fit_contours.C CVN combo true
cafe -bq joint_fit_contours.C CVN combo false

Wave plot: nue + numu + reactor constraint, with correlated systematics

cafe -bq joint_fit_contours.C CVN combo true  true joint #create file
cafe -bq joint_fit_contours.C CVN combo false true joint false # plot no FC
cafe -bq joint_fit_contours.C CVN combo false true joint true  # plot with FC

Note: to get the 0.2-0.75 axis range, set zoomIn=true in joint_fit_contours.C

Same macro can also run nue or numu only (e.g. with systematics, no FC). Some functionality is redundant with contours.C :/

cafe -bq joint_fit_contours.C CVN combo true  true nueOnly 
cafe -bq joint_fit_contours.C CVN combo false true nueOnly 

cafe -bq joint_fit_contours.C CVN combo true  true numuOnly 
cafe -bq joint_fit_contours.C CVN combo false true numuOnly

Joint fit slices¶

Requires the same experiment file as the contours

Usage
```
cafe -bq joint_fit_slices( PID, decomp = "prop", bool createFile, bool corrSysts, bool th23Fixed, bool octantConstr,
                       bool fccorr, bool fourierFit)
```
createFile = true runs the fit, =false makes the plots
Set corrSysts = false for stats only.
th23Fixed = true uses the numu best fit results for th23, does not fit th23
octantConstr = true seeds from the numu best fit; fits th23 but stays in the same octant (no jump).
If both are false, it fits th23 normally.
fourierFit will soften the FC-corrected IH curve, esp. important in region where significance reaches 4 sigma

Slice for the paper, nue+numu joint fit with octant constraint, correlated systematics, and FC corrections

cafe -bq joint_fit_slices.C CVN combo false true false true 
cafe -bq joint_fit_slices.C CVN combo false true false true true true #joint slices, octant constraint, FC

FC¶

We ran several million FC pseudo-experiments to properly translate a deltachisquare into a significance. There are macros for running a job of a user-specified number of pseudo-experiments.
Each run only produces points in a user-specified bin, so that we may produce more FC points in certain regions where our analysis is more sensitive.

Running a FC job on a gpvm¶

Set up the secondana.a release

setup_nova -b maxopt -r R16-07-15-secondana.a

Navigate to a test release with CAFAna checked out. Navigate to the nue SecondAna folder

cd CAFAna/nue/SecondAna

Contour FC¶

Run

cafe -bq make_fc_surf.C <int NPts> <int bin> <bool NH> <int iteration>

where NPts specifies the number of FC experiments to generate with this job, bin is the analysis bin that you want to correct, NH is a bool for deciding which true hierarchy to throw experiments in, and iteration is a bookkeeping parameter, insuring multiple FC output files have unique, human-readable names.

Slice FC¶

Run

cafe -bq make_fc_slice.C <int NPts> <double sinsqth23> <int bin> <bool NH> <int iteration>

where NPts specifies the number of FC experiments to generate with this job, sinsqth23 specifies the value of th23 to use when throwing your pseudo-experiments, bin is the analysis bin that you want to correct, NH is a bool for deciding which true hierarchy to throw experiments in, and iteration is a bookkeeping parameter, insuring multiple FC output files have unique, human-readable names.

Running many FC experiments¶

Submit many instances of make_fc_{surf,slice}.C on a grid node. It will take a few kiloCPU-days on the OSG to generate a large enough sample of FC experiments to be useful. Since that's hard, I'll point people to the files we used, all files (appropriately named) in

/nova/ana/nu_e_ana/SecondAna/FC/ForPaper/

If you do want to generate an independent set of FC points, you will need to ifdh cp a few files over in your grid submission script:

/pnfs/nova/persistent/analysis/nue/nu_e_ana/SecondAna/NueSA_numi_data.root // Numu SA data file, used for cosmic spectrum
/pnfs/nova/persistent/analysis/nue/nu_e_ana/SecondAna/Predictions/NueSA_prediction_reduced_CVN_Combo.root // File with nue prediction
/pnfs/nova/users/dpershey/InputForFC/pred2_numu_sa_123b_3c.root // File with numu prediction

Then wait several days, and you'll have FC pseudo-experiments.

Files

Project

General

Profile

NOvA » NOvA-ART

Wiki