Project

General

Profile

Reproducing nus Ana01 results » History » Version 15

« Previous - Version 15/18 (diff) - Next » - Current version
Gareth Kafka, 08/19/2016 10:44 AM
Add systematic section


Reproducing nus Ana01 results

Work in progress

Set up the appropriate release

Setup the second analysis tag (that nue and numu also use - unfortunate naming scheme!). Note this is Ana01 for nus but SecondAna for nue/numu...

setup_nova -r R16-07-15-secondana.a -b maxopt

Producing ND spectra

There ND spectra results were all data/MC comparisons, either with or without systematics included.
The basic data/MC comparisons just take a Spectrum object for the data and CheatDecomp for the MC.
The scripts DataMCLoad.C and DataMCAna.C make all of these comparisons, for each level of the analysis cut flow.
As a result, they require the full CAFs and take many hours to complete. DataMCLoad.C should be run first;
this makes and saves the CAFAna objects to the file $NOVA_ANA/steriles/Ana01/DataMC.root and takes nearly all of the running time.
DataMCAna.C is run next and creates and saves the stylized plots to $NOVA_ANA/steriles/Ana01/DataMCAna.root.

The plots with systematics included are made in the scripts NDDataMCSystBandLoad.C and NDDataMCSystBandAna.C.
The basic idea is to create two vectors of Spectrum objects, one that shifts a given systematic up 1 sigma, the other down 1 sigma.
The systematic band adds in quadrature all of the differences from nominal (and looks for tricky corner cases like when both +/-1 sigma shifts shift a bin the same direction).
Before running NDDataMCSystBandLoad.C, there is a line in CAFAna/Systs/NusSysts.cxx that must be commented out (this requires a make CAFAna.all).
The specific line is mentioned in a comment at the top of the macro. The macro itself takes about an hour to run,
creating and saving the Spectrum objects to $NOVA_ANA/steriles/Ana01/NDDataMCSystBand.root.
NDDataMCSystBandAna.C is run next and creates and saves the stylized plots to $NOVA_ANA/steriles/Ana01/NDDataMCSystBandAna.root.
Don't forget to recomment the line in CAFAna/Systs/NusSysts.cxx, and rebuild CAFAna!

Producing the FD Prediction objects

The NC prediction object is PredictionSterile, compatible with sterile neutrino oscillations.
It requires one to decompose the ND spectra. We use ProportionalDecomp to do that making a separate one for numu components that applies the kNumuND cut. Otherwise we apply the kNusND cut.
One then needs to extrapolate using ModularExtrapSterile and in particular the member NCDisappearance to extrapolate for the NC analysis. This is all done by period; period 1, period 2 and epoch 3b as one prediction and epoch3c as a second. Subsequently the two separate PredictionSterile objects are combined (scaled appropriately by POT) into a PredictionCombinePeriods object.
The prediction objects are saved in the file $NOVA_ANA/steriles/Ana01/nus_ana01_prediction.root.

In order to recreate this file just run:

cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/BlessedPlotsLoadByPeriod.C <output_filename>.root

and wait a few hours (specify your <output_filename> with the extension .root as indicated).

If you want to make the Prediction object with systematics included then that differs slightly by using PredictionInterp and saving the individual predictions.
These predictions are what we used for the final systematics-included contours.

To create that prediction run:

cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/MakeNusPredictionSystsAna01.C <output_filename>.root

The prediction objects for the final nus results are saved in the file $NOVA_ANA/steriles/Ana01/nus_ana01_prediction.root.
This contains the predictions for the two periods, nus_pred_p1p2e3b and nus_pred_e3c.

Producing the FD cosmic and data spectra

The FD cosmic and data spectra are just Spectrum objects. To make the cosmic prediction, use either the blind NuMI dataset,
fnameblind_concat, found in NuSLoadMacroDefs.h, or the unblind NuMI dataset, fnamefardata_unblind, found in Ana01/Ana01Data.h.
Apply the cut kInTimingSideband with any other cuts (i.e., kNusFD) to only select out of time data (still necessary even with the blind dataset!),
and apply the weight kTimingSidebandWeight to automatically scale the timing sideband to the in spill livetime.

To make the in time data prediction, apply the cut kInBeamSpill with any other cuts to only select in time data. No weight is needed.

nus_ana01_box_opening_macro.C makes in-time and out-of-time spectra for box opening. ETA 5 min

cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/nus_ana01_box_opening.C 

The output file, fout_nus_ana01_box_opening_restricted.root, contains a series of spectra named spec_nus_{cvn,time,slctime,cale}_{numi,cosmic}. Of special interest for the next steps in the analysis are spec_nus_cale_numi and spec_nus_cale_cosmic. This macro will also produce 2 text files (numi or cosmic selected events) with the values of &kRun, &kSubrun, &kEvt, &kSlice, &kSliceMeanTime, &kCVNnc, &kCaloE for the selected events.

TODO: Caveat is that there's a tweak to MakeText{Event}ListFile functions in CAFAna/Core that I needed for a vector<std::string> wildcard input. Need to commit!

You can find copies of these outputs in /nova/ana/steriles/Ana01/evds/.

Evaluating Systematics

The first step in systematic evaluation is running the Systs<specific>Load/Ana.C macros in the Ana01 directory.
For each pair of macros, they are run with the following commands

cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/Ana01/Systs<specific>Load.C+
cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/Ana01/Systs<specific>Ana.C+

where <specific> is the specific systematic to evaluate.

There are separate scripts for each type. The loader macro for Birks, Calibration, and Noise require non standard datasets which we do not have concats for.
Consequently, they can take many hours to run.

The loader macro will create the necessary Decomposition (ND) and Prediction (FD) objects, and save the output to $NOVA_ANA/steriles/Ana01/Systematics/Systs<specific>.root.
The ana macro will create stylized plots and evaluate the percentage differences for the systematic in $NOVA_ANA/steriles/Ana01/Systematics/Systs<specific>Ana.root and $NOVA_ANA/steriles/Ana01/Systematics/Systs<specific>Ana.txt.

Once all of the macros are finished from above, the second step is to run NusSystMaker.h.
This script is run with the following command.

cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/NusSystMaker.C+

This script will take the output from the macros above, and encodes the systematics as ratio histograms in a single file, $NOVA_ANA/steriles/Ana01/Systematics/NusSystsAna01.root.
(NusSystsAna01.root is opened and read by the NusSystFromHist class in NusSysts.{h, cxx} to make the actual Syst objects.)

Finally, there are a few systematics that are treated as flat, i.e., with a ratio at a single value. These are hardcoded in Systs/NusSysts.h.
The values are separate for NC and CC events, and the values used are the overall error for these components from the Systs<specific>Ana.C results.
For Ana01, the MC stats, ND containment, and normalization were treated as flat. Make sure the values in Systs/NusSysts.h are correct.
Note that normalization is several errors in quadrature: 0.5% for POT counting, 0.7% for fiducial mass, 0.7% for intensity uncertainty,
2.9% for reconstruction efficiency between ND data/MC, the results of the noise systematic, and 3.7% cosmic overlay study results,
that totals 4.9% for both signal and background. See doc 15330 for references.

Producing Side Band distributions

Side band distributions are made in SideBandLoad.C and SideBandAna.C.
The loader macro makes FD Predictions and data/cosmic Spectra using different cuts than for the regular analysis.
The specific cuts and axes are made using the MakeSample function, and added to the sidebands dictionary.
The loader macro is run by the command

cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/SideBandLoad.C+ <filename>.root

where <filename>.root is the file that stores the CAFAna objects.
SideBandAna.C is run next and makes the stylized plots. Make sure the file path in this macro matches the path to <filename>.root above.
The macro is then run with the following command.
cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/Ana01/SideBandAna.C+

Producing variable distributions of FD data

All of the scripts in this section come in two parts, a Load.C script and an analyzer script that makes the stylized plots.
The Load scripts create and save the necessary Prediction and cosmic and data Spectrum objects, and are run with the command

cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/Ana01/<macro>Load.C+ <filename>.root

where <macro> is the base macro name to be run and <filename>.root is the file that holds the CAFAna objects.
The analyzer script is run with the command
cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/Ana01/<macro_ana>.C+

where <macro_ana> is the companion macro name to the Load.C counterpart.
While the analyzer scripts do not take an input root file as argument, there is a path to <filename> in these macros, so be sure that the path matches!

Ana01/AnaResultsLoad.C and Ana01/AnaResultsSpectra.C make FD data/MC distributions with the full cut flow.
Ana01/ExtendedAxesLoad.h and Ana01/ExtendedAxesAna.h make distributions of the same variables as N-1 plots.
That is, the CVN NC classifier distribution does not apply a cut on CVN, and similar for each other variable.
The distributions also show dashed lines and arrows denoting the regions of each distribution kept for the final results.

Plots that show the vertex distributions of data events inside the detector are made in Ana01/DataVtxDistributionLoad.C and Ana01/DataVtxDistributionLoad.C.
These only have to load data files, so the Load scripts takes a much shorter time of 15 minutes to run.

FD data/MC energy distributions with a systematic band are made in Ana01/FDDataMCSystBandLoad.C and Ana01/FDDataMCSystBandAna.C.
These require loading through the full set of concats so the Load script takes about an hour to run.

Producing the final 1D and 2D contours

Taking the FD cosmic and data spectra and the prediction file, the 1D/2D fits are produced by running:

cafe -bq $SRT_PUBLIC_CONTEXT/CAFAna/nus/PlotNusSensAna01.C <input_filename>.root

where <input_filename>.root is the file that holds the prediction objects.
The macro is already set up to read in the cosmic and numi data spectra.
It will produce an output file called nus_ana01_results_final.root with stats-only and systs 1D Slices for the sterile mixing angles as well as 2D contours.

Producing final 2D contour w/ MINOS comparison

Run the following:

cafe $SRT_PUBLIC_CONTEXT/CAFAna/nus/NusContoursPlusMINOS.C true

Passing value true includes the MINOS 2011 2D contour for t24 vs. t34.
This will then produce the 2D contour for the analysis plus the 2D sensitivity and compare to MINOS contour.

Event list and event displays

TODO