Project

General

Profile

Fixing the "many vectors" problem

Description of the problem:

Many difficulties have arisen due to the fact that we nest vectors more than 2 layers deep in the CAFs (just search the root forums for something like "more than 2 vectors deep" and you'll find lots of people complaining about how it doesn't work.) My personal impression from root developers is that this is at best a low priority for them to fix (more like something they don't intend on fixing.)

A recent NOvA specific talk that summarizes this issue and the proposed solution is here: docDB 40077
Another talk giving the final steps of validation is here: https://nova-docdb.fnal.gov/cgi-bin/private/ShowDocument?docid=40558

Proposal of the fixes:

So we have decided to propose restructuring our CAFs to make this problem go away for us. Specifically, there are 2 unnecessary structures in the CAFs that we can remove that will minimize this problem:

1. There is a vector of 'elastic' vertices hanging off the 'vtx' branch. This vector ONLY ever has a size of zero or one. So we propose removing the vector of SRElastic objects hanging off of the SRVertexBranch and replacing it with a single SRElastic object. We will also add an IsValid flag to indicate if the single SRElastic object has been filled (to replace instances where sr->vtx.nelastic != 0 was checked before.)
2. There is a vector of SRBPFTracks hanging off of the prong branch that only ever has a maximum size of three. So we propose removing this vector and replacing it with 3 SRBpfTrack objects called muon, pion, and proton (contained in a new class to hold these three objects.) We will also add an IsValid flag to indicate if each SRBpfTrack object has been filled.

There will be a significant amount of CAFMaker and CAFAna fallout from these 2 changes, so step 3 will of course be to fix everything that breaks.

Please see docDB 40077 for more technical details on the implementation of this.

How to fix your code not found in CAFAna:

First, this all comes with the caveat that these fixes are guides to how to fix your code. There will inevitably be some user specific use cases that will require more thought as to how best to handle them. Please reach out if you think you are in one of these situations and need help interpreting how to make your fixes.

fixing the vertex vector

Since the vector of elastic vertices was always size 0 or 1 it has been removed from the CAF structure and replaced with a single elastic vertex object. The following are some examples of code you might need to change.

For situations where you have:

 vtx.elastic[0].  

you would want to replace this with

 vtx.elastic.

as there is no longer a vector of elastic vertices.

For situations where you were checking to see if the elastic vertex vector had a non zero size, such as in:

 
if( vtx.elastic.size() != 0){
  do...
}

or in:

 
if( vtx.elastic.nelastic != 0){
  do...
}

you will want to replace either of these with

 
if(vtx.elastic.IsValid == true){
  do....
}

The member nelastic has been removed entirely since we no longer have a vector and size() can no longer be used for similar reasons. Instead we have a member called IsValid that is set to true when an elastic vertex exists.

Loops over vertices are no longer needed such as in:

 
for(unsigned int vtx_idx = 0; vtx_idx < sr->vtx.elastic.size(); vtx_idx++) {

    do something...

}

Instead you will want to just check if a vertex exists using the IsValid member as shown above.

FIXING THE BPF VECTOR:

Fixing loops over the vector of BPF tracks

Since there is no longer a vector of BPF tracks, you will now have to access directly the track that you wanted (muon, pion, proton). Assuming you have some loop like this in your code:

unsigned int muIdx = (unsigned int)kCVNMuonIdx(sr);
// loop over tracks to make sure we only get the track fit under the muon assumption
for(unsigned int t = 0; t < sr->vtx.elastic[0].fuzzyk.png[muIdx].bpf.size(); ++t) {
     // skip this track if it is not the muon assumption
     if(sr->vtx.elastic[0].fuzzyk.png[muIdx].bpf[t].pdg != 13) continue;
     Nhits = sr->vtx.elastic[0].fuzzyk.png[muIdx].bpf[t].nhit;
}

this loop will now be replaced with:
unsigned int muIdx = (unsigned int)kCVNMuonIdx(sr);
if(sr->vxt.elastic.IsValid && sr->vtx.elastic.fuzzyk.png[muIdx].bpf.muon.IsValid) Nhits = sr->vtx.elastic.fuzzyk.png[muIdx].bpf.muon.nhit;

Replacing access to bpf.size()

Besides access to bpf.size() in loops like the above, I ran into a few instances of querying bpf.size() in the code. I don't think there is a fool proof thing to swap out for this, it may depend on each individual use case. In general, any new logic can probably be replaced with querying the IsValid flag for each of the three BPF tracks. So here is an example assuming you are requiring all 3 BPF tracks to be present:

if(png.bpf.size() == 3) {

     (do something...)

}

should be replaced with:
if(png.bpf.muon.IsValid == true && png.bpf.pion.IsValid == true && png.bpf.proton.IsValid == true) {

     (do something...)

}

...and another example requiring there to be at least one BPF track:

if(png.bpf.size() != 0) {

     (do something...)

}

should be replaced with:
if(png.bpf.muon.IsValid == true || png.bpf.pion.IsValid == true || png.bpf.proton.IsValid == true) {

     (do something...)

}

General checks on track quality

If you are checking track specific variables to see if the track is valid (for example):

if(png.bpf[i].energy > 0.0) (do something...)

these checks could stay as they are (provided you replace bpf[i].energy with bpf.{muon,pion,proton}.energy) or you could check the IsValid flag for that track.

Various other Standard Record Changes - Removing various branches

In combination with the changes above it was also decided to remove numerous branches from Standard Record. Below is hopefully a complete list;
  • Various 2017 braches, eg remid2017
  • Various CVN branches, eg cvn2017 and cvnProd3Train
    There were also edits to files like CAFMakerParams.h to remove fhicl parameters called 2017.

There were also changes made to the SRCosRej package to accommodate the new NuMu CosRej BDT for the 2020 analysis.

For the removal of the 2017 branches if a CAFAna Var accesses those data products it will fail.
Effort was made to add abort() statements wherever this was the case in CAFAna. If you see this then you are using an old Variable. Ultimately there are two choices;
  • Create a new variables which accesses the branch which is used for Prod5.
  • There may be cases where the fallback was implemented too stringently. In this case remove the abort() and fix the code.