Project

General

Profile

CAFAna bulletin board

This page is modeled after the shift bulletin board. Most recent CAFAna changes you should be aware of are at the top, older ones further down.

CAFAnaCore part 1 (28th September 2020)

In order to facilitate sharing CAFAna components between experiments, an effort is underway to move non-NOvA-specific parts out to external packages.

Earlier phases were the creations of the srproxy and osclib externals. The current step consists of moving (much of) the existing CAFAna/Core to a ups product called cafancore.

This should have no observable effect on users. Some headers, e.g. CAFAna/Core/Spectrum.h are now being looked up within the cafanacore product rather than in $SRT_PUBLIC_CONTEXT or $SRT_PRIVATE_CONTEXT but this ought to be transparent, and not require any updates of user code.

The most likely cause of problems is if you use an old base release, still containing the headers, with an updated test release that expects them to have been moved. Or vice versa.

If you are intending to work with development CAFAna, you should also be using development as your base release. If you don't intend to update your local CAFAna and need a stable base from before these changes, S20-09-25 is the last no-cafanacore snapshot. There is now a version-checking mechanism that will hopefully alert you at compile- or run-time in case there is a mismatch between the required and actually set-up cafanacore versions. Most users should be working in 3FlavorAna/NuXAna/NDAna though, and have no need to have CAFAna checked out.

The second most likely problem is failure to find the new headers or libraries. In SRT you might need to add include SoftRelTools/arch_spec_cafanacore.mk to your GNUmakefile.

Technical details

  • A series of "stub" headers are left behind within CAFAna/Core/. These forward to the file of the corresponding name in the external product. In principle these shouldn't be necessary. In practice they are likely to help smooth other some problems with inconsistent base/test releases. They will be removed in a week or so.
  • For non-SRT users the additional compilation flags are -L$CAFANACORE_LIB -lCAFAnaCoreExt -I$CAFANACORE_INC

Future plans

  • Future steps will consist of moving more code to cafanacore, including anything non-nova-specific that might exist in Cuts/Vars/Systs. Pieces that are required for oscillation analysis, but not for general event selection / histogram manipulation will go into a new cafanafit. Many CAFAna users in the codebase will only need cafanacore. Further libraries, such as cafanabayes are a possibility. NOvA-specific code will remain in 3FlavorAna/NuXAna/NDAna. The ultimate home of NOvA-specific but not analysis-specific code has not yet been determined. Currently it is remaining within novasoft's CAFAna/

Covariance matrix code (15th September 2020)

Covariance matrix code developed in the nux-dev branch has been reintegrated into trunk, including changes in the CAFAna, 3FlavorAna and NuXAna packages. The majority of the changes are new files; some existing files have been updated or removed, but all of these are specifically within the domain of the covariance matrix framework, so no code seeing general use should be affected at all. Regardless, if you see any errors or strange behaviour that you think might be related to these changes, please reach out and let me know – j hewes

3FlavorAna (10th September 2020)

CAFAna has historically contained all the compiled code and analysis macros from the various published analyses.
This has two drawbacks:
  • Anyone who wants to work on CAFAna is forced to compile all of the analysis-specific code, and vice versa.
  • It stands in the way of the extraction of CAFAna into an experiment-independent codebase.

As of September 10th, the 3-flavor and NuX components have been moved to their own standalone packages, 3FlavorAna and NuXAna.
(The NuX code has also been moved because it has dependencies on 3FlavorAna that would otherwise cause a dependency loop.)
This solves the above issues.

A wholehearted attempt to adapt code checked into the repository (including analysis macros) to the new header locations.

What you need to do

  • The cafe executable has been updated to automatically load the 3-flavor libraries, since most NOvA users of CAFAna will likely need something from them.
    If for some reason you don't need or want to load them, you may invoke cafe with the --no3flavor argument.
  • If you have macros that aren't checked into the repository, you will need to adjust your #include's for the many headers that were formerly in CAFAna/{Cuts,Vars,etc.} and now have been moved into 3FlavorAna/{Cuts,Vars,etc.}.

Troubleshooting

(fill in as needed)

SpectrumLoader changes (9th September 2020)

The internals of SpectrumLoader have been refactored. This should allow Cool Things in the future, but should have no observable consequences for now.

If you do observe SpectrumLoader weirdness, and/or assertions about fReferences they're probably due to this. Please shout!

Unified Spectrum (Eigen/stan) (25th August 2020)

There are two main, interrelated, changes:

  • There is no longer a separate SpectrumStan. Instead, Spectrum will automatically have derivatives if created from something with derivatives. In case you need to know (you likely don't) use HasStan() to check.
  • The internal storage for Spectrum and OscillatableSpectrum is now Eigen rather than TH1. This should be more efficient, and allows us to get rid of HistCache. ToTH1() should only be used for plotting. If you need the bin data (implementing a Decomp or something) use GetEigen() (or possibly GetEigenStan()).

This also moves us much closer to the state of DUNE CAFAna which will facilitate the CAFAnaCore work going forwards.

Testing so far: the 2020 3flav fit macro finds exactly the same best fit, in 2/3 the time and 1/2 the memory. We will do more testing on trunk.

Potential problems: the memory improvement is mainly due to not carrying statistical uncertainties around - it's possible some plot that needs to have correctly computed stat errors no longer does. There is an environment variable to re-enable this computation. It has not been well-tested. Likewise, the conversion of non 3-flav code (eg nux, xsecs) has not yet been as thoroughly tested.

Fitter return value (17th August 2020)

To accommodate the different methodologies of our variety of fitters (minimizing chi2 vs. maximizing LL), I changed the signature returned by IFitter::Fit() to give you back a new type, IFitSummary. From one of these you can get back the value the fitter was optimizing EvalMetricVal(), as well as be compared with another to tell you which one was better (see IsBetterThan()), since the sign which is a better values of the "eval metric value" differs depending on whether the fit was maximizing or minimizing.
I tried to update all of the macros in SVN so that they will "just work" as they did before, though as usual if you have any that aren't committed, you'll need to update them yourself.

MinuitFitter and StanFitter both have derived types of IFitSummary which can do further things specific to each fit; you can see MinuitFitSummary or StanFitSummary if you're curious.

Signature of LoadFrom/SaveTo (12th August 2020)

I've updated the way LoadFrom/SaveTo work. It should reduce the number of TDirectories open at any one time, and hence speed up loading/saving large structures. I'd be interested to hear if this has any effect on the (pretty slow) loading of the 3flav prediction objects. The new format is also a little bit less typing for you.
SaveTo(dir->mkdir("foo")) becomes SaveTo(dir, "foo") and LoadFrom(dir->GetDirectory("foo")) becomes LoadFrom(dir, "foo")
I updated all the compiled code, but some of the stuff in Analysis/ and XSec/ is unfamiliar to me. If I updated something you're responsible for, please check I didn't break it?
I also attempted to update all the macros. This was an automated process, and since they're not compiled there's no easy way to check them. The new label argument is a std::string, so there are lots of .c_str() s that are now superfluous. There are also, I think, places that try to pass a TString that need to add .Data(). These should be pretty straightforward to fix when we run into them. I tried to update all the NDAna macros, but only the ones included in the build.

External OscLib (12th August 2020)

I merged a change so that OscLib now comes from an external package. You should see essentially no effect, but I took the opportunity to shorten the names, so if you have scripts that aren't in version control you will need to update

#include "OscLib/func/OscCalculatorFoo.h" 
osc::OscCalculatorFoo calc;
to
#include "OscLib/OscCalcFoo.h" 
osc::OscCalcFoo calc;

To make OscLib visible in packages where the Makefile has not already been adjusted you will need to include SoftRelTools/arch_spec_osclib.mk

External SRProxy (22nd June 2020)

SRProxy now comes from an external package rather than being built into novasoft. This should have no effect on you.

For non-CAFAna packages you may need to add arch_spec_srproxy.mk to the Makefile

Prod5 CAF structure ("vector of vectors")

We have made major structural changes to the CAFs which will be included in prod5. These changes have been summarized and reported here:
initial proposal of changes: https://nova-docdb.fnal.gov/cgi-bin/private/ShowDocument?docid=40077
general announcement: https://nova-docdb.fnal.gov/cgi-bin/private/ShowDocument?docid=41310
validation of changes: https://nova-docdb.fnal.gov/cgi-bin/private/ShowDocument?docid=40558
All of these changes are considered “breaking changes” and are currently sitting in a branch which we intend to merge into development on Wednesday (10/30) morning. After we do that, all CAF-based scripts running out of development that are running over prod4 or miniprod5 files will no longer work. A few key points if you are someone that will be affected by this:
1. Prior to doing this we will of course cut a tag of development for people to continue to work in.
2. A wiki has been created to help guide individuals through updating their code to be in line with these changes here:
https://cdcvs.fnal.gov/redmine/projects/nova_reconstruction/wiki/Fixing_the_%22many_vectors%22_problem
But note that no one needs to make these changes until they are ready to run over new prod5 files (which we have none of yet) AND all of the code in CAFAna has already been fixed to be in alignment with these changes.
3. As discussed with the ND group, we will take the NDAna package out of the nightly build so that ND-analyzers can continue to commit their code to development while working in the tag we make tomorrow.

Stan (July 2019)

(UNDER CONSTRUCTION)

Learn how to fit w STAN https://cdcvs.fnal.gov/redmine/projects/novaart/wiki/Using_Stan_Bayesian_MCMC_for_fitting

Merged into development on July 10-11, 2019.

From J. Wolcott:

You may have noticed a slew of commits from me (mostly to `CAFAna`, but a few to `Utilities` and `OscLib`) late this evening. After Chris & I spent a while reviewing and testing the changes, the work I've been doing to support Stan (Bayesian MCMC software)---see DocDB 33995/35236/35743/38205 if you don't know what that means and are interested---has finally been merged into the trunk.

This is supposed to be mainly transparent to users: everything should continue to work as usual. (For CAFAna, specifically, I've already verified that the `demoN.C` macros in `tute` still function as expected.) The main place where things may break is in macros you have -- there's been some shuffling around of directories (in particular: much of the fitting machinery has been consolidated into a `Fit/` directory), so you may experience some `include not found` errors from headers jumping to a different place.

Fitter changes

Old class Old include New class New include Notes
IFitter CAFAna/Fit/IFitter.h
Fitter CAFAna/Analysis/Fit.h MinuitFitter CAFAna/Fit/MinuitFitter.h
StanFitter CAFAna/Fit/StanFitter.h
Surface CAFAna/Analysis/Surface.h FrequentistSurface CAFAna/Fit/FrequentistSurface.h
Profile CAFAna/Analysis/Fit.h Profile CAFAna/Fit/Fit.h Uses MinuitFitter

Additional changes (that shouldn't have an impact)

  • Spectrum Sth sth template _Spectrum, sth sth no adjustments.
  • IOscCalculator

fatal error: 'stan/math/rev/scal.hpp' file not found

Try echo $STAN_MATH_INC. If it doesn't return anything, it means that your release doesn't have the necessary libraries. Use a tagged release post-July 11.

$ echo $STAN_MATH_INC
/cvmfs/nova.opensciencegrid.org/externals/stan_math/v2.18.0

CMakeBuild on OSX

It's complicated.

Introduction of SRProxy (2017)

The latest SRProxy-based version of CAFAna should make many macros run faster, and frees us from the error-prone tyranny of the Requires() line. It does take a few syntactic adjustments to your macro though. These should only take a few minutes.

Header inclusion

Adjust any #include lines like this

#include "StandardRecord/StandardRecord.h" 

to include SRProxy.h instead

#include "StandardRecord/Proxy/SRProxy.h" 

If you include any of the individual StandardRecord classes (e.g. SRSlice.h) you now only need SRProxy.h

Signature of Cut and Var

Variable definitions like

Var kMyVar({"my.var"},
           [](const caf::StandardRecord* sr)
           {
             return sr->my.var;
           });

should now look like this

Var kMyVar([](const caf::SRProxy* sr)
           {
             return sr->my.var;
           });

that's right. No need to specify your Requires() line anymore!

and the same for Cut or MultiVar.

SpillCuts now take a caf::SRSpillProxy argument.

If you assign to a temporary variable inside your Var, like const SRSlice& slc = sr->slc;, that type now gains a Proxy too: const SRSliceProxy& slc = sr->slc;

Similarly ISyst now takes an SRProxy, no need for Restorer anymore, and no need for a Requires() function.

"inconsistent types ‘float’ and ‘caf::Proxy<float>’ deduced for lambda return type"

You may have run into this previously. C++ is very picky about return values from lambdas and won't apply its normal type conversion rules. So, if you have an `if()` returning a default value, or something like that, you need to force your main return value taken from the proxy to be the same type. Just add a cast to float or whatever type you need to make things match.