V06260105 » History » Version 41

« Previous - Version 41/42 (diff) - Next » - Current version
Daniel Devitt, 07/11/2017 02:28 PM

Uboonecode v06.26.01.05 (MCC8.1) validation

See mcc8-validation Slack channel

June, 2017 Mini-Retreat

Validation plots from mini-retreat held on June 19-20, 2017.

  • 3D vertex resolution plots in CPM analysis (Aleena), (docdb 9509)
    • MCC8.1 shows worse neutrino vertex resolution than MCC7.
      • Due to space charge? Prescription for correcting for space charge exists. Apply correction and regenerate plots.
      • Due to more broken tracks in MCC8.1 vs. MCC7? May be the case but selection II also shows disagreement and includes an angle cut for multiplicity = 2 events.
  • SelectionII Comparisons between MCC8.0 and MCC8.1 (Ariana), (docdb 9482)
    • Passing rate tables seem to indicate that difference is in on-beam data only. However, a 20% increase in the CC cross section is expected in MCC8.1 vs. MCC7 and so this could be compensating for a lower efficiency in MC.
      • Also, Andy checked the overlap between MCC7 events passing selection II and those passing MCC8.1 and found 1.6% of MCC8.1 events were also selected by MCC7 for off-beam data and 23% of MCC8.1 events were also selected by MCC7 for on-beam data.
        • Re-run overlap analysis on the updated selection II, which is the basis for the current selection II (Andy)
        • Check the fcl parameters used for the beam window in the cosmic removal pass and in the neutrino selection filters for both on-beam and off-beam data and MC (Joseph)
        • Plot the time distribution of flashes accepted and rejected as in the neutrino event selection (Ariana)
        • Plot time distribution of accepted and rejected flashes in the cosmic removal pass (Marco)
    • Noise filtering has changed
      • Validate integration of wirecell noise filtering into LArSoft since some differences seen in the event display (Mike M.)
      • Run rawdigitfilter instead of wirecell noise filter (Adam)
      • Modify wirecell noise filter to compute the truncated RMS for each wire to aid in ROI finding (Mike M.)
      • Add code to CalwireROI to compute the truncated RMS for ROI finding (Tracy) DONE
    • Track/shower classification in Pandora has changed dramatically
      • Run Giuseppe's track fitter over track-like and shower-like PFPs and then use those tracks in the selection (Ariana)
    • Cosmic removal pass has changed
      • Run selection II with cosmic removal pass exactly as it was in MCC7 and re-run selection II (David C.)
  • Comparison of MCC7, MCC8, & MCC8.1 CC Selection I Passing Rates (Aleena), (docdb 9476), (docdb 9644)
    • Use tables to compare on-off beam data pass rates and BNB+cosmics pass rates (David C.) DONE
    • Plot the time distribution of flashes accepted and rejected as in the neutrino event selection (Aleena)
    • Cut on opflash instead to see if pass rate fractions in MCC7 and MCC8.1 come in better agreement (Aleena)
      • For this to be an apples-to-apples comparison, one needs to also revert back to a MCC7-like cosmic removal path (David C.)
    • If disagreement in track containment still exists after going back to opFlash, look at events rejected by the track containment cut in the event display for MCC7 and MCC8.1 (Aleena)
  • MCC8 Validation: position dependent response (David C.), (docdb 7171)
    • Bug found in ACPT t0 tagging module and fix committed DONE
  • CPM Analysis in MCC8.1 (Aleena), (docdb 9389), (docdb 9644)
    • Print out event lists for the selection I anatree and uboonecode modules and then investigate what cuts are causing them to pass different events and why. (Aleena)
    • Understand why there is no efficiency in the backwards direction for DIS events (see DocDB-7607, slide 19)
    • Aleena uses the original (unmodified) version of selection 1, which does not preferentially select events with forward-going tracks.
    • There is a big difference in the efficiency for selecting backward-going cosmics in MCC8.1 relative to MCC7
      • Look at off-beam data events selected by the neutrino filter in the backward direction in the event display (Aleena)
    • There is a much larger "broken track" issue in MCC8.1 relative to MCC7 (manifested as a excess of events with tracks that have an opening angle of costh=-1)
      • Note that if this broken track issue is solved then Aleena's plots start to look much better.
        • Cut out events that have two tracks with an opening angle of costh=-1 and remake plots (Aleena)
        • Look at off-beam data events selected by the neutrino filter with multiplicity=3 in the event display to see if trains of hits fit to a post-pulse baseline shift is the source of the problem (Aleena)
  • Studies on MCC8.1 reconstructed data events (Marco), (docdb 9446)
    • One bug found in geometric cosmic tagging and fix has been committed (Herb to verify it goes on MCC8.x branch)
    • Add two additional columns to the table: nShowers, nDaughterPFPs (Marco)
      • Repeat this study on MC (Marco)
    • Issue with reconstructed flash position moving by >1 m causing MCC7-selected neutrino candidates to not pass flash/track matching selection cut in MCC8.1.
      • Print out timing (in addition to position and PEs) of opHits for the events which had a shifted flash z position causing the event to not be selected in MCC8.1 (Marco)
      • Look at optical waveforms for these events (Marco)
  • Hit level MCC8.1 (Brooke), (docdb 9431)
    • Some disagreements remain, though these are perhaps unrelated to the data-driven responses that were put in to better match data.
  • Data/MC comparisons for MCC8.1 validation (Adam L. and Danny D.), (docdb 9401)
    • Need to run over a dataset who list of dead channels matches the MC simulation in order to compare residual "hot spots" (Adam)
    • How do chirping-correlated excess noise hits appear as a function of run number and are they already excluded from the good run list (Adam)?
    • In simpleBeamFlash, there are data events which have photoelectrons/flash below the flash cut of 20 PE. DONE
      • Bug found in which MCC8.1 analysis trees for data were generated using wrong version LArSoft. MCC8.1 data analysis trees are being regenerated.
      • There should be more events with hit multiplicities > 4 in data. This may be solved with the above bug fix. DONE
    • Upward-going muons not expected in MCC8.1 from PandoraCosmic and needs to be investigated (Andy B.) DONE
    • PandoraNuPMA shows very different track start X and track theta behavior in MCC8.1 vs. MCC7 that must be understood before using this producer (Tingjun)
    • The PEs/flash plots appear to have a different threshold in data and MC. Not confirmed by Kazu's study. However, the PE distribution of ophits in data and MC may be different due to differences in the baseline overshoot and/or background light levels in data and MC (Kolahal)

POT accounting and absolute-normalization DATA/MC comparison investigations:

  • Trying to answer: does the POT scaling we apply now show good absolute-normalization agreement between DATA and MC after the very first few cuts in the selections? (docdb 9614)
  • Scaling Modified Selection 1 pass rates cut-by-cut to 5E19 POT: (DocDB 9644)
  • Does the average protons per pulse used in GENIE match Run 1 (or 5e19) data (Joseph)?

Upward going PandoraCosmic tracks:

Hi Matt,

Yes, I think I understand the upward-going tracks now. They're all really short tracks (mostly fake?). When I cut out tracks shorter than 10cm, the distributions return to normal.

I'm not sure why this is new in MCC8, as the pandoraCosmic reconstruction has hardly changed since MCC7. But I think the problem is that the pandoraCosmic thresholds are very low, and can easily make tracks out of noise hits. MCC8 has "horizontal" noise (i.e. chains of hits along a single wire). For MCC9, I think I'll need to work on tightening up the thresholds again to remove some of the junk.

It might be better to select clean samples of cosmic muon tracks for many of our data/MC comparisons. Of course, we can't do that all the time, as we also need to study the reconstruction of very short proton tracks.

Hope that helps.

Best Wishes,

(Danny) I quickly had a look at this myself (DocDB 9902). It looks like an even shorter cut might be reasonable, though one would have to play around with it to figure out what's best.

Other known issues in MCC8.x

Various and sundry items

  • Janet suggests everyone use a "front porch" cut on the optical hits before the beam window.