Project

General

Profile

Feature #7449

Analysis support requires ability to store objects in file in branch other than Run, SubRun, or Event

Added by Brian Rebel almost 6 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Category:
I/O
Target version:
Start date:
12/05/2014
Due date:
10/01/2015
% Done:

100%

Estimated time:
112.00 h
Spent time:
Scope:
Internal
Experiment:
NOvA
SSI Package:
art
Duration: 301

Description

I am currently working on performing an analysis on NOvA and would like to use art's provenance tracking feature when producing and storing the final output of the analysis.

The problem is that to do a neutrino oscillation analysis, we do a fit over the energy spectra of all selected events from all runs taken. That is, we cannot store the analysis result in any of the event, sub run, or run areas of the output art file because the analysis must be done at the endJob portion of the state. To store the result, we would need to do it at a level that is outside of the run process.

I think it would be good to organize a meeting with the art stakeholders to discuss the use case of art for performing an analysis and come up with requirements. Doing so would also allow users to move away from making TTrees/ntuples outside of art for doing analysis and make use of art's provenance tracking.

I have a path forward for now, so this is not a critical issue, but it should be given attention soon.

History

#1 Updated by Christopher Backhouse almost 6 years ago

We also wanted this for the attenuation calibration. In that respect it was filed as Issue 4324. The solution there with the database I guess is working for the drift, but it's inapplicable for the attenuation, where we're still hacking around the absence of this.

#2 Updated by Christopher Green almost 6 years ago

  • Category set to I/O
  • Status changed from New to Feedback
  • Assignee set to Christopher Green
  • SSI Package art added
  • SSI Package deleted ()

We believe it is a reasonable way forward to send a message to the stakeholders list requesting that experiments put together their thoughts on this subject, for a meeting to discuss them at next week's stakeholders meeting (18th December). Does this work for you?

#3 Updated by Brian Rebel almost 6 years ago

Sounds like a plan to me.

#4 Updated by Kyle Knoepfel over 5 years ago

  • Target version set to 521

#5 Updated by Marc Paterno over 5 years ago

We need to evaluate whether the technology preview of the histogram and ntuple handling meets the need expressed here.

#6 Updated by Brian Rebel over 5 years ago

I don't think histograms and ntuples are going to get us where we want to go. We want to be able to define a data product that is neither of those and store them in the output file with full provenance.

#7 Updated by Kyle Knoepfel about 5 years ago

  • Due date set to 10/01/2015

#8 Updated by Marc Paterno about 5 years ago

  • Estimated time set to 80.00 h

#9 Updated by Christopher Green about 5 years ago

  • Status changed from Feedback to Assigned
  • Estimated time changed from 80.00 h to 112.00 h

Adjusted estimated time to including planning and group analysis time.

#10 Updated by Kyle Knoepfel about 5 years ago

  • Target version changed from 521 to 1.16.00

#11 Updated by Christopher Green about 5 years ago

  • Status changed from Assigned to Resolved
  • % Done changed from 0 to 100

#12 Updated by Christopher Green about 5 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF