Project

General

Profile

Feature #8402

Feature #8593: Improve flexibility of output file handling

Request art have the ability to created a new output file for every input file.

Added by Michael Kirby over 5 years ago. Updated almost 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
I/O
Target version:
Start date:
04/23/2015
Due date:
08/01/2015
% Done:

100%

Estimated time:
4.00 h
Spent time:
Scope:
Internal
Experiment:
MicroBooNE
SSI Package:
art
Duration: 101

Description

I would like to request a new feature for art such that for each input file a new output file is created. This is very useful for production running of binary data file swizzling to art/root format and reconstruction running for production. Then for every file produced by the DAQ of an experiment there would be a single corresponding file containing all of the information from that input file. Please let me know if there is additional information needed or other requirements that you might foresee.

- Kirby

Associated revisions

Revision 491fe406 (diff)
Added by Kyle Knoepfel almost 5 years ago

Add test for issue #8402, and some tidying

Revision 753d128b (diff)
Added by Kyle Knoepfel almost 5 years ago

Add test for issue #8402, and some tidying.

Revision 139089c2 (diff)
Added by Kyle Knoepfel almost 5 years ago

Add test for issue #8402, and some tidying

Revision a3ef3b31 (diff)
Added by Kyle Knoepfel almost 5 years ago

Add test for issue #8402, and some tidying

History

#1 Updated by Christopher Green over 5 years ago

  • Status changed from New to Accepted
  • Estimated time set to 4.00 h

We believe, subject to vague memories, that this ability may already exist, and a minor change to the user's Source may provide the functionality requested. This is far from certain, however, and the time estimate of 4 hours is merely that time required to ascertain whether this is in fact the case.

#2 Updated by Marc Paterno over 5 years ago

  • Parent task set to #8593

#3 Updated by Christopher Green over 5 years ago

  • Target version set to 1.18.00

#4 Updated by Christopher Green over 5 years ago

  • Due date set to 08/01/2015

#5 Updated by Kyle Knoepfel almost 5 years ago

Indeed, this feature does already exist in art. Consider the following configuration:

process_name: mergerTest

services : {
   scheduler: {
      fileMode: NOMERGE
   }
}

source: { 
   maxEvents: 10
   module_type: RootInput
   fileNames : [ "ToyProductProducer_t_01.root",
                 "ToyProductProducer_t_02.root" ]
}

physics: {

   stream1  : [ out1 ]
   end_paths: [ stream1 ]

}

outputs: { 

   out1: { 
      fileName: "ToyProductMerger_%#.root" 
      module_type: "RootOutput" 
   } 

}  

For this particular example, %# will increment the file number written to the out1 stream. Two output files are thus produced:

  • ToyProductMerger_1.root
  • ToyProductMerger_2.root

which correspond, respectively, to the first and second input files.

Summary: The important ingredients here are the NOMERGE file mode specification, and the output file renaming specifiers (see here). Without the specifiers, the same output file will be overwritten with the events corresponding to the current input file.

Although this feature has existed for some time, I will not mark this issue as resolved until I have verified that the appropriate tests exist.

#6 Updated by Kyle Knoepfel almost 5 years ago

  • % Done changed from 0 to 100

#7 Updated by Kyle Knoepfel almost 5 years ago

  • Assignee set to Kyle Knoepfel

#8 Updated by Christopher Green almost 5 years ago

  • Target version changed from 1.18.00 to 834

#9 Updated by Marc Paterno almost 5 years ago

  • Target version changed from 834 to 2.01.00

#10 Updated by Kyle Knoepfel almost 5 years ago

  • Status changed from Accepted to Resolved
  • SSI Package art added
  • SSI Package deleted ()

Tests implemented with art:491fe40. Documentation added here.

#11 Updated by Kyle Knoepfel almost 5 years ago

  • Target version changed from 2.01.00 to 1.17.04

#12 Updated by Kyle Knoepfel almost 5 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF