Project

General

Profile

Support #16766

Empty/depopulated events getting through filter stages?

Added by Will Foreman over 3 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
I/O
Target version:
-
Start date:
06/05/2017
Due date:
% Done:

100%

Estimated time:
Spent time:
Scope:
Internal
Experiment:
LArIAT
SSI Package:
art
Duration:

Description

Hi,

In lariatsoft v06_38_00, I run an event filter followed by various reco modules over data files. However, when looking at the output either with "count_events" or "lar -c eventdump.fcl ..." I still see entries corresponding to the events that were supposed to have been filtered out. For example:

lar -c eventdump.fcl /pnfs/lariat/persistent/users/wforeman/lariat_pbars/pbar_run2_pickytrks/beamline/20611676_0/lariat_r8777_20170602T032436_1.root

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: Run
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE | SIZE
EventBuilderNoMerge | daq......... | ..................... | sumdata::RunData. | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

Total products (present, not present): 1 (1, 0).

PRINCIPAL TYPE: SubRun
PROCESS NAME....... | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE..... | SIZE
EventBuilderNoMerge | daq......... | ..................... | ldp::ConditionsSummary | ...-

And so on...

This makes it hard to judge if my reco is working because I have to scroll over many hundreds of these Run and Subrun principals to find actual Event output products to look at. Any idea how to modify my filter/workflow to exclude all these empty Run/SubRun principals? Or are there options I can provide to eventdump to skip these? I tried reading through the art workbook but couldn't find this mentioned anywhere.

Thanks,
Will

History

#1 Updated by Kyle Knoepfel over 3 years ago

Hi Will,

By default, art processes empty (Sub)Runs. However, there is an option to disable processing of empty (Sub)Runs by the FHiCL configuration:

services.scheduler.handleEmptySubRuns: false
services.scheduler.handleEmptyRuns: false

If you do not own the eventdump.fcl file, you can adjust the behavior by doing:

fhicl-dump eventdump.fcl -o my_eventdump.fcl
# ... then add the relevant configuration to my_eventdump.fcl
art -c my_eventdump.fcl ...

Please let us know if that does not fit your need.

#2 Updated by Kyle Knoepfel over 3 years ago

On second thought, Will, the approach I suggested will likely not work because the event-dumper functionality relies on the write* output-module function calls, which are always called, even if the handleEmpty(Sub)Runs parameters are set to false.

Another possibility is that for your event-dumper job, you can drop some products on input so that they're not even read by the framework:

source.inputCommands: ["keep *", "drop * InSubRun"]

#3 Updated by Kyle Knoepfel over 3 years ago

Okay, more typos in the above edit. My apologies. I was able to get the following to work:

source: {
  module_type: RootInput
  inputCommands: ["keep *", "drop * InSubRun", "drop * InRun"]
  fileNames: ["a.root"]
}

physics.e1: [o1]
outputs.o1.module_type: FileDumperOutput

#4 Updated by Kyle Knoepfel over 3 years ago

  • Category set to I/O
  • Status changed from New to Assigned
  • Assignee set to Kyle Knoepfel
  • SSI Package art added

#5 Updated by Kyle Knoepfel over 3 years ago

  • Status changed from Assigned to Resolved
  • % Done changed from 0 to 100

#6 Updated by Kyle Knoepfel over 3 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF