Project

General

Profile

Event filtering and selection

EventID pattern-based filter and comparator

As of art 2.07.01, a framework-provided module called EventIDFilter is available to select events whose EventIDs match a sequence of configured patterns. Should a given EventID match at least one of the patterns, the EDFilter::Pass value is returned from the module's filter function, accepting the event1. The format of a pattern string consists of three fields:

"<Run-part>:<SubRun-part>:<Event-part>" 

Each of the Run, SubRun, and Event fields can be represented by a number, or set of numbers. The '*' wildcard can be used to represent any number, and the ',' and '-' characters can be used to specify sets or ranges of numbers. For example:

"1:*:*"     // Accept Run 1, any SubRun, any Event
"1:2:*"     // Accept Run 1, SubRun 2, any Event
"1:2:3"     // Accept Run 1, SubRun 2, Event 3
"1:*:4"     // Accept Run 1, any SubRun, Event 4
"1:2-5:*"   // Accept Run 1, SubRuns 2 through 5 (inclusive), any Event
"*:9:10,11" // Accept any Run, SubRun 9, Events 10 and 11
"7:2-5,8:*" // Accept Run 7, SubRuns 2 through 5 (inclusive) and 8, any Event

When specifying multiple patterns, each pattern is tried in the specified order, and if any pattern matches the EventID, then true is returned. In other words, if the event in question matches any (not all) of the patterns, the event is accepted.

1 For configuration details, type art --print-description EventIDFilter.

Users who wish to use the above functionality outside of the art-supplied module may do so through the EventIDMatcher class, found in canvas. The public interface is:

explicit EventIDMatcher(std::string const& pattern);
explicit EventIDMatcher(std::vector<std::string> const& patterns);

bool operator()(EventID const&) const;
bool match(EventID const&) const;

There are two constructors, the first takes one argument which is the pattern the matcher should use, while the second takes one argument which is a list of patterns the matcher should use. The match and operator() member functions are synonyms and take an EventID to be matched against the patterns.

Event selection

Output modules and analyzers can be configured so that events for that module will be processed only if they satisfy a given trigger criterion. For each analyzer or output module, the events processed are determined based on the SelectEvents specification. The default specification is:

SelectEvents: [] 

where the empty sequence implies that all events should be processed.

N.B. For art versions older than 2.01.00, see here.

To select events that have passed a given filter path, the name of the path is specified (e.g.):

physics : {

   filters : {
      f1 : { module_type : MyFilter1 }
      f2 : { module_type : MyFilter2 }
   }

   fpath: [f1, f2]

   analyzers : {
      a1 : { 
        module_type : MyAnalyzer
        SelectEvents: [fpath]
      }
   }

   epath: [a1]

}

The analyzer with the module label 'a1' will see the events for which the filters in fpath both yield true.

Negated filter paths

If it is desired to run over events that fail the filter criterion fpath (instead of passing the criterion), the SelectEvents clause can be replaced by:

SelectEvents: ["!fpath"]

N.B. It is necessary for the path specification to appear in quotes whenever the negation symbol ('!') is used. Not using the quotation marks would result in a parsing error.

Inclusive OR of filter criteria

If multiple trigger paths are included in the configuration file, the inclusive OR of the trigger paths is indicated by separating the OR'd path-names by comma:

physics : {

   filters : {
      f1 : { module_type : MyFilter1 }
      f2 : { module_type : MyFilter2 }
      f3 : { module_type : MyFilter3 }
   }

   fpathA: [f1, f2]
   fpathB: [f3]

   analyzers : {
      a1 : { 
        module_type : MyAnalyzer
        # Select events that pass either 'fpathA' or 'fpathB' or both.
        SelectEvents: [fpathA, fpathB]
      }
   }

   epath: [a1]

}
The following specifications yield identical behavior:
  • SelectEvents: [] -- process all events
  • SelectEvents: [fpathA, "!fpathA"] -- process all events that either pass fpathA or fail fpathA
  • No SelectEvents clause at all -- interpreted by art to mean 'SelectEvents: []'

Wildcards

For convenience, the wildcards '*' and '?' can be used to simplify the trigger-path specifications. Suppose a given configuration has the following trigger paths:

  • fpathA
  • fpathB
  • foopath
  • path

Then the following translations are performed:

This configuration is equivalent to this configuration.
SelectEvents: ["*"] SelectEvents: [fpathA, fpathB, foopath, path]
SelectEvents: ["f*"] SelectEvents: [fpathA, fpathB, foopath]
SelectEvents: ["*path"] SelectEvents: [foopath, path]
SelectEvents: ["*path*"] SelectEvents: [fpathA, fpathB, foopath, path]
SelectEvents: ["fpath?"] SelectEvents: [fpathA, fpathB]
SelectEvents: ["?path*"] SelectEvents: [fpathA, fpathB]
SelectEvents: ["fpath?","!*path"] SelectEvents: [fpathA, fpathB, "!foopath", "!path"]

Wildcards can also be used when specifying the requested behavior for exceptions (see below).

Events with/without exceptions

If art is configured so that an event is skipped if an exception was thrown during its processing, then the user can filter events based on whether an exception was thrown. For example, suppose it is desired to write to a file only those events that passed trigger path fpath and were not skipped due to an exception being thrown. The appropriate output module configuration would be:

outputs.o1 : {
   module_type : RootOutput
   SelectEvents: [ "fpath&noexception" ]
}

Furthermore, a user can specify that events should be written to a file if they were not skipped due to an exception throw, and failed trigger path fpath:

outputs.o1 : {
   module_type : RootOutput
   SelectEvents: [ "!fpath&noexception" ]
}

Lastly, a user can request that events should be written to a file if they were skipped due to an exception throw:

outputs.o1 : {
   module_type : RootOutput
   SelectEvents: [ "exception@fpath" ]
}

N.B. The ampersand ('&') is used for the noexception case, and the "at"-symbol ('@') is used for the exception case.

The following specifications are errors:

SelectEvents: ["exception@fpathA&noexception"]  # Error - conflicting 'exception' vs. 'noexception'

SelectEvents: ["!exception@fpathA"]             # Error - 'exception' already indicates a failure. 
                                                #         Cannot select an event that failed a filter criterion
                                                #         in addition to having an exception being thrown

Filter paths from previous processes

Restored in art 2.07.01.

Since filter information from each process is stored on disk, it is possible to apply trigger criteria from a given process to the current process. This is achieved by prepending the process name (and a delimiting colon ':') to the path specification that represents the desired event selection. For example, suppose you needed to output events for which a track was not reconstructed (as determined from a trigger path recoTracks in a previous process named "Produce"), but the energy of the event was high enough to satisfy the highEnergy trigger criterion. The configuration would look like:

process_name: Select
physics: {
  filters: { ... }
  highEnergy: [...]
  epath: [o1]
}

outputs.o1: {
  module_type: RootOutput
  SelectEvents: ["Produce:!recoTracks", highEnergy]
}

Notice the colon (:) that delimits the process name 'Produce' from the path specification '!recoTracks'. Qualifying an event selection by its process name can similarly be done for analyzers.

Further examples

In what follows, various examples are given only for output modules. However, the same support exists for any analyzers, as described above.

# this is only a fragment of a full configuration ...
physics: {

  path1 : [ ... ]  # producers and filters are put in this path
  path2 : [ ... ]  # other producers, other filters are put in this path

  observers : [ a1, a2 ] # analyzers
  outputs   : [ o1, o2, o3, o4 ]

  trigger_paths: [ path1, path2 ] # declare that these are "trigger paths" 
  end_paths: [ observers, outputs ] # declare these are "end paths" 

}

outputs: {

  o1 : {
    module_type : RootOutput
    fileName : "pathA_passes.root" 
    # Write all the events for which pathA ended with 'true' from filtering.
    SelectEvents: [ pathA ]
  }

  o2 : {
    module_type : RootOutput
    fileName : "pathA_passes_noexceptions.root" 
    # Write all the events for which pathA ended with 'true' from filtering.
    # Events which caused an exception throw will not be written.
    SelectEvents: [ "pathA&noexeception" ]
  }

  o3 :  {
    module_type : RootOutput
    fileName : "pathA_failures_noexceptions.root" 
    # Write all the events for which pathA ended with 'false' from filtering.
    # Events which caused an exception throw will not be written.
    SelectEvents: [ "!pathA&noexception" ]
  }

  o4 :  {
    module_type : RootOutput
    fileName : "pathAorB_exceptions.root" 
    # Write all the events for which pathA or pathB ended because an exception was thrown.
    SelectEvents: [ "exception@pathA", "exception@pathB" ]
  }
}

Legacy configurations

For art versions older than 2.01.00, the 'SelectEvents' sequence needs to be wrapped in a 'SelectEvents' table:

SelectEvents: { SelectEvents: [...] }