Allowed product lookups

Consider the following configuration:

physics: {
  producers: {
    p1: {}
    p2: {}
    p3: {}
  filters: {
    f1: {}

  # Trigger paths
  tp1: [p1, p3]
  tp2: [f1, p2]

where the specific configurations for the modules have been suppressed. As discussed on the Paths page, the following behaviors apply:

  1. The order in which trigger paths (e.g. tp1 and tp2) are executed is unspecified in a single-threaded art job. In a multi-threaded art job, trigger paths will be executed concurrently.
  2. Modules in a trigger path are always executed in the order specified. In single-threaded process, entire paths are executed sequentially. However, in a multi-threaded process, the execution of modules in different paths can be interleaved.
  3. End paths are always executed after all trigger paths have executed. The order of execution for all end-path modules is unspecified.

Suppose that filter f1 depends on a data product pr1 that has been produced by the producer p1. Because of behavior 1, in the configuration above it is not guaranteed that the pr1 product will be available by the time f1 processes the event. Because of this, the above configuration is ill-defined even in a single-threaded art job, despite the event-processing implementation of older art versions executing trigger paths in alphabetical order according to the trigger-path name. With such an implementation, a change as simple as relabeling the paths can affect the output. In art 3, such a configuration is illegal.

The solution to this problem (for both single- and multi-threaded art jobs) is to list the p1 producer as the first module in path tp2:

tp1: [p1, p3]
tp2: [p1, f1, p2]

With this configuration, the producer p1 is guaranteed to execute before the filter f1, and art ensures that a module shared across paths (i.e. p1) is executed only once.

Circular data-dependencies within a path

Consider the situation where the producer p3 consumes product pr1 created by producer p1. In such a case, the following configuration is an error:

tp: [p3, p1]

Because of behavior 2 above, p3 will execute before p1, and pr1 will not be available. This configuration error has not been caught in the past, but if the user invokes the consumes interface as discussed here, such an error will be caught by the framework.

The corresponding rule is: a module in a trigger path may consume a product that is either provided by the input source, or by an upstream module in that same path.

This rule is strictly enforced for art 3: if a module on a trigger path attempts to retrieve a product that is not from the input source or from an upstream module on the same path, the returned/filled art::Handle will be invalid.

Behavior of getMany and getManyByType

For art versions older than 3.00.00, the behavior of Event::getMany(ByType) is to return any products in memory that match the given criteria. For art 3, this behavior is restricted so that the only products returned to the user will be those matching the given criteria for (a) modules on that path and (b) any products provided by the source. Consider the following configuration:

tp1: [p1, p2, p3, retrieveProducts]
tp2: [p1, p2, p4, p5]

If module retrieveProducts calls Event::getManyByType, then any products produced by modules p1, p2, and p3, are candidates for being provided to retrieveProducts. art guarantees that any products produced by p4 and p5 are not provided during the getMany call, even if p4 and p5 happened to executed before retrieveProducts.

Different modules before call to getMany(ByType)

Consider this variant:

tp1: [p1, p2, retrieveProducts]
tp2: [p4, p5, retrieveProducts]

Here, retrieveProducts still calls getManyByType, but the modules that precede it are different for paths tp1 and tp2. Because of behavior 1 above, such a configuration is an error. If the consumes interface is used, this error will be caught at a job start-up time.

Product lookups in analyzers and output modules

Because art analyzers and output modules do not create products, there are no path-ordering behaviors that require special attention. It is still to the user's benefit to make the 'consumes' calls in analyzer constructors. Doing so makes it apparent which products will be retrieved in the module, and in future versions of art, the consumes information can be used to optimize the efficiency of the job.

Graphical representation of data-product dependencies

Please see the documentation here for a facility to help you understand what the data dependencies are among your modules.