Notes from the parallel art work

  • Formulate guidelines for the decision to use such a thread-safe data structure, rather than the standard library data structures.
    • Example: a map that is constructed in single-threaded mode, and read only after multi-threading begins, need not be replaced.
    • Example: a map that must be both read and updated in multi-threaded mode is a strong candidate for a thread-safe replacement.
  • We need to be careful in our parallel code to:
    • Protect (e.g., with mutexes) all direct access to shared writable data structures.
    • Prevent unprotected indirect access to shared data via any proxy (reference, pointer, iterator, index, etc.):
      • We shouldn’t return any proxy from a function, and ...
      • We shouldn’t pass any proxy as a function argument.
      • The constness of the shared object or proxy thereto does not affect this, since concurrently-executing code may still be updating the original.

Proposal to resolve the module type erasure problem in the worker hierarchy and module construction.

Currently, the four non-input module base classes (EDAnalyzer, EDProducer, EDFilter, OutputModule) all define a type WorkerType which is, respectively, WorkerT<EDAnalyzer>, WorkerT<EDProducer>, WorkerT<EDFilter> and OutputWorker<OutputModule> and is used therefore to construct a worker in the macro-defined function make_temp in ModueMacros.h. here is some type erasure here, which makes things difficult if we are to hold a module and copy it faithfully without slicing.

Our tentative proposal (which will require some fleshing-out) is to take advantage of the fact that we are maintaining a separate list of OutputWorker in Schedule to break the relationship between Worker and OutputWorker, and introduce an OutputWorkerT, with a non-owning relationship to the module in the WorkerT from which it is created.

In summary:

  • make_temp() will always make a WorkerT<T>, where T is the most-specific subclass.
  • Schedule (or eventually, EventProcessor) will construct its list of output workers by consulting a trait provided by the four module base classes to determine whether an OutputWorkerT<T> should be made.
  • WorkerT<T> (inheriting from Worker) will hold its module by value.
  • OutputWorkerT<T> (inheriting from OutputWorker, which will not inherit from anything else) will hold its module by exempt_ptr<T>.
  • Lifetime will be controlled as appropriate to make this work.

This will require some analysis of how workers and output workers are invoked to be sure that this will work, but this will I believe ensure that modules are copied as necessary invoked as required, and the different worker types will carry the most specific information applicable with them without their interface having anything not suited to the character of the underlying module type.

Some issues and questions to consider

  1. What different collections of modules are currently in the system?
  2. Are all of these collections useful?
  3. Is the ownership model clear?
  4. Is their any advantage to having a common base class for all modules? Should we move in the direction of having the Schedule deal directly with base classes like ProducerWorker and FilterWorker?
  5. Should the Schedule deal with output at all, or should handling of both input and output be done outside of Schedule, and thus explicitly in single-threaded code?
  6. How do we order the modifications we choose to make in such a way that progress on the DS50 project can continue? Or should we revert DS50 to using the singly-threaded version of art, so that we do not impede progress on the multi-threaded version of art?
    • Minimum requirements on ART from DS50: must retain the ability to run a single producer and analyzer with one schedule and use of openmp directives for task-level parallelism within those modules.


  1. Using the art-openmp product name and separate tag / version stream, we can "release" art-openmp as appropriate during multi-threaded ART development while ensuring that it continues to satisfy the requirements of the artdaq product.