Notes from the parallel art work¶
- Consider locating a library of thread-safe data structures (for example, https://parasol.tamu.edu/stapl/), and then ...
- Formulate guidelines for the decision to use such a thread-safe data structure, rather than the standard library data structures.
- Example: a map that is constructed in single-threaded mode, and read only after multi-threading begins, need not be replaced.
- Example: a map that must be both read and updated in multi-threaded mode is a strong candidate for a thread-safe replacement.
- We need to be careful in our parallel code to:
- Protect (e.g., with mutexes) all direct access to shared writable data structures.
- Prevent unprotected indirect access to shared data via any proxy (reference, pointer, iterator, index, etc.):
- We shouldn’t return any proxy from a function, and ...
- We shouldn’t pass any proxy as a function argument.
- The constness of the shared object or proxy thereto does not affect this, since concurrently-executing code may still be updating the original.
Proposal to resolve the module type erasure problem in the worker hierarchy and module construction.¶
Currently, the four non-input module base classes (
OutputModule) all define a type
WorkerType which is, respectively,
OutputWorker<OutputModule> and is used therefore to construct a worker in the macro-defined function
ModueMacros.h. here is some type erasure here, which makes things difficult if we are to hold a module and copy it faithfully without slicing.
Our tentative proposal (which will require some fleshing-out) is to take advantage of the fact that we are maintaining a separate list of
Schedule to break the relationship between
OutputWorker, and introduce an
OutputWorkerT, with a non-owning relationship to the module in the
WorkerT from which it is created.
make_temp()will always make a
Tis the most-specific subclass.
EventProcessor) will construct its list of output workers by consulting a trait provided by the four module base classes to determine whether an
OutputWorkerT<T>should be made.
Worker) will hold its module by value.
OutputWorker, which will not inherit from anything else) will hold its module by
- Lifetime will be controlled as appropriate to make this work.
This will require some analysis of how workers and output workers are invoked to be sure that this will work, but this will I believe ensure that modules are copied as necessary invoked as required, and the different worker types will carry the most specific information applicable with them without their interface having anything not suited to the character of the underlying module type.
Some issues and questions to consider¶
- What different collections of modules are currently in the system?
- Are all of these collections useful?
- Is the ownership model clear?
- Is their any advantage to having a common base class for all modules? Should we move in the direction of having the
Scheduledeal directly with base classes like
- Should the
Scheduledeal with output at all, or should handling of both input and output be done outside of
Schedule, and thus explicitly in single-threaded code?
- How do we order the modifications we choose to make in such a way that progress on the DS50 project can continue? Or should we revert DS50 to using the singly-threaded version of art, so that we do not impede progress on the multi-threaded version of art?
- Minimum requirements on ART from DS50: must retain the ability to run a single producer and analyzer with one schedule and use of openmp directives for task-level parallelism within those modules.
- Using the art-openmp product name and separate tag / version stream, we can "release" art-openmp as appropriate during multi-threaded ART development while ensuring that it continues to satisfy the requirements of the