Feature #3956: We should protect against all module failures at end run so that files get closed correctly
Avoid situations in which exceptional conditions in online monitoring module cause DAQ processes to crash [art-related]
This issue was originally part of Issue #3956, but that Issue has become a little muddled over time, so I'm creating this Issue expressly for dealing with the topic of handling exceptions. Of course, it will be useful to look at the notes in #3956.
Over time, in various discussions, we have learned that art currently supports the configuration of whether exceptions can be caught and reacted to by exception type. It would be more useful for artdaq/ds50daq, if this configurability were on a module-by-module basis.
Here are some notes on this topic from a 02-July discussion between Chris, Marc, and Kurt:
2) Chris suggested that "workers" could be changed to optionally swallow exceptions based on a configuration parameter. We talked through various types of exceptions, and we agreed that there will probably always need to be a catch-all case (since we can't foresee all exceptions that user code might throw). I made my usual request for module-based configurability (in addition to the exception-based configurability that already exists).
2.1) Possible action items
2.1.1) a quick solution might be to allow all non-art exceptions to be handled in the same way that we handle art exceptions. This would be configurable and would allow "next event" and "next module" options.
2.1.2) a medium-term solution might be to allow module-based configurability of handling exceptions and continuing