Redesign the model that is used to provide events to online monitoring
At the moment, the online monitoring Aggregator is run as part of the MPI program. This means that if it crashes (for example, because of a bug in a user-provided monitoring module), there is a good chance that the entire system stops.
We should design and implement a system that allows multiple online monitoring applications to come and go as needed without disturbing the primary function of the DAQ system to collect and log the data to disk.
It would be great for the final solution to provide monitoring information on the fraction of events that are seen by a particular consumer. If this were included in the data stream, then an online monitor could do the appropriate normalization to scale its results up to the full data set.