Make it possible an experiment to filter on the events artdaq sends out to its monitoring art process(es)
Redmine Issue #13360, "Create an art source capable of receiving events sent via a Transfer plugin from an artdaq aggregator", has decoupled monitoring from the main artdaq data flow. In particular, it's possible to launch multiple art processes running different art modules for different monitoring tasks. Not all - or even, perhaps, any - of the art monitoring processes may need every single event passing through the DAQ. A very probable scenario moving forward is that while artdaq is running, different art processes with different monitoring goals (modules) will want different subsets of the events- e.g., every Nth event, or only events which pass a certain set of triggers. It should be possible for experiments to specify in the FHiCL document passed to the art process which subset of events they want passed to it according to filters they've defined; by providing only the events that monitors care about, artdaq will create less of a load on the network it's on.
#1 Updated by John Freeman about 4 years ago
- % Done changed from 0 to 60
The following comments concern artdaq commit 49ce50068f3b4ae279e35c23d8c31cfb148a92ec and artdaq-demo commit fd668fed0ae21be81688442af26df19f1802ec25 :
In artdaq, the register_monitor function now expects that the string it's passed as an argument is the FHiCL definition of a transfer plugin. When an art process using a Transfer source is launched, the FHiCL code inside of a table called "transfer_plugin" is passed to register_monitor; the dispatcher aggregator will proceed to create a transfer plugin instance and add it to a queue. Whenever the queue has been seen to increase in size since the last iteration through the process_fragments loop, the new transfer plugins in the queue send out the Init fragment. Also, for every event received by the dispatcher, it broadcasts out that event using every transfer plugin in the queue.
In essence, each art process has its own transfer plugin managed by the dispatcher sending it events. What this allows is for different filtering algorithms to be applied depending on the art process/transfer plugin pair in question. A new transfer plugin called "moduloTransfer" has been added to artdaq-demo; it uses the Decorator pattern to add filtering logic to whatever transfer plugin from artdaq it uses to physically copy out events. In the case of moduloTransfer, the filtering logic is very simple - it only copies out an event if that event's sequence ID modulus some value (set by the parameter "modulus") is zero. To see all this in action, after installing and building artdaq-demo using quick-start.sh off of the fd668fed0ae21be81688442af26df19f1802ec25 commit on the feature/dispatcher branch, and running the 2x2x2 demo as normal, from another terminal, after setting up the artdaq-demo environment, run "art -c fcl/TransferInput.fcl" out of build_artdaq-demo - what will happen is that this art process will only receive events with even-numbered sequence IDs, since "modulus" is set to "2" in TransferInput.fcl .
#2 Updated by John Freeman about 4 years ago
- Status changed from New to Resolved
- % Done changed from 60 to 100
With artdaq commit ecce15b4f1428b8ef5132b8c02e831f4820beb9b, installed when running the quick-start.sh script off of artdaq-demo commit 18133ee52967baa8c52b6ce3ea8161b989e37cdf, support for art monitors requesting filtered events is complete. The following changes since the September 1 entry on this Issue apply:
- Some simple instructions on how to create, destroy and configure a monitor is provided at https://cdcvs.fnal.gov/redmine/projects/artdaq-demo/wiki/Dispatcher_Instructions .
- The concept of a monitor being identified by a unique label and being something that isn't just registered with the dispatcher but also unregistered has been added. Previously, you could launch a pair of identical monitors in two terminals using the same FHiCL document, and the dispatcher would end up with two essentially identical transfer plugins with which it sent out events to the art processes. This would created collisions in the event of shared memory, and in the case of multicast, both monitors would receive duplicate copies of events. Now, when "Ctrl-C" is hit in an art process which serves as an artdaq monitor, the monitor is unregistered with the dispatcher. If an attempt is made to register a monitor with a label which duplicates an already-registered monitor's label, the attempt fails.
- "moduloTransfer" has been renamed "NthEventTransfer", and its primary parameter renamed from "modulus" to "nth"
#3 Updated by John Freeman about 4 years ago
Further testing revealed a bug which related to the is_dispatcher_ boolean not being treated on equal footing with the is_online_monitor_ boolean in AggregatorCore. That's been fixed; the new artdaq commit, at the head of its develop branch, is 5c92d89171ac70d13e371154ecc52cf5cba11206, also used by the artdaq-demo installation scripts at the head of artdaq-demo's develop branch.