Feature #13357

Create a plugin allowing users a choice in the method by which data is sent out from the diskwriting aggregator for monitoring purposes

Added by John Freeman over 4 years ago. Updated over 4 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
50.00 h
Duration: 75


Traditionally, monitoring in artdaq has been done by having the diskwriting aggregator push events into a shared memory buffer, and the monitoring aggregator read events from the buffer. In light of a discussion held May-14-2016 ( it was decided that greater flexibility in monitoring was desirable; in particular, the desire to have monitoring processes running on hosts different than the host running the diskwriting aggregator meant that users would need the option to select methods other than shared memory by which to have the diskwriting aggregator send out data. For this reason, a new type of plugin called Transfer was created; while an implementation of Transfer using shared memory (shmemTransfer) effectively reproduces the previous functionality of artdaq, other implementations using RTI-DDS and multicast currently exist in artdaq as well. Available on the head of the develop branch as of this writing (commit 3a62fc5fedc9eb63e271fdafc5721f29139fd567), the interface which needs to be implemented by a Transfer plugin can be found in artdaq/TransferPlugins/TransferInterface.h :

  virtual size_t receiveFragmentFrom(artdaq::Fragment& fragment,
                                     size_t receiveTimeout) = 0;

  virtual void copyFragmentTo(bool& fragmentHasBeenCopied,
                              bool& esrHasBeenCopied,
                              bool& eodHasBeenCopied,
                              artdaq::Fragment& fragment,
                              size_t send_timeout_usec = std::numeric_limits<size_t>::max()) = 0;

where these functions have (almost) the same signatures as the shared-memory specific functions found in traditional versions of artdaq, artdaq::AggregatorCore::receiveFragmentFromSharedMemory_ and artdaq::AggregatorCore::copyFragmentToSharedMemory_ - the difference being that the timeout parameter for the copy function can default to a "very large number". AggregatorCore now loads in a FHiCL-defined instance of a Transfer plugin and uses it to copy and receive events; in particular, running for artdaq-demo at the current head of its develop branch (commit 065de77bb610135e1bb40969aa00616429fa0319) with the --run-demo option will run a 2x2x2 system in which the shmemTransfer plugin is used to mediate between the diskwriting aggregator and online monitoring aggregator.

For developers wishing to create new implementations of the Transfer plugin, the executables "transfer_plugin_sender" and "transfer_plugin_receiver" have been added to the proto/ directory in artdaq, making it possible to test the Transfer plugin without running a full-blown artdaq system. transfer_plugin_sender will send N artdaq fragments with payloads of M bytes apiece, and transfer_plugin_receiver will check to make sure they've been received correctly. Assuming artdaq is built, then if you have two terminals where in each terminal you've run setup_for_development and are in the build directory, in the first terminal you can type

transfer_plugin_sender fcl/shmem.fcl 100 10000

which will commence the sending of 100 fragments with payloads of 10k bytes each (as well as a 12-byte metadata struct, which aligns to 16 bytes in the fragment) and then in the second terminal you can type
transfer_plugin_receiver fcl/shmem.fcl

if things are working as they should, you'll see messages like
Returned from call to transfer_->receiveFragmentFrom; fragment with seqID == 14, fragID == 0 has size 10040 bytes
Success: do_check indicates fragment transmitted correctly

and if not, along with any errors printed by the Transfer plugin itself, you'll see
Error: do_check indicates fragment failed to transmit correctly

Please note that in the case of the shmem Transfer plugin, you need to launch the sender first so the shared memory buffer is created; if you try launching the receiver first you'll get an error. This means the first few fragments will typically have been sent by the time you start up the receiver- i.e., don't expect that the first fragment on screen has sequence ID 1. The check that's performed is simply that the payload of each fragment sent by transfer_plugin_sender consists of a monotonically increasing set of unsigned 64-bit integers, so this is what transfer_plugin_receiver checks for.

Lastly, for the record, the following were the main commits (preceded by dates) which implemented the Transfer plugin:

5/18 375971d25afb86ed5a4fd847e104c9e08eddd97f
5/20 a12127ee4549a079f65ce09d323d2bd25a0586a4
5/20 f26430ad522ff7331af62b00ed079c71b272a097
5/20 32d22966cd0b7dcbf35297456ea2ea332c0bdcf6
6/9 b8b5b4e260bc168744e4e11a0d80c61645273721
6/9 621bb98c4b2c78ad7a3611b90aacbcfddaf853cc
6/12 14fde3f6b9e72b41b46a73476ef27360ed4a2e8a
6/14 137420b26bcccbe69d0fa7105673ecfa785a41a3
6/14 e2d5159330454d0ab2575af0d179acc65c33f1f1
6/15 eee2b90effb758f6ed4bff7a2468dd3c7be276b0
7/1 7a72f7f4a64fa93cd0e42890df52e370b8ae0bb1
7/2 8e05b4d7388977e6a8142135d91bbcbead48b14c
7/7 d4ec054e0052f2519de014a4b152dfc8383406f1
7/25 3a62fc5fedc9eb63e271fdafc5721f29139fd567


#1 Updated by Eric Flumerfelt over 4 years ago

  • Status changed from Resolved to Closed
  • Target version set to v1_13_01

Also available in: Atom PDF