artdaq

Last September, we discovered that the feature that allows us to group several consecutive events into a bunch in the EventStore that runs in the Aggregator causes noticeable performance issues. (The reason for grouping consecutive events together is to allow online monitoring to do event-to-event analyses.)

Here are some notes from September:

email from me

I was able to improve system performance by configuring the Aggregator so that it does not group N>1 events together before passing them to art. Some performance results are shown in the following table - these numbers have large error bars, but hopefully they show that an improvement has been made.

These results were achieved with a system with 6 boardreaders (5 1720s, one 1495), 16 eventbuilders, and two aggregators. Disk writing and online monitoring were turned off in these tests.

I haven't looked into why the grouping of N>1 events could be slowing things down, but I see a call to the sort function on line 110 of NetMonTransportService_service.cc that might be a place to start looking.

email from Mike Wang on the subject of reproducing the problem at the FNAL WH14 teststand

Using -g 600 -p 2000, here are the event rates for the first six samples printed to the console for a grouping of 4 and 1:

grouping = 4 (events/sec)
26.0122
25.6448
27.8478
29.3477
31.9142
30.831

grouping = 1 (events/sec)
33.5078
34.1949
34.477
34.864
34.9473
34.7972