Project

General

Profile

Bug #20403

SBND - SHM - artdaq::SharedMemoryManager::GetBufferForReading() skipping/dropping buffers/fragments/events

Added by Ron Rechenmacher 12 months ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Known Issues
Target version:
Start date:
07/23/2018
Due date:
% Done:

0%

Estimated time:
Experiment:
-
Co-Assignees:
Duration:

Description

When looking into an SBND/lariat subrun issue, we noticed missing events. I've seen GetBufferForReading() skip an event.
The issue does not appear to happen when TCP transfers are force.


Related issues

Related to artdaq - Idea #20442: Evaluate the shared memory interplay between "broadcast," "reliable," "overwrite" and "drop" design/functionality (and corresponding fcl config)New2018-07-25

Related to artdaq - Bug #20373: SBND VST - subrun change from fragment generator not workingNew2018-07-19

History

#1 Updated by Ron Rechenmacher 12 months ago

/home/ron/work/artdaqPrj/demo21-om/DAQInterface
mu2edaq10 :^) pidlist="36559 36556 39357 36557 39305"   # PIDs of BR, EB, EBart, DL, DLart
--2018-07-22_16:08:38--
/home/ron/work/artdaqPrj/demo21-om/DAQInterface
mu2edaq10 :^) first=0
--2018-07-22_16:08:38--
/home/ron/work/artdaqPrj/demo21-om/DAQInterface
mu2edaq10 :^) for PID in $pidlist;do
>   if [ $first -eq 0 ];then
>     first=1
>   else
>     echo ======== $PID Read=========
>     tshow\
>     | egrep "[0-9]  *$PID .*ReadFragmentHeader: read active_buffer_=[0-9]* sequence_id=(41|42|43|44)\$"\
>     | tdelta -d 1 -ct 1 | cat
>   fi
>   echo ======== $PID Write=========
>   tshow\
>   | egrep "[0-9]  *$PID .*Sending fragment with seqID=(41|42|43|44)(\$| )"\
>   | tdelta -d 1 -ct 1 | cat
> done
======== 36559 Write=========
4536491 07-22 15:39:37.353500         0 36559 39408  19             SharedMemoryFragmentManager d13 . Sending fragment with seqID=44 using buffer 5
4536816 07-22 15:39:37.353243       257 36559 39408  19               component01_ShmemTransfer d05 . transfer_between_0_and_1_SEND: Sending fragment with seqID=44
4537056 07-22 15:39:37.353085       158 36559 39408  19             SharedMemoryFragmentManager d13 . Sending fragment with seqID=43 using buffer 4
4537104 07-22 15:39:37.353060        25 36559 39408  19               component01_ShmemTransfer d05 . transfer_between_0_and_1_SEND: Sending fragment with seqID=43
4537376 07-22 15:39:37.352905       155 36559 39408  19             SharedMemoryFragmentManager d13 . Sending fragment with seqID=42 using buffer 3
4537431 07-22 15:39:37.352875        30 36559 39408  19               component01_ShmemTransfer d05 . transfer_between_0_and_1_SEND: Sending fragment with seqID=42
4537688 07-22 15:39:37.352680       195 36559 39408  19             SharedMemoryFragmentManager d13 . Sending fragment with seqID=41 using buffer 2
4537741 07-22 15:39:37.352639        41 36559 39408  19               component01_ShmemTransfer d05 . transfer_between_0_and_1_SEND: Sending fragment with seqID=41
======== 36556 Read=========
4525695 07-22 15:39:37.360928         0 36556 39404  23             SharedMemoryFragmentManager d22 . ReadFragmentHeader: read active_buffer_=5 sequence_id=44
4526065 07-22 15:39:37.360708       220 36556 39404  23             SharedMemoryFragmentManager d22 . ReadFragmentHeader: read active_buffer_=4 sequence_id=43
4526405 07-22 15:39:37.360426       282 36556 39404  23             SharedMemoryFragmentManager d22 . ReadFragmentHeader: read active_buffer_=3 sequence_id=42
4526699 07-22 15:39:37.360250       176 36556 39404  23             SharedMemoryFragmentManager d22 . ReadFragmentHeader: read active_buffer_=2 sequence_id=41
======== 36556 Write=========
problem finding column associated with "1" in first line
======== 39357 Read=========
problem finding column associated with "1" in first line
======== 39357 Write=========
4509116 07-22 15:39:37.372705         0 39357 39357   6             SharedMemoryFragmentManager d13 . Sending fragment with seqID=44 using buffer 0
4509124 07-22 15:39:37.372695        10 39357 39357   6                          _ShmemTransfer d05 . transfer_between_1_and_2_SEND: Sending fragment with seqID=44
4509303 07-22 15:39:37.372335       360 39357 39357   6             SharedMemoryFragmentManager d13 . Sending fragment with seqID=43 using buffer 9
4509311 07-22 15:39:37.372326         9 39357 39357   6                          _ShmemTransfer d05 . transfer_between_1_and_2_SEND: Sending fragment with seqID=43
4509472 07-22 15:39:37.372010       316 39357 39357   6             SharedMemoryFragmentManager d13 . Sending fragment with seqID=42 using buffer 8
4509480 07-22 15:39:37.371997        13 39357 39357   6                          _ShmemTransfer d05 . transfer_between_1_and_2_SEND: Sending fragment with seqID=42
4509696 07-22 15:39:37.371652       345 39357 39357   6             SharedMemoryFragmentManager d13 . Sending fragment with seqID=41 using buffer 7
4509732 07-22 15:39:37.371633        19 39357 39357   6                          _ShmemTransfer d05 . transfer_between_1_and_2_SEND: Sending fragment with seqID=41
======== 36557 Read=========
4479856 07-22 15:39:38.003701         0 36557 39352  21             SharedMemoryFragmentManager d22 . ReadFragmentHeader: read active_buffer_=0 sequence_id=44
4493909 07-22 15:39:37.726294    277407 36557 39352  21             SharedMemoryFragmentManager d22 . ReadFragmentHeader: read active_buffer_=8 sequence_id=42
4509644 07-22 15:39:37.371688    354606 36557 39352  21             SharedMemoryFragmentManager d22 . ReadFragmentHeader: read active_buffer_=7 sequence_id=41
======== 36557 Write=========
problem finding column associated with "1" in first line
======== 39305 Read=========
problem finding column associated with "1" in first line
======== 39305 Write=========
4397168 07-22 15:39:38.798920         0 39305 39305   7             SharedMemoryFragmentManager d13 . Sending fragment with seqID=44 using buffer 5
4397227 07-22 15:39:38.798889        31 39305 39305   7                          _ShmemTransfer d05 . transfer_between_2_and_3_SEND: Sending fragment with seqID=44
4398350 07-22 15:39:38.798028       861 39305 39305   7             SharedMemoryFragmentManager d13 . Sending fragment with seqID=42 using buffer 4
4398398 07-22 15:39:38.798003        25 39305 39305   7                          _ShmemTransfer d05 . transfer_between_2_and_3_SEND: Sending fragment with seqID=42
4399583 07-22 15:39:38.797330       673 39305 39305   7             SharedMemoryFragmentManager d13 . Sending fragment with seqID=41 using buffer 3
4399619 07-22 15:39:38.797311        19 39305 39305   7                          _ShmemTransfer d05 . transfer_between_2_and_3_SEND: Sending fragment with seqID=41
--2018-07-22_16:13:37--

#2 Updated by Ron Rechenmacher 12 months ago

  • Subject changed from artdaq::SharedMemoryManager::GetBufferForReading() skipping buffer to SBND - SHM - artdaq::SharedMemoryManager::GetBufferForReading() skipping/dropping buffers/fragments/events

#3 Updated by Ron Rechenmacher 12 months ago

  • Related to Idea #20442: Evaluate the shared memory interplay between "broadcast," "reliable," "overwrite" and "drop" design/functionality (and corresponding fcl config) added

#4 Updated by Ron Rechenmacher 12 months ago

  • Related to Bug #20373: SBND VST - subrun change from fragment generator not working added

#5 Updated by Eric Flumerfelt 10 months ago

  • Category set to Known Issues
  • Status changed from New to Closed
  • Target version set to artdaq_core v3_04_02

We have removed buffer sequence ID checking when in non-broadcast modes.



Also available in: Atom PDF