Data flow indicators stop, but data appears to be flowing
Shifter saw data flow indicators stop, but it appeared to still be flowing from DCMs and buffer nodes and data logger (as far as I could tell from ganglia). From the shifter's description below it looks like the indicators failed, not the DAQ.
5505, Tony Mann (mann), 04/16/2011 08:22:53 General
I stopped the Run (since there was no data flow) and tried to start a new run. dcm's 02-01, 02-02 did not come back on and the TriggerScalars box has stayed at pink (not gone to green.)
I'll repeat the new run start; if that fails I will power-cycle the two dcm's.
Leon Mualem (mualem) 04/16/2011 08:40:15
What was the evidence/indicator that data flow stopped. From Ganglia, it looks like data was still flowing from all DCMs.
5507, Tony Mann (mann), 04/16/2011 08:55:23 General
Answer to Leon's question: Flow on the Raw Data and Event Viewer stopped; the Event Display stopped; and the TriggerScalars stopped.
5506, Tony Mann (mann), 04/16/2011 08:49:27 General
I power-cycled the DCMs and started a new run (11995/0). The DAQ came up normally. Actually, I had Bill Oliver do this under my supervision -- was good training for him.
#1 Updated by Leon Mualem over 8 years ago
- Status changed from New to Closed
On Further inspection of ganglia, it appears that this was a typical corruption crash. All DCMs except dcm-03 started showing corruption, dcm-03 did show a lower data rate, but no time marker corruption until the run stopped.
I'll close this issue.