Bug #6795

MC7 Stalled wire chamber read out

Added by Geoff Savage over 6 years ago. Updated over 6 years ago.

Start date:
Due date:
% Done:


Estimated time:


Email from Jason St. John titled "MC7 support stuff".

Hey Geoff,

The second one is much more you: The will run for a bit, then it chokes with an error like the one below, and exits. Any idea what we messed up? Out of memory? This thing was really useful!


Traceback (most recent call last):
File "./", line 20, in <module> # Application loops here collecting data.
File "/fnal/products/ftbfwirechamberdaq/v01-00/py/", line 129, in run
if c.checkForNewData():
File "/fnal/products/ftbfwirechamberdaq/v01-00/py/", line 777, in checkForNewData
File "/fnal/products/ftbfwirechamberdaq/v01-00/py/", line 788, in checkForReadoutError
raise RuntimeError('Wire chamber readout has stalled.')
RuntimeError: Wire chamber readout has stalled.


#1 Updated by Geoff Savage over 6 years ago

Methods from

def checkForNewData(self):
bit = self.readoutDoneBit
self.readoutDoneBit = self.tdcReadoutDone()
self.readoutBusyCount= self.readoutBusyCount + self.tdcReadoutBusy()
if self.readoutDoneBit - bit > 0:
return 1
return 0
def checkForReadoutError(self):
if self.readoutBusyCount > 10:
raise RuntimeError('Wire chamber readout has stalled.')

self.tdcReadoutDone() - check bit 1 in spill state register (addr=0xB)
self.tdcReadoutBusy() - check bit 0 in spill state register (addr=0xB)

#2 Updated by Geoff Savage over 6 years ago

Bill Badgett readout the wire chambers with his daq code yesterday (8/13) afternoon. Here are his results.

"The Lariat readout worked OK for a few spills; switching to the python readout, the first spill (~4000 triggers) it crashed. Subsequent few spills with lower triggers (~1000) have not crashed."

It looks like ~10 seconds is not long enough to transfer all the TDC data to the controllers when the spill ends.

#3 Updated by Geoff Savage over 6 years ago

Here is the logic Bill uses in his DAQ code to see when the wire chamber data is ready in the controller.

  bool tdcBusy = true;
  bool tdcDone = false;
  bool eotComplete = false;
  int ctr = 0;
  while ( !tdcDone && !eotComplete && tdcBusy && ( ctr<1000))
    uint16_t data = read(SPILL_STATE_ADDR);
    tdcBusy = ( data >> TDC_READOUT_BUSY_S ) & 0x1;
    tdcDone = ( data >> TDC_READOUT_DONE_S ) & 0x1;
    eotComplete = ( data >> EOT_COMPLETE_S ) & 0x1;

This is called after the end-of-spill flag is found to go from 1 to 0.

I'm relying on the network latency of the read() call to be about one millsecond (OOM) so the timeout would be on the order of second(s). I have seen timeouts in the past, but I don't remember the circumstances.

Also available in: Atom PDF