NDOS Commissioning Meeting 2011/01/03

Attendance: Jon, Leon, Susan, Ryan, Peter, Kurt, Genaidy, Andrew, Mark Messier, Qiming, Holger, Rick, Denis, Mat, Sue, Alena, Art, Ron

  1. DCS Status & Plans
  2. DAQ/Electronics Update & Plans
  3. Shifters' report
  4. AOB


  • no update, Athans continues his work on coding up the DCM DCS monitor


  • Peter:
    • turned on MB triggers
    • there's a problem with copying the raw data off of the local DAQ cluster
      • we are rsync'ing to BlueArc (new temporary directory, Andrew will send the location out)
      • we are not writing to tape (that will come later)
      • Jarek will be back next week and will work on this, in the meantime, the offline crew can access the raw data in the temporary directory
    • tried some updates to DCMApp & ProgUtils, but still have stability issues
    • both DSO & SDP modes are currently broken; these modes were not tested before the DAQ software was updated... a checklist is needed for when DAQ software is updated. Peter will put this together.
  • Ron & Rick:
    • will continue to beat on the firmware and the DCM data rate
    • need to read FEB registers while data is being taken
    • the missing channels in a DCM is being worked on; some of this issue has been resolved.
  • Kurt is able to create the XML files for DCM hardware configuration at the 90% level.
  • Peter added quite a bit of documentation on the Shift wiki that helped the shift crew through the break. Thanks to Peter, calls to Jon and the DAQ on-call experts were kept to a reasonable level.
  • The TDU rebooted at some point over the break, but the time was never reset. Holger will modify the spill eavesdropper to look for a time difference between the local unix time and the [trigger?] time.
  • The datamon machine has shown signs that there is a memory leak in both the onmon viewer and the evd. The leak rate is ~8 Gb/12 hours. In the short term, shifters will be asked to kill and restart these applications periodically. Mark Messier will do a search and destroy for the memory leak.
  • Jon requested that more documentation be added about the online monitor. Alena has agreed to do that.
  • Sue:
    • noted that the microslice size is large (~700b) on DCM12, and that the CPU is nearly saturated. 700b is about twice what we can handle currently on a DCM.
    • most of the time on DCM12 is spent in system-time.
    • the copy of data from system to user is possibly the cause of this. Ron's code could be used to resolve this performance issue; Sue will look into this today.
  • Ron:
    • noticed that logging in or executing any simple command on a DCM can cause data to be dropped.
    • is there a way to start-up the DCMApp on the DCMs in the real-time scheduler?
    • Sue will work on this.
  • Mark: would like to see a list of non-operational channels. Mat & Denis will make a table and put it in the wiki
  • Rick:
    • an automated procedure now exists to collect I/V curves directly from APDs on the detector. Mat is fully trained in this procedure.
    • is talking with the supervisor at SciDet about cleaning the APDs and optical connector w/ propenol.
  • The Plan:
    • some APDs have oil on them and need to be cleaned. This involves cleaning both the optical connector and the APDs themselves. Seven eboxes have been removed from the detector; Leon does not see a need to remove more until more is known about the ones that have been removed.
    • the DAQ software crew will continue to do the work outline above.

Shifters' report: nothing specific, the shifts over the holiday break went relatively smoothly