Run plan » History » Version 1383

« Previous - Version 1383/1457 (diff) - Next » - Current version
Ralitsa Sharankova, 10/18/2019 05:45 PM

Run Plan - last updated: (check history)

All shifters are required to be very familiar with this page.
Do not hesitate to contact the Run Cos with questions.


Contact Number Alt Number
Shifter 1 (937) 582-6663 1(937) 5-UBOONE
Backup Shifter 1 (978) 822-2582 9788 BACK UB
ON-CALL: RunCo (Zarko) (630) 276-9061
ON-CALL: Deputy RunCo (Ralitsa) (617) 230-2978

Refresh this page Now. Click here: Run plan

Link to which experts to call in case of other alarms

Current Activities

  • A valve on a liquid nitrogen line at LArTF developed a leak and there are issues with oxygen monitors. The work is in progress to have the valve replaced.
    • The drift HV and PMTs are off.
    • The DAQ is not running.
    • Instead of the usual checklist, Cryo recovery checklist should be filled every 2 hours.
    • At the end of shift save a plot of the LArLevel_cryostat_dewar and N2Analyzer_O2Analyzer tabs in SlowMon for last 8hrs and post it in elog
  • Impedance monitor uB_ZMON_ZMON_1/short_ok is oscillating in and out of major alarm. Operations team is aware of the problem and working with experts to resolve it. No need to call or email the expert about it.
  • Shifters are requested to contact by email CRT expert Elena Garmellini for any CRT crate rail alarm and for LOLO DAQ alarms.
  • Beam is off for the summer shutdown as of 8 AM Saturday, July 6.
    ** We will continue taking cosmic data during this time. Please check that the DAQ is working and that we are collecting data even while the beam is off.
    • There are several alarms that occur when then beam is off. These can be acknowledged. You do not need to notify the run-cos or experts. Please see Running while beam is off for a list of these alarms.
      • A "big red box" may appear in the online monitor related to RWM Timing. This is also to be expected while beam is off.
    • Some work is planned during this time. The run coordinators will notify the shifters when the work is beginning.
    • If you are on shift when the beam goes on or off, please elog it. Otherwise please continue taking data and monitoring the detector normally.
  • If you see a uB_DAQStatus_DAQX/sn_read_lag_multiplicity alarm:
  • If you are local and you notice the yellow computer slowing down (cr-02):
    1. Check the "about:performance" tab in the browser. (If the tab is not open, just type "about:performance" in the address line of a new tab). See (and make note of) if any of the web apps are performing poorly.
    2. Close and restart any pages that are performing poorly, or just close and reopen all Firefox windows.
    3. Email or Slack message (don't call) the run coordinator and tell them which app was causing the slow down.

Run Control

Configuration Config ID Time [min]
BeamOff_EXT1Hz 815 420

Notes on current operations

  • General procedure: some troubleshooting instructions advocate to stop and restart the run to fix a problem. It is good to do so only once. If stopping and restarting didn't work at the first try, please get in touch with the relevant experts (or the Run Co) and wait for their clearance before stopping and restarting again.
  • Before resetting the PUBS, please ensure there are no PUBS errors being displayed - if there are, please report this to the Data Management expert (via email) before resetting.
  • If you make an elog entry about online monitor or runcat checks failing, upload a screenshot of the plot with the entry.

Additional Shifter Requirements:

  • Please inform the Run Coordinator(s) if
    • you see any DAQ instabilities, or
    • the mean run duration gets smaller than 2h, or
    • you receive any information from MCR about the beam, or
    • there is any mismatch between beam triggers and beam spills (dots and lines of the same color in the nearline trigger monitor).
  • Please do not forget to check your runs in the RunCat.
  • Please fill out the Offline Production Checklist at 9am and 9pm every day.
    There are instruction on how to fill this out on this page:
    • Please note that the offline production checklist has been modified. The content is essentially the same, but if you've done it before, it will look different now. Also, now you email the checklist itself to the offline production listserv rather than composing a separate email.
  • During access to the detector:
    • Make sure the Run Co's were informed of the access (Slack chat or call).
    • Detector platform access must be cleared by Run Co's.
    • Please fill out a Beginning of Detector Access and End of Detector Access Form.
    • If the "keytree" alarm occurs, and the persons accessing the platform have not called or do not call the shifter within ~5 minutes, then call LArTF to find out who is accessing the detector. If no one answers, call the runco.
    • Please remind to the people making the access to call you back when they are done with their work in the pit.
    • At the end of detector access, please acknowledge the keybank incomplete alarm.
  • If you happen to see a pretty event in the event display, take a screenshot!

What should I do if I see alarms during my shift?

Have a look at this wiki page: What to do in case of alarms during a shift

Expert contact information: