Project

General

Profile

History

Wiki »

Run Plan - last updated: (check history)

All shifters are required to be very familiar with this page.
Do not hesitate to contact the Run Cos with questions.

Contacts

Contact Number Alt Number
Shifter 1 (937) 582-6663 1(937) 5-UBOONE
Backup Shifter 1 (978) 822-2582 9788 BACK UB
OFF-CALL: RunCo (Sophie) (630) 446-0767
ON-CALL: Deputy RunCo (Zarko) 630-276-9061

Refresh this page Now. Click here: Run plan

Link to which experts to call in case of other alarms

Current Activities

  • Glenn Horton-Smith will be the slow control expert from Saturday August 24-Sunday August 25. (He will start at midnight on Friday night, and continue through the weekend.)
  • Please fill out the "Cryostat Argon Purity" form in the elog every two hours: 12pm, 2pm, 4pm, 6pm, 8pm, 10pm, 12am, 2am, 4am, 6am, 8am, 10am. You can do the run cat part of the checklist using an ongoing run if it is more than 30 minutes.
  • Remote shift stations will be re-certifying their stations with the new procedures that do not require VPN. Please expect to hear from remote shift stations that they are starting a one hour shadow period. Please write an elog when the one hour shadow period begins, and when it ends, based on the information given by the shift station.
    • Instructions from Thomas Strauss on Remote Shift Station Re-Certification:
      • The remote station needs to coordinate with a shifter on duty to ensure that we have a ‘record’ of the qualification for the 1 hour ‘holding’ test to ensure communication works properly. Upon finishing the setup, the regular shifter notes in the ECL ‘Remote Station XYZ Call In starts/stops'. These entries are preferably 60min apart.
      • The other items the REMOTE STATION adds in the ECL is a SECOND dedicated entry (Remote Station Certification Checklist).
        • For the ECL entry you will need the following:
          • Screenshot of the Ifix Display (clearly showing date&time, status)
          • Screenshot of the DAQ vncviewer (clearly showing date&time and open terminal “STATION XYZ" is here)
          • Check of at least one run in the normal run catalog and screenshot of the supernova run catalog (showing the run#)
          • Screenshot of the Slow Monitor with TPC drift variation plot (clearly showing date&time on the plot access)
          • Screenshot of the Online Monitor (clearly showing run#)
          • Screenshot of the Pubs Display (clearly showing date&time)
          • Slack Channel communication utilizing the uboone shifter account (or remote shift station account)
  • MiniBooNE turned off on the morning of Wednesday, July 17. There will be no elogs posted, and the RWM plot will not update.
  • Beam is off for the summer shutdown as of 8 AM Saturday, July 6.
    • We will continue taking cosmic data during this time. Please check that the DAQ is working and that we are collecting data even while the beam is off.
    • There are several alarms that are thrown when then beam is off. These can be acknowledged. You do not need to notify the run-cos or experts. Please see Running while beam is off for a list of these alarms.
      • A "big red box" may appear in the online monitor related to RWM Timing. This is also to be expected while beam is off.
    • Some work is planned during this time. The run coordinators will notify the shifters when the work is beginning.
    • If you are on shift when the beam goes on or off, please elog it. Otherwise please continue taking data and monitoring the detector normally.
  • The uB_PCStatus_PCXX_pdu07/load, uB_PCStatus_PCXX_pdu04/load, uB_Cryo_IFIX_1_0/AT608, and uB_Cryo_IFIX_1_0/AT610 alarms are known issues and can be acknowledged. Please notify the run-co immediately if any other PDU alarms appear.
  • If you see a uB_DAQStatus_DAQX/sn_read_lag_multiplicity alarm:
  • If you are local and you notice the yellow computer slowing down (cr-02):
    1. Check the "about:performance" tab in the browser. (If the tab is not open, just type "about:performance" in the address line of a new tab). See (and make note of) if any of the web apps are performing poorly.
    2. Close and restart any pages that are performing poorly, or just close and reopen all Firefox windows.
    3. Email or Slack message (don't call) the run coordinator and tell them which app was causing the slow down.

Run Control

Configuration Config ID Time [min]
BeamOff_EXT16Hz 813 420

Notes on current operations

  • General procedure: some troubleshooting instructions advocate to stop and restart the run to fix a problem. It is good to do so only once. If stopping and restarting didn't work at the first try, please get in touch with the relevant experts (or the Run Co) and wait for their clearance before stopping and restarting again.
  • Before resetting the PUBS, please ensure there are no PUBS errors being displayed - if there are, please report this to the Data Management expert (via email) before resetting.
  • If you make an elog entry about online monitor or runcat checks failing, upload a screenshot of the plot with the entry.

Additional Shifter Requirements:

  • Two small but important questions were added to the checklists:
    • In the beginning of shift checklist, a reminder to log-in to the RunCat.
    • In the general checklist, a reminder to check the nearline trigger monitor to be sure our triggers match our spills.
  • Please inform the Run Coordinator(s) if
    • you see any DAQ instabilities, or
    • the mean run duration gets smaller than 2h, or
    • you receive any information from MCR about the beam, or
    • there is any mismatch between beam triggers and beam spills (dots and lines of the same color in the nearline trigger monitor).
  • Please do not forget to check your runs in the RunCat.
  • Please fill out the Offline Production Checklist at 9am and 9pm every day.
    There are instruction on how to fill this out on this page: https://cdcvs.fnal.gov/redmine/projects/uboone-operations/wiki/Offline_Production_Checklist_-_Shifters
    • Please note that the offline production checklist has been modified. The content is essentially the same, but if you've done it before, it will look different now. Also, now you email the checklist itself to the offline production listserv rather than composing a separate email.
  • During access to the detector:
    • Make sure the Run Co's were informed of the access (Slack chat or call).
    • Detector platform access must be cleared by Run Co's.
    • Please fill out a Beginning of Detector Access and End of Detector Access Form.
    • If the "keytree" alarm occurs, and the persons accessing the platform have not called or do not call the shifter within ~5 minutes, then call LArTF to find out who is accessing the detector. If no one answers, call the runco.
    • Please remind to the people making the access to call you back when they are done with their work in the pit.
    • At the end of detector access, please acknowledge the keybank incomplete alarm.
  • If you happen to see a pretty event in the event display, take a screenshot!

What should I do if I see alarms during my shift?

Have a look at this wiki page: What to do in case of alarms during a shift