Wiki » History » Version 217

« Previous - Version 217/370 (diff) - Next » - Current version
Michael Mooney, 10/15/2015 11:34 PM

Before you start your shift

  • Make sure you have a current Fermilab ID, Fermilab Kerberos account, and Fermilab Services account (and go to "Request New/Renew Accounts" or call 630-840-2345 - 24 hours a day).
  • Ask a control room expert or the Run Coordinator to add your Kerberos principal to the .k5login files for the uboonedaq and ubooneshift users on uboone-cr-01, uboone-cr-02, uboone-cr-03, and uboone-cr04.

Shift Procedure

Review Operations Wiki Updates page

  • Changes in shift procedure and important notices for shifters are listed by date in the Operations Wiki Updates page. Please review this list at the beginning of every shift and contact the relevant subsystem expert or Run Coordinator if you are not clear on new procedures.
  • Be sure to reread parts of the Operations Wiki that have been updated (or are new) since the last shift you took. The Operations Wiki Updates page should make it very simple to determine which material should be reviewed.
  • If your last shift was before October 14th, 2015 (the date the Operations Wiki Update page was created), please read the entire Operations Wiki. Ideally this should be done before the beginning of your shift.

Log on to the control room computers

  • On the mac, the user is ubooneshift, and the password is the same as the MicroBooNE DocDB password.
  • On the Linux boxes, there are two cases (You must be added to the .k5login file. Contact us to help):
    • Case 1. The screen has been locked. In this case enter your Kerberos password, followed by your Keberos principal (user name).
    • Case 2. You have been logged out. In this case first enter the user as "ubooneshift", then enter your Kerberos password, and finally enter your Keberos principal (user name).

REFRESH ALL OPEN BROWSERS and sign in to E-log, Google+

  • Browsers open with instructions may have been changed! Make sure you have the latest instructions.

Check for special run plans, else follow the routine shift procedure

  • At the beginning of the shift call MCR (x3721) and let them know that the new MicroBooNE shifters are in ROC West.
  • At the beginning of shift, fill out the Beginning of Shift form on the E-log, and enter it under topic "Shift/Beginning of shift".
  • During shift:
    • See Run Plan.
    • Execute the noisy channel analysis script (see Getting the list of noisy channels) and post the list of very noisy and very quiet channels to the E-log under topic "Shift".
    • Look at the Online Monitor plots for the run you just took. See the prototype checklist here: OM - Checklist
    • Monitor the slow control display. Check for alarming or tripping Wiener power supplies (see here for instructions). Call the DAQ expert on call if this happens. Monitor the pressure in the cryostat. There is a saved plot "filling.plt". If you see the pressure rise to 15 psig and no corresponding e-log entry from the cryogenics team that they are working on the issue immediately page the cryo on-call expert (630-255-1324) and call the run coordinator. If you do not receive a response within 10 min, call cryogenics experts directly. Check the slow control for recent alarms (see instructions).
    • Monitor data processing, a.k.a. PUBS. Make sure PUBS is running and that the computers on which PUBS is running are not overloaded. See the specific instructions at DM - Shifters.
    • Please promptly update the Run Information Document with run information as the runs are taken during shift.
      • This document includes ASIC configuration settings as well as information about activity at LArTF during the collection of data. Please continue to update this document during the commissioning process as it will be very helpful in helping us diagnose noise-related concerns. Please update this document during your shift and copy your updates into an E-log entry at the end of your shift.
      • Also note that noting operations-related items in the Run Information Document or making a note in the shifter Google chat window is not a replacement for an E-log entry. Any operations-related items of interest must be noted in the E-log, both promptly and with specific information (such as which alarms went into a bad state).
    • Shifters need to notify Run Coordinator when experts are going to work in LArTF and also when the shifters are asked to help the experts in LArTF.
      • Shifters must also fill out an E-log entry using the Start of Detector Access form, and fill out the End of Detector Access form when the work is complete.
      • LArTF access procedures for shifts
  • At the end of shift, fill out the End of Shift form on the E-log, and enter it under topic "Shift/End of shift". Communicate with the incoming shifters (who should arrive at least 10 minutes before the end of the shift) recent events, things they should be monitoring, and changes in the Run Plan communicated to you by the Run Coordinator or DAQ experts.

Run Control

Slow control

  • SLC - Shifters: SHIFTER INSTRUCTIONS (Shifters are strongly instructed to ONLY follow instructions in this link. Ignore any other instructions you might see in the slow monitoring tabs below, you are welcome to read them for informative purposes ONLY!)
  • SLC - Guide: What should I look at? How should it look like?

Online Monitoring

Data Management and PUBS

Laser System

Light Collection/PMT System

Beam Monitoring

Raspberry Pi Webcam

Cathode High Voltage

Readout Electronics

Time Projection Chamber (TPC)

  • TPC-Overview: This is a shifter guide to what are the relevant systems associated with the TPC
  • TPC-Guide: Here is a step-by-step guide to monitoring things related to the operation of the TPC
  • TPC-Troubleshooting: Quick guide to previously experienced problems with the TPC (When in doubt call the TPC expert)

Purity Monitors


Example System

  • Example - Overview: This is basically what should the shifter be looking at (not necessarily touching), try to be explicit about what give what information about what to the shifter.
  • Example - Guide: How should the shifter navigate through the different screens to retrieve the necessary information
  • Example - Experts Only: This is where the "Expert Documentation" should live, but only for the time being, we will want to move this behind a password wall. For the time being please start to build the documentation here now but be mindful that it is public viewable.

Expert call list

Useful Links

Expert actions

  • k5login Adding shifters to the .k5login
  • Keytab: Automatically renew the kerbero tickets on evb and control room machines.
  • MicroBooNE spares: Follow this link to see a list of MicroBooNE spares, location, and date last checked.