Project

General

Profile

Shift Handbook

Welcome to the NOvA test beam shift wiki! Stop here for all information relating to shifts, what to do on shift, who to call, when to be worried, etc.

NOvA Test Beam

=====================

Please consult the shift bulletin board for up-to-date information on current running conditions and operations:
Test Beam Shift Bulletin Board

=====================

Environment Emergency Procedures

=====================

Important Information

See the Useful information page. Useful numbers:
  • MCR x3721
  • ROC-MCenter: x3726
  • ROC-West-TB: x5413
  • MC7b enclosure: x2135
  • NOvA test beam Zoom: 356-255-7103

Expert contacts see Blue Sheet here: https://nova-docdb.fnal.gov/cgi-bin/private/ShowDocument?docid=37734

Shift Locations

NOvA test beam shifts will be taken either at the MCenter control room (ROC-MCenter) or in ROC-West (ROC-West-TB). Information for each location is below.

Each shift station has eight monitors with two keyboards and mice. In an identical set up to the other NOvA detectors, they run VNC sessions hosted on one of the remote DAQ machines.

ROC-MCenter

The ROC-MCenter control room is located at the Fermilab Test Beam Facility (FTBF). Upon entering the facility, turn left and the room is on the left. The shift station is on the left as you walk in.

Since this is part of FTBF, all persons taking shifts at ROC-MCenter must have valid FTBF training. There are kitchen-like facilities in the other control room at FTBF, MTest, to the right and in front of you as you enter the building. Bathrooms are to the right of MTest control room.

Day shifts and some swing shifts will be taken from ROC-MCenter.

ROC-West-TB

The ROC-West control room is located on WH1W, the first floor of Wilson Hall. NOvA test beam has its own shift station (note: this is different to the regular NOvA shifting space!), which is at the center of the front row as you enter the room. ROC-West is shared by multiple experiments and has the following rules:
  • Public space
  • Keep room clean
  • NuMI beam spill beep must remain audible at all times
  • Entry doors must remain closed at all times

The ROC-West wiki has more information on the facility, including what to do in the event of an emergency.
https://cdcvs.fnal.gov/redmine/projects/remote-operations-centers/wiki

Night shifts, and likely swing shifts, will be taken from ROC-West-TB.

Shift Procedures

Here will you find checklist for things to do when beginning, during, and after your shift. This will be expanded over time...

Beginning Your Shift

Welcome to your exciting shift! When arriving at the control room, do the following:
  • Fill out the shift check-in form in the ECL;
  • Debrief with the previous shifter to understand the current status of operations and complete required knowledge transfer;
  • Call MCR (x3721) to report the start of your shift (give them your location); make it clear you are the NOvA test beam shifter, not just NOvA shifter;
  • IMPORTANT If there was no beam through MCenter before you started your shift, you will need to ask MCR for beam. Instructions are at Requesting MCenter Beam;
  • Ensure you logged into the machines (see Control Room Machines);
  • Ensure all screens are showing the correct VNCs and you can see all required information (see Control Room VNCs);
  • Ensure you are connected to the NOvA test beam Zoom meeting, with video and audio turned on;
  • Read through the Test Beam Shift Bulletin Board.
  • Fill out the Start Shift form in the ECL.

During Your Shift

During your shift, you must do the following:
  • Monitor everything!
  • Ensure both DAQs are running and collecting data;
  • Check the data to ensure it is of good quality;
  • Fill out the shift check form in the ECL every hour; Hint: you can use the snapshot server to capture the images to include on this form or for other entries where screenshots may be relevant: http://novadaq-test-master.fnal.gov:8083/snapshot/ShowImageList.jsp
  • Document everything in the ECL;
  • Coordinate with collaborators via phone and Zoom.

Towards the end of your shift, fill out the shift summary form in the ECL.

Ending Your Shift

Congrats! You have successfully navigated the NOvA test beam experiment to dry lands. But wait, you're not done yet...
  • Say hello to the next shifter (don't leave without the next shifter being present, call run coordinator if no one shows);
  • Debrief with the next shifter so they know what happened during your shift and what to expect during theirs;
  • IMPORTANT If there is no shifter after you, call the MCR and ask to end beam through MCenter and for the magnet current to be turned down (see Requesting MCenter Beam);
  • Fill out the shift check-out form in the ECL.

How to Collect Data

This section contains basic information for operating back DAQs. Review complete instructions at Operating the DAQ.

Detector DAQ

The detector DAQ is controlled and monitoring on VNCs 1 and 2. VNC 1 contains run control and detector monitoring. VNC 2 contains data quality monitoring.

Launch Run Control using the desktop icon on VNC 1.
Use Run Control to configure the DAQ and start data-taking. In general, you should just use the defaults when selecting components to include in the run, the configuration and other details.
Step through the successive transitions (Discover Resources -> Select Resources -> Reserve Resources -> Select Configuration -> Prepare Configuration -> Load Connections -> Make Connections -> Load Hardware Config -> Configure Hardware -> Load Run Config -> Configure Run -> Begin Run).
If the DAQ crashes and you are unable to recover the run, contact the on-call expert.

Beamline DAQ

The beamline DAQ is controlled and monitored on VNC 4. The run control consists of a terminal and message viewer window. Data quality monitoring is also available.

Log into machine novabeamlinedaq00 in a terminal (as user novadaq). To start a run, execute `go'. This will begin the run and launch a message viewer to display log messages during the data taking.
There are plenty of different configurations and this will use the previous way the DAQ was set up.

Things to Monitor on Shift

Data Taking

  • Ensure the DAQs are running and taking data.
  • Monitor the data quality plots to ensure the data are of sufficient quality.

Detector Monitoring

  • Make sure the detectors and hardware (e.g. TDUs, FEBs) are running without problems. Ensure everything is green on the run control GUIs and in the Test Beam Detector Synoptics.
  • Monitor the temperature and field of the magnet.
  • Ensure the dry air and water supplies for the detector are operating correctly. The dryer system dew point ('25') should be significantly below -30 C (typically around -40 C). The water inlet temperature should be around 15 C.

MC7b Environment

  • Monitor the temperature and dew point in the experimental hall. If the temperature gets significantly above 20 C and the dew point at all above 10 C, contact an expert.

M1 Magnet

  • The Magnet Synoptics display shows various readings off of the Test Beam Magnet. Make sure the current reading is stable. Contact an expert if there are significant deviations in the current readout.
  • On occasion, you may be asked by the Run Coordinator to ask MCR to change the magnet current. You will then call MCR (x3721) and tell them to set the magnet "F:MC7AN1" to the value the RunCo told you.

How to Monitor Data

This section will need to be expanded to contain just relevant information needed for the standard shift. For the moment, review instructions at Monitoring the Data.

Running into issues?

Refer to Troubleshooting on Shift to see if you can solve your issue.

Useful Links

Notes for Early Shifts

Thank you so much for taking one of the early NOvA test beam shifts. We are still getting everything set up properly and we appreciate your patience as we figure things out!
A big part of your job is to help work the system and smooth out any problems. Please, please, please update the wiki with any information you think is useful and may be relevant for you or another shifter in the future! And don't worry about calling the run coordinator about things if they're not clear.

At the start of the swing shift, we will need to ask the MCR to turn up the current on the magnet and to send beam through MCenter. This is requested by called MCR (you can request it when checking in for your shift). At the end of the night shift, you will need to ask the MCR to stop sending MCenter beam and to turn the magnet current down. See instructions at Requesting MCenter Beam.