Off-Site Procedures¶

Before familiarizing yourself with the procedures please read the Off-Site Shifting Requirements.

• Table of contents
• Off-Site Procedures
  • Off-Site Shifting Procedure
    • Starting your Off-Site Shift
    • During your Off-Site Shift
  • Start-up Guide
    • Establish Connection to FNAL
    • Start-up of the Run Control VNC
    • Start-up the Slow Monitoring/Control VNC
    • Start-up PUBS Monitoring GUI
    • Open SSH Tunnel for Web Interfaces
    • Open Web Pages
    • Communication with ROC West
    • Get ready for Shift
  • Quick Start-up Guide

Off-Site Shifting Procedure¶

In general an off-site shifter is also obliged to follow the procedures described on the Wiki main page. Still there are some special procedures an off-site shifters need to follow in addition.

Starting your Off-Site Shift¶

First of all arrange enough time to start up your off-site shift system. Under normal conditions it takes about 15 minutes to set everything up and you also want to have an overlap with previous shifters.

• Follow the start-up or quick start-up guide below.

• Call ROC West using Vidyo, Skype or your landline.

• Exchange information with previous shifters.

• Even if the shifter in ROC West called MCR already you need to do so too (use your landline). Inform them that you are off-site and leave them your number.

If you run into problems while setting up your system, contact your co-shifter in ROC West and inform him about your problem. Maybe they even have the same issues. If you have severe connection problems inform the Run Coordinator.

During your Off-Site Shift¶

Although the shifters are separated by thousands of miles it is very important that the communication between the shifters works out well. Therefore try to stick to these rules

• Off-Site shifters and ROC West shifters need to maintain at least one mean of communication.

• Communicate all steps you are taking while facing alarms or unexpected detector behavior.

• While troubleshooting, assign tasks, e.g. one shifter calls the expert the other writes an elog entry

• Inform each other if you need to step out for a short time.

Start-up Guide¶

This guide lists all steps necessary for starting up your off-site remote shift system. All steps in this guide might be subject to changes If steps in this guide do not work, please look up the information on the Wiki main page.

Establish Connection to FNAL¶

1. Start the VPN client. For this you need to use your FNAL services account credentials.
2. Get a Kerberos Ticket by using your FNAL computing account (e.g. kinit <username>@FNAL.GOV)

It is very important to connect by VPN before any SSH tunnel is opened. If you do it later, you will end up having broken pipes and VNC freezes after some time.

Start-up of the Run Control VNC¶

Before you start up the run control VNC for the first time please check out RC - Guide.

Open a terminal/shell

ssh -L 5903:localhost:5903 uboonedaq@ubdaq-prod-ws01.fnal.gov

and on ws01, do

ssh -L 5903:localhost:5903 ubdaq-prod-evb.fnal.gov

now, connect your VNC client to localhost:5903. In case you are using RealVNC open a new terminal and start the VNC session locally on your machine with the command

vncviewer localhost:5903

Enter standard DAQ password and wait for the VNC to appear.

If you are facing any problems read RC - Guide and RC - Troubleshooting before calling the expert.

Start-up the Slow Monitoring/Control VNC¶

Also here, if you are operating Slow Monitoring/Control for the first time make sure you read SLC - Guide.

Open a terminal/shell

ssh -K -L 5902:localhost:5902 ubooneshift@ubdaq-prod-ws01

now, connect your VNC client to localhost:5902. In case you are using RealVNC open a new terminal and start the VNC session locally on your machine with the command

vncviewer localhost:5902

Enter the standard uBooNE password+smc

If you face any issues while starting up or during operations check SLC - Guide and SLC - Troubleshooting before calling the expert.

Start-up PUBS Monitoring GUI¶

For more information about PUBS read DM - Overview and DM - Shifters

Open a terminal/shell

ssh -X uboonedaq@ubdaq-prod-ws01.fnal.gov

while on ws01 do

ssh -X uboonedaq@ubdaq-prod-evb.fnal.gov

on this machine execute the following lines

cd pubs
source config/setup.sh
setup pyqtgraph
python $PUB_TOP_DIR/pub_mongui/mongui.py

When problems occur follow DM - Troubleshooting before calling an expert.

Open SSH Tunnel for Web Interfaces¶

In order to view the Slow Monitoring Lizard, Ganglia and the PUBS status pages one needs to open the following SSH tunnel. This can be done in a separate Terminal by

ssh -D 8881 uboonedaq@ubdaq-prod-ws01.fnal.gov

Now you are ready to start-up the web pages.

Open Web Pages¶

The following web pages need to be open during the whole shift.

In the Opera browser open

• uBooNE Online Monitor Lizard

• PUBS Resource Monitoring

• MicroBooNE Trigger Page

• MicroBooNE Run Summary

• MicroBooNE Run Statistics

• Weather Map

In any browser of your choice open

• Ganglia Shifer View

• MiniBooNE Monitor

• BNB Status Display

• NuMi Status Display

• Accelerator Division Notification Page

• Slow Monitoring Archiver Status

• MicroBooNE Run Information Document

• MicroBooNE Alarm Page

• MicroBooNE Operations Wiki

• MicroBooNE Run Plan

• MicroBooNE Expert Call List

• Google+ Hangout

• MicroBooNE Elog

If you face any problems with the online monitoring Lizard look into OM - Checklist, OM - Guide and OM - Troubleshooting.

When you have trouble with the BNB Beam monitor checkout Beam - Guide.

Communication with ROC West¶

Communication with ROC West is an important part of your off-site shift, therefore:

• Connect to MicroBooNE's CERN Vidyo room using the instructions in Off-Site System Setup.

• Login to Google+ and join the MicroBooNE shift hangout.

If the Vidyo connection fails, connect to Skype using your local shifting account and call ubooneshift. If both fail, use your landline to communicate with ROC West. Also inform ROC West if you have trouble with your off-site system.

Get ready for Shift¶

If you successfully performed all of the above steps, you are almost ready for shift. Even if your co-shifter already called MCR you have to do so too. Call MCR using your landline, give inform them about the off-site shift and leave them your phone number.

Now, you are ready for shift!

Quick Start-up Guide¶

Open a terminal/shell

ssh -L 5903:localhost:5903 -L 5902:localhost:5902 -D 8881 uboonedaq@ubdaq-prod-ws01.fnal.gov

and on ws01, do

ssh -L 5903:localhost:5903 ubdaq-prod-evb.fnal.gov

Now connect yout VNC client to localhost:5903 and localhost:5902. After that follow the steps to start up PUBS above and open the all the needed pages with your browser(s).