Update to contact information:
When trying to contact the shifter, please use: 1 (937) 582-6663 (or, 1(937) 5-UBOONE)
When trying to contact the run coordinator, please use: 1 (607) 862-6820 (60 RUNCO UB 0)
When trying to contact the backup shifter, please use: 1 (978) 822-2582 (9788 BACK UB)
If the above numbers don't work to contact the person you need, you can resort to the call list as per usual.
Shifter: Please post a plot of the cryostat pressure (uB_Cryo_IFIX_1_0/PT102) for the past 8 hours at the end of your shift in the e-log. Please e-log any
correlations of cryostat pressure blip with pickoff point voltage blip (and email to Jen Raaf). If pickoff point voltage blip exceeds threshold (generating a minor alarm, and just after double check that there is cryostat pressure blip at the same time of the pickoff point voltage blip), please phone to Jen Raaf. For a detailed example please see e-log entry 47871.
Please follow this procedure for the shift turnover: Shift_Turnover
If there are any blips, acknowledge the alarm and make an elog entry with a picture of the blip. Include the pickoff point in the plot, please. The blip alarm is latching, so if you don't acknowledge it, we will miss the next one.
Run config 558 for 420 minutes. If there are instabilities, call the DAQ expert immediately.
*BNB RWM Timing (Update) *
The BNB RWM timing plot has shifted ( instead of bin 335, 336 now the plot will have occupancy in bins 337 and 338) due to recent work of Zarko and Mike Backfish in the electronics (they have replaced a discriminator due to the RWM started flaking out couple of days ago, more details on MCR April 11th 2017).
Beam Conduit (Important)
We have not observed issues regarding the beam conduit damage, but it is still important to monitor. Shifters please pay close attention to Zarko's trigger rate plot (http://www-microboone.fnal.gov/at_work/beam/performance/trigger_nearline.html), as well as the beam plot in the OM (the RWM timing plot, "beam" -> "h_bnb_rwm_timing"). Please call the Runco and email Zarko, David C, Andy F and firstname.lastname@example.org immediately if you see anything abnormal.
The MiniBooNE checklist is having trouble updating due to a new firewall in the MiniBooNE system. Experts are working on it, so if you see the checklist not updating this is why. No need to email the experts - they know already.
We are having some DAQ issues. The symptom is a bunch of DAQStatus major alarms, such as the SEB0X circular buffer occupancy and evb/Builder_TimeDifference_GPS_Local would go off. If you see these alarms, please restart the run immediately. It may require several restarts to make the alarms go away. Please call the DAQ experts when this happens, and make an elog including a screen shot of the trigger rate plot.
Unless approved by the run coordinator, access to the platform while the beam is running if prohibited.
Currents on Feedthroughs
- The feedthrough currents recently had an increase in the monitoring resolution. If you autoscale, don't be alarmed to see variations on a very small scale. Mostly, don't be concerned if there are fluctuations that are below 2 microamps. On some feedthroughs, this is higher - contact the experts if you get an alarm on the feedthrough currents, but do not panic if the plots show variability.
- Run Cat checks are part of the shift requirement. Please do so whenever a run is finished. This window is typically open in the yellow screened desktop, and you can follow the instructions inside the runcat. If you have questions, please email the run coordinators.
IMPORTANT: Current procedure in case of cryo alarms (Starting from Dec. 6th 2016)¶
In the case that iFIX overall cyro alarm is not Green, Cryo experts should be notified by auto dialer from iFIX directly and they will call shifters that they are aware of the problem and troubleshooting is ongoing. If the shifter doesn't get the phone call within 15 mins after seeing the non-Green cryo alarms from iFix, please call the cryo experts with following orders. The next person in the list should only be called if you can't reach the previous person after 5 mins.
Cryo experts calling order: Fritz Schwartz, Michael Zuckerbrot, Michael Geynisman. You can find the experts' number here http://www-microboone.fnal.gov/at_work/expert_call_list.html
For cryo related major alarms from slow control monitor, if occurs between 8am to 10pm, call run coordinators and cryo experts with above order (again, next cryo expert only need to be called if you can't reach the first person after 5 mins). For cryo related minor alarms from slow control monitor or any alarms outside of 8am-10pm, call runcos and email cryo experts.
All the cryo related alarms from either iFix or slow control should be documented by shifters in ECL with the alarm variable names.
IMPORTANT: Changes to "Start of detector access" form¶
The "Start of detector access" form has been updated. It now includes a check for training for each person accessing below ground level (pit or platform). You must fill out the names of each person accessing (no "+ 1", be specific with first name and at least last initials). After getting their names you will ask for their Fermilab ID numbers and check their training using the Fuzzy tool: http://www-numi.fnal.gov/fuzzy.html. The page will be completely green if they are permitted to access below ground level. If it is not, tell them to contact the Run Coordinator immediately and not to proceed.
Latching Keytree Alarm in Slow Controls (March 29th, 2016)¶
- Please note that there is a "latching" keytree alarm in the slow controls now, which will go into minor alarm when someone removes a key from the keytree at LArTF (with some delay) and will remain in alarm even after the key is replaced unless it is acknowledged. Simply "acknowledging" this alarm will make it go away after all keys are replaced. If you are unaware of what access is being made in conjunction with this alarm, please contact LArTF and find out who is making the entrance (see the building phone numbers posted above the phonebook). Then let the Run Coordinator know immediately of the access (by phone), if it was not cleared beforehand. Then "acknowledge" the alarm and make it disappear.