Project

General

Profile

Power Outage Planning » History » Version 7

David Caratelli, 08/14/2018 01:50 PM

1 1 David Caratelli
h1. Power Outage Planning
2 1 David Caratelli
3 1 David Caratelli
h2. Prep work:
4 1 David Caratelli
5 3 David Caratelli
* System experts need passwords: 
6 3 David Caratelli
> # PMT: Slow Controls HV power-supply control.
7 3 David Caratelli
> # TPC: glassman connection.
8 1 David Caratelli
9 1 David Caratelli
* We need a DAQ configuration that configures the ASICs. Examples from the past is config 768. Take data for any amount of time. Once run starts ASICs are configured. Make sure run does not re-start or you will re-configure the ASICs.
10 1 David Caratelli
11 1 David Caratelli
* If you want to take data during ramp-up or ramp-down and the PMTs are off you will need a config which has an unbiased trigger and goes to the TestRun folder.
12 1 David Caratelli
13 2 David Caratelli
A power-outage schedule to guide planning can be found below. Outages generally start early in the morning, so this guide assumes the power-down procedure will take place the day before. In the plan below
14 2 David Caratelli
15 2 David Caratelli
Guidelines:
16 2 David Caratelli
17 2 David Caratelli
* The cryo system should be down for as little time as possible to prevent buildup of impurities and a degradation of our electron lifetime. 
18 2 David Caratelli
19 2 David Caratelli
* The ASIC LV should be powered up as soon as this is possible. The ASIC LV powers the diodes which protect our electronics, hence the urgency.
20 2 David Caratelli
21 4 David Caratelli
* Rack power-up/down instructions are on the [[Rack Power ONOFF procedures]] page.
22 4 David Caratelli
23 2 David Caratelli
* Subsystem dependencies: 
24 3 David Caratelli
> # TPC HV, ASIC LV, and wire-bias HV should come up only after the Cryo system is on. 
25 3 David Caratelli
> # Data taking in BeamOff or Physics mode requires PMTs to be on. This is because otherwise PUBS will not be able to process incoming files.
26 4 David Caratelli
27 4 David Caratelli
Day before outage:
28 4 David Caratelli
29 4 David Caratelli
# 14:25 Stop data-taking - shifter (5 min.)
30 4 David Caratelli
# 14:30 Turn off PUBS - DM expert (30 min.)
31 4 David Caratelli
# 14:30 Turn off CRT remotely - CRT expert (30 min.)
32 4 David Caratelli
# 14:30 power down PMT HV - TPC/HV expert (30 min.)
33 4 David Caratelli
# 15:00 ramp down TPC HV - TPC/HV expert (2-3 hours)
34 4 David Caratelli
# 17:00 ramp down wire-bias - TPC/HV expert (1 hour)
35 4 David Caratelli
# 18:00 power off ASIC LV - TPC/HV expert (30 min.)
36 4 David Caratelli
# 18:30 power down racks - RunCo + RO expert. CRT/MuCS expert also may be necessary. (1 hour.)
37 4 David Caratelli
> * Make sure data-taking has stopped before powering off any TPC/PMT rack.
38 4 David Caratelli
> * Make sure PMTs are powered off before touching the PMT-Trig crate. Check explicitly with PMT expert via phone.
39 4 David Caratelli
> * Make sure the TPC wire-bias and ASIC LV are powered off before powering down the TPC-PS rack. Check explicitly with the TPC-HV expert via phone.
40 4 David Caratelli
> * Make sure the TPC HV is powered down before turning off the TPC HV racks. Check explicitly with the TPC-HV expert via phone.
41 4 David Caratelli
42 4 David Caratelli
Day of outage, before:
43 4 David Caratelli
44 4 David Caratelli
# T-30 min. Turn off slow-controls. - SMC expert (10 min.)
45 4 David Caratelli
# T-20 min. Turn off non-evb machines - SLAM team (5 min.)
46 4 David Caratelli
# T-10 min. Turn off evb - SLAM team (5 min.)
47 4 David Caratelli
# T-5 min. Power down cryo - Cryo engineers, remotely (5 min.)
48 4 David Caratelli
49 4 David Caratelli
Day of / after outage, power-up:
50 4 David Caratelli
51 4 David Caratelli
# T+5 min. Power up cryo - Cryo engineers (5 min.)
52 5 David Caratelli
# T+5 min. Turn on slow-controls 
53 7 David Caratelli
     a. Make sure Keythleys in TPC-HV rack are configured. See e-log 77738: http://dbweb5.fnal.gov:8080/ECL/uboone/E/show?e=77738.
54 4 David Caratelli
# T+5 min. Power-up racks in LArTF - RunCo + RO expert (1 hour)
55 4 David Caratelli
# T+5 min. Power-on machines @ LArTF - SLAM team (5 min.)