Project

General

Profile

Power Outage Planning » History » Version 12

David Caratelli, 08/16/2018 01:43 PM

1 1 David Caratelli
h1. Power Outage Planning
2 1 David Caratelli
3 1 David Caratelli
h2. Prep work:
4 1 David Caratelli
5 3 David Caratelli
* System experts need passwords: 
6 3 David Caratelli
> # PMT: Slow Controls HV power-supply control.
7 3 David Caratelli
> # TPC: glassman connection.
8 1 David Caratelli
9 1 David Caratelli
* We need a DAQ configuration that configures the ASICs. Examples from the past is config 768. Take data for any amount of time. Once run starts ASICs are configured. Make sure run does not re-start or you will re-configure the ASICs.
10 1 David Caratelli
11 1 David Caratelli
* If you want to take data during ramp-up or ramp-down and the PMTs are off you will need a config which has an unbiased trigger and goes to the TestRun folder.
12 9 Pawel Guzowski
** If it is not in the TestRun folder, it will cause PUBS errors.
13 1 David Caratelli
14 2 David Caratelli
A power-outage schedule to guide planning can be found below. Outages generally start early in the morning, so this guide assumes the power-down procedure will take place the day before. In the plan below
15 2 David Caratelli
16 2 David Caratelli
Guidelines:
17 2 David Caratelli
18 2 David Caratelli
* The cryo system should be down for as little time as possible to prevent buildup of impurities and a degradation of our electron lifetime. 
19 2 David Caratelli
20 2 David Caratelli
* The ASIC LV should be powered up as soon as this is possible. The ASIC LV powers the diodes which protect our electronics, hence the urgency.
21 2 David Caratelli
22 4 David Caratelli
* Rack power-up/down instructions are on the [[Rack Power ONOFF procedures]] page.
23 4 David Caratelli
24 2 David Caratelli
* Subsystem dependencies: 
25 3 David Caratelli
> # TPC HV, ASIC LV, and wire-bias HV should come up only after the Cryo system is on. 
26 3 David Caratelli
> # Data taking in BeamOff or Physics mode requires PMTs to be on. This is because otherwise PUBS will not be able to process incoming files.
27 4 David Caratelli
28 4 David Caratelli
Day before outage:
29 4 David Caratelli
30 4 David Caratelli
# 14:25 Stop data-taking - shifter (5 min.)
31 4 David Caratelli
# 14:30 Turn off PUBS - DM expert (30 min.)
32 4 David Caratelli
# 14:30 Turn off CRT remotely - CRT expert (30 min.)
33 4 David Caratelli
# 14:30 power down PMT HV - TPC/HV expert (30 min.)
34 4 David Caratelli
# 15:00 ramp down TPC HV - TPC/HV expert (2-3 hours)
35 4 David Caratelli
# 17:00 ramp down wire-bias - TPC/HV expert (1 hour)
36 4 David Caratelli
# 18:00 power off ASIC LV - TPC/HV expert (30 min.)
37 4 David Caratelli
# 18:30 power down racks - RunCo + RO expert. CRT/MuCS expert also may be necessary. (1 hour.)
38 8 David Caratelli
 a Make sure data-taking has stopped before powering off any TPC/PMT rack.
39 8 David Caratelli
 b Make sure PMTs are powered off before touching the PMT-Trig crate. Check explicitly with PMT expert via phone.
40 8 David Caratelli
 c Make sure the TPC wire-bias and ASIC LV are powered off before powering down the TPC-PS rack. Check explicitly with the TPC-HV expert via phone.
41 8 David Caratelli
 d Make sure the TPC HV is powered down before turning off the TPC HV racks. Check explicitly with the TPC-HV expert via phone.
42 4 David Caratelli
43 4 David Caratelli
Day of outage, before:
44 4 David Caratelli
45 4 David Caratelli
# T-30 min. Turn off slow-controls. - SMC expert (10 min.)
46 4 David Caratelli
# T-20 min. Turn off non-evb machines - SLAM team (5 min.)
47 4 David Caratelli
# T-10 min. Turn off evb - SLAM team (5 min.)
48 4 David Caratelli
# T-5 min. Power down cryo - Cryo engineers, remotely (5 min.)
49 4 David Caratelli
50 4 David Caratelli
Day of / after outage, power-up:
51 4 David Caratelli
52 12 David Caratelli
# T+5 min. Power up cryo - Cryo engineers (up to 1 hour)
53 8 David Caratelli
 a. Make sure Keythleys in TPC-HV rack are configured. See e-log 77738: http://dbweb5.fnal.gov:8080/ECL/uboone/E/show?e=77738.
54 4 David Caratelli
# T+5 min. Power-up racks in LArTF - RunCo + RO expert (1 hour)
55 1 David Caratelli
# T+5 min. Power-on machines @ LArTF - SLAM team (5 min.)
56 1 David Caratelli
# T+15 min. Power-on slow controls - Slow Controls (30 min.)
57 12 David Caratelli
# T+1 hour. Turn on ASIC LV. - TPC/HV expert (30 min.)
58 12 David Caratelli
# T+1 hour. Turn on PUBS - DM expert (30 min.)
59 10 David Caratelli
# T+1 hour Turn on CRT - CRT expert (30 min.)
60 12 David Caratelli
# T+1:30 hour Turn on wire-bias HV - TPC/HV expert (1 hour)
61 12 David Caratelli
# T+2:30 hours Ramp up TPC HV - TPC/HV expert (2/3 hours)
62 11 David Caratelli
# Start taking data. First run should use config ID that re-configures the ASICs.