Project

General

Profile

Using Synoptic for monitoring » History » Version 101

Flor de maria Blaszczyk, 03/31/2017 05:26 AM

1 1 Flor de maria Blaszczyk
{{toc}}
2 1 Flor de maria Blaszczyk
3 42 Jason St.john
*Synoptic plots are interactive:* 
4 42 Jason St.john
Zoom in by selecting a box around the area you're interested in.
5 42 Jason St.john
Zoom out by right-clicking on the plot.
6 42 Jason St.john
7 57 Flor de maria Blaszczyk
%{color:red}"Troubleshooting":https://cdcvs.fnal.gov/redmine/projects/lariat-online/wiki/Using_Synoptic_for_monitoring#Troubleshooting%
8 42 Jason St.john
9 1 Flor de maria Blaszczyk
h1. 1) How to start the Synoptic slow-control monitoring
10 39 Jason St.john
11 44 Flor de maria Blaszczyk
1) Open a console.
12 56 Flor de maria Blaszczyk
2) Run the *LArIATViewer.sh* script. This will get automatically a kerberos ticket with settings permissions and open the beamline overview.
13 44 Flor de maria Blaszczyk
14 56 Flor de maria Blaszczyk
*From ROC West:*
15 55 Flor de maria Blaszczyk
<pre>
16 55 Flor de maria Blaszczyk
cd /home/lariat/acnet
17 1 Flor de maria Blaszczyk
./LArIATViewer.sh
18 55 Flor de maria Blaszczyk
</pre>
19 55 Flor de maria Blaszczyk
20 56 Flor de maria Blaszczyk
21 56 Flor de maria Blaszczyk
*From the MCenter control room:*
22 54 Jason St.john
<pre>
23 54 Jason St.john
setup ftbf
24 54 Jason St.john
cd /home/nfs/ftbf_user/acnet
25 54 Jason St.john
./LArIATViewer.sh
26 54 Jason St.john
</pre>
27 44 Flor de maria Blaszczyk
28 44 Flor de maria Blaszczyk
29 44 Flor de maria Blaszczyk
Back up option: ACNET page *D99* launches Synoptic for Lariat.
30 1 Flor de maria Blaszczyk
31 43 Jason St.john
Detailed instructions (starting with launching ACNET): 
32 43 Jason St.john
If the Synoptic viewer is not opened already or if it has crashed, please follow the "steps 1 to 6 here":http://cdcvs.fnal.gov/redmine/projects/lardbt/wiki/How_to_use_the_online_monitoring#From-the-FTBF-MCenter-Control-Room-computers-any-desktop-computer
33 32 Flor de maria Blaszczyk
34 1 Flor de maria Blaszczyk
35 52 Flor de maria Blaszczyk
h1. 2) The displays main menu - Beamline overview
36 8 Flor de maria Blaszczyk
37 52 Flor de maria Blaszczyk
!BeamOverview.png!
38 1 Flor de maria Blaszczyk
39 52 Flor de maria Blaszczyk
This is the main menu. All displays/controls a shifter or expert needs can be accessed through this menu. 
40 1 Flor de maria Blaszczyk
41 52 Flor de maria Blaszczyk
* Click inside any of the frames (mouse-over until you get a small hand for a pointer) for each sub-system part to open the history plots for the corresponding devices. 
42 52 Flor de maria Blaszczyk
* The small gears on the top right corner inside the frames open the control panels when controls are available.
43 52 Flor de maria Blaszczyk
* Red boxes will appear to indicate a value is out of the allowed range.
44 52 Flor de maria Blaszczyk
* On top of the cryostat drawing, you might get alarm messages such as LAr dewar level too low, HV interlock or cryostat LAr level too low. Please pay attention to these.
45 1 Flor de maria Blaszczyk
46 52 Flor de maria Blaszczyk
* Click on the cryostat drawing to open the cryogenic system overview.
47 52 Flor de maria Blaszczyk
* Click on the TPC drawing to open the TPC overview.
48 1 Flor de maria Blaszczyk
49 1 Flor de maria Blaszczyk
*Beam related information*
50 1 Flor de maria Blaszczyk
* MC6CV indicates the opening a vertical collimator in the secondary beamline(see "Switchyard Layout":https://cdcvs.fnal.gov/redmine/attachments/download/23879/Switchyard-Current.pdf ). When the beam profiles do not look “good” (off-centered or too wide), MCR can change this value to improve the beam profile.
51 1 Flor de maria Blaszczyk
* MCENRG states the energy of the secondary beam at the LArIAT target.
52 1 Flor de maria Blaszczyk
53 1 Flor de maria Blaszczyk
*Tertiary magnets*
54 1 Flor de maria Blaszczyk
* MC7AT1 states the temperature of the coil of magnet 1. Alarm above 175º F.
55 1 Flor de maria Blaszczyk
* MC7AT2 states the temperature of the coil of magnet 2. Alarm above 175º F.
56 22 Flor de maria Blaszczyk
* +MC7AN+ is the current of the magnets . +Remember to report this number when starting a run+.
57 1 Flor de maria Blaszczyk
* MC7ANB is a probe inserted in our magnets to measure the polarity of the field. The actual value stated is not important as only the sign matters. +Double check that the polarity is the right one (positive or negative)+ required for the run and +remember to report its value in the elog when starting a run+.
58 2 Flor de maria Blaszczyk
59 20 Flor de maria Blaszczyk
*Scintillator counters*
60 2 Flor de maria Blaszczyk
* MC7SC1  states the number of counts for the scintillator counter before the LArIAT target.  Intensity of the secondary beam. 1E5 or more is pretty good.
61 8 Flor de maria Blaszczyk
* MC7SC2  states the number of counts for the scintillator counters after the last magnet.
62 1 Flor de maria Blaszczyk
* MC7SC3  states the number of counts for the scintillator counters after the Wire Chamber 4.
63 3 Flor de maria Blaszczyk
64 52 Flor de maria Blaszczyk
65 52 Flor de maria Blaszczyk
h1. 3) For shifters
66 52 Flor de maria Blaszczyk
67 52 Flor de maria Blaszczyk
Unless you need to change the settings on one of the power supplies / tertiary beam magnet, these are the displays you will need to look at.
68 52 Flor de maria Blaszczyk
69 52 Flor de maria Blaszczyk
h2. Beam related information
70 52 Flor de maria Blaszczyk
71 52 Flor de maria Blaszczyk
h3. MCenter Supercycle
72 52 Flor de maria Blaszczyk
73 52 Flor de maria Blaszczyk
To access this display, click on the "Beam info" frame on the beam-line overview.
74 52 Flor de maria Blaszczyk
75 52 Flor de maria Blaszczyk
!MCenterSupercycle.png!
76 52 Flor de maria Blaszczyk
77 52 Flor de maria Blaszczyk
On this display you can see at which stage of the beam cycle you are. The plots evolve continuously, wait for the spill (circled) to appear before stopping / starting a run (on the screenshot above you can see 2 spills). The full cycle is 1mn.
78 52 Flor de maria Blaszczyk
79 52 Flor de maria Blaszczyk
80 1 Flor de maria Blaszczyk
h3. Beam profiles
81 10 Flor de maria Blaszczyk
82 3 Flor de maria Blaszczyk
_Not available through Synoptic_ 
83 8 Flor de maria Blaszczyk
To get these for now you have to use ACNET. See "How to display the wire chambers positioned before the LArIAT target":https://redmine.fnal.gov/redmine/projects/lariat-online/wiki/Using_ACNET_for_monitoring#How-to-display-the-wire-chambers-positioned-before-the-LArIAT-target-along-the-secondary-beam-line .
84 8 Flor de maria Blaszczyk
85 28 Flor de maria Blaszczyk
h3. Beam-line detectors: Upstream and Downstream TOF, Halo and Punch-through, Aerogel Cerenkov
86 3 Flor de maria Blaszczyk
87 46 Sarah Lockwitz
!AGPMTsMonitor.png!
88 1 Flor de maria Blaszczyk
89 1 Flor de maria Blaszczyk
Because most of the PMTs for these systems are powered by the Zener voltage divider (aka COW) in the control room, controls and current monitoring are only available for the halo PMTs. These displays are similar, above is the one for the Aerogel Cerenkov.
90 3 Flor de maria Blaszczyk
91 1 Flor de maria Blaszczyk
_Note: If you cannot do settings on a page, it simply means no setting is possible. Pages where settings are possible are listed as "controls" in the main menu. Otherwise they are called "monitor"._ 
92 46 Sarah Lockwitz
93 52 Flor de maria Blaszczyk
h2. TPC Overview
94 12 Flor de maria Blaszczyk
95 52 Flor de maria Blaszczyk
To open this display, click on the TPC drawing in the beamline overview. Click inside the frames to get the history plots or on the gears to access the control panels.
96 9 Flor de maria Blaszczyk
97 52 Flor de maria Blaszczyk
!TPCOverview.png!
98 1 Flor de maria Blaszczyk
99 1 Flor de maria Blaszczyk
100 52 Flor de maria Blaszczyk
h2. Cryogenic system overview
101 21 Flor de maria Blaszczyk
102 52 Flor de maria Blaszczyk
To open this display, click on the cryostat drawing in the beamline overview. Click inside the frames to get the history plots.
103 21 Flor de maria Blaszczyk
104 52 Flor de maria Blaszczyk
!CryoOverview.png!
105 3 Flor de maria Blaszczyk
106 52 Flor de maria Blaszczyk
* An alarm will be triggered when the level inside the cryostat is below 2.5"Ar. 
107 52 Flor de maria Blaszczyk
* The level meter inside the LAr dewar does not cover the full range of the dewar. The maximum reading is 72"Ar which represent ~ 79% full. When there is a refill, the value should read 79% for a while, until the level reached the active range of the instrument.
108 52 Flor de maria Blaszczyk
* An alarm will be triggered when the LAr dewar level drops below 30%. 
109 52 Flor de maria Blaszczyk
110 59 Flor de maria Blaszczyk
_Note: Not all of the devices of the cryogenic system are shown, only a selection relevant for the LArIAT shifter._
111 1 Flor de maria Blaszczyk
112 77 Flor de maria Blaszczyk
h1. 4) Synoptic power supplies controls
113 1 Flor de maria Blaszczyk
114 75 Flor de maria Blaszczyk
These instructions apply to the power supplies for the wire chambers, the bias voltage, and the cryo PMTs, some of the muon range stack paddles, halo veto. 
115 12 Flor de maria Blaszczyk
Before you do anything... some useful information:
116 1 Flor de maria Blaszczyk
117 78 Jason St.john
* *You cannot change anything unless you have "settings" enabled.* Settings are disabled by default. 
118 93 Jennifer Raaf
* %{color:red}*If a VME PS has to be turned off in an emergency situation (e.g. problem with TPC PMTs) - turn off the regulate function first, then +put the decreasing voltage values in the quick ramp window, go to 0 and then turn off!+*%
119 60 Flor de maria Blaszczyk
120 98 Jennifer Raaf
&nbsp;
121 98 Jennifer Raaf
&nbsp;
122 98 Jennifer Raaf
&nbsp;
123 99 Jennifer Raaf
124 96 Jennifer Raaf
h2. *%{color:green}Turning ON the wire chambers, the bias voltage, or the cryo PMTs%*
125 78 Jason St.john
126 82 Jason St.john
Nominal Voltages:
127 82 Jason St.john
|_. WC1 | -2425 |
128 82 Jason St.john
|_. WC2 | -2400 |
129 82 Jason St.john
|_. WC3 | -2425 |
130 82 Jason St.john
|_. WC4 | -2450 |
131 82 Jason St.john
132 62 Flor de maria Blaszczyk
!>MWPCHV_animated.gif!
133 62 Flor de maria Blaszczyk
1) To enable settings, click on the button on the bottom right hand corner. 
134 62 Flor de maria Blaszczyk
2) Select for how long you wish to enable the settings (Disable, 10mn, 1h or 8h). Do not forget to disable the settings once you are done! 
135 1 Flor de maria Blaszczyk
136 62 Flor de maria Blaszczyk
_There are pop-up confirmation windows that will prevent you from changing anything by mistake but it is still better to disable the settings once you have finished._
137 94 Jennifer Raaf
138 96 Jennifer Raaf
3) To turn on the power supply, click the appropriate ON button near the top of the page. A window will pop-up for confirmation. When the supply is on, the light should be green. There is no voltage yet.
139 62 Flor de maria Blaszczyk
140 1 Flor de maria Blaszczyk
4a) Turn off the regulate function if it's on (the light is dark red when off, green otherwise).
141 1 Flor de maria Blaszczyk
142 94 Jennifer Raaf
4b) Set the nominal voltage *in the RAMP frame* (left panel) by clicking on the bright button next to the power supply you would like to set a voltage to. A window will pop up, enter the value *including the "-" sign* and validate (hitting the enter key doesn't work). If you enter a positive value and the supply is negative only, or if you enter a value out of range, nothing will happen. You will have to re-enter a value. 
143 1 Flor de maria Blaszczyk
144 63 Flor de maria Blaszczyk
*WAIT until the readout value has stabilized and is around the nominal value you set.*
145 94 Jennifer Raaf
* FYI, ramp times from 0 to nominal: ~ 2 minutes for the wire chambers, ~2 hours for the cryo PMTs, ~2 minutes for the TPC bias voltage
146 94 Jennifer Raaf
* %{color:blue}You can stop the ramp at anytime by clicking on the stop button.%
147 80 Flor de maria Blaszczyk
148 94 Jennifer Raaf
%{color:red}Keep an eye on the current draw when ramping up the wire chambers. If current > 15 uA, the power supply will trip. If this happens, repeat instructions from step 3.%
149 1 Flor de maria Blaszczyk
150 63 Flor de maria Blaszczyk
There will be an offset between the readout and the setting, you can fine tune as follows.
151 63 Flor de maria Blaszczyk
152 1 Flor de maria Blaszczyk
5a) Turn on regulate (light should be green when on). 
153 1 Flor de maria Blaszczyk
154 94 Jennifer Raaf
5b) Set the nominal voltage (again) *in the quick settings frame* (right panel) by clicking on the bright button. Wait for ~30 seconds to 1 minute until the voltage stabilizes. *Make sure regulate is on BEFORE setting a voltage, it will not do anything if turned on after the setting is done.*
155 1 Flor de maria Blaszczyk
156 83 Flor de maria Blaszczyk
6) Check if the readout value matches what you were expecting within ~ 5V.
157 1 Flor de maria Blaszczyk
158 1 Flor de maria Blaszczyk
7) Turn regulate off. 
159 1 Flor de maria Blaszczyk
160 1 Flor de maria Blaszczyk
161 1 Flor de maria Blaszczyk
*For the muon range stack / halo veto, skip step 4 since there is no ramp feature for those power supplies.*
162 1 Flor de maria Blaszczyk
163 98 Jennifer Raaf
&nbsp;
164 98 Jennifer Raaf
&nbsp;
165 98 Jennifer Raaf
&nbsp;
166 99 Jennifer Raaf
167 96 Jennifer Raaf
h2. *%{color:green}Turning OFF the wire chambers, the bias voltage, or the cryo PMTs%*
168 96 Jennifer Raaf
169 96 Jennifer Raaf
1) To enable settings, click on the button on the bottom right hand corner. 
170 96 Jennifer Raaf
2) Select for how long you wish to enable the settings (Disable, 10mn, 1h or 8h). Do not forget to disable the settings once you are done! 
171 96 Jennifer Raaf
172 96 Jennifer Raaf
_There are pop-up confirmation windows that will prevent you from changing anything by mistake but it is still better to disable the settings once you have finished._
173 96 Jennifer Raaf
174 96 Jennifer Raaf
3) Turn off the regulate function if it's on (the light is dark red when off, green otherwise).
175 96 Jennifer Raaf
176 96 Jennifer Raaf
4) Set the nominal voltage *in the RAMP frame* (left panel) by clicking on the bright button next to the power supply you would like to set a voltage to. For example, if you're ramping down the wire chambers, you would set the desired voltages to 0 and validate the entries (hitting enter does not work). 
177 96 Jennifer Raaf
178 96 Jennifer Raaf
5) After the voltages have smoothly ramped down to 0V, turn off the power supply by clicking the appropriate OFF button at the top of the page.
179 94 Jennifer Raaf
180 100 Jennifer Raaf
&nbsp;
181 74 Flor de maria Blaszczyk
182 71 Flor de maria Blaszczyk
Notes:
183 78 Jason St.john
* *Closing a window restarts the plots, but the settings are NOT changed.* This means that if you close a window (on purpose or by mistake) or if the display crashes (it can happen from time to time) the power supply remains in whichever state it was before the closure. If it was on, it remains on at whatever voltage it was set. 
184 78 Jason St.john
* To power off a supply, please set the voltage to 0 first and then turn it off. *Do not forget to turn it off!*
185 62 Flor de maria Blaszczyk
* Current can be readout in the bottom left hand side of the display. The top plot shows the voltage as a function of time, the bottom plot is the current as a function of time.
186 62 Flor de maria Blaszczyk
* When regulate is off, there is an offset between the value you set and the actual readout. The lower the voltage you set (with respect to the range of the PS, ie 0-2kV or 0-500V), the larger the offset. 
187 62 Flor de maria Blaszczyk
* In particular, *for the induction plane bias, set a value of ~ 36-38 V to get 18V on the output*. Do not use regulate for this value, it will not work.
188 62 Flor de maria Blaszczyk
* The VME power supplies are not stable below a certain a value: for the 0-2kV ones (all PMTs, including cryo ones), do not set voltages below 200V; for the 0-500V (bias PS), do not set values below 36V (which will give you an 18V output).
189 93 Jennifer Raaf
* After a power supply is turned off (by the user or a power outage), the voltage setting is reset to 0 and should remain at 0 after turning it back on. If the power supply was mid-ramp, the ramp status will also be cleared.
190 1 Flor de maria Blaszczyk
191 1 Flor de maria Blaszczyk
h1. 5) For experts: Controlling the power supplies with ACNET
192 1 Flor de maria Blaszczyk
193 1 Flor de maria Blaszczyk
*Please use this method ONLY if the Synoptic displays are not working even after restarting the Synoptic Viewer* (and contact F. Blaszczyk to report the issue). Once ACNET settings permissions are enabled, it is very easy to make mistakes: there is no confirmation window so you could cause a power supply to turn off or trip if you are not familiar with ACNET.
194 1 Flor de maria Blaszczyk
195 1 Flor de maria Blaszczyk
[[Using ACNET to power up the power supplies]]
196 1 Flor de maria Blaszczyk
197 1 Flor de maria Blaszczyk
h1. Troubleshooting
198 14 Flor de maria Blaszczyk
199 1 Flor de maria Blaszczyk
If Synoptic crashes or hangs up, do not panic! Only the application is affected, not the devices themselves. You only need to close the application and restart it. The non-historical plots will be reset.
200 30 Flor de maria Blaszczyk
201 27 Flor de maria Blaszczyk
*The most common reasons why the Synoptic viewer does not start, crashes or gives you random errors are:*
202 58 Flor de maria Blaszczyk
203 1 Flor de maria Blaszczyk
* You are not opening Synoptic correctly, i.e. you forgot to get a Kerberos ticket or you didn't open Synoptic with the script (or through the ACNET console) when working on a control room desktop computer. Please read the instructions carefully. 
204 58 Flor de maria Blaszczyk
205 101 Flor de maria Blaszczyk
* You get a kerberos ticket error message. The ticket has probably expired. Just go to a terminal and type kticket. This will recreate a ticket. Click on "Proceed" on the error message.
206 101 Flor de maria Blaszczyk
207 13 Flor de maria Blaszczyk
* From your personal computer: you don't have the right Java environment. Make sure you have the "Sun / Oracle Java Runtime Environment (JRE)":http://www.oracle.com/technetwork/java/javase/downloads/index.html installed and up-to-date. OpenJRE / IcedTea does not seem to work well with Synoptic (unless you're lucky).
208 58 Flor de maria Blaszczyk
209 30 Flor de maria Blaszczyk
* From your personal computer: your Java security parameters are not configured correctly. Because of some recent changes on the Java platform, you might have to do some tweaks to the Java security parameters. If you get an error that says that the application cannot be launched because it is from an unknown source / signature, please do the following:
210 1 Flor de maria Blaszczyk
211 1 Flor de maria Blaszczyk
1) Open the Java control panel. How to access it varies according to the OS (Google is your friend). You might have to launch it with admin privileges.
212 1 Flor de maria Blaszczyk
213 1 Flor de maria Blaszczyk
2) Select the security tab.
214 1 Flor de maria Blaszczyk
215 1 Flor de maria Blaszczyk
3) Drop the security to medium level.
216 1 Flor de maria Blaszczyk
*Note: This might not be possible in recent java versions. If you only have the high and very high options, choose high.
217 1 Flor de maria Blaszczyk
218 1 Flor de maria Blaszczyk
4) Add the following links to the exceptions list:
219 1 Flor de maria Blaszczyk
http://synoptic.fnal.gov
220 1 Flor de maria Blaszczyk
http://www-bd.fnal.gov
221 31 Flor de maria Blaszczyk
222 1 Flor de maria Blaszczyk
5) Validate. You might have to restart the computer for these changes to take effect. _Note: Make sure you type "http" and not "https". The latter will not work._
223 30 Flor de maria Blaszczyk
224 30 Flor de maria Blaszczyk
225 30 Flor de maria Blaszczyk
*The most common reasons why you cannot do settings are:*
226 30 Flor de maria Blaszczyk
227 30 Flor de maria Blaszczyk
* You forgot to enable the settings or you went past the time the settings were enabled.
228 30 Flor de maria Blaszczyk
229 30 Flor de maria Blaszczyk
* There are no possible settings on the display (monitor only plots).
230 35 Flor de maria Blaszczyk
231 34 Flor de maria Blaszczyk
* You are not including the "-" sign in front of the value you are trying to set. All the negative VME power supplies require you to include the sign, while the Glassman does not.
232 36 Flor de maria Blaszczyk
233 36 Flor de maria Blaszczyk
* You are trying to set a value larger than the allowed limit. For security reasons or physical reasons (maximum PS voltage), some settings have been restricted to a particular range.
234 30 Flor de maria Blaszczyk
235 30 Flor de maria Blaszczyk
* Your Kerberos account has not been granted setting permissions. There is only a few people who have settings permissions on their individual accounts. If you really need to do settings, all desktop computers in the control room can do settings, provided you started Synoptic correctly.
236 30 Flor de maria Blaszczyk
237 30 Flor de maria Blaszczyk
* You are off-site. Even if you follow the procedure, and even if you're connected using the Fermilab VPN, you will not be able to enable settings. 
238 37 Flor de maria Blaszczyk
239 33 Flor de maria Blaszczyk
* We (as in LArIAT users) do not have settings permissions for that particular device. This shouldn't happen since all the devices used on Synoptic are LArIAT devices so we should have permissions for all of them. If it does happen, the device might be a recent one so we haven't been granted permissions yet. Please let Flor Blaszczyk and Charlie Briegel know. If you are doing settings on ACNET, make sure it is one of our devices and refrain from trying to change other devices (unless you are 100% sure you should). 
240 30 Flor de maria Blaszczyk
241 91 Hans-Joachim Wenzel
*Clearing the red Beam Check Button in the Alarms window*
242 91 Hans-Joachim Wenzel
First click on the button a new window (BeamWatch.xml) should pop up. Follow the advice: " go to acnet window 129 --> restart persistent--> select LArIAT_beamcheck.acl --> select restart
243 91 Hans-Joachim Wenzel
if it ask kill the process and restart press ok and if acnet asks for username and password leave both blank. If the button on the alarms page is still red kill the alarms window and restart it from the synoptic overview page.  
244 91 Hans-Joachim Wenzel
245 30 Flor de maria Blaszczyk
*Common web-viewer issues:*
246 91 Hans-Joachim Wenzel
247 91 Hans-Joachim Wenzel
* "I know the device is on, why does it show as if it were off?" Because of Fermilab firewalls or your respective institution firewalls, it might take couple minutes for the displays to show up. If you see little "stars" instead of values with a unit in the readout sections of the displays and that nothing shows on the time plots, it means you're not getting any information from the devices. Only way to go around or speed things up: use the Fermilab VPN.
248 1 Flor de maria Blaszczyk
249 1 Flor de maria Blaszczyk
h1. Contact
250 1 Flor de maria Blaszczyk
251 1 Flor de maria Blaszczyk
*LArIAT contact*: Flor de Maria Blaszczyk (Boston University), fblaszcz@bu.edu
252 1 Flor de maria Blaszczyk
Office phone #: 617-358-0989
253 1 Flor de maria Blaszczyk
Cell-phone # (also on the Control Room white board): 225-280-0536
254 1 Flor de maria Blaszczyk
Skype: flor.de.maria.blaszczyk
255 1 Flor de maria Blaszczyk
256 1 Flor de maria Blaszczyk
_Need a new display? Don't understand something? Please let me know! All suggestions and comments are welcome._
257 7 Flor de maria Blaszczyk
258 19 Flor de maria Blaszczyk
*Synoptic support*: synoptic-support@fnal.gov
259 53 Flor de maria Blaszczyk
260 19 Flor de maria Blaszczyk
*ACNET controls and VME power supplies emergencies*: Charlie Briegel, briegel@fnal.gov
261 53 Flor de maria Blaszczyk
262 1 Flor de maria Blaszczyk
*Synoptic only emergencies*: If no matter what you do, you cannot restart the OM, please contact *Linden Carmichael* (carmichael@fnal.gov) x5036 or 630-728-7059 (cell) or *Denise Finstrom* x4688