Project

General

Profile

Wiki » History » Version 281

Jason St. John, 03/08/2017 09:04 AM

1 104 Jennifer Raaf
!https:/redmine/attachments/download/17373/Lariat_Logo_256.jpg!  
2 257 Jason St. John
3 272 Jennifer Raaf
LArIAT shift: *+1 9293 LARIAT* (= +1 929 352 7428)
4 218 Jason St. John
ROC West x5413
5 1 William Badgett
MCenter Control Room (MC-CR) x3726 or x8871
6 1 William Badgett
Internationally: prefix *+1 630 840* to the above
7 257 Jason St. John
8 241 Jason St. John
9 1 William Badgett
h1. Shifter Stuff
10 1 William Badgett
11 147 Jason St. John
* [[Before your first shift at LArIAT]]
12 254 Jason St. John
* If remote or starting local shift from scratch: "How to Set up your Shift Station":https://redmine.fnal.gov/redmine/projects/lariat-online/wiki/How_to_set_up_your_shift_station
13 279 Andrea Falcone
* +[[LArIAT Run 3 Shifter Instructions]]+ (April 2016)
14 207 Luke Batten
** [[Run Plan]]
15 202 Jason St. John
** "LArIAT Expert Call List":http://lariat.fnal.gov/internal/internal.html  (User is "lariat", password same as docdb)
16 248 Jason St. John
17 248 Jason St. John
{{toc}}
18 1 William Badgett
19 171 Jason St. John
h2. Link summary ( [[quick start]] )
20 166 Steven Dytman
21 245 Jason St. John
* "Live event viewer":http://lariat-wbm.fnal.gov/lariat/evd.html snapshots
22 197 Jason St. John
* "Accelerator Schedule":http://www-ad.fnal.gov/ops/schedule.html  (power outages, etc)
23 270 William Badgett
* "LArIAT eLog":http://dbweb0.fnal.gov/ECL/lariat/E/index  For making/ending access, starting/ending shifts.
24 210 Luke Batten
* [[LArIAT DAQ Instructions]] (February 2016)
25 237 Jason St. John
* "LArIAT Run Status Webpage":http://lariat-wbm.fnal.gov/lariat/run-alert.html
26 196 Ryan Linehan
* "LArIAT Run Summary Webpage":http://lariat-wbm.fnal.gov/wbm/servlet/LariatRunSummary
27 213 Luke Batten
* "LArIAT Online DQM":http://lariat-dqm.fnal.gov (works with VPN or VNC or on-site only, for now)
28 164 Johnny Ho
* How to use the [[event viewers]]
29 1 William Badgett
* "Synoptic":https://cdcvs.fnal.gov/redmine/projects/lariat-online/wiki/Using_Synoptic_for_monitoring
30 250 Jason St. John
* "Beam monitoring (ACNET)":https://redmine.fnal.gov/redmine/projects/lariat-online/wiki/Using_ACNET_for_monitoring?parent=Wiki
31 247 Jason St. John
* "Beams Division E-Log":https://www-bd.fnal.gov/Elog/?logNames=Booster%2C+External+Beamlines%2C+Linac%2C+Main+Injector%2C+Operations
32 1 William Badgett
* "MC7 WebCam":http://131.225.176.26/
33 1 William Badgett
34 151 Jason St. John
h3. Important knowledge
35 1 William Badgett
36 149 Jason St. John
* "List of MC7 ACNET devices":https://redmine.fnal.gov/redmine/projects/lariat-online/wiki/MC7_in_ACNET
37 149 Jason St. John
* [[MC-7 Enclosure Layout]]
38 1 William Badgett
* "In case the beam TVs are off":http://www-bd.fnal.gov/notifyservlet/www?project=&refresh=on&infolinks=top
39 1 William Badgett
* [[The LArIAT supercycle]] _How DAQ operations relate to the main injector supercycle_ 
40 151 Jason St. John
* "Hardware-Connections-and-Configurations":https://redmine.fnal.gov/redmine/projects/lariat-online/wiki/Wiki#Hardware-Connections-and-Configurations
41 151 Jason St. John
* "Electronic Hardware Documentation":https://redmine.fnal.gov/redmine/projects/lariat-online/wiki/Wiki#Electronic-Hardware-Documentation
42 132 Jason St. John
43 149 Jason St. John
h3. Obsolete links
44 1 William Badgett
45 202 Jason St. John
* [[LArIAT Run 1 Shifter Instructions]] (June 2015)
46 149 Jason St. John
* -LArIAT Engineering Run (Aug 2014) [[How to Run the Lariat Data Acquisition]]- (OBSOLETE)
47 1 William Badgett
* -"Shifter Instructions":https://docs.google.com/document/d/1mlN_UIOCqwhe0cFiU7mRc3-lxDZhY-4TmIy23rOzILI/pub August 2014- (Some obsolete information)
48 164 Johnny Ho
* -How to make [[Lariat DAQ Plots]]- (OBSOLETE)
49 149 Jason St. John
50 83 Jason St. John
h1. Troubleshooting 
51 1 William Badgett
52 152 Jason St. John
Any time day or night, please call experts if you are in need of help and the answer is not provided below.
53 152 Jason St. John
54 274 Jason St. John
h2. *VNC Shared Desktop*
55 274 Jason St. John
56 276 Jason St. John
h3. Mouse clicks not working. Keyboard remapped. Arrow keys type numbers.
57 276 Jason St. John
58 277 Jason St. John
*Solution:* Kill synoptic, restart it.
59 277 Jason St. John
*Procedure:* 
60 277 Jason St. John
* Open a new terminal window on @lariat-cr-02@ (or log in remotely).
61 277 Jason St. John
* Find the process ID's to kill by issuing the command 
62 277 Jason St. John
<pre>ps aux | grep javaws</pre>
63 277 Jason St. John
* The integer after @lariat@ is the process ID.  For example:
64 277 Jason St. John
<pre>
65 277 Jason St. John
[lariat@lariat-cr-02 ~]$ ps aux | grep javaws
66 277 Jason St. John
lariat    1319 19.3 12.2 4049912 471044 pts/1  Sl   Jul15 777:36 /home/lariat/jdk1.8.0_91/jre/bin/java -classpath /home/lariat/jdk1.8.0_91/jre/lib/deploy.jar -Djava.security.policy=file:/home/lariat/jdk1.8.0_91/jre/lib/security/javaws.policy -DtrustProxy=true -Xverify:remote -Djnlpx.home=/home/lariat/jdk1.8.0_91/jre/bin -Djava.security.manager -Djnlpx.origFilenameArg=/home/lariat/acnet/LArIAT_mainmenu -Djnlpx.remove=true -Dsun.awt.warmup=true -Xbootclasspath/a:/home/lariat/jdk1.8.0_91/jre/lib/javaws.jar:/home/lariat/jdk1.8.0_91/jre/lib/deploy.jar:/home/lariat/jdk1.8.0_91/jre/lib/plugin.jar -Xincgc -Djavaws.gov.fnal.controls.kerberos.enable=true -Dsun.java2d.noddraw=true -Xms32m -Xmx256m -Djnlpx.heapsize=32m,256m -Djnlpx.splashport=38857 -Djnlpx.jvm=/home/lariat/jdk1.8.0_91/jre/bin/java -Djnlpx.vmargs=LVhpbmNnYwAtRGphdmF3cy5nb3YuZm5hbC5jb250cm9scy5rZXJiZXJvcy5lbmFibGU9dHJ1ZQAtRHN1bi5qYXZhMmQubm9kZHJhdz10cnVlAA== com.sun.javaws.Main /tmp/javawO3BJLo
67 277 Jason St. John
lariat    9599  4.0 12.5 4048312 482184 pts/1  Sl   Jul14 243:28 /home/lariat/jdk1.8.0_91/jre/bin/java -classpath /home/lariat/jdk1.8.0_91/jre/lib/deploy.jar -Djava.security.policy=file:/home/lariat/jdk1.8.0_91/jre/lib/security/javaws.policy -DtrustProxy=true -Xverify:remote -Djnlpx.home=/home/lariat/jdk1.8.0_91/jre/bin -Djava.security.manager -Djnlpx.origFilenameArg=/home/lariat/acnet/LArIAT_mainmenu -Djnlpx.remove=true -Dsun.awt.warmup=true -Xbootclasspath/a:/home/lariat/jdk1.8.0_91/jre/lib/javaws.jar:/home/lariat/jdk1.8.0_91/jre/lib/deploy.jar:/home/lariat/jdk1.8.0_91/jre/lib/plugin.jar -Xincgc -Djavaws.gov.fnal.controls.kerberos.enable=true -Dsun.java2d.noddraw=true -Xms32m -Xmx256m -Djnlpx.heapsize=32m,256m -Djnlpx.splashport=52805 -Djnlpx.jvm=/home/lariat/jdk1.8.0_91/jre/bin/java -Djnlpx.vmargs=LVhpbmNnYwAtRGphdmF3cy5nb3YuZm5hbC5jb250cm9scy5rZXJiZXJvcy5lbmFibGU9dHJ1ZQAtRHN1bi5qYXZhMmQubm9kZHJhdz10cnVlAA== com.sun.javaws.Main /tmp/javawBDjN44
68 277 Jason St. John
lariat   13226  0.0  0.0 103312   868 pts/3    S+   10:53   0:00 grep javaws
69 277 Jason St. John
lariat   22629  9.9  7.7 4044088 298244 pts/1  Sl   Jul17  91:52 /home/lariat/jdk1.8.0_91/jre/bin/java -classpath /home/lariat/jdk1.8.0_91/jre/lib/deploy.jar -Djava.security.policy=file:/home/lariat/jdk1.8.0_91/jre/lib/security/javaws.policy -DtrustProxy=true -Xverify:remote -Djnlpx.home=/home/lariat/jdk1.8.0_91/jre/bin -Djava.security.manager -Djnlpx.origFilenameArg=/home/lariat/acnet/LArIAT_mainmenu -Djnlpx.remove=true -Dsun.awt.warmup=true -Xbootclasspath/a:/home/lariat/jdk1.8.0_91/jre/lib/javaws.jar:/home/lariat/jdk1.8.0_91/jre/lib/deploy.jar:/home/lariat/jdk1.8.0_91/jre/lib/plugin.jar -Xincgc -Djavaws.gov.fnal.controls.kerberos.enable=true -Dsun.java2d.noddraw=true -Xms32m -Xmx256m -Djnlpx.heapsize=32m,256m -Djnlpx.splashport=35726 -Djnlpx.jvm=/home/lariat/jdk1.8.0_91/jre/bin/java -Djnlpx.vmargs=LVhpbmNnYwAtRGphdmF3cy5nb3YuZm5hbC5jb250cm9scy5rZXJiZXJvcy5lbmFibGU9dHJ1ZQAtRHN1bi5qYXZhMmQubm9kZHJhdz10cnVlAA== com.sun.javaws.Main /tmp/javaw8bLkDI
70 277 Jason St. John
</pre>
71 277 Jason St. John
* Kill all of these processes: (you may ignore the one for your @grep@) 
72 277 Jason St. John
<pre>kill -9 1319
73 277 Jason St. John
kill -9 9599
74 277 Jason St. John
kill -9 22629</pre>
75 277 Jason St. John
* Check that your keyboard and mouse function are normal again. If not, call an expert.
76 277 Jason St. John
* Restart DAQ or Event Viewer if needed. THEN re-launch Synoptic.
77 276 Jason St. John
78 274 Jason St. John
h3. *'VNC server closed connection'* might mean that the vnc server on lariat-cr-02 is not running. 
79 274 Jason St. John
80 274 Jason St. John
Log on to lariat-cr-02 and check if the X11 vnc sever is running:
81 274 Jason St. John
82 274 Jason St. John
<pre>
83 274 Jason St. John
$ ps aux | grep vnc 
84 274 Jason St. John
lariat 31537 0.0 0.0 103308 860 pts/5 S+ 00:12 0:00 grep vnc 
85 274 Jason St. John
</pre>
86 274 Jason St. John
87 274 Jason St. John
88 275 Jason St. John
Re-launched it:
89 275 Jason St. John
90 275 Jason St. John
[lariat@lariat-cr-02 ~]$ x11vnc -create -localhost -xinerama -display :0 -shared -many -forever -bg -rfbauth ~/.x11vnc/passwd
91 275 Jason St. John
92 275 Jason St. John
Checked that it is running now:
93 275 Jason St. John
94 275 Jason St. John
[lariat@lariat-cr-02 ~]$ ps aux | grep vnc
95 275 Jason St. John
lariat 31670 8.4 1.6 194032 61944 ? Ss 00:14 0:10 x11vnc -create -localhost -xinerama -display :0 -shared -many -forever -bg -rfbauth /home/lariat/.x11vnc/passwd
96 275 Jason St. John
lariat 31753 0.0 0.0 103308 860 pts/5 S+ 00:16 0:00 grep vnc
97 275 Jason St. John
98 275 Jason St. John
99 275 Jason St. John
...and shifter is able to connect (the most important part!)
100 275 Jason St. John
101 152 Jason St. John
h2. *Trigger and DAQ system*
102 152 Jason St. John
103 188 Jason St. John
When the DAQ crashes, please post the relevant information to [[DAQerrors]] for experts to study.
104 188 Jason St. John
105 187 Jason St. John
h3. Common failures:
106 187 Jason St. John
107 258 Jason St. John
* Communication failure with the front ends. Trying clearing it with 
108 258 Jason St. John
<pre>lariatReset</pre>
109 258 Jason St. John
Wait a few minutes and then try the @go 480@ command again.
110 243 Jason St. John
* Unable to start a new run: <pre>lost connection
111 243 Jason St. John
Permission denied (gssapi-keyex,gssapi-with-mic)</pre>
112 271 Roberto Acciarri
** Log out of the DAQ machine, and use the command *@kticket@* to pick up a fresh kerberos ticket with the special principal that we use for the DAQ: @lariat/lariat@lariat-daq00.fnal.gov@FNAL.GOV@. Check for this success with @klist@. Then log back in to the DAQ machine and start a new run.
113 187 Jason St. John
* DAQ fails with @ERROR: ReadRegisgter ACQUISITION_STATUS CAEN_DGTZ_CommError BoardId 0@  wait 2 minutes, then try to start again.
114 187 Jason St. John
** Try issuing the command @lariatReset 0 1@ waiting more then 5 seconds, and trying again.
115 187 Jason St. John
This command is appropriate when one sees any type of "CAEN_DGTZ" error.  This command will NOT help the persistent ports error.
116 187 Jason St. John
* DAQ fails with @Found persistent communication port still open@
117 187 Jason St. John
** Wait 8 minutes for the ports to clear, then try again.
118 224 Pawel Kryczynski
* DAQ fails with "An exception occurred when trying to send a message to ..." 
119 223 Pawel Kryczynski
** Check that the terminal you are running the DAQ from is indeed lariat-daq00 (or a screen within it).  Output from @klist@ should show the default principal as "lariat/lariat/lariat-cr-02.fnal.gov@FNAL.GOV" or similar if running from a screen off lariat-daq00 from the ROC-West station.
120 242 Jason St. John
** Run number failed increment or has reset to a low number such as 1: Stop the run and then on the DAQ terminal edit @/opt/lariat-online/config/runNumber.dat@ to correct the run number. Start new run after.
121 187 Jason St. John
122 187 Jason St. John
h3. Other failures: 
123 187 Jason St. John
124 1 William Badgett
* If the DAQ is having trouble talking to the crates, try issuing the command @lariatReset 0 1@ waiting more then 5 seconds, and trying again.
125 187 Jason St. John
This command is appropriate when one sees any type of "CAEN_DGTZ" error.  This command will NOT help the persistent ports error.
126 227 Jason St. John
* *No triggers at all* If the trigger crate has been power cycled and the V1495 is not emitting triggers: [[Reset1495]]
127 226 Jason St. John
* *Run Status page not updating* 
128 226 Jason St. John
** Try refreshing the page. 
129 227 Jason St. John
** If the ACNET node has gone down (as in a power loss) may need to re-start the tomcat servlet: @ssh tomcat@lariat-wbm.fnal.gov@, then @kitty@.  (Yep, "kitty.")
130 83 Jason St. John
131 152 Jason St. John
h2. *Wire Chambers*
132 152 Jason St. John
133 228 Jason St. John
Nominal Voltages:
134 229 Flor de maria Blaszczyk
|_. WC1 | -2425 |
135 228 Jason St. John
|_. WC2 | -2400 |
136 229 Flor de maria Blaszczyk
|_. WC3 | -2425 |
137 228 Jason St. John
|_. WC4 | -2450 |
138 228 Jason St. John
139 269 Jason St. John
* If the wire chambers HV trips: "instructions to control wire chamber voltages":https://cdcvs.fnal.gov/redmine/projects/lariat-online/wiki/Using_Synoptic_for_monitoring#4-Synoptic-power-supplies-controls
140 135 Jason St. John
* If the wire chambers are acting strangely: "remote power cycling instructions":https://redmine.fnal.gov/redmine/projects/lariat-online/wiki/Communicating_with_the_wire_chambers including mini-DAQ and telnet interface.
141 83 Jason St. John
* If the wire chamber system is skipping spill numbers, it may be a high trigger rate (> 30 k per spill).  The wire chamber controller may still be reading out the hits from the last spill.
142 1 William Badgett
143 231 Flor de maria Blaszczyk
h2. *High voltage for the wire chambers, TPC wireplane bias, cryo PMTs*
144 230 Flor de maria Blaszczyk
145 256 Jason St. John
%{color:red}ICathode HV trips off due to LAr level interlock.%
146 240 Jason St. John
[[Drift HV]] TPC Cathode Voltage
147 240 Jason St. John
148 256 Jason St. John
"*General Instructions for controls via Synoptic*":https://cdcvs.fnal.gov/redmine/projects/lariat-online/wiki/Using_Synoptic_for_monitoring#section-10
149 230 Flor de maria Blaszczyk
150 236 Flor de maria Blaszczyk
Also valid for some of the muon range stack paddles (1-8), the halo veto and 1 aerogel PMT.
151 236 Flor de maria Blaszczyk
152 230 Flor de maria Blaszczyk
*ONLY IF SYNOPTIC DOESN'T WORK* "Instructions for controls via ACNET":https://cdcvs.fnal.gov/redmine/projects/lariat-online/wiki/Using_ACNET_to_power_up_the_power_supplies
153 230 Flor de maria Blaszczyk
154 230 Flor de maria Blaszczyk
* The readback voltage doesn't make any sense, is of the wrong polarity and/or is way above the nominal value, but just for a fraction of time.
155 230 Flor de maria Blaszczyk
This is most likely just a glitch in the readout, please ignore if it happens once or twice. If it happens too frequently, set voltage to 0, turn off the power supply and report it. 
156 230 Flor de maria Blaszczyk
157 230 Flor de maria Blaszczyk
* The readback voltage doesn't make any sense, is of the wrong polarity and is way above the nominal value, and the value is stable~ish (you might see small fluctuations over time)
158 230 Flor de maria Blaszczyk
This is a typical symptom of a bad power supply. Try to set it back to 0, turn it off and disconnect the SHV cable. If the readout values remain the same, it confirms there is a hardware problem. Please contact Brian Fellenz for repairs if we don't have a spare for that unit.
159 230 Flor de maria Blaszczyk
160 230 Flor de maria Blaszczyk
FYI, the power supplies have a given voltage rating and cannot physically deliver voltages beyond that range. Also, their polarities cannot be swapped so if you see the wrong sign on the readout ie a positive voltage on a negative power supply or vice versa, then you can be sure the unit is broken or you have a problem somewhere else in the system.
161 230 Flor de maria Blaszczyk
- Bias voltage rating: - 500 V for the shield and induction planes, + 500 V for the collection plane.
162 230 Flor de maria Blaszczyk
- ELT and Hamamatsu PMTs: +2000 V
163 253 Flor de maria Blaszczyk
- Wire chambers: - 5000 V
164 230 Flor de maria Blaszczyk
- All of the other units: -2000 V
165 230 Flor de maria Blaszczyk
166 230 Flor de maria Blaszczyk
* Something went wrong during a ramp, I need to turn off the supply!
167 230 Flor de maria Blaszczyk
There is an interlock that prevents you from using the normal fast controls on the power supply when a ramp is on-going. First, *stop the ramp by clicking on the STOP button*. This will stop the ramp and clear the interlock. If you need to go back to 0 quickly, use the fast controls to set the voltage to 0 and *don't forget to turn off the supply!*
168 230 Flor de maria Blaszczyk
169 230 Flor de maria Blaszczyk
*Even if the controls appear not to respond, set the voltage to 0 and turn the supply off*. Contact Charlie Briegel first and describe the problem in detail. If you can and if it's safe, disconnect the SHV cable from the power supply. If the readback values remain the same, then the power supply is broken.
170 230 Flor de maria Blaszczyk
171 232 Flor de maria Blaszczyk
* *ACNET controls expert: Charles (Charlie) Briegel, x4510 briegel@fnal.gov*  
172 232 Flor de maria Blaszczyk
* *VME power supplies repairs / replacements: Brian Fellenz, x2512 fellenz@fnal.gov*
173 230 Flor de maria Blaszczyk
174 225 Roberto Acciarri
h2. *Event Viewer*
175 225 Roberto Acciarri
176 225 Roberto Acciarri
* If the event viewer stops cycling through the events and the terminal from which it is running shows a list of lines saying something like _"event XX in file, but on event 0"_ with _XX_ any number,
177 225 Roberto Acciarri
** _*ctrl+z*_ 
178 225 Roberto Acciarri
** _*ps -def*_ to look at the processes list. Look for the job ID of the process _"python /lariat/app/users/EventViewer/..."_ 
179 225 Roberto Acciarri
** _*kill -9 JobID*_ to kill the event viewer
180 268 Jason St. John
** open a new event viewer <pre>ssh lariat@lariatgpvm01.fnal.gov 
181 268 Jason St. John
cd /lariat/app/users/EventViewer/; source SETUP_ONLINE_EVENTVIEWER.sh</pre>
182 225 Roberto Acciarri
183 268 Jason St. John
* If the problem persists and/or become more frequent, exit the session you are in and try to connect from a different gpvm machine
184 181 Jason St. John
185 181 Jason St. John
h2. Power Cycle the ASICs (TPC Wire Noise)
186 181 Jason St. John
187 217 Flor de maria Blaszczyk
* Anecdotal evidence has shown that analog noise on the TPC wire readout channels can be 
188 217 Flor de maria Blaszczyk
alleviated by power cycling the TPC electronics: ASICs and Warm Receiver/Drivers
189 278 Jason St. John
* You can remotely power cycle the TPC electronics.  Log onto lariat-gateway00.fnal.gov with X-forwarding enabled, type "firefox", and go to http://lariat-pdu02.fnal.gov.  Usename "ftbf" (password posted on Post-It on side of ROC monitor, and whiteboard in MC7).  Choose the *CYCLE* option, which will automatically give a 60 second powerless interlude.
190 220 Flor de maria Blaszczyk
* Further important informations can be found on the white board in MC-CR.
191 219 Flor de maria Blaszczyk
* This mechanism is controlled by a smart power distribution unit (PDU) located under the 
192 220 Flor de maria Blaszczyk
TPC electronics power supply near the cryostat in the enclosure.   Due to electrical safety 
193 220 Flor de maria Blaszczyk
rules you must *NOT* use any of the other outlets on this PDU.  Do *NOT* remove the protective 
194 219 Flor de maria Blaszczyk
outlet covers at any time.
195 220 Flor de maria Blaszczyk
!https://redmine.fnal.gov/redmine/attachments/download/25467/lariat-pdu02-cycle.png!
196 220 Flor de maria Blaszczyk
197 1 William Badgett
h2. Recover from a LAr level interlock
198 220 Flor de maria Blaszczyk
199 221 Flor de maria Blaszczyk
Assuming the LAr Level dropped low enough to trip off the TPC Cathode power supply, and maybe you even powered down the PMTs.
200 221 Flor de maria Blaszczyk
201 221 Flor de maria Blaszczyk
# Be sure HV is set to zero for these: Cathode Drift, Wire Bias, cryo-PMTs
202 220 Flor de maria Blaszczyk
How to control the [[Drift HV]]
203 219 Flor de maria Blaszczyk
# Power down the ASICs. (They should be off when TPC voltages change.)
204 220 Flor de maria Blaszczyk
# Bring up the Cathode
205 219 Flor de maria Blaszczyk
# Bring up the Wire Bias
206 221 Flor de maria Blaszczyk
# Bring up the ASICs
207 219 Flor de maria Blaszczyk
# Bring up the PMTs
208 219 Flor de maria Blaszczyk
209 219 Flor de maria Blaszczyk
210 219 Flor de maria Blaszczyk
211 217 Flor de maria Blaszczyk
h2. Synoptic troubleshooting
212 217 Flor de maria Blaszczyk
213 217 Flor de maria Blaszczyk
%{color:red}"Synoptic Troubleshooting":https://cdcvs.fnal.gov/redmine/projects/lariat-online/wiki/Using_Synoptic_for_monitoring#Troubleshooting%
214 217 Flor de maria Blaszczyk
215 266 Johnny Ho
h2. DQM troubleshooting
216 266 Johnny Ho
217 266 Johnny Ho
If the DQM is %{color:red}flashing red%...
218 267 Johnny Ho
219 266 Johnny Ho
* Check to see if the DAQ is running. If it isn't running, you should start a new run!
220 266 Johnny Ho
* Check the "LArIAT Run Status page":http://lariat-wbm.fnal.gov/lariat/run-alert.html. If the number under *Complete Files Waiting* is greater than 10, please call a data handling expert!
221 267 Johnny Ho
* Check the disk space on the @/lariat/data@ mount with the command @df -h /lariat/data@ on a @lariatgpvm@ or @lariat-daq@ machine. If the disk space is 100% full, please call a DQM expert!
222 266 Johnny Ho
223 260 Monica Nunes
224 260 Monica Nunes
h2. PB S60 PPD Params
225 261 Monica Nunes
226 261 Monica Nunes
If F:MC7AN , F:MC7ANB and F:MC7ANP show the values 1  -6 (in red), this is probably a communication problem, where we are not receiving data from them.
227 261 Monica Nunes
Usually it comes back in a few minutes. 
228 265 Monica Nunes
If not, call the MCR and ask them if they can see those values.
229 217 Flor de maria Blaszczyk
230 68 Michael Kordosky
h1. Hardware Connections and Configurations
231 1 William Badgett
232 141 William Badgett
* [[TPC Channel Mapping]]
233 51 Jason St. John
* [[Non-TPC Detectors]] *Naming and numbering in MC7*
234 281 Jason St. John
* "Cable Patch Panel Spreadsheet":http://goo.gl/JXiH8Y as well as high voltage settings, power supplies, and warm PMT operating curves
235 54 Jason St. John
* [[Connections on the Control Room Patch Panel]]
236 54 Jason St. John
237 51 Jason St. John
* [[Logical inputs to the CAEN V1740]]
238 51 Jason St. John
* [[Connections into the WUT TDC]]
239 51 Jason St. John
* [[Connections into the V1751 Digitizer]]
240 1 William Badgett
* [[Trigger inputs|Connections in/out of the V1495 Trigger]]
241 68 Michael Kordosky
* [[Logic pulse widths and discriminator thresholds]]
242 68 Michael Kordosky
* [[WUT configuration]]
243 74 William Badgett
* [[Timing of ACnet TCLK Events]]
244 140 William Badgett
* The [[Teensy]] LAr ASIC controller's configuration
245 50 Michael Kordosky
246 138 Roberto Acciarri
* [[Punch Through counters voltages and rates ]]
247 138 Roberto Acciarri
248 102 Jason St. John
h1. Electronic Hardware Documentation
249 45 Michael Kordosky
250 51 Jason St. John
* Link to LArIAT eLog http://dbweb0.fnal.gov/ECL/lariat
251 51 Jason St. John
* [[LariatOnlineGit]] Accessing the LArIAT Online GIT repository
252 98 Brian Rebel
* [[Building LArIATOnline with CMake]]
253 44 Michael Kordosky
254 51 Jason St. John
* [[LArIAT TPC Readout Parameters]]
255 185 William Badgett
* [[LArASIC Preamp Documentation from BNL]]
256 51 Jason St. John
* [[LArIAT DAQ Commercial Products]] Technical Documentation and Firmware Files
257 101 William Badgett
* [[LArIAT CAEN Products]] Technical Documentation and Firmware Files
258 51 Jason St. John
* Test Beam Wire Chamber Sten Hensen's [[TDC Readout Documentation]]
259 89 William Badgett
* WUT Wave Union TDC: [[Jinyuan's TDC Documentation]]
260 51 Jason St. John
* [[v1495 notes]] Notes and working area for the V1495 and V2718
261 273 James Zhu
* "V1495 docs on DocDB":http://lartpc-docdb.fnal.gov:8080/cgi-bin/ShowDocument?docid=1698, built from source:fpga/doc/
262 195 Johnny Ho
* [[A3818 PCI optical controller]] links
263 35 Michael Kordosky
264 1 William Badgett
* [[DAQ Configuration]] settings, parameters, starting a run
265 51 Jason St. John
* [[Raw Data Format and Readout Source code]]
266 100 John Freeman
* [[All things lariat-artdaq]] (UPDATED 11/5/14)
267 154 Jason St. John
* [[Interpreting Wire Chamber TDC Data]]
268 36 Michael Kordosky
269 54 Jason St. John
* [[LArIAT Test Stand]] Photographs
270 47 Michael Kordosky
271 72 Michael Kordosky
h1. Diagnostic and Control Software
272 72 Michael Kordosky
273 73 Michael Kordosky
_Test programs in the_ lariat-online/daq/src _directory_
274 73 Michael Kordosky
275 72 Michael Kordosky
* [[DumpDigitizerData]]
276 72 Michael Kordosky
* [[SummarizeDAQOutput]]
277 72 Michael Kordosky
* [[Reset1495]]
278 72 Michael Kordosky
* [[ScalarReadout]]
279 97 Pawel Kryczynski
* [[Data_quality_monitoring_module]]
280 72 Michael Kordosky
281 139 William Badgett
* CAEN's [[WaveDump]] utility
282 139 William Badgett
283 75 Jason St. John
h1. Examining the WC mini DAQ ASCII data
284 75 Jason St. John
285 75 Jason St. John
The Processor makes TTrees from the ASCII files saved by @mc7-daq.py@
286 82 Jason St. John
[[WC MiniDAQ Data Processor]]
287 75 Jason St. John
288 153 Jason St. John
h1. Data Files from the year 2013
289 76 Will Flanagan
290 85 William Badgett
Data files for various beam conditions can be found in the [[DataFiles]] page.
291 85 William Badgett
292 51 Jason St. John
h1. Meetings
293 51 Jason St. John
294 51 Jason St. John
* [[Connecting to LArIAT ReadyTalk]]
295 198 Johnny Ho
296 1 William Badgett
h1. Miscellaneous
297 198 Johnny Ho
298 216 Luke Batten
* The control room shifter computer (MCenter) can be logged into as ftbf_user@ftbflx18.fnal.gov after passing through lariat-gateway.fnal.gov
299 198 Johnny Ho
* [[Proxy using SSH tunnel]]
300 280 Andrea Falcone
301 280 Andrea Falcone
302 280 Andrea Falcone
h1. Commissioning tests before Run III
303 280 Andrea Falcone
304 280 Andrea Falcone
* TOF calibration with electrons - Robb Carey
305 280 Andrea Falcone
306 280 Andrea Falcone
If others want to do calibration, please tell me.