Project

General

Profile

Data Files from NDOS » History » Version 11

Martin Frank, 01/17/2013 11:43 AM

1 9 Brian Rebel
{{toc}}
2 9 Brian Rebel
3 1 Martin Frank
h1. Data Files from NDOS
4 1 Martin Frank
5 4 Martin Frank
*CHECK WITH THE RUN COORDINATOR BEFORE DOING ANY OF THIS!*
6 4 Martin Frank
7 10 Martin Frank
h2. Running on One Buffer Node
8 11 Martin Frank
9 1 Martin Frank
This wiki entry describes how to collect data from NDOS and store it as a data file that can be used with our DDT framework.
10 2 Martin Frank
11 3 Martin Frank
First, you need to make sure that data is being collected.  Check the MicroSlice rate of the DCMs on "Ganglia":https://novadaq-ctrl-datamon.fnal.gov/ganglia/?r=2hr&cs=&ce=&m=MicroSlice%2520Rate_P0&s=by+name&P=0&c=DCM&h=&host_regex=&max_graphs=0&sh=1&z=small&hc=4&q= , it should be 20 kHz.
12 1 Martin Frank
13 3 Martin Frank
Once that is verified you can log into one of the NDOS buffer nodes:
14 3 Martin Frank
15 2 Martin Frank
<pre>
16 2 Martin Frank
ssh novadaq@novadaq-ctrl-farm-01.fnal.gov
17 2 Martin Frank
</pre>
18 2 Martin Frank
19 2 Martin Frank
NDOS has 16 buffer nodes (01 - 16).  Note that the @novaddt@ user does not have permissions on the buffer nodes.
20 2 Martin Frank
21 2 Martin Frank
Now you just need to set up the environment and run the job:
22 2 Martin Frank
23 1 Martin Frank
<pre>
24 2 Martin Frank
setup_novaddt
25 3 Martin Frank
ddt-filter -c ddt-writeFile.fcl
26 2 Martin Frank
</pre>
27 2 Martin Frank
28 3 Martin Frank
This will run on the data stream and dump the data into a file called @ddt.root@.
29 5 Jan Zirnstein
30 5 Jan Zirnstein
Files from live data should be stored in
31 5 Jan Zirnstein
32 5 Jan Zirnstein
<pre>
33 5 Jan Zirnstein
/nova/ana/trigger/data/
34 5 Jan Zirnstein
</pre>
35 5 Jan Zirnstein
36 5 Jan Zirnstein
with format
37 5 Jan Zirnstein
38 5 Jan Zirnstein
<pre>
39 5 Jan Zirnstein
DDTData-<det>-<size>-<date>-<run>.root
40 5 Jan Zirnstein
</pre>
41 5 Jan Zirnstein
42 5 Jan Zirnstein
where 
43 5 Jan Zirnstein
* <det> is one of NDOS, ND, or FD
44 5 Jan Zirnstein
* <size> is number of milislices (1000,10000,etc)
45 5 Jan Zirnstein
* <date> is of format yymmdd
46 5 Jan Zirnstein
* <run> is 1, 2, 3, 4, etc.
47 6 Martin Frank
48 6 Martin Frank
h2. Running on Several Buffer Nodes
49 6 Martin Frank
50 6 Martin Frank
When you run on several nodes, make sure that you do not write to the shared disk because the parallel jobs will then write to the same location.  You should write to an area that is local to the buffer nodes (e.g. @/scratch/mfrank@).  
51 1 Martin Frank
52 10 Martin Frank
Now that we have the nifty DAQApplicationManager for our DDT processes (see [[Running_the_Trigger_using_the_DAQApplicationManager]]), we can control all twelve buffer nodes at once.  First, you need to edit the DAQApplicationManager FHICL file and enable the output stream.  Now, you can start and stop your processes.
53 10 Martin Frank
54 10 Martin Frank
After you collected your data, you can transfer it to the VMs.  Go to the following directory:
55 10 Martin Frank
56 1 Martin Frank
<pre>
57 10 Martin Frank
/nova/ana/trigger/data/
58 10 Martin Frank
</pre> and use the following script to transfer your data from all the buffer nodes:
59 10 Martin Frank
<pre>
60 10 Martin Frank
./transfer_files.sh <file name>
61 7 Martin Frank
</pre>
62 8 Martin Frank
63 10 Martin Frank
After all of the data is transferred, you can clean up with the following script run from @novadaq-ctrl-master@:
64 7 Martin Frank
<pre>
65 7 Martin Frank
rgang.py --rsh=ssh 'novadaq-ctrl-farm-{01-10}' 'cd /scratch/mfrank;rm ddt.root; ls -lh'
66 1 Martin Frank
</pre>