Project

General

Profile

Data Files from NDOS » History » Version 10

Martin Frank, 01/17/2013 11:43 AM

1 9 Brian Rebel
{{toc}}
2 9 Brian Rebel
3 1 Martin Frank
h1. Data Files from NDOS
4 1 Martin Frank
5 4 Martin Frank
*CHECK WITH THE RUN COORDINATOR BEFORE DOING ANY OF THIS!*
6 4 Martin Frank
7 10 Martin Frank
h2. Running on One Buffer Node
8 1 Martin Frank
This wiki entry describes how to collect data from NDOS and store it as a data file that can be used with our DDT framework.
9 2 Martin Frank
10 3 Martin Frank
First, you need to make sure that data is being collected.  Check the MicroSlice rate of the DCMs on "Ganglia":https://novadaq-ctrl-datamon.fnal.gov/ganglia/?r=2hr&cs=&ce=&m=MicroSlice%2520Rate_P0&s=by+name&P=0&c=DCM&h=&host_regex=&max_graphs=0&sh=1&z=small&hc=4&q= , it should be 20 kHz.
11 1 Martin Frank
12 3 Martin Frank
Once that is verified you can log into one of the NDOS buffer nodes:
13 3 Martin Frank
14 2 Martin Frank
<pre>
15 2 Martin Frank
ssh novadaq@novadaq-ctrl-farm-01.fnal.gov
16 2 Martin Frank
</pre>
17 2 Martin Frank
18 2 Martin Frank
NDOS has 16 buffer nodes (01 - 16).  Note that the @novaddt@ user does not have permissions on the buffer nodes.
19 2 Martin Frank
20 2 Martin Frank
Now you just need to set up the environment and run the job:
21 2 Martin Frank
22 1 Martin Frank
<pre>
23 2 Martin Frank
setup_novaddt
24 3 Martin Frank
ddt-filter -c ddt-writeFile.fcl
25 2 Martin Frank
</pre>
26 2 Martin Frank
27 3 Martin Frank
This will run on the data stream and dump the data into a file called @ddt.root@.
28 5 Jan Zirnstein
29 5 Jan Zirnstein
Files from live data should be stored in
30 5 Jan Zirnstein
31 5 Jan Zirnstein
<pre>
32 5 Jan Zirnstein
/nova/ana/trigger/data/
33 5 Jan Zirnstein
</pre>
34 5 Jan Zirnstein
35 5 Jan Zirnstein
with format
36 5 Jan Zirnstein
37 5 Jan Zirnstein
<pre>
38 5 Jan Zirnstein
DDTData-<det>-<size>-<date>-<run>.root
39 5 Jan Zirnstein
</pre>
40 5 Jan Zirnstein
41 5 Jan Zirnstein
where 
42 5 Jan Zirnstein
* <det> is one of NDOS, ND, or FD
43 5 Jan Zirnstein
* <size> is number of milislices (1000,10000,etc)
44 5 Jan Zirnstein
* <date> is of format yymmdd
45 5 Jan Zirnstein
* <run> is 1, 2, 3, 4, etc.
46 6 Martin Frank
47 6 Martin Frank
h2. Running on Several Buffer Nodes
48 6 Martin Frank
49 6 Martin Frank
When you run on several nodes, make sure that you do not write to the shared disk because the parallel jobs will then write to the same location.  You should write to an area that is local to the buffer nodes (e.g. @/scratch/mfrank@).  
50 1 Martin Frank
51 10 Martin Frank
Now that we have the nifty DAQApplicationManager for our DDT processes (see [[Running_the_Trigger_using_the_DAQApplicationManager]]), we can control all twelve buffer nodes at once.  First, you need to edit the DAQApplicationManager FHICL file and enable the output stream.  Now, you can start and stop your processes.
52 10 Martin Frank
53 10 Martin Frank
After you collected your data, you can transfer it to the VMs.  Go to the following directory:
54 10 Martin Frank
55 1 Martin Frank
<pre>
56 10 Martin Frank
/nova/ana/trigger/data/
57 10 Martin Frank
</pre> and use the following script to transfer your data from all the buffer nodes:
58 10 Martin Frank
<pre>
59 10 Martin Frank
./transfer_files.sh <file name>
60 7 Martin Frank
</pre>
61 8 Martin Frank
62 10 Martin Frank
After all of the data is transferred, you can clean up with the following script run from @novadaq-ctrl-master@:
63 7 Martin Frank
<pre>
64 7 Martin Frank
rgang.py --rsh=ssh 'novadaq-ctrl-farm-{01-10}' 'cd /scratch/mfrank;rm ddt.root; ls -lh'
65 1 Martin Frank
</pre>