Project

General

Profile

Running DAQ Interface » History » Version 21

John Freeman, 07/08/2014 06:03 PM

1 1 John Freeman
h1. Running DAQ Interface
2 1 John Freeman
3 1 John Freeman
DAQ Interface is designed to be run, along with rest of the run control code, on lbnedaqtest01.fnal.gov . To obtain an account on this system, contact John Freeman, jcfree@fnal.gov . Once you have an account, you may do the following:
4 1 John Freeman
5 1 John Freeman
* *Check out the run control software*:
6 1 John Freeman
7 12 John Freeman
 Create a new directory, cd into it, and execute <pre>git clone ssh://p-lbnerc@cdcvs.fnal.gov/cvs/projects/lbnerc</pre>
8 1 John Freeman
9 2 John Freeman
* *Make sure you're on the feature/DAQInterface branch*
10 3 John Freeman
cd into lbnerc/, and execute
11 2 John Freeman
<pre>git checkout feature/DAQInterface </pre>
12 2 John Freeman
13 1 John Freeman
* *Set up the environment*: 
14 10 John Freeman
From the lbnerc/ directory, execute <pre>source source_me</pre> This will do two things: first, if it sees that the lbne-artdaq package isn't installed, it will install it. The lbne-artdaq installation will take around 10 minutes, and will take up about 3.3 Gb of space. There will be a prompt early in the download as to whether you want to continue; hit "y" to do so.  "source_me" will search for the lbne-artdaq package in the directory /data/YOURUSERNAME/lbne-artdaq-base ; if it sees that /data/YOURUSERNAME doesn't exist, it will notify you and exit. Contact John Freeman (jcfree@fnal.gov) or someone else with root privileges on lbnedaqtest01 to create this directory for you. After installing lbne-artdaq, it then sets up the Python virtual environment needed by the LBNE RC code in the parent directory of lbnerc, in a directory call "env" (in other words, "env" and "lbnerc" are at the same level of the directory hierarchy on the system). If this is the first time you've set up the Python virtual environment, the process will take roughly two minutes. Note that while there will be a few error/warning messages displayed at different points of the setup, at the end you should see <pre>Environment ready; consider running the unit tests via command nosetests</pre>
15 17 John Freeman
n.b. As of 7/8/14 if you run <code>nosetests</code> 4 of the 65 tests will fail; more than this, and there may be a problem which will affect the running of DAQInterface.  
16 13 John Freeman
17 18 John Freeman
* *Start LBNE run control*: <pre> lbnecontrol & </pre>. Note this won't work if lbnecontrol is already running; to find this out, run "<code>ps -A | grep lbnecontrol</code>" 
18 1 John Freeman
19 1 John Freeman
* *Start DAQ Interface*: <pre> daqinterface -n daqint -r 5570 -c localhost -H localhost & </pre> . Like lbnerc, this also won't work if daqinterface is already running
20 1 John Freeman
21 1 John Freeman
* *Take DAQ Interface through the standard transitions* : 
22 20 John Freeman
Fire up a new shell/terminal in which the artdaq processes are launched, and initialize them with FHiCL documents, by executing the following:
23 1 John Freeman
<pre>
24 1 John Freeman
lbnecmd init daq
25 1 John Freeman
</pre>
26 1 John Freeman
Start the toy fragment generator, which produces simulated CAEN board data, and plot the data using an Art module:
27 1 John Freeman
<pre>
28 1 John Freeman
lbnecmd start daq
29 1 John Freeman
</pre>
30 4 John Freeman
Pause it, ending the subrun but not the run:
31 4 John Freeman
<pre>
32 4 John Freeman
lbnecmd pause daq
33 4 John Freeman
</pre>
34 4 John Freeman
Resume DAQ running:
35 4 John Freeman
<pre>
36 4 John Freeman
lbnecmd resume daq
37 4 John Freeman
</pre>
38 1 John Freeman
Halt the running of the DAQ:
39 1 John Freeman
<pre>
40 1 John Freeman
lbnecmd stop daq
41 1 John Freeman
</pre>
42 1 John Freeman
Kill all the artdaq processes:
43 1 John Freeman
<pre>
44 1 John Freeman
lbnecmd terminate daq
45 1 John Freeman
</pre>
46 4 John Freeman
47 21 John Freeman
* *If Problems Arise*
48 1 John Freeman
49 21 John Freeman
As of this writing (7/8/14) there as not yet been extensive user feedback concerning DAQInterface; despite this, certain potential problems have been anticipated and are handled within DAQInterface. These problems include:
50 21 John Freeman
51 21 John Freeman
-An artdaq process returns an error state after a transition request, or an exception is thrown by the XML-RPC library during the request
52 21 John Freeman
53 21 John Freeman
-During periodic checks, one or more artdaq processes expected to exist are not found
54 21 John Freeman
55 21 John Freeman
In either case, an error is reported via 0MQ to run control, and the "Recover" transition is automatically triggered. This transition is a fairly blunt instrument: it will kill any remaining artdaq processes and return DAQInterface to its original state of "stopped" (i.e., one in which it requires the "init" transition before anything else is done). 
56 21 John Freeman
57 21 John Freeman
58 21 John Freeman
In order to see this for yourself, you can deliberately sabotage one of the transitions.  E.g., during the "init" transition, FHiCL documents located in /data/fcl/daqinterface are used to initialize the artdaq processes after these processes have been started. You can replace one of these filenames listed in the lbnerc/rc/control/daqinterface.py file with one of your own files intentionally designed to be improper FHiCL; this will then trigger a "recover" transition automatically when an "init" transition is requested. You can then use the "lbnecmd check" command to see for yourself that DAQ Interface has returned to its original state. Another thing to do is, after the init transition, once the artdaq process terminal pops up, close it -- this will terminate the artdaq processes, triggering a call to "Recover".
59 18 John Freeman
60 19 John Freeman
Please note that if you issue an "init" transition and then follow it with a "terminate" transition, you'll see an exception in the artdaq terminal window which looks like the snippet below; this is because statistics collection which occurs during termination will fail if no data's been collected, which is expected:
61 18 John Freeman
62 18 John Freeman
<pre>
63 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014:  Time Summary: 
64 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014:  Min: 0
65 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014:  Max: 0
66 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014:  Avg: inf
67 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: %MSG-s ArtException:  Aggregator-lbnedaqtest01-5265 JobSetup
68 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: cet::exception caught in art
69 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: ---- DataCorruption BEGIN
70 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014:   NetMonInputDetail: Could not receive message!
71 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: ---- DataCorruption END
72 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: %MSG
73 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: %MSG-s ArtException:  Aggregator-lbnedaqtest01-5266 JobSetup
74 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: cet::exception caught in art
75 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: ---- DataCorruption BEGIN
76 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014:   NetMonInputDetail: Could not receive message!
77 18 John Freeman
Tue Jul 08 14:16:33 -0500 2014: ---- DataCorruption END
78 18 John Freeman
</pre>