Project

General

Profile

UCL Test Bench » History » Version 5

Mark Lancaster, 03/07/2017 10:37 AM

1 1 Thomas Stuttard
h1. UCL Test Bench
2 1 Thomas Stuttard
3 1 Thomas Stuttard
{{>toc}}
4 1 Thomas Stuttard
5 1 Thomas Stuttard
h1. IP addresses
6 1 Thomas Stuttard
7 1 Thomas Stuttard
|Device|IP address|
8 2 Mark Lancaster
|MCH|192.168.200.34|
9 5 Mark Lancaster
| FC7 (slot-4) | 192.168.200.71 |
10 1 Thomas Stuttard
11 1 Thomas Stuttard
h1. Labgate network
12 1 Thomas Stuttard
13 1 Thomas Stuttard
Info here: https://www.hep.ucl.ac.uk/twiki/bin/view/Computing/LabComputingIPList?twiki_redirect_cache=5d3215cac6755a11dead45911ea56434
14 1 Thomas Stuttard
15 1 Thomas Stuttard
35 is the AMC13
16 1 Thomas Stuttard
17 1 Thomas Stuttard
h1. AMC13 / FC7
18 1 Thomas Stuttard
19 3 Mark Lancaster
FC7 (fc7-gm2-ucl-blue, IP 192.168.200.71, MAC fc:07:00:00:00:33) is connected to AMC13 channel 4. This channel must be enabled (and only this one) on the AMC13.
20 1 Thomas Stuttard
21 1 Thomas Stuttard
A GLIB (*glib-gm2-ucl-lbemulator*) is emulating 3 logic boards. Each is connected via a fiber channel to the FC7 inputs 0,1,2.
22 1 Thomas Stuttard
23 1 Thomas Stuttard
Set the number of hits in the emulated logic boards by writing to the GLIB registers *tdcwords.reg2* (this sets number of words in all the emulated logic boards).
24 1 Thomas Stuttard
25 1 Thomas Stuttard
h1. Logic boards
26 1 Thomas Stuttard
27 1 Thomas Stuttard
USB Devices:
28 1 Thomas Stuttard
29 1 Thomas Stuttard
* FTWYN4RJ
30 1 Thomas Stuttard
* FTWYN43B
31 1 Thomas Stuttard
32 1 Thomas Stuttard
h1. DAQ Software
33 1 Thomas Stuttard
34 1 Thomas Stuttard
h2. Configuration
35 1 Thomas Stuttard
36 1 Thomas Stuttard
Configure using the XML file ${DAQ_CONFIG_XML} if running using python, or using the ODB if running with MIDAS.
37 1 Thomas Stuttard
38 1 Thomas Stuttard
Some important config params:
39 1 Thomas Stuttard
40 1 Thomas Stuttard
* *"/Equipment/StrawTrackerDAQ/Settings/Globals/Sim Mode"*
41 1 Thomas Stuttard
** Use hardware emulators instead of talking to real hardware
42 1 Thomas Stuttard
* *"/Equipment/StrawTrackerDAQ/Settings/Globals/External trigger"*
43 1 Thomas Stuttard
** If in normal mode:
44 1 Thomas Stuttard
*** If true then expects external trigger (TTC if AMC13, TTL or IPBus if GLIB/FC7) 
45 1 Thomas Stuttard
*** If false then will self-generate a signal at the period specified by *"/Equipment/StrawTrackerDAQ/Settings/Globals/Trigger period clocks"* (units = clocks) (using local loop back in AMC13, or with software thread sending IPBus gen_spill if GLIB/FC7)
46 1 Thomas Stuttard
** If in sim mode:
47 1 Thomas Stuttard
*** If true then expects external trigger (either "SEND TRIGGER" button in GUI, or one will be sent every EOF if running in MIDAS) 
48 1 Thomas Stuttard
*** If false then will self-generate a signal at the period specified by *"/Equipment/StrawTrackerDAQ/Settings/Globals/Trigger period clocks"* (units = clocks) (using T1 emulator for AMC13, or with software thread sending IPBus gen_spill if GLIB/FC7)
49 1 Thomas Stuttard
50 1 Thomas Stuttard
h1. Using the calo frontends
51 1 Thomas Stuttard
52 1 Thomas Stuttard
The calo frontends are in gm2daq. They are:
53 1 Thomas Stuttard
54 1 Thomas Stuttard
* CaloReadoutAMC13
55 1 Thomas Stuttard
** The readout frontend
56 1 Thomas Stuttard
* AMC13Simulator
57 1 Thomas Stuttard
** The emulator
58 1 Thomas Stuttard
59 1 Thomas Stuttard
The guide below assumes you have NOT setup the gm2-tracker-readout-daq environment (using trackerDaq-setup.sh). For the time being this is incompatible with gm2daq due to different amc13 tool versions (this will be resolved in the future).
60 1 Thomas Stuttard
61 1 Thomas Stuttard
To build them follow these steps:
62 1 Thomas Stuttard
63 1 Thomas Stuttard
<pre>
64 1 Thomas Stuttard
cd <your install location>/gm2daq
65 1 Thomas Stuttard
. setup.sh                                                 #Set environment (edit this script to match your system)
66 1 Thomas Stuttard
cd $AMC13_STANDALONE_ROOT/amc13StandaloneMAN_2014-05-12
67 1 Thomas Stuttard
make                                                       #Build the calo modified AMC13 libs
68 1 Thomas Stuttard
cd $GM2DAQ_DIR/frontends/AMC13Simulator
69 1 Thomas Stuttard
make                                                       #Build the simulator frontend
70 1 Thomas Stuttard
cd $GM2DAQ_DIR/frontends/CaloReadoutAMC13
71 1 Thomas Stuttard
#If not using MC1 DAQ machines, edit Makefile to set "USE_GPU" to "0" (as don't have GPUs/CUDA on e.g. UCL machines)
72 1 Thomas Stuttard
make                                                       #Build the readout frontend
73 1 Thomas Stuttard
</pre>
74 1 Thomas Stuttard
75 1 Thomas Stuttard
The following ODB params need settings:
76 1 Thomas Stuttard
77 1 Thomas Stuttard
* */Equipment/AMC1301/Settings/Link01/readout ip*
78 1 Thomas Stuttard
** IP adress of machine running CaloReadoutAMC13 frontend (use localhost e.g. 127.0.0.1 if running all on same machine)
79 1 Thomas Stuttard
* */Equipment/AMC1301/Settings/Link01/source ip*
80 1 Thomas Stuttard
** IP adress of machine running AMC13Simulator frontendd (use localhost e.g. 127.0.0.1 if running all on same machine)
81 1 Thomas Stuttard
82 1 Thomas Stuttard
To run the frontends, use the gm2-tracker-readout-daq environment (using trackerDaq-setup.sh) to get all our MIDAS experiment variables. Use the following commands or the buttons in the "Programs" tab in the MIDAS web GUI:
83 1 Thomas Stuttard
84 1 Thomas Stuttard
<pre>
85 1 Thomas Stuttard
$GM2DAQ_DIR/frontends/AMC13Simulator/frontend -i <fronted index>
86 1 Thomas Stuttard
$GM2DAQ_DIR/frontends/CaloReadoutAMC13/frontend -i <fronted index>
87 1 Thomas Stuttard
</pre>
88 1 Thomas Stuttard
89 1 Thomas Stuttard
where *<frontend index>* is in th range 1-24.
90 1 Thomas Stuttard
91 1 Thomas Stuttard
Note that AMC13Simulator must be started *BEFORE* CaloReadoutAMC13.
92 1 Thomas Stuttard
93 1 Thomas Stuttard
94 1 Thomas Stuttard
Note that the calo data volume is large, so if not using GPUs then you either need to run with a low trigger rate (0.5Hz seems fine in MasterGM2) or prescale the data (TODO how?).
95 1 Thomas Stuttard
96 1 Thomas Stuttard
h2. Troubleshooting
97 1 Thomas Stuttard
98 1 Thomas Stuttard
h3. Missing readline
99 1 Thomas Stuttard
100 1 Thomas Stuttard
When the building the calo AMC13 stuff, you might get errors like:
101 1 Thomas Stuttard
102 1 Thomas Stuttard
<pre>
103 1 Thomas Stuttard
include/hcal/amc13/FilePrompt.hh:11:31: error: readline/readline.h: No such file or directory
104 1 Thomas Stuttard
include/hcal/amc13/FilePrompt.hh:12:30: error: readline/history.h: No such file or directory
105 1 Thomas Stuttard
</pre>
106 1 Thomas Stuttard
107 1 Thomas Stuttard
This means that readline-devel isn't installed on your system. Do this using (requires root):
108 1 Thomas Stuttard
109 1 Thomas Stuttard
<pre>
110 1 Thomas Stuttard
yum install readline-devel
111 1 Thomas Stuttard
</pre>
112 1 Thomas Stuttard
113 1 Thomas Stuttard
h3. Missing CUDA
114 1 Thomas Stuttard
115 1 Thomas Stuttard
When the building the CaloReadoutAMC13 frontend, you might get errors like:
116 1 Thomas Stuttard
117 1 Thomas Stuttard
<pre>
118 1 Thomas Stuttard
gpu_thread.cpp:35:18: error: cuda.h: No such file or directory
119 1 Thomas Stuttard
</pre>
120 1 Thomas Stuttard
121 1 Thomas Stuttard
This means that the CUDA GPU packages can't be found. If you have the GPUs (e.g. the MC1 DAQ machines), you probably need to check your "CUDASYS" environment variable is correcta nd that your machine has CUDA installed. If not at MC1 (e.g. UCL), we need to build without GPUs so make sure that $GM2DAQ_DIR/frontends/CaloReadoutAMC13/Makefile has "USE_GPU" set to 0 to avoid building with these packages.
122 1 Thomas Stuttard
123 1 Thomas Stuttard
h3. Fixing permissions
124 1 Thomas Stuttard
125 1 Thomas Stuttard
The permissions are weird on /unix/muons/g-2, and the g-2 group stuff isn't working. This means that common repos on here (gmwmidas,gm2daq,etc) need their permissions updating manually so that all users can build.
126 1 Thomas Stuttard
127 1 Thomas Stuttard
Need to do this each time the repo is pulled.
128 1 Thomas Stuttard
129 1 Thomas Stuttard
The following command sets the correct permissions (and only on directories so git doesn't think all files are modified):
130 1 Thomas Stuttard
131 1 Thomas Stuttard
<pre>
132 1 Thomas Stuttard
chmod 771 `find <repo> -type d`
133 1 Thomas Stuttard
</pre>
134 1 Thomas Stuttard
135 1 Thomas Stuttard
e.g.
136 1 Thomas Stuttard
137 1 Thomas Stuttard
<pre>
138 1 Thomas Stuttard
chmod 771 `find gm2midas -type d`
139 1 Thomas Stuttard
</pre>