Project

General

Profile

History

Wiki ยป

Installing the full DAQ application suite on a host

JCF, Feb-28-2017: The instructions below are obsolete- click here to see the new instructions

n.b. JCF, Apr-1-2016: the writing of these instructions took place at the same time that I successfully installed the DAQ application suite on woof.fnal.gov, a system unrelated to the DUNE project. Note that as it didn't have 0MQ and I didn't have root access to the machine, I couldn't test the logging of messages up to run control or to messagefacility. I was, however, able to successfully produce a Root file using the "demo" configuration with "component01" selected. As explained below, some of the code you can obtain assumes it will run on woof.fnal.gov; you'll be pointed to places where this is the case so you can make your own system-specific modifications. If you have any questions, I can be reached at

Nice-to-haves and must-haves for your host

Must-haves

- Scientific Linux
- bash
- git
- python 2.6.6+
- python's virtualenv package, needed by RunControl (installation described below under description of the "lbnerc" package)

Nice-to-haves

- Root access to your system if, e.g., you want to install some of the nice-to-haves below. Also, you may need this if you run into issues elsewhere related to non-existent packages (see comment below on pg_config, e.g.)
- 0MQ (if you want to have lbne-artdaq code capable of sending metrics, messages, etc. up to RunControl and the messageviewer window)
- A disk mounted on /data as there are places in the various applications which refer to /data/lbnedaq/config, /data/lbnedaq/daqarea, etc.
- XWindows (so you can see the messageviewer window)

Directories to create

mkdir -p <appropriate_path>/run_records     # DAQInterface will store run-by-run metadata in subdirectories of this directory
mkdir -p <appropriate_path>/daqlogs            # lbne-artdaq will write to logfiles in subdirectories of this directory

Things to install

lbne-artdaq

Already covered here

The lbnerc package

This contains both RunControl and DAQInterface (basically, DAQInterface is defined in rc/control/daqinterface.py, and RunControl encompasses the rest of the code in the package). To obtain it, execute the following:

 git clone http://cdcvs.fnal.gov/projects/lbnerc

Then, you'll want to execute the following commands:
curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz
tar xvfz virtualenv-1.10.1.tar.gz
python virtualenv-1.10.1/virtualenv.py env
source ./env/bin/activate
cd lbnerc
python ./setup.py develop
nosetests # Optional, but lets you know if things are working as they should

To see what the output of these commands looks like, take a look at RunControl installation output

Tip
While the above commands worked fine on woof.fnal.gov, on cluck.fnal.gov I found the following when I ran python ./setup.py develop:

Error: pg_config executable not found

Apparently, pg_config is installed on woof in /usr/bin, but isn't on cluck. Poking around on the internet, it appears to be part of the postgresql-devel package (see http://stackoverflow.com/questions/11618898/pg-config-executable-not-found for more).

RunControl will examine a file called $HOME/.lbnerc-components to determine which components are available. Its contents should look like the following:

component01: <hostname> 5205
component02: <hostname> 5206

where "component0{1,2}" refers not to a fragment generator linked to an actual part of the hardware (e.g., an RCE), but rather a fragment generator which creates simulated data. <hostname> is the name of the system on which you want that component's fragment generator to run.

Finally, RunControl code requires some alteration when running on a host that's not lbne35t-gateway01; from lbnerc/ you'll want to run

git checkout feature/run_on_different_host

and then you can see the changes to RunControl if from lbnerc/ you run 
git diff master feature/run_on_different_host -- rc/control/control.py

In essence, the RunControl on the master branch expects a database which is installed on lbne35t-gateway01 (ask Erik Blaufuss for more details). The branch "feature/run_on_different_host" contains modifications to the RunControl / DAQInterface code which were necessary to get them to run on woof.fnal.gov, a computer which is unrelated to the DUNE or the 35t project. To see the full set of changes that were necessary to get RunControl and DAQInterface to run on woof.fnal.gov, execute: 
git diff master feature/run_on_different_host

which should also provide guidance on where in the code you need to make changes for your own system (a bit more on this later).

The config package

This is simply the collection of FHiCL documents used to initialize the artdaq processes

git clone http://cdcvs.fnal.gov/projects/lbne-daq-config config

The configuration manager

To set up the configuration manager, go to your home directory ($HOME), and first get the configuration manager's repo:

git clone http://cdcvs.fnal.gov/projects/fhicl_cfgmgr

and then create a directory in which to build it:
mkdir cfgmgrDevel

However, before building it, observe the following: the configuration manager relies on some older packages which wouldn't have been pulled in when lbne-artdaq was set up. cd into the products/ directory which was created when lbne-artdaq was installed, copy the following into a file, and source that file - what this will do is download and install the products the configuration manager requires:

if [[ $(basename $PWD) != "products" ]]; then
    echo "Need to be in products/ directory" >&2
    return
fi

tmpfile=$( uuidgen )

cat > $tmpfile <<EOF
http://scisoft.fnal.gov/scisoft/packages/cmake/v3_0_1/cmake-3.0.1-slf6-x86_64.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/gcc/v4_9_1/gcc-4.9.1-slf6-x86_64.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/cetbuildtools/v4_02_02/cetbuildtools-4.02.02-noarch.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/boost/v1_56_0/boost-1.56.0-slf6-x86_64-e6-prof.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/cetlib/v1_07_03/cetlib-1.07.03-slf6-x86_64-e6-prof.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/cetpkgsupport/v1_08_07/cetpkgsupport-1.08.07-noarch.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/cpp0x/v1_04_08/cpp0x-1.04.08-slf6-x86_64-e6-prof.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/fhiclcpp/v3_01_02/fhiclcpp-3.01.02-slf6-x86_64-e6-prof.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/messagefacility/v1_11_15/messagefacility-1.11.15-slf6-x86_64-e6-prof.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/sqlite/v3_08_05_00/sqlite-3.08.05.00-slf6-x86_64.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/xmlrpc_c/v1_25_30/xmlrpc_c-1.25.30-slf6-x86_64-e6-prof.tar.bz2
EOF

for file in $( cat $tmpfile ); do 
    curl -O $file
    gtar xjvf $( basename $file )
done

rm -f $tmpfile

with these products now installed, we can build the configuration manager:

cd ~/cfgmgrDevel
source <lbne-artdaq products directory>/setup
source ~/fhicl_cfgmgr/ups/setup_for_development -p 
buildtool -c

where <lbne-artdaq products directory> is a placeholder for the actual directory its name suggests.

Preparing DAQInterface

Installing packages for messageviewer

When DAQInterface is launched, it pops up a messageviewer window to which log messages are sent by the fragment generators, etc., assuming 0MQ is installed on the system. You'll want to install the packages messageviewer relies on: 

cd  <lbne-artdaq products directory>
curl -O http://scisoft.fnal.gov/scisoft/packages/artdaq_mfextensions/v1_01_00/artdaq_mfextensions-1.01.00-slf6-x86_64-e10-s35-prof.tar.bz2
gtar xjvf artdaq_mfextensions-1.01.00-slf6-x86_64-e10-s35-prof.tar.bz2
curl -O http://scisoft.fnal.gov/scisoft/packages/qt/v5_4_2b/qt-5.4.2b-slf6-x86_64-e10.tar.bz2
gtar xjvf qt-5.4.2b-slf6-x86_64-e10.tar.bz2

Modifying DAQInterface code

To see the modifications which were made to DAQInterface to get it to run on woof.fnal.gov, you can run the following from lbnerc/ :
git diff master feature/run_on_different_host -- rc/control/daqinterface.py

and you'll see that the major changes included:
  • Change from setting up the lbne35t-gateway01.fnal.gov products/ area to the products/ area created when lbne-artdaq was installed on woof
  • Commenting out of Jonathan Insler's lcmControl.exe program
  • Change of configuration server's address from gateway01 to woof
  • Commenting out of configuration manager's fetch of the FHiCL document directory and replacing it with a manual assignment of the location of the config package-- will need to determine why this is necessary, but since the name of the game here is to tell DAQInterface where it can find the necessary FHiCL documents, this should work for now.

Changing the DAQInterface configuration file

To see the modifications made to the DAQInterface configuration file for running on woof, from lbnerc/, run the following :

git diff master feature/run_on_different_host -- docs/config.txt

Described in detail in the DAQ wiki here , the DAQInterface configuration file tells DAQInterface where the run records directory is located, how many eventbuilders to use and where to run them, etc., etc. You'll of course want to modify this file for your own system.

Obtaining the launch / check / kill scripts

As described on the DAQ wiki, https://cdcvs.fnal.gov/redmine/projects/lbne-daq/wiki/Running_DAQ_Interface, there are scripts which launch, kill, and check the status of the major DAQ applications. They had to be tweaked in order to run on woof:
  • The check of the TDUControl application, assumed to be running on lbnedaq1, was disabled in check_daq_applications.sh
  • The listing of the most recently modified logfile, which requires knowledge of where the logfiles are written to, was also disabled
  • The base directory variable at the top of the scripts, describing the location of lbnerc/, had to be changed
Things to change:
  • You'll want to modify the base directory variable yourself to wherever it was you installed lbnerc.
  • You'll want to edit the line ". $HOME/scratch/developarea_lbne-artdaq/products/setup" in launch_daq_applications.sh to refer to your products/ directory
  • You'll also want to edit the line "CfgMgrApp $HOME/scratch/lbnedaq/config/ &" in launch_daq_applications.sh to refer to where you put your configuration directory

You can get the tweaked scripts at the bottom of this page, in "utility_scripts.tar"; you'll of course want to install them in a directory which is pointed to in your system's PATH environment variable.

A final comment

When running your newly installed system, just be aware that the equivalent of the source of the "fireup" script described on https://cdcvs.fnal.gov/redmine/projects/lbne-daq/wiki/Running_DAQ_Interface is the following:

. ./env/bin/activate
cd lbnerc