Project

General

Profile

Installation Guide

DEPRECATED

Operating system requirements

The only supported operating system is SLF6 (64bit).

The following packages must be installed that are not always included in the default installation:

  • screen
  • gcc-c++
  • boost-devel
  • python-devel
  • PyQt4-devel
  • bzip2-devel
  • libzip-devel
  • zlib-devel
  • libxml2-devel
  • libcurl-devel
  • ncurses-devel
  • mysql-devel
  • mysql-server (if you want to host a local slow control database for testing)
  • tcsh
  • gcc-gfortran
  • gtkterm (needs epel repo)
  • libX11-devel
  • libXpm-devel
  • libpng-devel
  • fftw
  • fftw-devel
  • readline-devel

Install packages on SLF6 as root using:

yum install <package name>
yum --enablerepo=epel install <package name>

If the simple yum install doesn't work e.g. for gtkterm.

Also need the following python packages:

easy_install argparse
easy_install pyserial

If you don't have easy_install, get it with:

wget https://bootstrap.pypa.io/ez_setup.py
python ez_setup.py

Note that we are using python 2.6.

Virtual machines

A SLF6 virtual machine (VM) with a number of g-2 dependencies installed is available at https://cdcvs.fnal.gov/redmine/projects/gm2public/wiki/VirtualBoxImage. This is designed for the g-2 offline software, but can equally be used as a base system for a gm2-tracker-readout-daq installation (although the VM is primarily designed for offline work).

Alternatively a VM with the whole gm2-tracker-readout-daq software suite (including dependencies) is available on the Fermilab /gm2/data/ directory (accessible from the gm2gpvm02.fnal.gov, etc). It can be found at:

/gm2/data/TRACKER_DAQ_SLF6.tar.bz2​

Warning, it is big (>10Gb). Once you have set it up (e.g. in virtualbox), you should do a "git pull" in all repository products to get the latest versions.

Dependency installation

The default installation format for gm2-tracker-readout-daq is for all the dependencies listed below to be installed in a single directory. The gm2-tracker-readout-daq configuration script trackerDaq-setup.sh (see here) assumes this, and sets the GM2_INSTALL_ROOT environment variable to point to this location.

The following is a list of dependencies that must be installed to be able to use all features of gm2-tracker-readout-daq. The list indicates the specific make targets that require a dependency, and points to the relevant installation guides. Environment variable configuration is covered later.

By the time you're done the contents of your installation directory should look something like this:

gm2daq-gm2ipbus
gm2-tracker-readout-daq
root_v5.34.14_gcc_v4.4.7
gm2daq
gm2midas

Note that you need a FNAL kerberos principal set up for any git clone commands.

ROOT

Example configure:

./configure linuxx8664gcc --enable-mysql --enable-qt --enable-python --enable-fftw3

IPBus

Example (uses tag uhal_2_4_0):

cd <your install directory>
git clone ssh://p-gm2daq@cdcvs.fnal.gov/cvs/projects/gm2daq-gm2ipbus
cd gm2daq-gm2ipbus
sh scripts/user/install_tag.sh . uhal_2_4_0

Note that this is slow. Currently a few errors appear towards the end related to link file for some of the binaries. These don't seem to cause a problem, but will be fixed soon.

MIDAS

Example (assumes 64 bit system):

cd <your install directory>
git clone ssh://p-gm2midas@cdcvs.fnal.gov/cvs/projects/gm2midas
cd gm2midas/midas
export MIDASSYS=`pwd`
make
ln -s linux/ linux64

gm2daq

Example:

cd <your install directory>
git clone ssh://p-gm2daq@cdcvs.fnal.gov/cvs/projects/gm2daq
cd gm2daq
git checkout develop

Specific items to build are discussed later.

rootana

  • Follow installation guide here: https://bitbucket.org/tmidas/rootana ("Quick start guide" section)
  • Required for make midas target
  • Will soon be replaced with ROME, but for now is needed

Example

cd <your install directory>
git clone https://bitbucket.org/tmidas/rootana.git
export ROOTSYS=<wherever you installed ROOT>
export MIDASSYS=<wherever you installed gm2midas>/midas
cd rootana
make

Note that if you see make errors like:

XmlOdb.h:15:22: error: TXMLNode.h: No such file or directory

This means that your ROOT version built without XML components. Make sure you have libxml2-devel installed, then rebuild ROOT after running configure again (so that it knows that it has libxml2 headers now), e.g.

cd $ROOTSYS
./configure
make

gm2-tracker-readout-daq installation

Checkout the repository

Information on getting the repo gm2-tracker-readout-daq can be found here .

Setting the gm2-tracker-readout-daq environment

The script that sets up the environment for compiling and running the DAQ code is gm2-tracker-readout-daq/software/trackerDaq-setup.sh. It uses a particular .env file that contains a list of environment variables specific to your host, so you must create this file for your host.

To do so follow the steps below:

1) Copy an existing .env file (I recommend the environment-daq1.hep.ucl.ac.uk.env file) to create your own one, calling it environment-<your host>.env (where <your host> must match the output of "uname -n". See example below:

cd gm2-tracker-readout-daq/software/cfg
cp environment-daq1.hep.ucl.ac.uk.env environment-`uname -n`.env

2) Edit your new .env file so that the paths match your system (in particular where you have installed all dependencies). An example (with explanations) is:

#Path for output files to be written to
export GM2_TRACKER_DAQ_OUT=/tmp

#Area dependencies installed in (e.g. gm2midas, gm2daq, etc)
export GM2_INSTALL_ROOT=/unix/muons/g-2

#Outside dependencies
export PYTHONVERSION=2.6
export PYTHONINC=/usr/include/python$PYTHONVERSION #Python header path
export ROOTSYS=$GM2_INSTALL_ROOT/root #Path to your ROOT installation
export UHALVERSION=/tags/uhal_2_4_0/
export UHAL=$GM2_INSTALL_ROOT/gm2daq-gm2ipbus/install/$UHALVERSION #Path to your UHAL (IPBus) installation

#g-2 common DAQ
export GM2DAQ_DIR=$GM2_INSTALL_ROOT/gm2daq #Path to your gm2daq installation

#MIDAS system
export MIDASSYS=$GM2_INSTALL_ROOT/gm2midas/midas #Path to your MIDAS
export MIDAS_USER=$USER #User you will run MIDAS with
export MIDAS_HOST=$HOSTNAME #Machine you will run MIDAS on

#MIDAS experiment (shouldn't need to change these unless have multiple MIDAS experiments) (ignore if not using MIDAS)
export MIDAS_ONLINE=$GM2_TRACKER_DAQ_DIR/midas/online #Path to experiment directory
export MIDAS_ONLINE_BIN=$MIDAS_ONLINE/bin #Path to experiment bin directory (scripts)
export MIDAS_ONLINE_DATA=$MIDAS_ONLINE/data #Path to experiment data directory
export MIDAS_ONLINE_SHM=$MIDAS_ONLINE/shm #Path to experiment shared memory file directory
export MIDAS_EXPTAB=$MIDAS_ONLINE/exptab  #Path to experiment definition "exptab" 
export MIDAS_EXP_NAME="gm2StrawTracker" #Name of your experiment
export MIDAS_EXP_USER=$MIDAS_USER #Name of MIDAS user
export MHTTP_PORT=8080 #Port for mhttp
export MSERVER_PORT=7070 #Port for mserver
export TRIGGER_SERVER_HOST=$MIDAS_HOST #Machine the trigger server will run on
export TRIGGER_SERVER_PORT=90000 #Port the trigger server will run on

#GLIB (ignore if not using GLIB or GLIB emulator)
export GLIB_ID=glib-gm2  #ID of your GLIB in connections.xml
export GLIB_EMULATOR_PORT=50001 #Port of your GLIB emulator in connections.xml (ignore if not using emulator)

#Raspberry PI trigger system (ignore if not using one)
export RPI_HOST=rp1 #Remote machine trigger server is running on
export RPI_USER=pi #User of remote machine trigger server is running on
export RPI_DIR=/home/$RPI_USER/trigger_server #Directory in remote machine trigger server is running on here server script is

#ROOT analyzer
export ROOTANASYS=$GM2_INSTALL_ROOT/rootana #Path to rootana installation

Once this is all configured, you can set your environment using:

cd gm2-tracker-readout-daq/software
source trackerDaq-setup.sh

This should be done after each new login.

An example output is:

Project top-level: /home/tstuttard/g-2/gm2-tracker-readout-daq
OS type: linux64
Using env file /home/tstuttard/g-2/gm2-tracker-readout-daq/software/cfg/environment-localhost.localdomain.env
Setting environment on localhost.localdomain...
ROOT environment set
uHAL (IPBus) environment set
MIDAS environment set
gm2-tracker-readout-daq setup complete

If you see >>ERROR<< in the output you have a serious problem and you won't be able to build/run gm2-tracker-readout-daq.

If you see >>WARNING<< in the output you will be able to build/run some of gm2-tracker-readout-daq. The output should indicate what you can and can't do, and you can decide if this is an issue for you.

Building gm2-tracker-readout-daq

First, the event builder and master trigger must be built in gm2daq. Follow these instructions:

source gm2-tracker-readout-daq/software/trackerDaq-setup.sh (if not done already)
cd $GM2DAQ_DIR/eventbuilderNew
make
cd $GM2DAQ_DIR/frontends/masterTriggerTracker
make 

Next build gm2-tracker-readout-daq according to these instructions:

source gm2-tracker-readout-daq/software/trackerDaq-setup.sh (if not done already)
cd $GM2_TRACKER_DAQ_SRC
make

Everything should now be built and ready to go. See the gm2-tracker-readout-daq overview or gm2-tracker-readout-daq user guide for more information.

Slow control SQL database

Presently we are using mySQL and accessing the database via the ROOT TSQLServer class. To setup the mySQL database locally for testing do the following:

As root (i.e. after "ksu" on FNAL machines)

service mysqld start
/usr/bin/mysql_secure_installation
Hit return for the mysql root (note not su root) password and then set the mysql root password as: gm2_t1042
- Remove anonymous users? [Y]
- Disallow root login remotely? [Y]
- Remove test database and access to it? [Y]
- Reload privilege tables now? [Y]

mysql -u root -p 
mysql> create database gm2sc;
mysql> use gm2sc;
mysql> grant usage on *.* to gm2user@localhost identified by 'alpha_2pi';
mysql> grant usage on *.* to gm2user@192.168.0.1 identified by 'alpha_2pi';
mysql> grant usage on *.* to gm2user@127.0.0.1 identified by 'alpha_2pi';
mysql> use mysql
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE ON gm2sc.* to gm2user@localhost;
mysql> quit

cd /home/nfs/gm2/mtest/daq/gm2-tracker-readout-daq/software/bin
# Create the schema (blank tables)
mysql --user=gm2user --password=alpha_2pi gm2sc < setup-slow-control-dbSchema.sql
# Test the ROOT interface
cd /home/nfs/gm2/mtest/daq/gm2-tracker-readout-daq/software
source trackerDaq-setup.sh 
root -l
 root> .x test_db.C
 root> quit

This will setup the database with the required tables and the test_db.C will populate a couple of the tables with dummy data in order
to check the client connection (via ROOT) is OK.

USB setup for the LV ardiuno and USB slow control (FTDI chip) cable

As root you need to install/modify the following files:
  • /etc/udev/rules.d/99-arduino.rules
    # Arduino Micro
    SUBSYSTEMS=="usb", ATTRS{idProduct}=="8037", ATTRS{idVendor}=="2341", MODE:="0666" 
    SUBSYSTEMS=="usb", ATTRS{idProduct}=="8037", ATTRS{idVendor}=="2341", RUN+="/sbin/modprobe cdc_acm
    " 
    SUBSYSTEMS=="usb", ATTRS{idProduct}=="8037", ATTRS{idVendor}=="2341", RUN+="/bin/sh -c 'echo 0x234
    1 0x8037 > /sys/bus/usb/drivers/cdc_acm/new_id'" 
    
    # Arduino Uno
    SUBSYSTEMS=="usb", ATTRS{idProduct}=="0043", ATTRS{idVendor}=="2341", MODE:="0666" 
    SUBSYSTEMS=="usb", ATTRS{idProduct}=="0043", ATTRS{idVendor}=="2341", RUN+="/sbin/modprobe cdc_acm
    " 
    SUBSYSTEMS=="usb", ATTRS{idProduct}=="0043", ATTRS{idVendor}=="2341", RUN+="/bin/sh -c 'echo 0x234
    1 0x0043 > /sys/bus/usb/drivers/cdc_acm/new_id'" 
    
  • /etc/udev/rules.d/99-libftdi.rules
    SUBSYSTEM=="usb", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6010", MODE="0664", GROUP="plugdev" 
    
  • /etc/udev/rules.d/99-usbtty.rules
    ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", MODE="666" 
    ATTRS{idVendor}=="103e", ATTRS{idProduct}=="0460", MODE="666" 
    

Then trigger these with

udevadm trigger

Xilinx setup

Xilinx_LabTools_14.6_P.68d_3.tar is in /unix/muons/g-2. Unpack this to your machine.

tar -xvf Xilinx_LabTools_14.6_P.68d_3.tar
cd Xilinx_LabTools_14.6_P.68d_3
./xsetup

—> only tick-box for the install onf “USB Cable Drivers” (ie not the license management/webpack)

Get fxload.rpm. As root

scp ucl:/etc/yum.repos.d/atrpms.repo /etc/yum.repos.d/.
scp ucl:/etc/pki/rpm-gpg/RPM-GPG-KEY-atrpms /etc/pki/rpm-gpg/.
yum --enablerepo atrpms-stable install fxload

Update /etc/udev/rules.d/xusbdfwu.rules to be

ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="0008", MODE="666" 
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="0007", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusbdfwu.hex -D $tempnode" 
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="0009", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_xup.hex -D $tempnode" 
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="000d", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_emb.hex -D $tempnode" 
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="000f", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_xlp.hex -D $tempnode" 
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="0013", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_xp2.hex -D $tempnode" 
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="03fd", ATTRS{idProduct}=="0015", RUN+="/sbin/fxload -v -t fx2 -I /usr/share/xusb_xse.hex -D $tempnode" 

Plug in jtag, you should see upon issuing "lsusb"

Bus 001 Device 010: ID 0403:6014 Future Technology Devices International, Ltd FT232H Single HS USB-UART/FIFO IC

To load a bitfile, make sure you have XILINX_INIT_SH set in your environment script to point to /home/nfs/gm2/mtest/daq/xilinx-tools/14.6/LabTools/settings64.sh.

Then run ./software/bin/bitfile_load.sh <name of .mcs file>

Trouble-shooting

There is some incompatibility between the system version of zlib (used by MIDAS) and the version packaged with IPBus. This can cause link errors like this (e.g. missing symbols in one relative to the other):

/unix/muons/g-2/gm2daq-gm2ipbus/install//tags/uhal_2_4_0//lib/libz.a(zutil.o): In function `zcalloc':
zutil.c:(.text+0x72): undefined reference to `sys_alloc'
/unix/muons/g-2/gm2daq-gm2ipbus/install//tags/uhal_2_4_0//lib/libz.a(zutil.o): In function `zcfree':
zutil.c:(.text+0x98): undefined reference to `sys_free'
collect2: ld returned 1 exit status
make1: *** [/unix/muons/g-2/scratch/tom/productionDAQ/online/gm2-tracker-readout-daq/software/bin/StrawTrackerDAQ_frontend.exe] Error 1

We seem to be able to remove this by ensuring that the MIDAS include libs are the first thing in the LIB path, but this needs fixing properly at some point.