Setting up the IpBus Code

This will only work on machines that have SLF6 or SLC6 installed. We have tried it successfully with both these Linux variants.

Install IpBus

We have done this in two steps.

1. As root install the IpBus and dependent rpms and their automatic updates.
2. As any user creating an area which has all the static IpBus files required.

We don't know whether 2. works without first doing 1. since the IPbus install also installs some other RPMs (e.g. zlib-devel ncurses-devel etc) that it needs. However you need only do step 1 once and then you can use the static version of IPBus in step 2. By using a static version of the IpBus code we then don't build against code that is changing via nightly yum updates from CERN. We have scripts in the repository and running under cron monitoring these updates with respect to the static version and we also have a script svn checking out the latest IpBus source code to monitor for significant VHDL code changes. As it stands we are presently building all our code against the version of IpBus from 24/10/2013.

Step-1 : Install the IpBus rpms

Depends on system type : either x86_64:

sudo cp cactus.slc6.x86_64.repo /etc/yum.repos.d/cactus.repo

Or i686

sudo cp cactus.slc6.i686.repo /etc/yum.repos.d/cactus.repo

sudo yum groupinstall uhal@

This installs the IpBus libraries in /opt/cactus and the system libraries needed by IpBus. These are then updated via nightly yum updates. We have chosen however to build our code against a fixed IpBus version which in installed in Step 2 below.

To double check that all external dependencies are installed also do:

sudo yum -u install bzip2-devel zlib-devel ncurses-devel python-devel curl curl-devel graphviz graphviz-devel qt qt-devel PyQt PyQt-devel wxPython
sudo yum install PyQt qt* *-qt*

Particularly PyQt is needed for the graphical run-control interface.

Step-2 : Install the fixed / static IpBus code

Create an appropriate directory for the code. In the following we will assume this is: /data/g-2/.

export GM2_ROOT=/data/g-2
mkdir -p $GM2_ROOT/cactus/24_10_2013
cd $GM2_ROOT/cactus/24_10_2013
tar -xzf cactus-24_10_2013.tgz

This installs all the IpBus files (headers, libraries and executables) that we will need in $GM2_ROOT/cactus/24_10_2013

Install the google-test suite

cd $GM2_ROOT
mkdir -p $GM2_ROOT/gtest
cd gtest
tar -xzf gtest-1.7.0.tgz
ln -sf gtest-1.7.0 latest

This installs the google-test suite in $GM2_ROOT/gtest/latest

Install a compatible version of ROOT

This may already be present but you need to make sure your version of ROOT is compiled with the same compiler you'll be using with the IpBus and readout code. IpBus and the readout is compiled against the default SLF6/SLC6 compiler i.e. gcc_v4.4.7. The following steps will build v5.30.06 of ROOT.

cd $GM2_ROOT
tar -xvzf root_v5.30.06.source.tar.gz
mv root root_v5.30.06_gcc_v4.4.7
ln -sf root_v5.30.06_gcc_v4.4.7 root
rm root_v5.30.06.source.tar.gz
export ROOTSYS=$GM2_ROOT/root

This will install ROOT in $GM2_ROOT/root

Setup a local copy of the git repository

For more details see here

But the essential steps are:

Get a kerberos ticket:

kinit -f username@FNAL.GOV

Then you need only do this ONCE to get a local copy of the repository:

cd $GM2_ROOT
git clone ssh://
cd gm2-tracker-readout-daq
git config --global "Your name"
git config --global
git remote rename origin fnal

These are then the commands you will use frequently:

git checkout develop

Switch to develop branch: Note all the most recent code is in the "develop" branch and so this is the version of the code to work with

git pull fnal develop

Get the latest code from the FNAL repository (develop branch) into the local repository (develop branch)

git add file

Add a file to the local repository:

git commit -m "blah" file

Commit a file/change to the local repository:

git push fnal develop

Send the changes from your local repository to the FNAL repository so that others can use them:

Using and compiling IpBus code

cd $GM2_ROOT
git checkout develop
cd gm2-tracker-readout-daq/software

Edit the setup file:
You should add the node you are running on that has the IpBus software installed such that you will then get a warning if you attempt to run on another node. You should also make sure that GM2_ROOT is setup appropriately.

Having edited this file, setup the environment:

cd $GM2_ROOT/gm2-tracker-readout-daq/software
source ./ [or] . ./

Build the C++/python code

cd src

The "make" without a target runs the header dependency generation step by default. This can take 30 secs or so. If you've done a "make" once without a target you can generally skip this unless you've added new code. And so:

make bin

just does the compiling and linking using the existing dependency files.


The VHDL for the ATLYS board is from two sources:

1. The repository directory: hardware/trm/firmware/vhdl i.e. the folder $GM2_ROOT/gm2-tracker-readout-daq/hardware/trm/firmware/vhdl
2. The repository directory: cactus/trunk/components/ipbus/firmware/ipbus_core/hdl i.e. the folder $GM2_ROOT/gm2-tracker-readout-daq/cactus/trunk/components/ipbus/firmware/ipbus_core/hdl

This is the VHDL that is loaded onto the Digilent ATLYS.

We have tested this against the TDC firmware from the BU page that we have installed on an ALTERA DE0.

The IpBUS configuration file

The file is in the repository as the file: software/cfg/connections.xml
This defines the IP addresses that the ATLYS boards have and the name we are using for the boards. In this case we have one ATLYS board
called "Board1" at and one called "Board2" at

This configuration file also defines the location of the XML file that has the actual address map for the ATLYS VHDL - this in the repository as software/cfg/atlys_board_address_table_trm.xml

Note the syntax for specifying this files is "file:///xxx" for a file at "/xxx"

Running some example code

Having downloaded the VHDL onto the ATLYS and TDC/DE0 board then the following executables can be run to test the boards.

cd $GM2_ROOT/gm2-tracker-readout-daq/software
bin/Register_RW.exe cfg/connections.xml Board1 control_0.run_mode W 31
bin/Register_RW.exe cfg/connections.xml Board1 control_0.run_mode R

The last two commands write 0x31 to the run_mode register of the control_0 block and then read it back. These are defined in the software/cfg/atlys_board_address_table_trm.xml file.

bin/test_TrackerDaqRoot.exe < cfg/

This writes to the registers, generates start of spill commands and reads fake data from the TDC via the ATLYS and puts it into a ROOT file.
cfg/ steers the execution of the code e.g. it defines the location of the connections.xml file, which ATLYS board we are talking to and the number of fake spills to generate.

There is a also a Python GUI run-control invoked via: python

IpBus: Information and lessons learned