Project

General

Profile

User guide to running the tracker DAQ code

Installation

For information on installing gm2-tracker-readout-daq and its dependencies, see here .

Setting the environment

At every new login, the environment script must be sourced:

source gm2-tracker-readout-daq/software/trackerDaq-setup.sh

See here for more information.

Building the code

Below is an overview of the main software build targets:

  • make all (or just *make")
    • Performs all core tasks, including checking dependencies, making libs etc
    • Must be used for the first build of a clean git clone, or if files are added,removed or renamed
    • Best to always use this after a new git pull
    • If only file contents changed then can use more specific targets below for speed
  • make online
    • Builds the main executables of the online DAQ software
  • make offline
    • Builds the offline analysis executables
    • Reduced dependencies compared to online, can be used on laptops etc
  • make python
    • Builds python library that wraps some of the C++ online DAQ classes
    • Required for the python DAQ scripts, GUIs and unit tests
    • Is called as part of "make" and "make online"
  • make midas
    • Builds MIDAS frontend for the straw tracker DAQ
    • Required for the python DAQ scripts, GUIs and unit tests
    • Is called as part of "make" and "make online"
  • make sim
    • Builds the simulation executables (GARFIELD++, 2014 test beam, EDMSim)
    • Won't build GARFIELD unless GARFIELD_HOME is set (see here)
    • More recent tracker simulation stuff is now in gm2tracker

These are contained in cd $GM2_TRACKER_DAQ_SRC/Makefile. The targets must be run from this directory, e.g.

cd $GM2_TRACKER_DAQ_SRC
make

Note that if no files have been added/remove/renamed, it is fastest to use make midas if (using MIDAS straw FE) or make python (if using python GUI as straw FE).

Configuration files

The run time configuration of the straw DAQ system is parsed from XML configuration files that can be found in $GM2_TRACKER_DAQ_CFG. The files of most interest are:

  • RunConfig.xml - run parameters such as data output type, file paths, logger level, etc
  • DAQConfig.xml - configuration of DAQ hardware and controllers (generally doesn't need changing unless DAQ system changes)
  • connections.xml - a list defining IPBus connections, e.g. links between software and hardware (or emulaltors), including specifying the address table for the connection (generally doesn't need changing unless DAQ system changes)
  • glib_address_table.xml - a list of the nodes for IPBus read/write for the GLIB (generally doesn't need changing unless GLIB firmware changes)

Note that currently these parameters are not saved to the output data stream, but this will be implemented soon. This information may in future be stored in the MIDAS online data base instead of in XML files to help facilitate this.

A list of important parameters and notes is given below.

Run config file

The most important parameters are:

  • loggerLevel
    • 0 is quietest, 4 is loudest
    • 2 recommended for most users
  • exitOnError
    • It this 1, the DAQ software will exit if an error is thrown (recommended, otherwise easy to miss them)
  • writeToRoot
    • It this 1, the DAQ software will write the TDC data to a ROOT file directly
    • Generally don't want this in MIDAS, as want the data to be written to a MIDAS bank (can't do both simultaneously due to nature of buffering)
    • Useful when using python tools to run DAQ (as no MIDAS banks)

DAQ config file

The most important parameters are:

  • usingBosTrigger
    • When 1, GLIB expects a HW signal indicating the Begin Of Spill (BOS)
    • When 0, DAQ software provides fake BOS triggers instead (must use this when using emulator)
  • usingTestBos
    • When 1, GLIB generates the HW BOS internally
    • When 0, must have HW signal connected
  • sendExtDetectorTriggers
    • When == 1, GLIB will send a trigger to external detectors that is high whist the TDCs are accumulating
  • simMode
    • Set to 1 to run a GLIB emulator instead of the real GLIB hardware

IPBus XML files

If you have a GLIB in your setup, you will need to add an entry to connections.xml specifying its IP address, etc.

There is a separate entry for the GLIB emulator.

IMPORTANT: The device ID for a connection to a GLIB should include the string "glib-gm2", as this toggles a few special behaviours in the device manager (removes some prefixes from reg names to make it easier for uses, and only manages certain "user" registers - see src/daq/IPBusDeviceManager.cc constructor for details).

Running the DAQ

MIDAS

The tracker DAQ system is based around MIDAS. The configuration of the tracker MIDAS experiment is outlined [[Midas|here].

To run the tracker DAQ with MIDAS, follow these steps.

If you have not yet set up the MIDAS experiment, follow these steps.

Python

To run the DAQ in a simpler stand-alone mode (with out MIDAS) python scripts can be used. Both a python GUI and python batch script are provided for this:

  • $GM2_TRACKER_DAQ_SRC/test/StrawDAQRun.py
    • Run straw DAQ from command line
    • Can specify run sequence, e.g. number of spills, spill duration, etc
    • Also has listen mode to start straw DAQ but not start any data runs
  • $GM2_TRACKER_DAQ_SRC/test/StrawDAQGUI.py
    • Python GUI for straw DAQ system
    • Can start and stop runs and spills, or more simply trigger a single accumulations for all TDCs

To see the command line options for running this python scripts, run them without arguments and look at the output. Some example use cases are presented below:

Launch the GUI

python $GM2_TRACKER_DAQ_SRC/test/StrawDAQGUI.py -c $GM2_TRACKER_DAQ_CFG/RunConfig.xml

Run the straw DAQ from the command line for 5 spills that occur with a 10s period and are 4s in length

python $GM2_TRACKER_DAQ_SRC/test/StrawDAQRun.py -c $GM2_TRACKER_DAQ_CFG/RunConfig.xml run -n 5 -d 4 -p 10

Start the DAQ in listen mode

python $GM2_TRACKER_DAQ_SRC/test/StrawDAQRun.py -c $GM2_TRACKER_DAQ_CFG/RunConfig.xml listen

Note that in listen mode the straw DAQ system will initialise but not run or spill commands will be sent. Other tools can interact with the hardware though (via IPBus for example). This is mainly used for testing, and in particular is useful to start up the emulator so that the DAQ unit tests can be run on the emulated system to check the behaviour matches the real hardware. Use ctrl+c to exit listen mode fom the command line.

Output data

There are two possible ways of getting straw data out of the DAQ system:

Writing to a MIDAS bank

When running the DAQ in MIDAS the event builder will collect the straw data from the straw DAQ system and write it to a .mid MIDAS file.

IMPORTANT: Set writeToRoot im *$GM2_TRACKER_DAQ_CFG/RunConfig.xml" to "0" when running with MIDAS as otherwise data is written to ROOT file rather than bank (unless this is what you want, e.g. debug).

The output MIDAS data file can be converted to a ROOT file using $GM2_TRACKER_DAQ_BIN/strawsMidasToRoot.exe. Example usage is:

$GM2_TRACKER_DAQ_BIN/strawsMidasToRoot.exe MIDAS_FILE=$MIDAS_ONLINE_DATA/run00001.mid.gz OUTPUT_DIR=. DEBUG=N NEVENTS=1000 STRAWS=Y SIDET=N

The MIDAS system displays the location of the data directory where the output data files can be found, or the environment variable $MIDAS_ONLINE_DATA can be used.

Direct ROOT file writing

Alternatively the data can be directly written to a ROOT file without MIDAS. This is useful when using pyton scripts to run the DAQ. To do this, set the following params in RunConfig.xml:

  • writeToRoot - set to "1" to write data to ROOT directly
  • rootFileStem - set to the file path and file name stem (e.g. without .root suffix)
    • e.g. "$GM2_TRACKER_DAQ_OUT/myRootFile"
    • The output ROOT file will be have "RunXXXXX.root" appended to its name

Output data format

The format of the output ROOT file (regardless of whether it come from direct writing or via a MIDAS bank) has all straw data in a directory called "STRAWS". Within this the following trees can be found:

  • StrawsTDCRaw - the straw TDC data exactly as it is read from the TDCs and GLIB (e.g. 32bit words)
  • StrawsTDCProcessed - an unpacked version of the straw data containing hit times, channels, etc
    • This is what most users will want
  • Straws*Config - various config parameters for individual controllers/hardware in the DAQ system
    • These are currently not being filled in preparation for a move to using the MIDAS ODB

Emulators

gm2-tracker-readout-daq includes code that emulates the DAQ hardware in software. This allows the DAQ system to be run without real hardware attached. The DAQ software requires no modifications to do this, it is able to execute fully on either real or emulated hardware.

Running the emulator

To run the emulator system, run the DAQ as usual but set the following configuration parameters in $GM2_TRACKER_DAQ_CFG/DAQConfig.xml* as follows:

  • simMode to "1" for each GLIB you want to emulate (currently only one)
  • *usingBosTrigger" to "0" (emulator can't see hardware triggers, so disabling this feature will generate fake software triggers instead)

Emulating a GLIB also emulates the TDCs, connections, etc connected to that GLIB (as defined in DAQConfig.xml). Note that it is in principle possible to run with a mixture of real and emulated GLIBs, but currently the DAQ system only has one anyway.

Details of emulator

The key features of the emulator are:

  • IPBus device emulator
    • This emulates the registers of the hardware and allows IPBus read/write to this emulated registers instead of a real device
    • Based on DummyHardware IPBus class
  • GLIB emulator
    • This emulates the GLIB firmware to produce realistic behaviour based on the read to / writes from the emulated registers in the IPBus device emulator
  • TDC emulator
    • This receives commands from the GLIB as per the real TDCs and generates fake data to return
  • 8b10b/C5 interfaces
    • This replaces the real command/data transfer between the real TDCs and GLIBs
    • The real protocols are not modelled, these are just used to allow the TDC and GLIB emulators to talk to each other

The data flow in the emulator is asynchronous, and is triggered by register reads/writes on the IPBus device emulator.

Some information on IPBus emulators, including the tracler DAQ emulator, is given in the talk GM2-doc-2544-v1 (can be found on doc DB at http://gm2-docdb.fnal.gov:8080/cgi-bin/ShowDocument?docid=2544).

Note that each emulated GLIB needs its own connection defined in the IPBus gm2-tracker-readout-daq/software/cfg/connections.xml file. In the repository currently one exists already called glib-gm2-dummy. This should specify the same address table as for the real GLIB, but will define the uri to point to localhost rather than the GLIB IP address. If using more than one emulated GLIB, each must be connected to a different UDP port (in both connections.xml and DAQConfig.xml).

See https://svnweb.cern.ch/trac/cactus/wiki/uhalQuickTutorial#WorkingwithDummyHardware for information on the generic IPBus DummyHardware emulator.

Note that it is possible to run just the IPBus device emulator (e.g. not specifically the GLIB emulator) using the executable gm2-tracker-readout-daq/software/bin/RunIPBusDeviceEmulator.exe. You will need to define an address table and device connection to do this (see IPBus documentation: https://svnweb.cern.ch/trac/cactus/wiki/uhalQuickTutorial).

Useful tools

The following is a list of tools provided in gm2-tracker-readout-daq that users might find useful:

  • $GM2_TRACKER_DAQ_SRC/daq/IPBusDeviceManager.cc
    • C++ class managing connection to an individual IPBus device
    • For each device you want to connect to in your code, instantiate one of these for it
    • A python wrapped version is available in the PyGm2.so library in gm2-tracker-readout-daq/software/lib
  • $GM2_TRACKER_DAQ_SRC//test/IPBusDeviceGUI.py
    • GUI "spy" for an IPBus connection (a single connection)
    • Displays all node values real time (polling)
    • Allows to write to nodes too, including single, port and block nodes (double click row in table to write to node)
    • Can be useful
    • Command line options displayed if you run the script with no args
  • $GM2_TRACKER_DAQ_SRC/test/test/IPBusDeviceRW.py
    • Command line node single R/W (single or block) to an IPBus device
    • Command line options displayed if you run the script with no args (has seprate options for read/write

Here are a few examples for running IPBusDeviceRW.py (all for R/W with IPBus device gm2-glib):

Read single node "control_0.software_ready"

python $GM2_TRACKER_DAQ_SRC/test/IPBusDeviceRW.py -c $GM2_TRACKER_DAQ_CFG/connections.xml -d glib-gm2 -n control_0.software_ready  read 

gives:

Mode:  read
READ : Node =  control_0.software_ready  : Block size =  1
Addr: 0x0 | Data = 0b0 | 0x0

Read 3 words from block node "ram"

python $GM2_TRACKER_DAQ_SRC/test/IPBusDeviceRW.py -c $GM2_TRACKER_DAQ_CFG/connections.xml -d glib-gm2 -n ram -s 3 read 

gives:

Mode:  read
READ : Node =  ram  : Block size =  3
Addr: 0x0 | Data = 0b0 | 0x0
Addr: 0x1 | Data = 0b0 | 0x0
Addr: 0x2 | Data = 0b0 | 0x0

Write "1" to single node "control_0.software_ready"

python $GM2_TRACKER_DAQ_SRC/test/IPBusDeviceRW.py -c $GM2_TRACKER_DAQ_CFG/connections.xml -d glib-gm2 -n control_0.software_ready  write -v 1

gives:

Mode:  write
WRITE : Node =  control_0.software_ready  : Block size =  1
Addr: 0x0 | Data = 0b1 | 0x1

Write "2" to first 5 words of block node "ram" (e.g. same value to each word)

python $GM2_TRACKER_DAQ_SRC/test/IPBusDeviceRW.py -c $GM2_TRACKER_DAQ_CFG/connections.xml -d glib-gm2 -n ram -s 5 read

gives:

Mode:  read
READ : Node =  ram  : Block size =  5
Addr: 0x0 | Data = 0b10 | 0x2
Addr: 0x1 | Data = 0b10 | 0x2
Addr: 0x2 | Data = 0b10 | 0x2
Addr: 0x3 | Data = 0b10 | 0x2
Addr: 0x4 | Data = 0b10 | 0x2

Write "1,2,3,4" to first 4 words of block node "ram" (e.g. specify each word value)

python $GM2_TRACKER_DAQ_SRC/test/IPBusDeviceRW.py -c $GM2_TRACKER_DAQ_CFG/connections.xml -d glib-gm2 -n ram  write -v 1 2 3 4

gives:

Mode:  write
WRITE : Node =  ram  : Block size =  4
Addr: 0x0 | Data = 0b1 | 0x1
Addr: 0x1 | Data = 0b10 | 0x2
Addr: 0x2 | Data = 0b11 | 0x3
Addr: 0x3 | Data = 0b100 | 0x4