Project

General

Profile

DTC Interface Library

Description

The DTC Interface library is a C++11 class library containing functions and data products for communicating with the DTC device driver. The goal is to provide a simple interface for ARTDAQ application programmers for retrieving data and settings from the DTC module. The library also has a built-in DTC simulator, allowing for development of upstream code without requiring the presence of a real DTC.

Usage

The library declares all of its data-handling functions in DTC.h. This file is the primary entry point for ARTDAQ Board Reader classes. It inherits from DTC_Registers.h, which defines the DTC register address space and helper functions to access these registers. Also of interest to programmers is DTC_Packets.h and DTC_Types.h which define, respectively, the packets used in the DTC communication protocol with the ROCs, and the types and enumerations used internally in the library for interpreting the data from the DTC.

Users of the library should familiarize themselves with the public member functions of these classes. DTC->GetData() should be the primary function called by ARTDAQ BoardReaders.

The DTC class is designed to return the value for simple operations such as register reads and writes, and to return the addresses of memory blocks for data transfers (to avoid unnecessary memory copy operations).
To enable the simulator, either run the code on Windows or set the environment variable "DTCLIB_SIM_ENABLE".

Possible values for DTCLIB_SIM_ENABLE:
  • 0: Disable the simulator
  • [1Tt].* (for example DTCLIB_SIM_ENABLE=1, DTCLIB_SIM_ENABLE=tracker, DTCLIB_SIM_ENABLE=Track, etc.): Enable the simulator in Tracker mode
  • [2Cc].*: Enable the simulator in Calorimeter mode
  • [3Vv].*: Enable the simulator in CRV mode

For more values, see the DTCLIB::DTC_SimMode enumeration or the DTC_SimModeConverter class.
If DTCLIB_SIM_ENABLE is not set, you can still enable the simulator by passing a DTCLib::DTC_SimMode value to the DTCLib::DTC constructor or by calling DTC::SetSimMode.

Example:

cd /some/working/directory/build
source /path/to/mu2e_artdaq/ups/setup_for_development -d
buildtool
DTCLIB_SIM_ENABLE=1 driver -c /path/to/mu2e_artdaq/tools/fcl/driverToy.fcl

Library API

The DTC.h file is broken into several sections, each containing functions related to a certain type of data.
  1. DTC General Status Functions -- these functions provide information about the DTC interface itself
  2. DMA Functions -- these functions perform DMA access to the DTC. Examples are DCS request-reply cycles and data read-out
  3. Register IO -- these functions are used to control and view the state of the DTC
  4. PCIe and DMA performance -- these functions are used for testing the DTC module and monitoring the data transfer rates

DTC General Functions

  • ReadSimMode
    Returns a DTCLib::DTC_SimMode value indicating the simulation mode this DTC instance was instructed to use (if any).
  • SetSimMode
    Sets the SimMode and re-initializes the mu2edev and mu2esim classes.
    • Inputs
      • DTC_SimMode mode: The SimMode (currently Disabled, Tracker, Calorimeter, CosmicRay, or Hardware) that the DTC should be put into.
    • Outputs:
      • DTC_SimMode mode: The resultant SimMode of the DTC.

DMA Functions

  • GetData
    The main function that should be called by a Fragment generator. Returns a vector of pointers to the memory locations of the DataHeaderPackets in memory. The DataPackets associated with a DataHeaderPacket immediately follow the DataHeaderPacket in memory, and may be accessed by iterator semantics on a (uint8_t)* pointer initialized from one of the void* pointers returned by this function.
    • Inputs
      • DTC_Timestamp when = DTC_Timestamp(): Value to fill the Timestamp fields of the DataRequestPacket (Optional, for use with sendDReq and sendRReq parameters)
      • bool sendDReq = false: Whether the DTC class should send a DataRequestPacket for DTC_Timestamp "when" (Optional, when the CFO is not present/in use)
      • bool sendRReq = false: Whether the DTC class should send a ReadoutRequestPacket for DTC_Timestamp "when" (Optional, when the CFO is not present/in use)
    • Outputs:
      • std::vector<void*> (Function return value): Vector of pointers to the DataHeaderPackets in-memory
  • GetJSONData
    A convenience function that performs a GetData call and returns a JSON-formatted string using the resulting data. Function parameters are the same as for GetData, returns a JSON string.
  • DCSRequestReply
    Performs a Request/Reply cycle on the DCS channel. Expects a pointer to a uint8_t[12] array that will also contain the output.
    • Inputs:
      • const DTC_Ring_ID& ring: Which ring to get data from
      • const DTC_ROC_ID& roc: Which ROC on that ring to get data from
    • Input/Output Parameters:
      • uint8_t* dataIn: Pointer to an size-12 array containing the DCSRequest data. Will be filled with the DCSReply data.
  • SendReadoutRequestPacket
    Sends a broadcast Readout Request to the specified Ring. If fewer than 5 ROCs are connected to that ring, be sure to set the number of ROCs in the Ring before calling this function
    • Inputs:
      • const DTC_Ring_ID& ring: Which ring to get data from
      • const DTC_Timestamp& when: Value to fill the Timestamp fields of the DataRequestPacket
  • WriteDMADAQPacket, WriteDMADCSPacket
    Test functions used to send a DMA Packet on the DAQ or DCS Channel.
    • Inputs:
      • const DTC_DMAPacket& packet: The packet to transmit to the DTC
  • ReadNextDAQPacket
    Function called by GetData. Reads the next packet from the DAQ channel, obtaining a new buffer from the DTC if necessary.
    • Outputs:
      • DTC_DataHeaderPacket: The next DataHeaderPacket in sequence
  • ReadNextDCSPacket
    Function called by DCSRequestReply. Reads the next packet from the DCS channel, obtaining a new buffer from the DTC if necessary.
    • Outputs:
      • DTC_DCSReplyPacket: The next DCSReplyPacket in the buffer

Register IO Functions

  • RegDump
    Returns a JSON-formatted string containing the current state of all DTC registers. This string can be (and has been) imported into a JavaScript application for live status displays.
  • ConsoleFormatRegDump
    Returns a pretty-print formatted string containing the current state of all DTC registers.
  • RegisterRead
    Reads the given register and returns the value as a hex-formatted std::string
    • Inputs
      • const DTC_Register& address: The register to read
    • Outputs
      • std::string (function return value): The current value of the register, as a 8-character hex string
  • ReadDesignVersion
    Reads the design version registers and returns a std::string in v<MAJOR>.<MINOR>_<YYYY>-<MM>-<DD>-<HH> format.
  • ReadDesignVersionNumber (v1_04_00)
    Reads the design version number register and returns a std::string in v<MAJOR>.<MINOR> format
  • ReadDesignVersionDate (v1_04_00)
    Reads the design version date register and returns a std::string in <YYYY>-<MM>-<DD>-<HH> format.
  • ResetDTC
    Performs a DTC Reset operation
  • Toggle(RingEnabled, ClearLatchedErrors (Removed as of v1_02_01), CFOEmulation)
    Sets the register to the opposite state from its current state and returns the new state as a bool value
  • Read(ResetDTC, ClearLatchedErrors (Removed as of v1_02_01), SERDESLoopback, SERDESOscillatorIICError, SERDESOScillatorInitializationComplete, RingEnabled, SERDESRXDisparityError, SERDESRXCharacterNotInTableError, SERDESUnlockError, SERDESPLLLocked, SERDESOverflowOrUnderflow, SERDESBufferFIFOHalfFull, SERDESBufferStatus, TimestampPreset, FPGAPROMProgramFIFOFull, FPGAPROMReady, ResetSERDES, ResetSERDESDone, TriggerDMATransferLength, MinDMATransferLength, CFOEmulation, PacketSize, DMATimeoutPreset, DataPendingTimer, FPGACoreAccessFIFOFull)
    Returns the current value of the register
  • ReadROCEmulator, ToggleROCEmulator
    I/O for the ROC Emulator Enable Register
    • Inputs
      • DTC_Ring_ID& ring: Ring to query the ROC Emulator for
    • Outpu
      • bool (function return value): Whether the ROC Emulator is enabled on the ring.
  • Clear(SERDESRXDisparityError, SERDESRXCharacterNotInTableError, SERDESUnlockError, SERDESEyescanError) (Added as of v1_02_01)
    Remove individual error bits
    • Inputs
      • DTC_Ring_ID& ring: Ring to reset error bits on
    • Outputs
      • boolean, DTC_SERDESRXDisparityError, DTC_CharacterNotInTableError (function return value): The value of the error register for the ring after the operation.
  • Set(TriggerDMATransferLength, MinDMATransferLength, PacketSize)
    Set the DMA parameters
    • Inputs
      • uint16_t length: The size of the DMA transfer limit
  • SetSERDESLoopbackMode
    Sets the SERDES Loopback mode on a given ring to the specified mode
    • Inputs
      • const DTC_Ring_ID& ring: The ring to set the loopback
      • const DTC_SERDESLoopbackMode& mode: The mode to set. See the DTC_SERDESLoopbackMode enumeration in DTC_Types.h for more information.
    • Outputs
      • DTC_SERDESLoopbackMode (function return value): The Loopback mode of the ring after the operation
  • EnableRing
    Enables the given Ring, optionally setting the number of ROCs on the ring
    • Inputs
      • const DTC_Ring_ID& ring: The Ring to enable
      • const DTC_ROC_ID& lastRoc: (optional) The last ROC on the ring. If omitted, it will not be set, and should be set by the SetMaxROCNumber function before any broadcast packets are sent.
    • Outputs
      • bool (function return value): The Ring's enable state (will be false if error occurred)
  • DisableRing
    Disables the given Ring
    • Inputs
      • const DTC_Ring_ID& ring: The Ring to disable
    • Outputs
      • bool (function return value): The Ring's enable state (will be true if error occurred)
  • ResetSERDES
    Performs a reset of the given Ring's SERDES. This function will block until the reset completes.
    • Inputs
      • const DTC_Ring_ID& ring: The Ring to reset
      • int interval: (optional) The interval to check the SERDESResetDone register and flip the ResetSERDES register. If omitted, defaults to 100 ms.
    • Outputs
      • bool (function return value): Whether the reset completed (will be true unless something exceedingly strange happened).
  • Write(DMATimeoutPreset, DataPendingTimer)
    Set the DMA timeout parameters
    • Inputs
      • uint32_t value: The value to set the register
  • SetMaxROCNumber
    Set the ID of the last ROC in the specified Ring. This quantity may also be specified during an EnableRing call.
    • Inputs:
      • const DTC_Ring_ID& ring: Which Ring to set
      • const DTC_ROC_ID& lastRoc: The ID of the last ROC in the Ring
    • Output:
      • DTC_ROC_ID (function return value): The actual value of the register after the operation.
  • GetRingROCCount
    Get the ID of the last ROC in the specified Ring.
    • Inputs:
      • const DTC_Ring_ID& ring: Which Ring to read
    • Output:
      • DTC_ROC_ID (function return value): The value of the register.
  • WriteTimestampPreset
    Write a new value to the TimestampPreset register. This register is only occasionally used (see the DTCHardwareUsersGuide for more information).
    • Inputs
      • const DTC_Timestamp& preset: The preset to set
    • Outputs
      • DTC_Timestamp (function return value): The value of the preset register afterwards
  • ReloadFPGAFirmware
    Forces the DTC to reload its firmware. The host system will have to be rebooted after this function is called.

PCIe/DMA Status and Performance

  • StartTest
    Start a test of the DMA board with the given options. See the Kintex-7 manual for more information.
    • Inputs
      • const DTC_DMA_Engine& dma: The DMA engine (either DTC_DMA_Engine_DAQ or DTC_DMA_Engine_DCS)
      • int packetSize: The packet size to use for the test
      • bool loopback: Whether the SERDES loopback should be used (incompatible with txChecker or rxGenerator)
      • bool txChecker: Whether the TXChecker should be used (incompatible with loopback)
      • bool rxGenerator: Whether the RXGenerator should be used (incompatible with loopback)
    • Outputs
      • DTC_TestMode (function return value): The TestMode struct for the given DMA engine after command completion. See the DTC_TestMode struct in DTC_Types.h for more information.
  • StopTest
    Stop any tests running on the given DMA engine
    • Inputs
      • const DTC_DMA_Engine& dma: The DMA engine (either DTC_DMA_Engine_DAQ or DTC_DMA_Engine_DCS)
    • Outputs
      • DTC_TestMode (function return value): The TestMode struct for the given DMA engine after command completion. See the DTC_TestMode struct in DTC_Types.h for more information.
  • ReadDMAState
    Reads the current state of a DMA channel/direction pair
    • Inputs
      • const DTC_DMA_Engine dma: The DMA engine (either DTC_DMA_Engine_DAQ or DTC_DMA_Engine_DCS)
      • const DTC_DMA_Direction dir: The direction to get stats for (either DTC_DMA_Direction_C2S for Card-2-Server or "Transmit", or DTC_DMA_Direction_S2C for Server-2-Card or "Recieve")
    • Outputs
      • DTC_DMAState (function return value): The current state of the given channel/direction pair. See the DTC_DMAState struct in DTC_Types.h for more information.
  • ReadDMAStats
    Read the statistics of a DMA channel/direction pair
    • Inputs
      • const DTC_DMA_Engine dma: The DMA engine (either DTC_DMA_Engine_DAQ or DTC_DMA_Engine_DCS)
      • const DTC_DMA_Direction dir: The direction to get stats for (either DTC_DMA_Direction_C2S for Card-2-Server or "Transmit", or DTC_DMA_Direction_S2C for Server-2-Card or "Recieve")
    • Outputs
      • DTC_DMAStats (function return value): A collection of statistics for the selected channel/direction pair. Each entry contains LBR (Last Byte Rate), LAT (Last active time) and LWT (last wait time). Access using at(index) and get the count using size() on the DTC_DMAStats object.
  • ReadPCIeState
    Reads the PCIe state
    • Outputs
      • DTC_PCIeState (function return value): A struct containing information about the current PCIe state. See the DTC_PCIeState struct in DTC_Types.h for more information.
  • ReadPCIeStats
    Reads the PCIe Stats register
    • Outputs
      • DTC_PCIeStat (function return value): A PCIeStat object containing a LTX (last Transmit) and a LRX (last Recieve) statistic.

Exceptions

The DTCInterface Library will throw an exception in the following cases:
  • The packet read from the DMA engine is of an unexpected type
  • An operation failed (in cases where this would not be otherwise evident by checking the function's output and is important to the continued operation of the DTC)
  • Data Corruption is detected by the Library (Causes at least one DMA operation to be invalidated)