artdaq-related work list, January 2017

[Initial version, 04-Jan-2017, KAB]

Enhance dataflow monitoring
  • Convert the existing monitoring to use separate threads so that it continues even when the dataflow is very slow or stopped (Issue #8892)
  • Add quantities to be monitored (i.e. Issue #11753)
  • Choose/develop inter-(artdaq)-process communication mechanism (so that a BoardReader can know when to throttle triggers)
  • Develop additional tools (e.g. beyond Ganglia) for visualization?
Implement DataFlow Orchestrator (Routing Master)
  • Choose/develop communication mechanisms (Trigger to DFO, DFO to EB)
    • Technology choices and message formats
  • Discuss/define/refine/document the dataflow control behavior
  • Develop DFO application
  • Add “release-buffer” message to RootMPIOutput and RootOutput
Improve Disk-Writing Capabilities, if needed
  • Provide assistance in evaluating hardware options
  • Modify artdaq and/or ROOT code, if needed (direct I/O?) (Issue #13657)
Assist in evaluating and choosing DAQ cluster hardware
  • Goal is to define the DAQ computing topology by 01-Mar-2017
Configuration Management
  • Work with protoDUNE DAQ and electronics folks to develop the online configuration management subsystem for protoDUNE
  • Design and develop the run number broker and FHiCL file generator
  • Develop/provide dataflow-configuration example(s) for 2 Central Trigger Board data paths (triggered stream to full event building and continuous stream to local disk)
Run Control
  • Decide on role for SDAQInterface, if any
  • Develop interface to deliver DAQ monitoring metrics to JCOP (Issue #15017)
  • Design and develop interaction of JCOP with DAQInterface
  • The organigram document talks about process management as part of Run Control – not sure what is needed there
  • Something functional is needed for APA testing – goal is mid-March 2017
  • Target completion of everything is July 2017
Assist in the development of experiment-specific FragmentGenerator code
  • Documentation, tutorials, training
  • Core artdaq changes to provide common functionality (if any)
  • protoDUNE-specific common functionality

Work with art team (or whomever) to get reconfiguration between runs working

Remove the old auto-file-closing logic from the Aggregator (Issue #14516)

Provide socket-based dataflow (as alternative to MPI over Ethernet) (Issue #15013, Issue #15015)

Assist with the setup of the DAQ software environment on official DAQ computers
  • Develop a standard deployment and usage pattern
  • Install the UPS-based software, when/where needed
  • Work with system administrators to specify system packages that are needed and get them installed

Assist with integration and system testing

Functionality requests/requirements:
  • Configurable ability to “time out” trigger requests in the FragmentGenerators using either a time interval or a trigger timestamp delta (Issue #13903)
  • Ability to mark various error conditions in artdaq::RawEvents and artdaq::Fragments (e.g. incomplete event), and support for special handling downstream (e.g. write to a special stream)
  • Ability to continue a run with a crashed BoardReader
  • Ability to continue a run with a crashed EventBuilder
  • (refer to the Dataflow Discussion Summary document)
Changes noted from other experiments:
  • Develop circular buffer for use in FragmentGenerators

Will we need to support the dispatching of DQM events from EBs (rather than an AG)?

“artdaq” needs to be ready for APA testing in April/May 2017. Goal is to have the full set of artdaq changes done by December 2017.