Run Control Workflow on the DCM

(2010-05-26, S. Kasahara)

This document is a DCM specific version of the more general [[kab-test-project:Proposed Run Control Flow]] document posted by Kurt Biery on 2010-05-25.
describing the proposed Run Control States and Actions.

The following describes the corresponding State or Action of the DCM Application during one of the proposed Run Control State/Actions.

No Resources State

The DCM application (dcmapplication) should be started on boot of a dcm so that during the NoResources State
the dcmapplication will already be running. The configuration information given to the dcmapplication on boot will be
minimal but should include at minimum:

  • Its application name.
    • In normal operation mode, the application name will correspond to the host name number using the format dcmNNN, e.g. dcm001 will be the application name on dcm node dcm-01.
    • There may be special circumstances in which the application name may vary from this scheme, for example in a test mode in which two dcm's are running on the same dcm node.
  • Its startup message facility configuration.

The DCM kernel device drivers should also be started on boot of the dcm node.

Reserve Resources Action

This action does not apply to the dcmapplication.

Resources Reserved State

Same as the "No Resources" State above.

Establish Partition Action

A DCM receiving an EstablishPartitionRequest message must also receive:

  • A partition number. This partition number is stored by the dcmapplication and distributed to the appropriate dcmapplication threads.

Actions taken by the dcmapplication:

  • Checks to see if it is already assigned to a partition. If so, it will return Error.
  • Sets up the RC listeners to listen for commands given to its assigned partition.

Partition Established State

  • With an assigned partition, the DCM is now in a wait state for the next command.

Configure Hardware Action

A DCM receiving a ConfigureHardwareRequest message must also receive:

  • A configuration file URI for the hardware configuration. This file is expected to be an .xml file, perhaps with the location of further hardware-specific configuration files embedded in it as part of its configuration instruction.

Actions taken by the dcmapplication:

  • Opens the configuration file. Failure to open returns Error.
  • If the configuration file contains no instruction (such as might be expected when running in simulation mode or test mode), return Success.
  • A configuration request to configure the dcm hardware requires the dcmapplication to interface with the /dev/dcm_control device driver.
    • A failure to open this device results in error.
    • Any other failure during hardware configuration returns error. The message returned on error will contain more specific information about the failed state.
  • Return Success when hardware configuration is complete.

Hardware Configured State

The DCM has now established a connection to the /dev/dcm_control device driver, configured the DCM FPGA as requested, and has an established partition.

Configure Software Action

A DCM receiving a ConfigureSoftwareRequest message must also receive:

  • A configuration file URI for the software configuration. This file is expected to be an .xml file. Some examples of configuration information which the dcmapplication receives:
    • A list of eventbuilder buffer nodes. For each buffernode, the information to be sent is:
    • Id - the EventBuilder indentification Id. These will just be numbered, e.g. 0,1,2,3,... and the sequential order of these be used to sequence the sending of the millislices to the eventbuilder buffer nodes. (The maximum Id number is 1023 as determined by the number of bits reserved for this field in the millisliceheader.)
    • Host - the EventBuilder host, e.g.
    • Port - the EventBuilder listening port, e.g. 7555
    It is possible for no eventbuilder buffer nodes to be in the list, which is useful in test environments in which the eventbuilder buffer nodes are not available.
    • Identification information for the dcm.
    • DetId - identifies the detector type (far,near,ndos)
    • Diblock Id - identifies the block within the detector.
    • DcmId - identifies the dcm on the block.
    The numbering scheme is described in DocDB #4390.
    • Whether the data taking mode is data or simulation, and the simulation type and location of input simulation files if appropriate.
    • Information about the length of a microslice in clock ticks and the length of a millislice in clock ticks.
    • Other operational information, e.g. whether empty microslices should be included in a millislice and whether a millislice index should be built, etc.
    The configuration data that is currently sent to the dcmapplication is described in DCMApplication/config/DcmConfiguration.xsd.

Actions taken by the dcmapplication:

  • If the dcmapplication is configured to:
  • Take real data, the /dev/dcm_data device will be opened. Failure to open this device will result in error.
  • Take simulated input data from a set of input files, the ability to open the list of input files will be tested. Failure to open one or more files will result in error.
  • Take simulated input data from the kernel device driver pattern generator, the pattern generator device driver will be opened. Failure to open this device will result in error.
  • If all actions succeed, return Success.

Fully Configured State.

The dcmapplication has successfully configured the dcm hardware and has initialized its connection to its input data source. It has not yet attempted to initialize connections with the event builder buffer nodes.

Make Connections Action.

A DCM receiving a MakeConnectionsRequest message requires no other information to be sent.

Actions taken by the dcmapplication:

  • For each event builder buffer node:
  • Establish a socket connection, using the host/port number associated with the event builder buffer node in the configuration list.
  • Send an empty MilliSlice with the InitializationRequest bit set and evb id, partition, and dcm id information set as part of the MilliSlice header.
  • If either one of these steps fails for any of the event builder buffer nodes in the configuration list, return error, else success.

Connections Made State.

The dcmapplication is fully ready to begin data taking, with connections established to the input data source and the output event builder buffer nodes.

Begin Run Action.

A DCM receiving a BeginRunRequest message must also receive:

  • A T0 start time for the run, measured in clock ticks. This is the T0 relative to which the construction of MilliSlices on all DCM's will be measured, and an identical value should be sent to all DCM's in the same partition. A failure to receive a T0 results in an error return.

Actions taken by the dcmapplication:

  • Set the T0 in necessary dcmapplication thread.
  • Data taking begins.

Running State.

Data is read from the input data source indefinitely. If the input data source runs dry (e.g. a limited set of input files), the dcmapplication just waits for further instruction. Sending millislice data to the event builder buffer nodes is carried out in round-robin fashion using an algorithm:
evb index in connection list = (millislice SEQ number) % (number of evb connections);
i.e. the evb is chosen by modulating the sequence number by the number of evb connections.

A procedure in the event of an evb buffer node connection failing has not been worked out, but will include attempts to reconnect with the unresponsive evb buffer node for some pre-determined time interval,and result in an error message sent to RunControl if such a failure persists beyond that. We will have to decide if this failure should result in a run stop/begin or design a work-around a broken evb buffer node such that the run can continue.

Pause Run Action.

The DCM takes no action to a PauseRunRequest, always returning Success.

Run Paused State.

The DCM continues to take and send data during a Run Paused State.

End Run Action.

A DCM receiving an EndRunRequest requires no other information to be sent.

Actions taken by the dcmapplication:

  • Finish creating the MilliSlice currently under construction and then stop the further receipt of new data.
  • Send all MilliSlices still in queue to the event builder buffer nodes before sending a EndRunResponse Success message to RunControl.

Run Ended State.

The dcmapplication has stopped taking new data and is not sending any data to the event builder buffer nodes. It maintains its connections and its configured state.