Generally, conditions data is an approximate representation of a set of quantities (or a tuple) as a function of observation (or validity) time. Condition database approximates such a function as a sequence of validity intervals of time, during which the function is considered to be constant.
There are 3 products used at FNAL for storing conditions data:
- Minerva conditions database, named after the the experiment it was implemented for initially * ConDB, or NOvA conditions database * UConDB or Unstructured Conditions Database
All these products are designed, developed and used to represent conditions data and have their own features to cover several classes of use cases. They all follow common design architecture. To provide scalable and high performance data access for off-line analysis, they have a HTTP REST interface and users are encouraged to use this interface as the access method for data processing jobs.
Minerva Conditions Database¶
It was developed for Minerva experiment in 2010. Its data model is designed to represent an array of tuples as a function of observation (or validity) time. The array is assumed to be indexed by an integer channel number. The channel space does not have to be contiguous. In Minerva Conditions Database, all the channels are updated at the same time and therefore have the same validity time.
ConDB was developed for NOvA in 2014 to meet requirements their requirements. ConDB is a generalization for Minerva Conditions Database, which assumes that all the channels have the same validity time. ConDB allows each channel to have its own validity timeline. This generalization allows the database to have another important feature. When new data is recorded in ConDB, new values are compared to previous values and new values are recorded only for those channels, for which the changes in values are greater than the user specified tolerance values. This feature is possible only if channels are allowed to have their own validity intervals and it saves the space in the database in case of large number of channels.
Minerva Conditions DB and ConDB operate with structured data. They represent an array of tuples, indexed by an integer channel number, as a function of validity time. Although, this data model covers a wide range of use cases, but unfortunately not all conditions data can be easily represented that way. There is a class of applications where the data has more complicated and often non-constant structure. For example, some conditions data comes in the format of JSON, XML or FHiCL files. In these cases, interpretation of the data is done by the application and the database does not need to be aware of the data structure. Therefore, from the database standpoint, it records a history of changes of an unstructured object as a function of validity time. Hence the name of the product - Unstructured Conditions Database.
UConDB is a collection of named folders. Folders contain named objects and each object can have one or more versions. Each version has its own validity time. So the sequence of the timestamped versions of an object is the record of its history over the validity time.