Project

General

Profile

Run Configuration Database Tool

The run configuration database includes all the fcl parameters used in the DAQ, and the configuration of the ASIC, the calibration pulser, and the external trigger. The status of the calibration fanout is not included.

Installation and Set Up

Installation

The run configuration database tool is now under Kazu's gitHub repository,

git@github.com:drinkingkazu/DBTool.git

It depends on ROOT, CMake, postgres, and both needs to be pre-installed on the machine.
One can clone by

 git clone https://github.com/drinkingkazu/DBTool 

Since it involves C++, one needs to build it. For a build procedure, refer to the set up section.

Set up for shifters (uboonedaq account on ubdaq-prod-X machine)

When logging into ubdaq-prod-X machine as a uboonedaq account, pre-installed DBTool is automatically set up.
You do not have to do anything to use it.

FYI: the tool of the run config database is installed at the home directory of uboonedaq,

/home/uboonedaq/DBTool

Set Up (non-uboonedaq account on ubdaq-prod-X machine)

For the first time to setup & build DBTool on ubdaq-prod-X machines, one need to call:

source DBTool/config/setup.sh

which set up necessary ups products and call cmake to prepare for a build.
After sourcing above script, you are placed under DBTool/build directory.
Then you can simply type
make

to build the DBTool

From the next time, you can either call setup.sh (which runs cmake unnecessarily) or

source DBTool/config/setenv_dbtool.sh

which does everything setup.sh except for moving into build directory and execute cmake.

Both setup.sh and setenv_dbtool.sh configures to use a test run config database by default.
If you would like to use a production version, currently only kterao, yuntse, and uboonedaq are allowed to do this.
This can be done by giving any argument to a script upon sourcing like

 source setup.sh 1

Set Up (in general, on your laptop or anywhere)

It is very similar to what is explained so far. You can do:

source config/setup.sh

to set shell environment and call cmake, or
source config/setup.sh

to just set shell environment variables.
Note that there is no sense of production vs. test DB (that is a specific set up of our online DB).
Also note your cmake may fail if you do not have postgres/ROOT/cmake pre-installed and configured to be used in your shell.

Basic functions

There are two categories of the configurations: one is main config, and the other is sub config. As the name indicate, the main config is the config including all the components, while the sub config is the config of a module, such as pmt, trigger, or assembler. A main config is uniquely identified by either its string name or associated integer ID key. A main config is meant to represent the configuration of a data taking run, and the ID key of a used main configuration for any run is stored in the MainRun database table.

Main Configuration: simple query examples

You can list a default set of main configs for data taking by

list_main_cfg

There are two additional categories than "default": "expert" and "archived".
The former is a main configuration used by experts and not meant to be exposed to a non-expert (such as shifters).
One can list experts' main configs stored in the database by

list_main_cfg expert

Both "default" and "expert" main config can be used for data taking.
The "archived" main configs, on the other hand, cannot be used for data taking.
You can list "archived" main configs by

list_main_cfg arxive

Finally, if you want to take a look at a list of included sub configuration,

print_main_cfg NAME_OR_ID 

where NAME_OR_ID should be either main configuration's string name or unique ID integer.

Sub Configurations: simple query examples

There is a similar query methods for listing/printing sub configs.
A sub config is uniquely identified by its string name AND integer ID key.
Sub configs carry no sense of "expert" nor "arxive" at the moment.

One can list sub configs stored in the database by

list_sub_cfg 

You can look at a specific sub config parameters by

print_sub_cfg <name> <id>

where NAME should be the string name and ID should be the corresponding ID key of the subject sub configuration.
For example,
list_sub_cfg pmt 1
list_sub_cfg assemblerAppevb 1

This command shows parameter key/value pair including nested children sub configs.
If you would like to list a full, extracted set of parameter values, you can do this by

list_sub_cfg <name> <id> detail

Dumping into FHICL file

To dump the content of a main config, you can do

print_fcl <main config name>

For example,

print_fcl external_trigger_v6_13_02_c0_noPMT_0_1Hz

It will generate external_trigger_v6_13_02_c0_noPMT_0_1Hz.fcl.

Running DAQ with the run configuration database tool

Currently we are using the runConsoleDAQ.sh script to take runs. We have included the run config database tools since uboonedaq v6_13_02. Shifters should do

runConsoleDAQ.py

It will connect to the run config database and show the list of the non-expert main configs. Shifters should enter the Config ID and the run config DB will assign the run number to this run according to the database. To take runs with the expert main configs, you just have to do

runConsoleDAQ.py expert

and choose the Config ID from the list.

Upload the configurations

The config files we use to upload the main and sub configs are located at the data/ folder under DBTool/. For convenience, the sub config files have the name convention data/*.cfg, while the main config files data/main/*main.

An example of the lowest layer of the sub configs is shown below,

SUB_CONFIG_START pmt 1
channels: 40
slots: 0x60
enable_top:    0xffff
enable_middle: 0xffff
enable_bottom: 0xffff
....
SUB_CONFIG_END

You have to specify the config name (pmt) and ID (1) at the beginning. Please note the pair of config name and config ID should be unique.

An upper layer of sub configs would look like

SUB_CONFIG_START sebAppseb01 1
   registration => 1
   xmit => 2
   tpc => 2
   neviscard_controller => 1
   neviscard_nu => 1
   nu_stream => 1
   controller => 1
SUB_CONFIG_END

where registration => 1 indicates that this sub config nests the sub config (registration, 1).

Below is an example of main configs,

MAIN_CONFIG_START external_trigger_v6_13_03_c8_0_1Hz PHYSICS
  sebAppseb01 => 1
  sebAppseb02 => 1
  sebAppseb03 => 1
  sebAppseb04 => 1
  sebAppseb05 => 1
  sebAppseb06 => 1
  sebAppseb07 => 1
  sebAppseb08 => 1
  sebAppseb09 => 1
  sebAppseb10 => 13
  dispatcherevb => 1
  online_monitornear1 => 1
  assemblerAppevb => 1
  ASIC => 1
  Pulser => 1
  ExtTrigger => 3
MAIN_CONFIG_END

The main config name, external_trigger_v6_13_03_c8_0_1Hz, should be unique, and the run type, PHYSICS, should be specified.
Other run type available include TEST and CALIBRATION.

After you update/create the sub config files, you can upload the sub configs to the database by

./sbin/upload_sub_cfg <sub config file>

such as

./sbin/upload_sub_cfg data/pmt.cfg
./sbin/upload_sub_cfg data/*.cfg

The script will detect the existing (sub config name, id) in the database and skip uploading these configurations.

Similarly, you can upload main configs by

./sbin/upload_main_cfg <main config file>

e.g.

./sbin/upload_main_cfg data/external_trigger_v6_13_02_c0_noPMT_reAllFT_1Hz_pulse.main

There are some checks during the uploading of upper layers of sub configs and main configs. For examples, the values of some parameters in different sub configs are required to be identical. If the check fails, the sub config or main config will not be uploaded. However, the checks haven't been completed yet.

The main configs which have been used in a run cannot be removed from the run config database.

HOW-TO: Quickly upload a configuration

Alternatively, you can also prepare a fcl file and upload the configuration by

./sbin/upload_fcl <fcl file>

e.g.

./sbin/upload_fcl PHYSICS_BNB_NUMI_MUCS_EXT9990mHz_SWTrigger_2Stream_c23.fcl

If the fcl file contains a subconfig with a name not existing in the run configuration database, such as eric_stream, you will not be able to upload the fcl with the function above. You will have to create a new subconfig with the function

./sbin/upload_sub_cfg <sub config file>

as described above. If you're merely changing the value of a parameter that already exists or even adding another one within an existing subconfig, certainly, this upload_fcl technique is the easiest way to make a new config. If you have permission issues, please contact Yun-Tse and Kazu.

HOW-TO: Modify the existing and upload

Often one wants to make a new sub config by modifying a few parameters of an existing sub config.
A recommended approach is to first generate a copy of sub config text file, formatted such that it can be uploaded.

print_sub_cfg <name> <id> format 

e.g.

$ print_sub_cfg sebAppseb10 1 format
SUB_CONFIG_START sebAppseb10 1
  controller => 1
  gps => 1
  neviscard_controller => 2
  neviscard_nu => 2
  nu_stream => 10
  pmt => 2
  registration => 10
  trg_stream => 1
  trigger => 1
  xmit => 4
SUB_CONFIG_END

Save this output to a text file with any name, say foo.txt.
Open foo.txt, and modify the contents and sub config ID.
Then upload it:

./sbin/upload_sub_cfg foo.txt

You can follow the same practice for main configuration.

Generating channel-wise parameters for zero-suppression algorithms in the continues (supernova) stream

Tools

The scripts to generate the channel-wise zero suppression parameters can be found here. The instruction is listed in the same page.

It is also installed at the home directory of uboonedaq,

/home/uboonedaq/ZSConfigGen

You have to input the script a fcl similar to

/home/uboonedaq/ZSConfigGen/config/2stream_5hz_100adc_allfem.fcl

as the reference of the other part of the DAQ configuration.

Once you have the reference fcl as the input, you can update the input and output fcl files in

GenConfig.py

and run
python GenConfig.py

Modifying uploaded contents

In general modification on the uploaded configuration (main or sub) cannot be modified except for special cases/components.

Main config "expert" and "archive" features

The uploaded main configs will be set to "expert" by default. You can move it to the non-expert list by

./sbin/nonexpert_main_cfg <main config name or id>

e.g.

./sbin/upload_main_cfg external_trigger_v6_13_02_c0_noPMT_reAllFT_1Hz_pulse

Any main config that should not be used for data taking in future should be archived (so that DB protects it from being used in data taking).
This can be done by

./sbin/arxive_main_cfg <main config name or id>

On the other hand, one can un-archive (i.e. activate) a main config by

./sbin/activate_main_cfg <archived main config name or id>

Renaming main config

One can rename a main config's string name (but not an integer ID key) by

./sbin/rename_main_cfg <main config current name or id> <new name>

Removing main/sub config

How dare you reading this section!
This is allowed by a DB super user (Kazu/YunTse/Andrzej) and is only allowed if a subject main/sub config is not used by anything else.
In case of main config, that means there is no official run taken with it.
In case of sub config, that means there is no other sub config including it.
If these conditions are met, uploaded configurations can be removed by contacting one of super users.
FYI the protection is embed in the database and not maintained by poor brains of super users.

Run info retrieval

This is the least explored area and peoples' suggestion/request is welcome.
DB is designed such that a query for a list/info of run by an arbitrary category should be easy and fast.

Run information

One can retrieve a run config information by

print_run_info <run> <subrun> 

Run list

Nothing is implemented but it is trivial to do so. Please send a request.
For instance we can make a run list by...

  • main config used
  • run type
  • data taking period
  • specific categories of sub configs used

or a combination of above. It will be implemented upon request.