Project

General

Profile

Sbndqm Workshop

Step 1: Building sbndqm

To setup a new environment with sbndqm, go to an empty directory in a fresh terminal. You should be on either sbnd-daq33.fnal.gov or icarus-gateway01.fnal.gov. Then, run:

source /software/products/setup
mkdir <sbndqm directory>
cd <sbndqm directory>
setup mrb
setup sbndqm v0_03_00 -q e17:prof:s83
export MRB_PROJECT=sbndqm
mrb newDev
source  localProducts_sbndqm_v0_03_00_e17_prof_s83/setup 
mrbsetenv

This will make a new environment for sbndqm. To build it, run “mrb i”. In the future, when logging into the server, run “source setup.sh” to activate the environment.

Step 2: Running an online monitor

NOTE: this next step will only work if you have an instance of artdaq running with the Dispatcher. Otherwise, you will not have a DAQ to connect to. In that event, you can still try out storing metric data in a database. In this case, run the run_example.fcl file and change the “metric” and “metric_config” blocks as specified below.

We are going to run the “example” online analysis program inside sbndqm (sbndqm/dqmAnalysis/example). This program does not inspect the data it receives from the DAQ. It just sends a set value to the database whenever it receives a new event from the DAQ.

We are going to need to change some things since everyone will be running the online analysis program at once. First, open the ExampleTimeStream_module.cc file and change:


// change this string to something different
const char *name = "example_metric”;

...

// change these strings to something different

// name of the metric -- should match the name we assigned in metric_config
const char *metric_name = "rms";
// name of the group -- should match the name we assigned in metric_config
const char *group_name = "example";

then rebuild. The names of these strings will set the names of values in the database, so if everyone uses the same names we will over-write each other’s data.

Next, we are going to to change some configuration. We are going to change the configuration for the service that makes a connection to redis.


// set for icarus-gateway
services.RedisConnection.host: “icarus-db01.fnal.gov”

// set for sbnd-daq33
services.RedisConnection.host: "sbnd-daq33.fnal.gov”
services.RedisConnection.passfile: "/var/adm/krb5/redis_password" 

This sets the hostname of the redis database and a file to lookup the password. The icarus redis instance does not have a password, so we do need to specify one.

Finally, you should change the the key in the “metric_config.groups” block from “example” to whatever you set “group_name” to. And, in the “metric_config.metrics” block, you should change “rms” to whatever you set “metric_name” to.

Next we will have to change the some configuration for connecting to the DAQ. First, change “dispatcherPort” to “6666” -- this is the port I have setup the artdaq Dispatcher to run on. Next change the “unique_label” to your own string and “shm_key” to your own hex number. Note that both of these are configured in two places -- they must match.

Now, you are ready to run the online monitor! Run, “lar -c run_example_daq_interface.fcl”.

Step 3: Inspecting the Database

Next, we are going to inspect the redis database to see how data is being stored. We will inspect the database using the python redis library. To setup the library, from a fresh terminal in an empty directory, run:


cd sbndqm/sbndqm/DatabaseTools/
virtualenv env
source env/bin/activate
pip install -r requirements.txt

Now, when you re-connect to the server, run “source env/bin/activate” to re-setup the environment.

Now that you have the environment setup, you can monitor the database using one of the provided python scripts in the DatabaseTools/ directory or be using the python command line steps below.

Scripts

The script monitor_streams.py will monitor the streams you have setup. Read the README file for instructions.

Python Command Line

Now, running the example online monitor set up two data streams to the DAQ -- one single metric and one group of metrics. We’ll first inspect the single metric. From a python command line, run:

import redis
r = redis.Redis(host=__host__) # whatever you configured hostname too earlier
r.keys(__name__*) # whatever you set name to before in the C file
# you should see a list of metrics with a name: __name__:__steam__ where __name__ is as before and __stream__ is one of slow, fast
r.xrange(__name__:__stream__)
# this will print out all of the data stored for this stream in the format (timestamp-0, {“dat”: __data__}) where timestamp is a millisecond UTC timestamp of when the data was stored

# Now we can examine the group of metrics. First we look at its configuration:
r.get(GROUP_CONFIG:__group_name__) # set __group_name__ as you did in the C file
# this is a JSON dictionary of the configuration for this group of metrics: it sets warning range, names of things, units, etc. The config is set in the fhicl file
r.lrange(GROUP_MEMBERS:__group_name__, 0, -1) # set __group_name__ as you did in the C file
# this is a list of all of the member metrics in this grouping. In this case, 1 - 10

# now, a specific instance of a metric in this group will be formatted as:
# __KEY__ == __group_name__:__instance_name__:__metric_name__:__stream_name__
# where: 
# __group_name__ is as before
# __instance_name__ is one of the items in the GROUP MEMBERS list
# __metric_name__ is what you set “metric_name” to in the C file
# __steam_name__ is one of the configured streams as in the example with the single metric

# so now we can inspect the values:
r.xrange(__KEY__)

Step 4: Viewing the website

Now, we can view the metrics plotted on the website. To see the single metric, from behind the FNAL proxy go to (SBND):


To see the single metric:
http://sbn-online.fnal.gov/cgi-bin/minargon/minargon.wsgi/single_stream/__name__/
__name__ is the name of your stream as set by the stream_name plus the postfix.

To see the group of metrics:
http://sbn-online.fnal.gov/cgi-bin/minargon/minargon.wsgi/online_group/__group_name__/
__group_name__ is whatever you set the group_name to in the fhicl file. 

or (ICARUS):


To see the single metric:
sbn-online.fnal.gov/cgi-bin/icarus-minargon/icarus-minargon.wsgi/single_stream/__name__/
__name__ is the name of your stream as set by the stream_name plus the postfix.

To see the group of metrics:
sbn-online.fnal.gov/cgi-bin/icarus-minargon/icarus-minargon.wsgi/online_group/__group_name__/
__group_name__ is whatever you set the group_name to in the fhicl file.