Project

General

Profile

Data Files from NDOS

CHECK WITH THE RUN COORDINATOR BEFORE DOING ANY OF THIS!

Running on One Buffer Node

This wiki entry describes how to collect data from NDOS and store it as a data file that can be used with our DDT framework.

First, you need to make sure that data is being collected. Check the MicroSlice rate of the DCMs on Ganglia , it should be 20 kHz.

Once that is verified you can log into one of the NDOS buffer nodes:

ssh novadaq@novadaq-ctrl-farm-01.fnal.gov

NDOS has 16 buffer nodes (01 - 16). Note that the novaddt user does not have permissions on the buffer nodes.

Now you just need to set up the environment and run the job:

setup_novaddt
ddt-filter -c ddt-writeFile.fcl

This will run on the data stream and dump the data into a file called ddt.root.

Files from live data should be stored in

/nova/ana/trigger/data/

with format

DDTData-<det>-<size>-<date>-<run>.root
where
  • <det> is one of NDOS, ND, or FD
  • <size> is number of milislices (1000,10000,etc)
  • <date> is of format yymmdd
  • <run> is 1, 2, 3, 4, etc.

Running on Several Buffer Nodes

When you run on several nodes, make sure that you do not write to the shared disk because the parallel jobs will then write to the same location. You should write to an area that is local to the buffer nodes (e.g. /scratch/mfrank).

Now that we have the nifty DAQApplicationManager for our DDT processes (see Running_the_Trigger_using_the_DAQApplicationManager), we can control all twelve buffer nodes at once. First, you need to edit the DAQApplicationManager FHICL file and enable the output stream. Now, you can start and stop your processes.

After you collected your data, you can transfer it to the VMs. Go to the following directory:

/nova/ana/trigger/data/
and use the following script to transfer your data from all the buffer nodes:
./transfer_files.sh <file name>

After all of the data is transferred, you can clean up with the following script run from novadaq-ctrl-master:

rgang.py --rsh=ssh 'novadaq-ctrl-farm-{01-10}' 'cd /scratch/mfrank;rm ddt.root; ls -lh'