Running a novaddt process online¶
This page explains how to spin up a DAQ run on NDOS and FD such that it can receive DDT messages.
Quick Setup Instructions¶
1. Spin down the current DAQ run and bring down the DAQ system (including "Stop Run Control" and "Stop Resource Manager").
2. Log into the desired :
NDOS: ssh firstname.lastname@example.org FD: ssh email@example.com
3. FD Only: Change the underlying DAQ test release by moving the following configuration file into the
config area (the first command makes a backup copy of what is currently in there, it might be that there is nothing in there in which case you don't have to worry about it):
cp ~/ddt/config/daq-operations.cfg ~/ddt/config/daq-operations.cfg.daq.original cp ~/ddt/daq-operations.cfg ~/ddt/config/.
4. Restart the DAQ and bring it all the way through the "Prepare Configuration" stage (on NDOS, you may only have to stop the run to do this, but I have boot tested it yet). Then copy the Global Trigger configuration into place:
cp ~/ddt/GlobalTriggerRunConfig.xml ~/ddt/config/.
5. Take the DAQ through the remaining stages and start the run.
6a. You can now send trigger messages from the :
NDOS: ssh novadaq@novadaq-ctrl-farm-##.fnal.gov FD: ssh novadaq@novadaq-far-farm-##.fnal.gov
6b. Here is how you would start the trigger process on the right out of the DDT base release:
setup_novaddt source ~/ddt/ospl_setup.sh ddt-filter -c NDOS-Trigger.fcl
7. After you are finished, switch back to the , bring the DAQ system all the way down again (including "Stop Run Control" and "Stop Resource Manager").
8. FD Only Wipe away the test release configuration file on the :
rm ~/ddt/config/daq-operations.cfg mv ~/ddt/config/daq-operations.cfg.daq.original ~/ddt/config/daq-operations.cfg
9. The DAQ system is now in its original state, so you can hand it back to the shifter to start a regular run.
- Remember that the standard partition for NDOS is 0 and for FD is 1, so make sure your FHICL file sets the partition number as follows:
NDOS: physics.analyzers.sender.rms_partition_number: 0 FD: physics.analyzers.sender.rms_partition_number: 1
- You can find the data here:
NDOS: novadaq@novadaq-ctrl-master:/data/NDOS/ FD: novadaq@novadaq-far-datadisk-03:/data-a/FarDet/
This note below include some working instructions for setting up a ddt process on NDOS. These instructions are a work in progress and will develop.
From a machine inside the FNAL.GOV domain connect to one of the NDOS buffer nodes:
Then setup the novaddt package, make and setup a test release:
setup_novaddt mkdir tamsett cd tamsett newrel -t development DDT
You then need to copy Martin Frank's custom DDT packages. These differ from the trunk package in that they're setup to use the version of rms which is needed to run online. Once you've copied these go into include and setup links to them:
cp -r ~/mfrank/ddt/DDTDemo . cd include ln -s ../DDTDemo . cd ..
Then link to his version of rms:
ln -s ~/mfrank/ddt/rms .
Then go into include again and setup the link for this:
cd include ln -s ../rms/cxx/include ./rms cd ..
You should check with the "ll" command that your include directory matches Martin's. Finally copy over the following script and source it:
cp ~/mfrank/ddt/ospl_setup.sh . source ospl_setup.sh
Now the DDT package is ready to go we can start a run on NDOS.
Once things are working in the head and you don't need to link against Martin's versions of things you can add in packages using CVS or SVN. For DAQ packages a command of the form:
addpkg -hd :pserver:firstname.lastname@example.org:/cvs/nova <package>
will work. Specifically for rms the command would be:
addpkg -hd :pserver:email@example.com:/cvs/nova rms make rms.all
For DDT packages do a kinit <username> then:
addpkg_svn -h <package>
Spinning up a run on NDOS¶
End the run:
- click end run in run control * click release resources * stop run control by clicking stop run control button (icon bottom left) * pop up appears and click partition-0, this is where run control is killed * check that resource manager shows null partitions (no partition 0), if it's there right click then release * stop resource manager (icon bottom left)
Now we're ready to add in our special DDT configuration, go to a terminal window and copy a config file that will overwrites which test release we'll use:
ssh novadaq@novadaq-ctrl-master cd ddt cp daq-operations.cfg_Partition0_stage config/daq-operations.cfg
Note this must be removed once we're done.
Start the run:
- get a kerberos ticket (icon left) * start resource manager (icon bottom left) * start run control * select Partition-0 * click discover resources, select resources comes up, click ok * reserve resources, DAQ application manager appears - trigger scalars etc, wait for everything in application manager to go green * select configurations, select top from drop down * prepare configuration
Next edit/replace the DDT configuration xml file produced in the "prepare config" step previously so that it includes and enables the DDSNSet triggers and disables the beam spills and pulsers:
cp GlobalTriggerRunConfig.xml config/.
Finally load, make, configure etc, click all buttons all down until we're ready to begin run.
Sending the trigger signals:¶
From the buffer node we previously setup and run the filter:
ddt-filter -c job/NDOS-Trigger.fcl
And voila. Finally the data ends up:
You can view memory from novacr02 and you can open the event view too (icon bottom right) (start auto advance).