Project

General

Profile

Building up the PUBS online testbed

Note that the PUBS testbed and daemon should only be run on ubdaq-prod-near2.fnal.gov so that there won't be conflicts with the production daemon.

Login into ubdaq-prod-near2 and you may also need a production proxy in order to do some of the data transfers and accessing dCache resources. In order to get a proxy, do the following:

get-cert
voms-proxy-init -hours 24 -voms fermilab:/fermilab/uboone/Role=Production -cert /tmp/x509up_uNNNN.p12

where NNNN is a number that should be displayed after the get-cert command. You will also need to have the production role in VOMS. Once that is complete, then you are ready to start running things. Next checkout the devel version of PUBS from github.

git clone https://github.com/drinkingkazu/pubs.git

Once you've downloaded the repo, switch over to the devel branch or the feature branch of your choice.

 git checkout devel

Next you want to configure the environment so that you are using the testbed database (testprocdb) instead of the production version.

 source ~/pubs/config/setup_test_online.sh 

That assumes that you've checkout the repo in your home area, and not that you may want to make changes to the version of products setup by that shell script (e.g. setup uboonecode from your own development area, change uboonecode version, etc). Your milage may vary. Note that you should use the following configuration files to register projects in the testbed:

pubs/dstream_online/cfg/proj_evb_testbed.cfg
pubs/dstream_online/cfg/proj_near1_testbed.cfg

These configuration files change the normal directories that are used for staging the data and swizzled files.

/data/uboonedaq/rawdata/       --> /data/uboonepro/pubs_testbed_evb/
/datalocal/uboonepro/rawdata/  --> /datalocal/uboonepro/pubs_testbed_near2/
/datalocal/uboonepro/swizzled/ --> /datalocal/uboonepro/pubs_testbed_swizzled/

At this point, you should be setup to start the daemon, or issue the command interactively (e.g. python dstream_online/<proj_script> <proj_table>). If you want to really truly want to start from scratch, then you need to first delete all of the projects from the testprocdb, then delete the MainRun table. This is done using:

list_projects

To get a list of the projects. Please be really, really certain your working on testprocdb before you do that. You can test with:

echo $PUB_PSQL_ADMIN_DB

It should be testprocdb and NOT procdb. Next you remove all existing projects with:

sbin/remove_all_projects
And then finally, recreate the MainRun table using:
sbin/remove_runtable MainRun
sbin/create_runtable MainRun

Then go through the process of creating and registering all of the projects:

sbin/register_project dstream_online/cfg/proj_evb_testbed.cfg
sbin/register_project dstream_online/cfg/proj_near1_testbed.cfg

You should note, that the regiser_new_runs project requires you to have runs already in the MainRun table, so those need to be added with dstream_online/temp_insert_into_mainrun.py which looks for a file RunSubrun.txt in the current directory. It's a flat text file which has a run and subrun on each line separated by a tab. It's pretty simple. Do this command:

python dstream_online/temp_insert_into_mainrun.py

And put in at least run 2 / subrun 0. Next, running the register_new_runs project looks for data files in /data/uboonepro/pubs_testbed_evb/ for *.ubdaq files. But since the real data files might be in the middle of being written, the highest run/subrun combination is ignored. So you have to have at least two files in that area for one of them to be register in the MainRun table. If you want to sync up all of the projects by hand, instead of having the daemon do it for you, you need these commands:

python
>>> from dstream.ds_api import ds_writer
>>> from dstream.ds_api import ds_master
>>> k=ds_writer()
>>> k.connect()
>>> l=ds_master()
>>> l.runsynch()

Note that the connect() command will tell you which database you're connected to and you should make sure that you are connected to testprocdb before issuing the runsynch() command. Also, if you only want to synch a single project, you can give the project name as an arguement to runsynch (e.g. l.runsynch('get_binary_checksum_evb'))

Questions? Ask Kirby