Project

General

Profile

Updating the UPS Product

This page describes how to build a new version of the novaddt UPS product together with the packages that it depends on and how to roll these new products out on the detectors.

Building the Products

Here's how to build NOvA DDT products

0. Go to /nova/app/users/<user> without executing setup_nova

cd /nova/app/users/<user>

1. Make changes and commit it to trunk

<changes changes changes...>
svn commit <some_file.cc>

2. Checkout NOvA DDT release:

svn co svn+ssh://p-novaddt@cdcvs.fnal.gov/cvs/projects/novaddt/novaddt.svn/trunk novaddt

3. Renaming version in trunk:

cd <your build dir>/novaddt/ups
chmod +x ups_product_tag.sh 
./ups_product_tag.sh <new version> (example: ./ups_product_tag.sh v03_04_06)

4. Build novaddt with mrb

mkdir <builddir>
cd <builddir>
. /grid/fermiapp/products/nova/externals/setup
setup mrb
export MRB_PROJECT=nova
mrb newDev -v <new version> -q e6:on:prof (example: mrb newDev -v v03_04_06 -q e6:on:prof)
source localProducts_nova_<new version>_e6_on_prof/setup (example: source localProducts_nova_v03_04_06_e6_on_prof/setup)
cd srcs
svn co svn+ssh://p-novaddt@cdcvs.fnal.gov/cvs/projects/novaddt/novaddt.svn/tags/<new version>  novaddt (example: svn co svn+ssh://p-novaddt@cdcvs.fnal.gov/cvs/projects/novaddt/novaddt.svn/tags/v03_04_06  novaddt)
>> [
>> if there's a problem with some dependencies, say *novaddtdeps*, you'll need to do ... and then go back to *mrbsetenv* step
>> useful commands:
>> ups list -aK+ <package name> (example: ups list -aK+ <novaddtdeps>) - to list versions of that package available
>> 
>> ]
mrb uc
cd $MRB_BUILDDIR
mrbsetenv
mrb i -j8 --tee

5. Test if new version works as intended

cd <your build dir>
. /grid/fermiapp/products/nova/externals/setup
export PRODUCTS=<your build dir>/localProducts_nova_<new version>_e6_on_prof/:$PRODUCTS
cd ddt_release_02.09.2015/product/
setup novaddt <new version> -q e6:on:prof (example: setup novaddt v03_04_06 -q e6:on:prof)
>> uncomment DDTGlobalConfiguration-FD.fcl after "# if using nova binary:" 
nova -c srcs/novaddt/DDTGlobalConfigurations/DDTGlobalConfiguration-FD.fcl

Deploying the Products

Install build to FD.

1. Ssh to FD server

ssh novadaq@novadaq-far-master

2. Copy new build ot FD server (from gpvm node, from your build directory)

rsync -avP localProducts_nova_v03_04_06_e6_on_prof/novaddt novadaq@novadaq-far-master:/ddt/externals/.

3. Check that code works

. /ddt/externals/setup
setup novaddt v03_04_06 -q e6:on:prof
>> (a bit questionable) nova -c srcs/novaddt/DDTGlobalConfigurations/DDTGlobalConfiguration-FD.fcl

Start changes in control room

(Make track of what you'r doing n ECL)
Stop run
Start DAQ config editor on Monitor1
Config is FarDetGlobalConfigP1
Change build version manually
Enter
Start new run

Building with Jenkins

Using Jenkins simplifies build process.

1. Get certificate to enter Jenkins if you don't have one

General instructions: https://fermi.service-now.com/kb_view_customer.do?sysparm_article=KB0010657
Get KCA server CA (https://security.fnal.gov/pki/fermi-kca-ca.pemcertificate)
Get KCA certificate. For SL6 x86 this link helped me, since yum install krb5-fermi-getcert didn't work. Install packages krb5-fermi-*. http://ftp.scientificlinux.org/linux/fermi/slf6x/i386/os/FermiPackages/

2. Go to Jenkins

https://buildmaster.fnal.gov/view/Nova/

3. Go to build configuration

NOvA -> novaddt_build -> Configure

4. Make changes in version. For unclear dependensies go to novaddt/novaddtdeps repo -> tags -> tag you search -> ups -> product_deps

5. Make build (build with parameters)

6. Debug if any problem appears

Deploying products after Jenkins at FD

Do same steps as deploying after mrb build, but downloading to FD server looks differently:
1. Ssh to FD server

ssh novadaq@novadaq-far-master

2. Get link from Jenkins

Jenkins->novaddt_build_output->Last successful atrifacts->link for zip)

3. Download build to download area

cd <download folder> (/ddt/externals/DOWNLOAD for FD)
wget <link from Jenkins>

4. Unzip and untar link from Jenkins to /ddt/externals

5. Check that code works

6. Go to control room to make changes

Deploying products after Jenkins at ND

1. Ssh to ND server

ssh novadaq@novadaq-near-master

2. Get link from Jenkins

Jenkins->novaddt_build_output->Last successful atrifacts->link for zip)

3. Download build to download area

cd <download folder> (/nova/novaddt/ups/DOWNLOAD)
wget <link from Jenkins>

4. Unzip and untar link from Jenkins to /nova/novaddt/ups

5. Check that code works

6. Go to control room to make changes

Extended version of how to deploy products to FD

1. Upload new build to novadaq@novadaq-far-master

2. svn is not installed on novadaq@novadaq-far-master. You need to use novadaq@novadaq-far-master-02 to use svn to update DDTGlobalConfiguration-FD.fcl
Note: you may need to log in to novadaq@novadaq-far-master-02 through novadaq@novadaq-far-master, since direct access is not allowed for some reason.

3. Update fcl using instructions from instruction on updating GlobalConfiguration

4. Pick a buffer node which is in the run (see DAQ application manager or ask RunCoordinator if needed) and log in there. By default, node 70 is ok.

ssh novadaq@novadaq-far-farm-70

5. Copy fcl file (it gives you name of the file after step 3) to your directory on ssh novadaq@novadaq-far-farm-70 and change it so it doens't run in verbose and not producing trigger messages.

6. Make setup

. /ddt/externals/setup
setup novaddt <version> -q <qualifier> (for example: setup novaddt v03_09_02 -q e6:on:prof)

7. Interprocess communication shell setup

ipcs (it shows you shared memory segments, you need to pick up most used one)
export SHMRW_KEY=0x0001700F

8. Run your fcl using command allowing to connect to shared memory segment

ddt-filter -c <DDTGlobalConfiguration.fcl in your folder>

9. If it runs that means it works. You want to make sure you see event-by-enent messages that says you it's looking at events

10. Stop the run, go to DAQConfigurationManager -> DDTConfiguration -> change fcl and ddt version to latest one -> restart the run

Accesing DB to see versions of DDTGlobalConfiguration.fcl versions used for run

(See https://cdcvs.fnal.gov/redmine/projects/novadaq-config-mgmt/wiki/Useful_Commands_and_Files_for_DB_Configuration for more info)

setup_online
source `which setupNovaDBEnvVars.sh`
runDBQuery "select * from fardet.ddt_manager_connect_parameters order by id desc"