Project

General

Profile

Microboone products maintainers guide

The instructions in this guide apply to microboone hand-maintained products, such as ubtools, ubfcl, and ubxml. These products have the following features.
  • They contain only data and/or scripts, but not compiled code.
  • They are not packaged for distribution (except via cvmfs).
  • They are available only on the microboone interactive login nodes, uboonegpvmXX, and possibily via cvmfs.

Repository organization

The sources for microboone products are in the ubooneoffline svn (for now) repository.

The head version of each product is stored at svn relative path

trunk/products/<product name>

Tagged versions of each product are stored at svn relative path

products/<product>/<version>

Updating the uboonecode repository

To build uboonecode go to
Building a new version of the uboonecode product

When we build a new official uboonecode version we want to tag it. Those instructions follow.

Get the fresh copy:

mrb newDev...
source localProductsXXX/setup
cd $MRB_SOURCE
mrb g uboonecode

or update the one you have
cd $MRB_SOURCE
mrb pull origin develop

Now, go make a new branch called release (where <tag>=vX_YY_ZZ), ala
git flow release start <tag>
Then edit ups/product_deps, changing the parent release number and the larsoft release number.
Then go do a build to make sure there are no problems.
Then go finish the git flow release, commit, which will merge your changes and tag to both origin and master.

cd $MRB_SOURCE/uboonecode
git flow release finish <tag>

Finally, push and push the tag. When you push here you may be told you're not current with master, and it will tell you how to rectify that. Follow those directions and resume your push.
git commit -a -m "Tagging vX_YY_ZZ" 
git push origin master develop
git push --tag

Updating the ubooneoffline repository

The instructions below work for ubtools, ubfcl, and ubxml. For the uboone product you must follow separate instructions.

Building a new version of the uboone product

If you want to update a product, check out the head version under trunk/products.

export CVSROOT=svn+ssh://p-ubooneoffline@cdcvs.fnal.gov/cvs/projects/ubooneoffline/
svn co $CVSROOT/trunk/products/<product>
cd <product>

Edit the product, then commit your changes using svn commit.

svn commit -m "A meaningful comment" 

Making a new version tag

Make a new version by doing an svn directory copy.

export CVSROOT=svn+ssh://p-ubooneoffline@cdcvs.fnal.gov/cvs/projects/ubooneoffline/
svn copy $CVSROOT/trunk/products/<product> $CVSROOT/products/<product>/<new version> -m "Comment" 

If you are updating an existing version (like version "devel") you should delete the existing version directory before copying.

export CVSROOT=svn+ssh://p-ubooneoffline@cdcvs.fnal.gov/cvs/projects/ubooneoffline/
svn delete $CVSROOT/products/<product>/<existing version> -m "Comment" 
svn copy $CVSROOT/trunk/products/<product> $CVSROOT/products/<product>/<existing version> -m "Comment" 

Updating the Microboone UPS products area

On the uboonegpvmXX machines, microboone products are stored in /grid/fermiapp/products/uboone. Files in this area are a direct copy of files from the tagged versions stored in svn. To update this area, log in to the uboone account.

ssh -l uboone uboonegpvm01
cd /grid/fermiapp/products/uboone/<product>
svn update <version>

Delcaring products

Before a new version of a product is available to be setup, it must be declared to the ups products database. This is also done from the uboone account.

ups declare -0 -z /grid/fermiapp/products/uboone -r <product>/<version> -m <product>.table <product> <declared-version>

Note that the argument of the -r option represents the relative path where the product actually lives (underneath /grid/fermiapp/products/uboone). The "declared version" (the final argument) represents the version that users should specify in the setup command, as "setup <product> <declared-version>." Usually, these will be the same.

To declare products as "current," "development," "test,", "new," or "old" (the product chain, in ups lingo), use options -c, -d, -t, -n, or -o, like this:

ups declare -c <product> <version>   # Current
ups declare -d <product> <version>   # Development
ups declare -t <product> <version>   # Test
ups declare -n <product> <version>   # New
ups declare -o <product> <version>   # Old

You do not need to redeclare a product if you update the contents of an existing version.https://ticket.grid.iu.edu/submit

Development versions of products

By convention, several microboone products have a tagged version with tag name "devel," which is also declared as the development version (setup -d <product>). The development version may be updated rather frequently, and may be updated in situ (the contents may change).

Uploading microboone products to oasis

Registering your certificate with oasis.

Start by installing your grid certificate in your web browser. You can do this using firefox on uboonegpvmXX as follows.

kinit <username>
get-cert -i
firefox&

Visit the web page https://twiki.grid.iu.edu/bin/view/Documentation/Release3/UpdateOasis. In the Requirements section of this web page, click on the link to register that certificate in OIM. Then click on the Login button. The first time you login to the OIM site with your grid certificate, you will br prompted to register. Fill out and submit the registration form. On subsequent visits with the same grid certificate, you will be recognized automatically.

After you have successfully registered on the OIM site, you should open a GOC ticket to request to be added to the microboone manager list. Select the "Membership Request" check box and select MicroBooNE VO from the drop down list. Your membership request will be forwarded to an experiment representative for approval. It may take one or several days to process your ticket.

Authenticating yourself to oasis.

You need to authenticate by obtaining a grid certificate each time you upload to oasis. Type the following on uboonegpvmXX.

kinit <username>
get-cert

Note that typing kx509 (after kinit) is not good enough. You need to use get-cert.

Logging in to oasis.

You usually don't need to log in to oasis. You might want to log in to manually update the staging area. You can also test your authentication by logging in. After obtaining your grid certificate as described above, type the following.

gsissh oasis-login.opensciencegrid.org

If the above command doesn't work, try the following command.

gsissh -o GSSAPIDelegateCredentials=yes ouser.microboone@oasis-login.opensciencegrid.org

You can also add the following lines in your ~/.ssh/config file.

Host oasis-login.opensciencegrid.org
GSSAPIDelegateCredentials yes

You should now be logged in as user ouser.microboone. The staging area is located in subdirectory ouser.microboone/products below the ouser.microboone login area.

Staging files to oasis.

This step runs rsync to synchronize your files with the staging area on oasis. The basic command is

~larsoft/code/laradmin/cvmfs/oasis-sync -vr relative/path

Specify as argument the path of your product relative to the products area base /grid/fermiapp/products.
Use option -n to do a dry run. Here are the microboone products that we currently are uploading to oasis.

~larsoft/code/laradmin/cvmfs/oasis-sync -vr uboone/uboone
~larsoft/code/laradmin/cvmfs/oasis-sync -vr uboone/ubfcl
~larsoft/code/laradmin/cvmfs/oasis-sync -vr uboone/ubxml
~larsoft/code/laradmin/cvmfs/oasis-sync -vr uboone/ubtools
~larsoft/code/laradmin/cvmfs/oasis-sync -vr uboone/uboonecode
~larsoft/code/laradmin/cvmfs/oasis-sync -vr uboone/libwda

NEVER EVER upload the entire uboone products area by specifying just "uboone" as argument of oasis-sync.

Publishing files on oasis.

Manual method:

Log in to oasis (as described above) and run the following command.

osg-oasis-update

Automatic method:

Run the following script on uboonegpvmXX.

~larsoft/code/laradmin/cvmfs/oasis-sync -p uboone/uboone

Publishing is a cpu- and i/o-intensive process that calculates a hash for every file in the staging area. You should not publish indescriminately. Don't publish until you are finished updating the staging area.

You can combine publish (-p) and rsync (-r) options. The oasis-sync script requires you to specify a relative path even though this path isn't used for anything if you are only publishing.