Project

General

Profile

Getting to Swizzled Data

The swizzled files are being transferred onto tape storage in the enstore library and registered within SAM File Catalog1. If there is a particular run and subrun that you would like to access and process locally, then using the following commands you can copy the file to /uboone/data/users/<uid>/ but be aware of quotas2:

$> ssh uboonegpvm[01|02|03|04|05|06].fnal.gov

> source /grid/fermiapp/products/uboone/setup_uboone.sh
> setup ifdhc
> setup sam_web_client
> ifdh cp `samweb get-file-access-url -e uboone NoiseRun-0001093-00000.root` /uboone/data/users/kirby/
LOCK - Thu Aug  6 19:29:34 UTC 2015 lock  /grid/data/microboone/LOCK/LOCKS/20150806.19:29:34.0.uboonegpvm06.13731.kirby.kirby
LOCK - Thu Aug  6 19:29:44 UTC 2015 freed /grid/data/microboone/LOCK/LOCKS/20150806.19:29:34.0.uboonegpvm06.13731.kirby.kirby

Note that the character ` that surrounds the "samweb get-file-access-url -e uboone NoiseRun-0001093-00000.ubdaq" must be that specific character3. If you would like to change the run number, you must pad out the run number to be 7 characters (e.g. run 1111 becomes 0001111) and similarly, the subrun must be 5 characters long. If you would like to list all of the files registered to SAM in a particular run number, you can use this command:

> samweb list-files "run_number=1093 and data_tier=raw and file_format=artroot”
NoiseRun-0001093-00000.root
NoiseRun-0001093-00001.root

Once you find the dataset dimension that gets you the files that you want (that’s the stuff in the quotes), and you can then create a SAM dataset definition of the files that you have found. The command for that is4:

> kx509 #you have to first create a kerberos certificate from your kerberos ticket
> samweb -e uboone create-definition swizzled_data_run_1093_kirby "run_number=1093 and data_tier=raw and file_format=artroot”

You should check to make sure that you got things right:

> samweb list-definition-files swizzled_data_run_1093_kirby
NoiseRun-0001093-00000.root
NoiseRun-0001093-00001.root

If you want to have this dataset be the input into a larbatch/project.py5 set of grid jobs, then you can modify your project.py xml file to have the inputdef variable set to the definition you defined earlier. So for my definition, it would look like:

<inputdef>swizzled_data_run_1093_kirby</inputdef>

Then you can submit with project.py, and your SAM project will automagically be started and SAM when send all of the files from that dataset into the larsoft process designated in your xml file.

The best place to find out what the latest run number from the DAQ running is to check the E-log: http://dbweb3.fnal.gov:8080/ECL/uboone/E/index

DANGER (See note below) If you want to be very hands on and have to track things down by yourself, you might also look for swizzled files in the following directory on uboonegpvm0X.fnal.gov:

/pnfs/uboone/data/uboone/raw/Swizzler/swizzle/<uboonecode_version>/00/00/AB/CD/

where <uboonecode_version> is the version of the swizzler (v04_18_00 is the current one) and /00/00/AB/CD/ is a directory structure built from the run number. (e.g. run number 1093 becomes /00/00/10/93/)

I should mention though that the access to files that are in SAM/enstore (e.g. /pnfs/uboone/data/uboone/) might appear to be like normal disk access, but you should be aware that it is definitely not!!! These files are in dCache and so are therefore immutable (meaning they can’t be modified) and there is a chance that a file isn’t located on disk and you may be asking for it to be staged from tape. Doing so in a random way, may cause serious problems with enstore. Using ifdh cp helps minimize this risk.

1 Much more thorough documentation is available here: https://cdcvs.fnal.gov/redmine/projects/uboonecode/wiki/Sam

2 Remember that every uboone collaborator has a quota of 1.5 TB of space on /uboone/data/ and so you should be aware of your usage before copying over a huge set of files. You also have a 200 GB quota on /uboone/app so you really shouldn’t put data files there.

3 http://www.howtogeek.com/howto/29980/whats-the-difference-between-single-and-double-quotes-in-the-bash-shell/

4 Please make sure to use a rather unique dataset definition name. You should recognize that there are many users on uboone, and naming your dataset “test” is neither descriptive not unique. At a minimum, include your user id and something about the selection or format of the files.

5 https://redmine.fnal.gov/redmine/projects/larbatch/wiki/User_guide