Project

General

Profile

Finding & Accessing Data with SAM

The task is simple, you know the name of a file and you want to find it in the data collection and run a job on it (maybe look at the events). Here's how you can easily get at ANY file that we have ever taken and cataloged:

First generate the proper certificates so you can get to our data (i.e. so you are authorized to see our neutrinos):

setup_fnal_security -k

Make the ${HOME}/.globus directory if it doesn't already exist.

XROOTD Access

For Interactive or small test job running:

Now at the moment samweb returns files which are on tape as "enstore" locations. This is inconvenient.
Set up the nova software and you pick up the "samweb2xrootd" script that is available in novaproduction/bin to fix that.
If you're using the CMake build, you'll need to:

setup novaproduction

Xrootd URLs and Paths

The xrootd URLs differ slightly from the file locations that are return by SAMweb. The general form of the xrootd url is:

root://<server.name>:<port>/<path prefix>/<experiment>/<dCache location>/<filename>

Currently for Fermilab dCache these are:

Server: fndca1.fnal.gov
Port: 1094
Path Prefix: /pnfs/fnal.gov/usr/
Experiment: nova (for nova obviously)

The dCache tape label is not used in the xrootd path.

Example:
For the file: fardet_r00014000_s11_t02_S14-01-20_v1_data.artdaq.root

The locations are:

samweb locate-file fardet_r00014000_s11_t02_S14-01-20_v1_data.artdaq.root
enstore:/pnfs/nova/production/raw2root/S14-01-20/fardet/000140/14000/02(1809@vpl063)

This would become the xrootd URL:

root://fndca1.fnal.gov:1094//pnfs/fnal.gov/usr/nova/production/raw2root/S14-01-20/fardet/000140/14000/02//fardet_r00014000_s11_t02_S14-01-20_v1_data.artdaq.root

Locating a file (with XROOTD access)

Use the SAM catalog to get the locations of your files. Specify that you want the location that is returned to be an XROOTD accessible location with the "schema" option and you will get a valid list of XROOTD URLs.

Example:

# Syntax
samweb2xrootd <file>
#
# Example
samweb2xrootd fardet_r00014000_s11_t02_S14-01-20_v1_data.artdaq.root
xroot://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/nova/production/raw2root/S14-01-20/fardet/000140/14000/02/fardet_r00014000_s11_t02_S14-01-20_v1_data.artdaq.root

Opening a file in ROOT

To open an xrootd based file in root you need to use the TFile::Open() syntax.

TFile* f = TFile::Open("root://fndca1.fnal.gov:1094/<path>/<filename>");

Using the constructor version of this DOES NOT WORK

// DO NOT USE THIS STYLE OF OPENING A FILE. IT WILL FAIL
TFile* f = new TFile("root://<server>:<port>/<path>/<filename>");

Running a Job or the Event Display Interactively

To run Now run your job (interactively) you can do with something like:

# Syntax:
# nova -c evd.fcl <file access url>
# nova -c evd.fcl `samweb2xrootd <filename>`
#
# Example:
nova -c evd.fcl `samweb2xrootd fardet_r00014000_s11_t02_S14-01-20_v1_data.artdaq.root`

The file will be streamed to you. No copying, no bluearc, just lots of data direct to you.
For more information on how to locate files see the SAMWeb Cookbook

Reading Metadata from a file on dCache

Files on dCache are not directly openable by software (i.e. you can not do a simple open() call on the file) but you can use the xrootd protocol to stream the files to applications that support xrootd.

The sam_metadata_dumper utility has been patched to support xrootd input. To dump the metadata from a file with an enstore/dCache location:

sam_metadata_dumper `samweb2xrootd <filename>`

Standard Copy From dCache

Use ifdh_fetch to copy the file to a local area (note this makes and actual copy of the file in the working directory area. XRootD does not)

ifdh_fetch <filename>