Questions and Answers

Data Buffer questions:
  • Will the 50 nodes allocated to the DAQ provide a distributed file system?
  • Are these 50 nodes local to the detector, in the sense that they provide the storage if transfers to CERN data center are not possible?
  • If so, what is that file system (eg CEPH)? If this is CEPH with an xrootd server interface:

Does it present a single source URL?
Does the CEPH plugin support (minimally) "ls" and "mv"
How are parallel transfers managed? ie Is there a redirection to one of a set of nodes to perform the transfer?
What authentication / authorization is required?

  • If not CEPH, what provisions are provided (eg xrootd server) for remote access?

Which protocols are supported?
What authentication / authorization is required?
How is scale achieved?
Here is a reference for CEHP plugin for xrootd

Is there a single or multiple access points?
Are there multiple transfer points?

  • Are demands of Online and Nearline monitoring, in addition to data logging and data transfer, consistent with capability of individual nodes and aggregate cluster?

From the DAQ to the Data Buffer questions:
  • Will the "back ends" of the artDAQ system, presumably Event Builders or Aggregators, write directly to this file system from local (ie within the node) processes or via a remote connection?

If remote, via which protocol (presumably xrootd)?
If remote, how is scale achieved? ie Does each (or several/many) nodes run a service (eg xrootd server)?

  • When the artDAQ component generates a file, how does it provide information about that file?

Is this done via a supplemental file or otherwise?
What information is available? eg Checksum, first/last event, size, ...

  • When the artDAQ component generates a file, can it provide a completion signal? Via what mechanism?
  • Should a SAM entry be created at the point the file is initially created, or await an F-FTS action?

EOS questions:
  • What is the version of EOS (that is utilized for protoDUNE storage)?
  • What is the URL for xrootd access?

A: The xrootd redirector is at:

  • What provisions are provided (eg xrootd server) for remote access?

Which protocols are supported?

A: The EOS documentation lists: XRootD, a POSIX-like FUSE client or HTTP & WebDav. Note that gridftp is not listed in the EOS documentation, but Xavier's reply lists gridftp.

What authentication / authorization is required?
How is scale achieved?

Is there a single or multiple access points?
Are there multiple transfer points?

  • What is the desired directory structure?
  • What is the mechanism for creating new directories (and which process is responsible for such)?

From the Data Buffer to EOS questions (assuming managed by an F-FTS instance):
  • Similar to above questions, how does F-FTS get notified that a new file is available for transfer?

If information is pushed from the source, how does an F-FTS instance register to receive information?

What information is sent?

If information is polled, what should be polled? An "ls" of data or supplemental file?

How is it known if a file has been closed?

  • Similar to above questions, which protocol should be utilized?
  • What is the url specification format for source and destination?
  • How are checksums utilized?

A: Use the checksum capability of xrootd, xrdcp -c

Xavier's response: xrdcp -c: obtains the checksum of type (i.e. adler32, crc32, or md5) from the source,
computes the checksum at the destination, and verifies that they are the same. If a value is specified, it is
used as the source checksum. When print is specified, the checksum at the destination is printed but is not
* Is F-FTS responsible for creating destination directories?

CASTOR questions:
  • tbd

From EOS to CASTOR questions:
  • How can files in EOS be stored in CASTOR?

A: This appears to be done using the EOS archive command. This seems to need to be done on a node where EOS is mounted - but to be confirmed.

 Xavier's response: EOS doesn’t know anything about Castor, it is all done via FTS.

From EOS to Remote (eg Fermilab) questions:
  • tbd

SAM Metadata questions
  • How does FTS get metadata for use with SAM?

A: There are two ways -- Either a file (in json format) containing the metadata is put in the dropbox directory along side each file to be registered (with an appropriate naming convention so when we find "" we know to look for "myfile.metadata") or we register a small program or script that can generate the metadata from the file itself. In the case of NOvA we do both. Files produced with art can be run through the metadata_dumper utility which spits any metadata that has been imbedded in the files. Or for other types of files we have python scripts or binaries that read and extract the relevant information. Minerva takes the other route and just makes a .metadata file for each of their files they upload
A: Or the third possibility is that the job declares the file's metadata to SAM when it generates the file, before copying it to a dropbox.

Using ssh tunnels to get access to FTS-Light and FTS

1. From your desktop:

ssh -L

2. Using your browser, goto:

If necessary, you can patch through a chain of nodes like this:

my_desktop> ssh -L 8080:localhost:8765
lxplus> ssh -L

Expectations from DAQ