ifdh commands¶

Note that many of these commands are new in ifdh v1_3_x and/or v1_7_x.

Many commands implicitly use a State Directory.

Environment variables¶

See Environment Variable Reference

Commands¶

• ifdh cp args

  general file copy using cpn locks dd, gridftp, or srmcp
  supports:

  • cp src1 dest1 [';' src2 dest2 [';'...]] -- basic source/dest filenames * cp -r src1 dest1 [';' src2 dest2 [';'...]] -- recursive directory copies * cp -D src1 src2 destdir1 [';' src3 src4 destdir2 [';'...]] -- copies to dest. directory * cp -f file_with_src_space_dest_lines -- copies to a list file * any of the above can take --force={cpn,gridftp,srmcp,expgridftp} * any of the file/dest arguments can be URIs

  Users might consider utilizing these environment variables to control the response of "ifdh cp" commands:

  • export IFDH_GRIDFTP_EXTRA="-st N" which sets the stall timeout at N seconds IFDH_XROOTD_EXTRA * export IFDH_CP_MAXRETRIES=N, we find 2 is usually a good number * note that IFDH_GRIDFTP_EXTRA can be used to pass many arguments to globus-url-copy http://toolkit.globus.org/toolkit/docs/3.2/gridftp/user/globusurlcopy.html * note that IFDH_XROOTD_EXTRA can be used to pass many arguments to xrootd/xrdcp actions https://linux.die.net/man/1/xrdcp * full environment variables here: Environment Variable Reference

• ifdh fetchInput src_uri

  get input file to local scratch, return scratch location

• ifdh localPath src_uri

  return scratch location fetchInput would give, without copying

• ifdh addOutputFile filename

  add output file to set

• ifdh copyBackOutput dest_dir

  copy output file set to destination with cpn or srmcp

• ifdh log args

  logging

• ifdh enterState state

  log entering/leaving states

• ifdh leaveState state

• ifdh createDefinition name dims user group

  make a named dataset definition from a dimension string

• ifdh deleteDefinition name

  remove data set definition

• ifdh describeDefinition name

  describe a named dataset definition

• ifdh translateConstraints dims

  give file list for dimension string

• ifdh locateFile name

  locate a file

• ifdh getMetadata name

  get a files metadata

• ifdh dumpStation name what

  give a dump of a SAM station status

• ifdh startProject name station defname_or_id user group

  start a new file delivery project

• ifdh findProject name station

  find a started project

• ifdh establishProcess projecturi appname appversion location user appfamily description filelimit

  set yourself up as a file consumer process for a project

• ifdh getNextFile projecturi processid

  get the next file location from a project

• ifdh updateFileStatus projecturi processid filename status

  update the file status (use: transferred, skipped, or consumed)

• ifdh endProcess projecturi processid

  end the process

• ifdh dumpProject projecturi

  say what the sam station knows about your process

• ifdh setStatus projecturi processid status

  set process status

• ifdh endProject projecturi

  end the project

• ifdh cleanup

  clean up any tmp file stuff

• ifdh renameOutput how

  give output files reported with addOutputFile a unique name

• ifdh mv args

  general file rename using mvn or srmcp

• ifdh ls loc recursion_depth force

  Get a list of directory contents, or check existence of files

• ifdh mkdir loc force

  make a directory (i.e. for file destination)

• ifdh rm loc force

  remove files

• ifdh rmdir loc force

  remove directories

• ifdh more loc

  view text files

• ifdh pin loc seconds

  pin file

• ifdh chmod mode loc force

  change file permissions

• ifdh rename loc loc2 force

  atomic rename items in same directory/fs

• ifdh ll loc recursion_depth force

  list files with long listing
  use recursion_depth== 0 to check directory without contents

• ifdh lss loc recursion_depth force

  list files with sizes
  use recursion_depth== 0 to check directory without contents

• ifdh findMatchingFiles path glob

  find filenames and sizes matching pattern

• ifdh fetchSharedFiles args

  filenames and sizes matching pattern moved locally enough to be seen

• ifdh locateFiles args

  locate multiple files

• ifdh checksum loc

  cheksum file