Project

General

Profile

Ifdh commands » History » Version 16

« Previous - Version 16/33 (diff) - Next » - Current version
Marc Mengel, 03/06/2014 02:54 PM


ifdh commands

Note that many of these commands are new in ifdh v1_3_x.

Environment variables

  • IFDH_DEBUG=1 for voluminous debug output
  • EXPERIMENT set to an experiment name ("minerva", "nova", etc.)
    if it can't be guessed from your group,etc.
  • IFDH_BASE_URI to pick a particular samweb server.
  • IFDH_GRIDFTP_EXTRA, IFDH_SRM_EXTRA, IFDH_DD_EXTRA flags to set options
    flor globus-url-copy, lcg_cp, dd, etc.
  • IFDH_WEB_TIMEOUT for samweb timeouts.

Commands

  • ifdh cp src dest
    • general file copy using cpn or srmcp
      ifdh cp [--force=srm|gridftp|cpn|expftp] [-r] src1 dest1 [ ";" src2 ... dest2 [ ";" ... ] ]
      ifdh cp [--force=srm|gridftp|cpn|expftp] -f listfile
      ifdh cp [--force=srm|gridftp|cpn|expftp] -D [-r] src1 src2 destdir1 [ ";" src3 src4 ... destdir2 [ ";" ... ] ]
      One can also set IFDH_FORCE= in the environment rather than using --force=
  • ifdh ls dest [ depth [--force=srm|gridftp|cpn]]
    • list file or directory
    • also confirms existence
  • ifdh mkdir dest [--force=srm|gridftp|cpn]
    • make directories
  • ifdh rm dest [--force=srm|gridftp|cpn]
    • remove files
  • ifdh rmdir dest [--force=srm|gridftp|cpn]
    • remove directories
  • ifdh fetchInput src_uri
    • get input file to local scratch, return scratch location
  • ifdh addOutputFile filename
    • add output file to set
  • ifdh renameOutput how
    • rename output files, how is one of
      • unique -- add time,host,pid text to filename to make it unique
      • s/this/that/ -- convert nearest input filename to output filename with substitution, now takes full perl substitution/regexp
      • exec:/some/script -- run /some/script filename list file as its first argument, file has lines of the form: currentname inputfilename, script should update the file to maintani currentname so copyBackOuput will work.
  • ifdh copyBackOutput dest_dir
    • copy output file set to destination with cpn or srmcp
  • ifdh log message
    • logging
  • ifdh enterState state
    • log entering/leaving states
  • ifdh leaveState state
    • log entering/leaving states
  • ifdh createDefinition name dims user group
    • make a named dataset definition from a dimension string
  • ifdh deleteDefinition name
    • remove data set definition
  • ifdh describeDefinition name
    • describe a named dataset definition
  • ifdh translateConstraints dims
    • give file list for dimension string
  • ifdh locateFile name
    • locate a file
  • ifdh getMetadata name
    • get a files metadata
  • ifdh dumpStation name what
    • give a dump of a SAM station status
  • ifdh startProject name station defname_or_id user group
    • start a new file delivery project
  • ifdh findProject name station
    • find a started project
  • ifdh establishProcess projecturi appname appversion location user appfamily description filelimit
    • set yourself up as a file consumer process for a project
  • ifdh getNextFile projecturi processid
    • get the next file location from a project
  • ifdh updateFileStatus projecturi processid filename status
    • update the file status (use: transferred, skipped, or consumed)
  • ifdh endProcess projecturi processid
    • end the process
  • ifdh dumpProject projecturi
    • say what the sam station knows about your process
  • ifdh setStatus projecturi processid status
    • set process status
  • ifdh endProject projecturi
    • end the project
  • ifdh cleanup
    • clean up any tmp file stuff