ifdh commands¶

Note that many of these commands are new in ifdh v1_3_x.

Environment variables¶

• IFDH_DEBUG=1 for voluminous debug output
• EXPERIMENT set to an experiment name ("minerva", "nova", etc.)
  if it can't be guessed from your group,etc.
• IFDH_BASE_URI to pick a particular samweb server.
• IFDH_GRIDFTP_EXTRA, IFDH_SRM_EXTRA, IFDH_DD_EXTRA flags to set options
  flor globus-url-copy, lcg_cp, dd, etc.
• IFDH_WEB_TIMEOUT for samweb timeouts.

Commands¶

• ifdh cp src dest
  • general file copy using cpn or srmcp
    ifdh cp [--force=srm|gridftp|cpn|expftp] [-r] src1 dest1 [ ";" src2 ... dest2 [ ";" ... ] ]
ifdh cp [--force=srm|gridftp|cpn|expftp] -f listfile
ifdh cp [--force=srm|gridftp|cpn|expftp] -D [-r] src1 src2 destdir1 [ ";" src3 src4 ... destdir2 [ ";" ... ] ]
    One can also set IFDH_FORCE= in the environment rather than using --force=
• ifdh ls dest [ depth [--force=srm|gridftp|cpn]]
  • list file or directory
  • also confirms existence
• ifdh mkdir dest [--force=srm|gridftp|cpn]
  • make directories
• ifdh rm dest [--force=srm|gridftp|cpn]
  • remove files
• ifdh rmdir dest [--force=srm|gridftp|cpn]
  • remove directories
• ifdh fetchInput src_uri
  • get input file to local scratch, return scratch location
• ifdh addOutputFile filename
  • add output file to set
• ifdh renameOutput how
  • rename output files, how is one of
    • unique -- add time,host,pid text to filename to make it unique
    • s/this/that/ -- convert nearest input filename to output filename with substitution, now takes full perl substitution/regexp
    • exec:/some/script -- run /some/script filename list file as its first argument, file has lines of the form: currentname inputfilename, script should update the file to maintani currentname so copyBackOuput will work.
• ifdh copyBackOutput dest_dir
  • copy output file set to destination with cpn or srmcp
• ifdh log message
  • logging
• ifdh enterState state
  • log entering/leaving states
• ifdh leaveState state
  • log entering/leaving states
• ifdh createDefinition name dims user group
  • make a named dataset definition from a dimension string
• ifdh deleteDefinition name
  • remove data set definition
• ifdh describeDefinition name
  • describe a named dataset definition
• ifdh translateConstraints dims
  • give file list for dimension string
• ifdh locateFile name
  • locate a file
• ifdh getMetadata name
  • get a files metadata
• ifdh dumpStation name what
  • give a dump of a SAM station status
• ifdh startProject name station defname_or_id user group
  • start a new file delivery project
• ifdh findProject name station
  • find a started project
• ifdh establishProcess projecturi appname appversion location user appfamily description filelimit
  • set yourself up as a file consumer process for a project
• ifdh getNextFile projecturi processid
  • get the next file location from a project
• ifdh updateFileStatus projecturi processid filename status
  • update the file status (use: transferred, skipped, or consumed)
• ifdh endProcess projecturi processid
  • end the process
• ifdh dumpProject projecturi
  • say what the sam station knows about your process
• ifdh setStatus projecturi processid status
  • set process status
• ifdh endProject projecturi
  • end the project
• ifdh cleanup
  • clean up any tmp file stuff