Scientific Data Management » REX » IF Data Handling Client Tools (ifdhc)

h1. ifdh commands

Note that many of these commands are new in ifdh v1_3_x.

h2. Environment variables

* IFDH_DEBUG=1 for voluminous debug output

* EXPERIMENT set to an experiment name ("minerva", "nova", etc.)

  if it can't be guessed from your group,etc.

* IFDH_BASE_URI to pick a particular samweb server.

* IFDH_GRIDFTP_EXTRA, IFDH_SRM_EXTRA, IFDH_DD_EXTRA flags to set options

  flor globus-url-copy, lcg_cp, dd, etc.

* IFDH_WEB_TIMEOUT for samweb timeouts.

h2. Commands

* *ifdh* *cp* _src  dest_

** general file copy using cpn or srmcp

@ifdh cp [--force=srm|gridftp|cpn|expftp] [-r]  src1 dest1 [ ";" src2 ...  dest2 [ ";" ... ]  ]@

@ifdh cp [--force=srm|gridftp|cpn|expftp] -f listfile@

@ifdh cp [--force=srm|gridftp|cpn|expftp] -D [-r]  src1 src2 destdir1 [ ";" src3 src4 ...  destdir2 [ ";" ... ]  ]@

One can also set IFDH_FORCE= in the environment rather than using --force=

* *ifdh* *ls* _dest_ [ _depth_ [--force=srm|gridftp|cpn]]

** list file or directory 

** also confirms existence

* *ifdh* *mkdir* _dest_ [--force=srm|gridftp|cpn]

** make directories

* *ifdh* *rm* _dest_ [--force=srm|gridftp|cpn]

** remove files

* *ifdh* *rmdir* _dest_ [--force=srm|gridftp|cpn]

** remove directories

* *ifdh* *fetchInput* _src_uri_

** get input file to local scratch, return scratch location

* *ifdh* *addOutputFile* _filename_

** add output file to set

* *ifdh* *copyBackOutput* _dest_dir_

** copy output file set to destination with cpn or srmcp

* *ifdh* *log* _message_

** logging 

* *ifdh* *enterState* _state_

** log entering/leaving states

* *ifdh* *leaveState* _state_

** log entering/leaving states

* *ifdh* *createDefinition* _name  dims  user  group_

** make a named dataset definition from a dimension string

* *ifdh* *deleteDefinition* _name_

** remove data set definition

* *ifdh* *describeDefinition* _name_

** describe a named dataset definition

* *ifdh* *translateConstraints* _dims_

** give file list for dimension string

* *ifdh* *locateFile* _name_

** locate a file

* *ifdh* *getMetadata* _name_

** get a files metadata

* *ifdh* *dumpStation* _name  what_

** give a dump of a SAM station status

* *ifdh* *startProject* _name  station  defname_or_id  user  group_

** start a new file delivery project

* *ifdh* *findProject* _name  station_

** find a started project

* *ifdh* *establishProcess* _projecturi  appname  appversion  location  user  appfamily   description   filelimit_

** set yourself up as a file consumer process for a project

* *ifdh* *getNextFile* _projecturi  processid_

** get the next file location from a project

* *ifdh* *updateFileStatus* _projecturi  processid  filename  status_

** update the file status (use: transferred, skipped, or consumed)

* *ifdh* *endProcess* _projecturi  processid_

** end the process

* *ifdh* *dumpProject* _projecturi_

** say what the sam station knows about your process

* *ifdh* *setStatus* _projecturi  processid  status_

** set process status

* *ifdh* *endProject* _projecturi_

** end the project

* *ifdh* *cleanup*

** clean up any tmp file stuff