Project

General

Profile

Ifdh commands » History » Version 25

Version 24 (Marc Mengel, 05/12/2014 05:00 PM) → Version 25/33 (Marc Mengel, 05/12/2014 05:00 PM)

h1. ifdh commands

Note that many of these commands are new in ifdh v1_3_x.

h2. Environment variables

* IFDH_DEBUG=1 for voluminous debug output
* EXPERIMENT set to an experiment name ("minerva", "nova", etc.)
if it can't be guessed from your group,etc.
* IFDH_BASE_URI to pick a particular samweb server.
* IFDH_GRIDFTP_EXTRA, IFDH_SRM_EXTRA, IFDH_DD_EXTRA flags to set options
flor globus-url-copy, lcg_cp, dd, etc.
* IFDH_WEB_TIMEOUT for samweb timeouts.

h2. Commands

* *ifdh* *cp* _args_

general file copy using cpn locks dd, gridftp, or srmcp
supports:

* @cp src1 dest1 [';' src2 dest2 [';'...]] -- basic source/dest filenames @
* @cp -r src1 dest1 [';' src2 dest2 [';'...]] -- recursive directory copies @
* @cp -D src1 src2 destdir1 [';' src3 src4 destdir2 [';'...]] -- copies to dest. directory @
* @cp -f file_with_src_space_dest_lines -- copies to a list file @
* any of the above can take --force={cpn,gridftp,srmcp,expgridftp}
* any of the file/dest arguments can be URIs


* *ifdh* *fetchInput* _src_uri_ _@src_uri@_

get input file to local scratch, return scratch location

* *ifdh* *localPath* _src_uri_ _@src_uri@_

return scratch location fetchInput would give, without copying

* *ifdh* *addOutputFile* _filename_

add output file to set

* *ifdh* *copyBackOutput* _dest_dir_ _@dest_dir@_

copy output file set to destination with cpn or srmcp

* *ifdh* *log* _args_

logging

* *ifdh* *enterState* _state_ _state _

log entering/leaving states

* *ifdh* *leaveState* _state_


* *ifdh* *createDefinition* _name dims user group_

make a named dataset definition from a dimension string

* *ifdh* *deleteDefinition* _name_

remove data set definition

* *ifdh* *describeDefinition* _name_

describe a named dataset definition

* *ifdh* *translateConstraints* _dims_

give file list for dimension string

* *ifdh* *locateFile* _name_

locate a file

* *ifdh* *getMetadata* _name_

get a files metadata

* *ifdh* *dumpStation* _name what_

give a dump of a SAM station status

* *ifdh* *startProject* _name station defname_or_id user group_

start a new file delivery project

* *ifdh* *findProject* _name station_

find a started project

* *ifdh* *establishProcess* _projecturi appname appversion location user appfamily description filelimit_

set yourself up as a file consumer process for a project

* *ifdh* *getNextFile* _projecturi processid_

get the next file location from a project

* *ifdh* *updateFileStatus* _projecturi processid filename status_

update the file status (use: transferred, skipped, or consumed)

* *ifdh* *endProcess* _projecturi processid_

end the process

* *ifdh* *dumpProject* _projecturi_

say what the sam station knows about your process

* *ifdh* *setStatus* _projecturi processid status_

set process status

* *ifdh* *endProject* _projecturi_

end the project

* *ifdh* *cleanup*

clean up any tmp file stuff

* *ifdh* *renameOutput* _how_

give output files reported with addOutputFile a unique name

* *ifdh* *mv* _args_

general file rename using mvn or srmcp

* *ifdh* *ls* _loc recursion_depth force_

Get a list of directory contents, or check existence of files

* *ifdh* *mkdir* _loc force_

make a directory (i.e. for file destination)

* *ifdh* *rm* _loc force_

remove files