Ifdh commands » History » Version 31
« Previous -
Version 31/33
(diff) -
Next » -
Current version
Michael Kirby, 03/20/2017 10:35 AM
ifdh commands¶
Note that many of these commands are new in ifdh v1_3_x and/or v1_7_x.
Many commands implicitly use a State Directory.
Environment variables¶
See Environment Variable Reference
Commands¶
- ifdh cp args
general file copy using cpn locks dd, gridftp, or srmcp
supports:cp src1 dest1 [';' src2 dest2 [';'...]] -- basic source/dest filenames
*cp -r src1 dest1 [';' src2 dest2 [';'...]] -- recursive directory copies
*cp -D src1 src2 destdir1 [';' src3 src4 destdir2 [';'...]] -- copies to dest. directory
*cp -f file_with_src_space_dest_lines -- copies to a list file
* any of the above can take --force={cpn,gridftp,srmcp,expgridftp} * any of the file/dest arguments can be URIs
Users might consider utilizing these environment variables to control the response of "ifdh cp" commands:
export IFDH_GRIDFTP_EXTRA="-st N"
which sets the stall timeout at N seconds IFDH_XROOTD_EXTRA *export IFDH_CP_MAXRETRIES=N
, we find 2 is usually a good number * note that IFDH_GRIDFTP_EXTRA can be used to pass many arguments to globus-url-copy http://toolkit.globus.org/toolkit/docs/3.2/gridftp/user/globusurlcopy.html * note that IFDH_XROOTD_EXTRA can be used to pass many arguments to xrootd/xrdcp actions https://linux.die.net/man/1/xrdcp * full environment variables here: Environment Variable Reference
- ifdh fetchInput src_uri
get input file to local scratch, return scratch location
- ifdh localPath src_uri
return scratch location fetchInput would give, without copying
- ifdh addOutputFile filename
add output file to set
- ifdh copyBackOutput dest_dir
copy output file set to destination with cpn or srmcp
- ifdh log args
logging
- ifdh enterState state
log entering/leaving states
- ifdh leaveState state
- ifdh createDefinition name dims user group
make a named dataset definition from a dimension string
- ifdh deleteDefinition name
remove data set definition
- ifdh describeDefinition name
describe a named dataset definition
- ifdh translateConstraints dims
give file list for dimension string
- ifdh locateFile name
locate a file
- ifdh getMetadata name
get a files metadata
- ifdh dumpStation name what
give a dump of a SAM station status
- ifdh startProject name station defname_or_id user group
start a new file delivery project
- ifdh findProject name station
find a started project
- ifdh establishProcess projecturi appname appversion location user appfamily description filelimit
set yourself up as a file consumer process for a project
- ifdh getNextFile projecturi processid
get the next file location from a project
- ifdh updateFileStatus projecturi processid filename status
update the file status (use: transferred, skipped, or consumed)
- ifdh endProcess projecturi processid
end the process
- ifdh dumpProject projecturi
say what the sam station knows about your process
- ifdh setStatus projecturi processid status
set process status
- ifdh endProject projecturi
end the project
- ifdh cleanup
clean up any tmp file stuff
- ifdh renameOutput how
give output files reported with addOutputFile a unique name
- ifdh mv args
general file rename using mvn or srmcp
- ifdh ls loc recursion_depth force
Get a list of directory contents, or check existence of files
- ifdh mkdir loc force
make a directory (i.e. for file destination)
- ifdh rm loc force
remove files
- ifdh rmdir loc force
remove directories
- ifdh more loc
view text files
- ifdh pin loc seconds
pin file
- ifdh chmod mode loc force
change file permissions
- ifdh rename loc loc2 force
atomic rename items in same directory/fs
- ifdh ll loc recursion_depth force
list files with long listing
use recursion_depth== 0 to check directory without contents
- ifdh lss loc recursion_depth force
list files with sizes
use recursion_depth== 0 to check directory without contents
- ifdh findMatchingFiles path glob
find filenames and sizes matching pattern
- ifdh fetchSharedFiles args
filenames and sizes matching pattern moved locally enough to be seen
- ifdh locateFiles args
locate multiple files
- ifdh checksum loc
cheksum file