Ifdh commands » History » Version 22
Marc Mengel, 05/12/2014 04:55 PM
Latest command help converted to textile
1 | 2 | Marc Mengel | h1. ifdh commands |
---|---|---|---|
2 | 1 | Marc Mengel | |
3 | 15 | Marc Mengel | Note that many of these commands are new in ifdh v1_3_x. |
4 | 15 | Marc Mengel | |
5 | 15 | Marc Mengel | h2. Environment variables |
6 | 15 | Marc Mengel | |
7 | 15 | Marc Mengel | * IFDH_DEBUG=1 for voluminous debug output |
8 | 15 | Marc Mengel | * EXPERIMENT set to an experiment name ("minerva", "nova", etc.) |
9 | 15 | Marc Mengel | if it can't be guessed from your group,etc. |
10 | 15 | Marc Mengel | * IFDH_BASE_URI to pick a particular samweb server. |
11 | 15 | Marc Mengel | * IFDH_GRIDFTP_EXTRA, IFDH_SRM_EXTRA, IFDH_DD_EXTRA flags to set options |
12 | 15 | Marc Mengel | flor globus-url-copy, lcg_cp, dd, etc. |
13 | 15 | Marc Mengel | * IFDH_WEB_TIMEOUT for samweb timeouts. |
14 | 15 | Marc Mengel | |
15 | 15 | Marc Mengel | h2. Commands |
16 | 9 | Marc Mengel | |
17 | 22 | Marc Mengel | * *ifdh* *cp* _args[0m_ |
18 | 22 | Marc Mengel | |
19 | 22 | Marc Mengel | general file copy using cpn locks dd, gridftp, or srmcp |
20 | 22 | Marc Mengel | supports: |
21 | 22 | Marc Mengel | |
22 | 22 | Marc Mengel | * cp src1 dest1 [';' src2 dest2 [';'...]] -- basic source/dest filenames |
23 | 22 | Marc Mengel | * cp -r src1 dest1 [';' src2 dest2 [';'...]] -- recursive directory copies |
24 | 22 | Marc Mengel | * cp -D src1 src2 destdir1 [';' src3 src4 destdir2 [';'...]] -- copies to dest. directory |
25 | 22 | Marc Mengel | * cp -f file_with_src_space_dest_lines -- copies to a list file |
26 | 22 | Marc Mengel | * any of the above can take --force={cpn,gridftp,srmcp,expgridftp} |
27 | 22 | Marc Mengel | * any of the file/dest arguments can be URIs |
28 | 22 | Marc Mengel | --- |
29 | 1 | Marc Mengel | |
30 | 22 | Marc Mengel | * *ifdh* *fetchInput* _src_uri [0m_ |
31 | 22 | Marc Mengel | |
32 | 22 | Marc Mengel | get input file to local scratch, return scratch location |
33 | 22 | Marc Mengel | |
34 | 22 | Marc Mengel | * *ifdh* *localPath* _src_uri [0m_ |
35 | 22 | Marc Mengel | |
36 | 22 | Marc Mengel | return scratch location fetchInput would give, without copying |
37 | 22 | Marc Mengel | |
38 | 22 | Marc Mengel | * *ifdh* *addOutputFile* _filename [0m_ |
39 | 22 | Marc Mengel | |
40 | 22 | Marc Mengel | add output file to set |
41 | 22 | Marc Mengel | |
42 | 22 | Marc Mengel | * *ifdh* *copyBackOutput* _dest_dir [0m_ |
43 | 22 | Marc Mengel | |
44 | 22 | Marc Mengel | copy output file set to destination with cpn or srmcp |
45 | 22 | Marc Mengel | |
46 | 22 | Marc Mengel | * *ifdh* *log* _args[0m_ |
47 | 22 | Marc Mengel | |
48 | 22 | Marc Mengel | logging |
49 | 22 | Marc Mengel | |
50 | 22 | Marc Mengel | * *ifdh* *enterState* _state [0m_ |
51 | 22 | Marc Mengel | |
52 | 22 | Marc Mengel | log entering/leaving states |
53 | 22 | Marc Mengel | |
54 | 22 | Marc Mengel | * *ifdh* *leaveState* _state [0m_ |
55 | 22 | Marc Mengel | |
56 | 22 | Marc Mengel | |
57 | 22 | Marc Mengel | * *ifdh* *createDefinition* _name dims user group [0m_ |
58 | 22 | Marc Mengel | |
59 | 22 | Marc Mengel | make a named dataset definition from a dimension string |
60 | 22 | Marc Mengel | |
61 | 22 | Marc Mengel | * *ifdh* *deleteDefinition* _name [0m_ |
62 | 22 | Marc Mengel | |
63 | 22 | Marc Mengel | remove data set definition |
64 | 22 | Marc Mengel | |
65 | 22 | Marc Mengel | * *ifdh* *describeDefinition* _name [0m_ |
66 | 22 | Marc Mengel | |
67 | 22 | Marc Mengel | describe a named dataset definition |
68 | 22 | Marc Mengel | |
69 | 22 | Marc Mengel | * *ifdh* *translateConstraints* _dims [0m_ |
70 | 22 | Marc Mengel | |
71 | 22 | Marc Mengel | give file list for dimension string |
72 | 22 | Marc Mengel | |
73 | 22 | Marc Mengel | * *ifdh* *locateFile* _name [0m_ |
74 | 22 | Marc Mengel | |
75 | 22 | Marc Mengel | locate a file |
76 | 22 | Marc Mengel | |
77 | 22 | Marc Mengel | * *ifdh* *getMetadata* _name [0m_ |
78 | 22 | Marc Mengel | |
79 | 22 | Marc Mengel | get a files metadata |
80 | 22 | Marc Mengel | |
81 | 22 | Marc Mengel | * *ifdh* *dumpStation* _name what [0m_ |
82 | 22 | Marc Mengel | |
83 | 22 | Marc Mengel | give a dump of a SAM station status |
84 | 22 | Marc Mengel | |
85 | 22 | Marc Mengel | * *ifdh* *startProject* _name station defname_or_id user group [0m_ |
86 | 22 | Marc Mengel | |
87 | 22 | Marc Mengel | start a new file delivery project |
88 | 22 | Marc Mengel | |
89 | 22 | Marc Mengel | * *ifdh* *findProject* _name station [0m_ |
90 | 22 | Marc Mengel | |
91 | 22 | Marc Mengel | find a started project |
92 | 22 | Marc Mengel | |
93 | 22 | Marc Mengel | * *ifdh* *establishProcess* _projecturi appname appversion location user appfamily description filelimit [0m_ |
94 | 22 | Marc Mengel | |
95 | 22 | Marc Mengel | set yourself up as a file consumer process for a project |
96 | 22 | Marc Mengel | |
97 | 22 | Marc Mengel | * *ifdh* *getNextFile* _projecturi processid [0m_ |
98 | 22 | Marc Mengel | |
99 | 22 | Marc Mengel | get the next file location from a project |
100 | 22 | Marc Mengel | |
101 | 22 | Marc Mengel | * *ifdh* *updateFileStatus* _projecturi processid filename status [0m_ |
102 | 22 | Marc Mengel | |
103 | 22 | Marc Mengel | update the file status (use: transferred, skipped, or consumed) |
104 | 22 | Marc Mengel | |
105 | 22 | Marc Mengel | * *ifdh* *endProcess* _projecturi processid [0m_ |
106 | 22 | Marc Mengel | |
107 | 22 | Marc Mengel | end the process |
108 | 22 | Marc Mengel | |
109 | 22 | Marc Mengel | * *ifdh* *dumpProject* _projecturi [0m_ |
110 | 22 | Marc Mengel | |
111 | 22 | Marc Mengel | say what the sam station knows about your process |
112 | 22 | Marc Mengel | |
113 | 22 | Marc Mengel | * *ifdh* *setStatus* _projecturi processid status [0m_ |
114 | 22 | Marc Mengel | |
115 | 22 | Marc Mengel | set process status |
116 | 22 | Marc Mengel | |
117 | 22 | Marc Mengel | * *ifdh* *endProject* _projecturi [0m_ |
118 | 22 | Marc Mengel | |
119 | 22 | Marc Mengel | end the project |
120 | 22 | Marc Mengel | |
121 | 22 | Marc Mengel | * *ifdh* *cleanup* _[0m_ |
122 | 22 | Marc Mengel | |
123 | 22 | Marc Mengel | clean up any tmp file stuff |
124 | 22 | Marc Mengel | |
125 | 22 | Marc Mengel | * *ifdh* *renameOutput* _how [0m_ |
126 | 22 | Marc Mengel | |
127 | 22 | Marc Mengel | give output files reported with addOutputFile a unique name |
128 | 22 | Marc Mengel | |
129 | 22 | Marc Mengel | * *ifdh* *mv* _args[0m_ |
130 | 22 | Marc Mengel | |
131 | 22 | Marc Mengel | general file rename using mvn or srmcp |
132 | 22 | Marc Mengel | |
133 | 22 | Marc Mengel | * *ifdh* *ls* _loc recursion_depth force [0m_ |
134 | 22 | Marc Mengel | |
135 | 22 | Marc Mengel | Get a list of directory contents, or check existence of files |
136 | 22 | Marc Mengel | |
137 | 22 | Marc Mengel | * *ifdh* *mkdir* _loc force [0m_ |
138 | 22 | Marc Mengel | |
139 | 22 | Marc Mengel | make a directory (i.e. for file destination) |
140 | 22 | Marc Mengel | |
141 | 22 | Marc Mengel | * *ifdh* *rm* _loc force [0m_ |
142 | 22 | Marc Mengel | |
143 | 22 | Marc Mengel | remove files |