Project

General

Profile

Ifdh commands » History » Version 29

Marc Mengel, 05/20/2015 10:52 AM

1 2 Marc Mengel
h1. ifdh commands
2 1 Marc Mengel
3 28 Marc Mengel
Note that many of these commands are new in ifdh v1_3_x and/or v1_7_x.
4 1 Marc Mengel
5 28 Marc Mengel
Many commands implicitly use a [[State Directory]].
6 28 Marc Mengel
7 1 Marc Mengel
h2. Environment variables
8 15 Marc Mengel
9 28 Marc Mengel
See [[Environment Variable Reference]]
10 15 Marc Mengel
11 15 Marc Mengel
h2. Commands
12 9 Marc Mengel
13 29 Marc Mengel
* *ifdh* *cp* _args_
14 22 Marc Mengel
	   
15 22 Marc Mengel
	 general file copy using cpn locks dd, gridftp, or srmcp 
16 22 Marc Mengel
	 supports:  
17 22 Marc Mengel
	   
18 23 Marc Mengel
	 * @cp src1 dest1 [';' src2 dest2 [';'...]]                     -- basic source/dest filenames @
19 23 Marc Mengel
	 * @cp -r src1 dest1 [';' src2 dest2 [';'...]]                  -- recursive directory copies @
20 23 Marc Mengel
	 * @cp -D src1 src2 destdir1 [';' src3 src4 destdir2 [';'...]]  -- copies to dest. directory @
21 23 Marc Mengel
	 * @cp -f file_with_src_space_dest_lines                        -- copies to a list file @
22 22 Marc Mengel
	 * any of the above can take --force={cpn,gridftp,srmcp,expgridftp} 
23 22 Marc Mengel
	 * any of the file/dest arguments can be URIs 
24 23 Marc Mengel
	 
25 1 Marc Mengel
26 25 Marc Mengel
* *ifdh* *fetchInput* _src_uri_
27 22 Marc Mengel
	   
28 22 Marc Mengel
	 get input file to local scratch, return scratch location
29 22 Marc Mengel
30 25 Marc Mengel
* *ifdh* *localPath* _src_uri_
31 22 Marc Mengel
	   
32 22 Marc Mengel
	 return scratch location fetchInput would give, without copying
33 22 Marc Mengel
34 24 Marc Mengel
* *ifdh* *addOutputFile* _filename_
35 22 Marc Mengel
	   
36 22 Marc Mengel
	 add output file to set
37 22 Marc Mengel
38 25 Marc Mengel
* *ifdh* *copyBackOutput* _dest_dir_
39 22 Marc Mengel
	   
40 22 Marc Mengel
	 copy output file set to destination with cpn or srmcp
41 22 Marc Mengel
42 24 Marc Mengel
* *ifdh* *log* _args_
43 22 Marc Mengel
	   
44 22 Marc Mengel
	 logging 
45 22 Marc Mengel
46 25 Marc Mengel
* *ifdh* *enterState* _state_
47 22 Marc Mengel
	   
48 22 Marc Mengel
	 log entering/leaving states
49 22 Marc Mengel
50 24 Marc Mengel
* *ifdh* *leaveState* _state_
51 22 Marc Mengel
	  
52 22 Marc Mengel
53 24 Marc Mengel
* *ifdh* *createDefinition* _name  dims  user  group_
54 22 Marc Mengel
	   
55 22 Marc Mengel
	 make a named dataset definition from a dimension string
56 22 Marc Mengel
57 24 Marc Mengel
* *ifdh* *deleteDefinition* _name_
58 22 Marc Mengel
	   
59 22 Marc Mengel
	 remove data set definition
60 22 Marc Mengel
61 24 Marc Mengel
* *ifdh* *describeDefinition* _name_
62 22 Marc Mengel
	   
63 22 Marc Mengel
	 describe a named dataset definition
64 22 Marc Mengel
65 24 Marc Mengel
* *ifdh* *translateConstraints* _dims_
66 22 Marc Mengel
	   
67 22 Marc Mengel
	 give file list for dimension string
68 22 Marc Mengel
69 24 Marc Mengel
* *ifdh* *locateFile* _name_
70 22 Marc Mengel
	   
71 22 Marc Mengel
	 locate a file
72 22 Marc Mengel
73 24 Marc Mengel
* *ifdh* *getMetadata* _name_
74 22 Marc Mengel
	   
75 22 Marc Mengel
	 get a files metadata
76 22 Marc Mengel
77 24 Marc Mengel
* *ifdh* *dumpStation* _name  what_
78 22 Marc Mengel
	   
79 22 Marc Mengel
	 give a dump of a SAM station status
80 22 Marc Mengel
81 24 Marc Mengel
* *ifdh* *startProject* _name  station  defname_or_id  user  group_
82 22 Marc Mengel
	   
83 22 Marc Mengel
	 start a new file delivery project
84 22 Marc Mengel
85 24 Marc Mengel
* *ifdh* *findProject* _name  station_
86 22 Marc Mengel
	   
87 22 Marc Mengel
	 find a started project
88 22 Marc Mengel
89 24 Marc Mengel
* *ifdh* *establishProcess* _projecturi  appname  appversion  location  user  appfamily   description   filelimit_
90 22 Marc Mengel
	   
91 22 Marc Mengel
	 set yourself up as a file consumer process for a project
92 22 Marc Mengel
93 24 Marc Mengel
* *ifdh* *getNextFile* _projecturi  processid_
94 22 Marc Mengel
	   
95 22 Marc Mengel
	 get the next file location from a project
96 22 Marc Mengel
97 24 Marc Mengel
* *ifdh* *updateFileStatus* _projecturi  processid  filename  status_
98 22 Marc Mengel
	   
99 22 Marc Mengel
	 update the file status (use: transferred, skipped, or consumed)
100 22 Marc Mengel
101 24 Marc Mengel
* *ifdh* *endProcess* _projecturi  processid_
102 22 Marc Mengel
	   
103 22 Marc Mengel
	 end the process
104 22 Marc Mengel
105 24 Marc Mengel
* *ifdh* *dumpProject* _projecturi_
106 22 Marc Mengel
	   
107 22 Marc Mengel
	 say what the sam station knows about your process
108 22 Marc Mengel
109 24 Marc Mengel
* *ifdh* *setStatus* _projecturi  processid  status_
110 22 Marc Mengel
	   
111 22 Marc Mengel
	 set process status
112 22 Marc Mengel
113 24 Marc Mengel
* *ifdh* *endProject* _projecturi_
114 22 Marc Mengel
	   
115 22 Marc Mengel
	 end the project
116 22 Marc Mengel
117 24 Marc Mengel
* *ifdh* *cleanup* 
118 22 Marc Mengel
	   
119 22 Marc Mengel
	 clean up any tmp file stuff
120 22 Marc Mengel
121 24 Marc Mengel
* *ifdh* *renameOutput* _how_
122 22 Marc Mengel
	   
123 22 Marc Mengel
	 give output files reported with addOutputFile a unique name
124 22 Marc Mengel
125 24 Marc Mengel
* *ifdh* *mv* _args_
126 22 Marc Mengel
	   
127 22 Marc Mengel
	 general file rename using mvn or srmcp
128 22 Marc Mengel
129 24 Marc Mengel
* *ifdh* *ls* _loc  recursion_depth  force_
130 22 Marc Mengel
	   
131 22 Marc Mengel
	 Get a list of directory contents, or check existence of files
132 22 Marc Mengel
133 24 Marc Mengel
* *ifdh* *mkdir* _loc  force_
134 22 Marc Mengel
	   
135 22 Marc Mengel
	 make a directory (i.e. for file destination)
136 22 Marc Mengel
137 24 Marc Mengel
* *ifdh* *rm* _loc  force_
138 22 Marc Mengel
	   
139 22 Marc Mengel
	 remove files
140 26 Dennis Box
141 26 Dennis Box
* *ifdh* *rmdir* loc force
142 26 Dennis Box
143 26 Dennis Box
         remove directories
144 26 Dennis Box
	
145 26 Dennis Box
*  *ifdh* *more*  loc 
146 26 Dennis Box
	   
147 26 Dennis Box
	 view text files
148 26 Dennis Box
149 26 Dennis Box
* *ifdh* *pin*  loc  seconds 
150 26 Dennis Box
	   
151 26 Dennis Box
	 pin file
152 26 Dennis Box
153 26 Dennis Box
*  *ifdh* *chmod*  mode  loc  force  
154 26 Dennis Box
	   
155 26 Dennis Box
	 change file permissions
156 26 Dennis Box
157 26 Dennis Box
* *ifdh* *rename*  loc  loc2  force  
158 26 Dennis Box
	   
159 26 Dennis Box
	 atomic rename items in same directory/fs
160 26 Dennis Box
161 26 Dennis Box
*  *ifdh* *ll*  loc  recursion_depth  force  
162 26 Dennis Box
	   
163 26 Dennis Box
	 list files with long listing 
164 26 Dennis Box
	 use recursion_depth== 0 to check directory without contents
165 26 Dennis Box
166 26 Dennis Box
*  *ifdh* *lss*  loc  recursion_depth  force  
167 26 Dennis Box
	   
168 26 Dennis Box
	 list files with sizes 
169 26 Dennis Box
	 use recursion_depth== 0 to check directory without contents
170 26 Dennis Box
171 26 Dennis Box
*	*ifdh* *findMatchingFiles*  path  glob 
172 26 Dennis Box
	   
173 26 Dennis Box
	 find filenames and sizes matching pattern
174 26 Dennis Box
175 26 Dennis Box
*	*ifdh* *fetchSharedFiles* args
176 26 Dennis Box
	   
177 26 Dennis Box
	 filenames and sizes matching pattern moved locally enough to be seen
178 26 Dennis Box
179 26 Dennis Box
*	*ifdh* *locateFiles* args
180 26 Dennis Box
	   
181 26 Dennis Box
	 locate multiple files
182 26 Dennis Box
183 26 Dennis Box
*	*ifdh* *checksum*  loc 
184 26 Dennis Box
	   
185 26 Dennis Box
	 cheksum file