Project

General

Profile

Ifdh commands » History » Version 27

Marc Mengel, 05/20/2015 10:50 AM

1 2 Marc Mengel
h1. ifdh commands
2 1 Marc Mengel
3 15 Marc Mengel
Note that many of these commands are new in ifdh v1_3_x.
4 15 Marc Mengel
5 15 Marc Mengel
h2. Environment variables
6 15 Marc Mengel
7 27 Marc Mengel
See [[Environment Variables]]
8 15 Marc Mengel
9 15 Marc Mengel
h2. Commands
10 9 Marc Mengel
11 22 Marc Mengel
* *ifdh* *cp* _args_
12 22 Marc Mengel
	   
13 22 Marc Mengel
	 general file copy using cpn locks dd, gridftp, or srmcp 
14 22 Marc Mengel
	 supports:  
15 22 Marc Mengel
	   
16 23 Marc Mengel
	 * @cp src1 dest1 [';' src2 dest2 [';'...]]                     -- basic source/dest filenames @
17 23 Marc Mengel
	 * @cp -r src1 dest1 [';' src2 dest2 [';'...]]                  -- recursive directory copies @
18 23 Marc Mengel
	 * @cp -D src1 src2 destdir1 [';' src3 src4 destdir2 [';'...]]  -- copies to dest. directory @
19 23 Marc Mengel
	 * @cp -f file_with_src_space_dest_lines                        -- copies to a list file @
20 22 Marc Mengel
	 * any of the above can take --force={cpn,gridftp,srmcp,expgridftp} 
21 22 Marc Mengel
	 * any of the file/dest arguments can be URIs 
22 23 Marc Mengel
	 
23 1 Marc Mengel
24 25 Marc Mengel
* *ifdh* *fetchInput* _src_uri_
25 22 Marc Mengel
	   
26 22 Marc Mengel
	 get input file to local scratch, return scratch location
27 22 Marc Mengel
28 25 Marc Mengel
* *ifdh* *localPath* _src_uri_
29 22 Marc Mengel
	   
30 22 Marc Mengel
	 return scratch location fetchInput would give, without copying
31 22 Marc Mengel
32 24 Marc Mengel
* *ifdh* *addOutputFile* _filename_
33 22 Marc Mengel
	   
34 22 Marc Mengel
	 add output file to set
35 22 Marc Mengel
36 25 Marc Mengel
* *ifdh* *copyBackOutput* _dest_dir_
37 22 Marc Mengel
	   
38 22 Marc Mengel
	 copy output file set to destination with cpn or srmcp
39 22 Marc Mengel
40 24 Marc Mengel
* *ifdh* *log* _args_
41 22 Marc Mengel
	   
42 22 Marc Mengel
	 logging 
43 22 Marc Mengel
44 25 Marc Mengel
* *ifdh* *enterState* _state_
45 22 Marc Mengel
	   
46 22 Marc Mengel
	 log entering/leaving states
47 22 Marc Mengel
48 24 Marc Mengel
* *ifdh* *leaveState* _state_
49 22 Marc Mengel
	  
50 22 Marc Mengel
51 24 Marc Mengel
* *ifdh* *createDefinition* _name  dims  user  group_
52 22 Marc Mengel
	   
53 22 Marc Mengel
	 make a named dataset definition from a dimension string
54 22 Marc Mengel
55 24 Marc Mengel
* *ifdh* *deleteDefinition* _name_
56 22 Marc Mengel
	   
57 22 Marc Mengel
	 remove data set definition
58 22 Marc Mengel
59 24 Marc Mengel
* *ifdh* *describeDefinition* _name_
60 22 Marc Mengel
	   
61 22 Marc Mengel
	 describe a named dataset definition
62 22 Marc Mengel
63 24 Marc Mengel
* *ifdh* *translateConstraints* _dims_
64 22 Marc Mengel
	   
65 22 Marc Mengel
	 give file list for dimension string
66 22 Marc Mengel
67 24 Marc Mengel
* *ifdh* *locateFile* _name_
68 22 Marc Mengel
	   
69 22 Marc Mengel
	 locate a file
70 22 Marc Mengel
71 24 Marc Mengel
* *ifdh* *getMetadata* _name_
72 22 Marc Mengel
	   
73 22 Marc Mengel
	 get a files metadata
74 22 Marc Mengel
75 24 Marc Mengel
* *ifdh* *dumpStation* _name  what_
76 22 Marc Mengel
	   
77 22 Marc Mengel
	 give a dump of a SAM station status
78 22 Marc Mengel
79 24 Marc Mengel
* *ifdh* *startProject* _name  station  defname_or_id  user  group_
80 22 Marc Mengel
	   
81 22 Marc Mengel
	 start a new file delivery project
82 22 Marc Mengel
83 24 Marc Mengel
* *ifdh* *findProject* _name  station_
84 22 Marc Mengel
	   
85 22 Marc Mengel
	 find a started project
86 22 Marc Mengel
87 24 Marc Mengel
* *ifdh* *establishProcess* _projecturi  appname  appversion  location  user  appfamily   description   filelimit_
88 22 Marc Mengel
	   
89 22 Marc Mengel
	 set yourself up as a file consumer process for a project
90 22 Marc Mengel
91 24 Marc Mengel
* *ifdh* *getNextFile* _projecturi  processid_
92 22 Marc Mengel
	   
93 22 Marc Mengel
	 get the next file location from a project
94 22 Marc Mengel
95 24 Marc Mengel
* *ifdh* *updateFileStatus* _projecturi  processid  filename  status_
96 22 Marc Mengel
	   
97 22 Marc Mengel
	 update the file status (use: transferred, skipped, or consumed)
98 22 Marc Mengel
99 24 Marc Mengel
* *ifdh* *endProcess* _projecturi  processid_
100 22 Marc Mengel
	   
101 22 Marc Mengel
	 end the process
102 22 Marc Mengel
103 24 Marc Mengel
* *ifdh* *dumpProject* _projecturi_
104 22 Marc Mengel
	   
105 22 Marc Mengel
	 say what the sam station knows about your process
106 22 Marc Mengel
107 24 Marc Mengel
* *ifdh* *setStatus* _projecturi  processid  status_
108 22 Marc Mengel
	   
109 22 Marc Mengel
	 set process status
110 22 Marc Mengel
111 24 Marc Mengel
* *ifdh* *endProject* _projecturi_
112 22 Marc Mengel
	   
113 22 Marc Mengel
	 end the project
114 22 Marc Mengel
115 24 Marc Mengel
* *ifdh* *cleanup* 
116 22 Marc Mengel
	   
117 22 Marc Mengel
	 clean up any tmp file stuff
118 22 Marc Mengel
119 24 Marc Mengel
* *ifdh* *renameOutput* _how_
120 22 Marc Mengel
	   
121 22 Marc Mengel
	 give output files reported with addOutputFile a unique name
122 22 Marc Mengel
123 24 Marc Mengel
* *ifdh* *mv* _args_
124 22 Marc Mengel
	   
125 22 Marc Mengel
	 general file rename using mvn or srmcp
126 22 Marc Mengel
127 24 Marc Mengel
* *ifdh* *ls* _loc  recursion_depth  force_
128 22 Marc Mengel
	   
129 22 Marc Mengel
	 Get a list of directory contents, or check existence of files
130 22 Marc Mengel
131 24 Marc Mengel
* *ifdh* *mkdir* _loc  force_
132 22 Marc Mengel
	   
133 22 Marc Mengel
	 make a directory (i.e. for file destination)
134 22 Marc Mengel
135 24 Marc Mengel
* *ifdh* *rm* _loc  force_
136 22 Marc Mengel
	   
137 22 Marc Mengel
	 remove files
138 26 Dennis Box
139 26 Dennis Box
* *ifdh* *rmdir* loc force
140 26 Dennis Box
141 26 Dennis Box
         remove directories
142 26 Dennis Box
	
143 26 Dennis Box
*  *ifdh* *more*  loc 
144 26 Dennis Box
	   
145 26 Dennis Box
	 view text files
146 26 Dennis Box
147 26 Dennis Box
* *ifdh* *pin*  loc  seconds 
148 26 Dennis Box
	   
149 26 Dennis Box
	 pin file
150 26 Dennis Box
151 26 Dennis Box
*  *ifdh* *chmod*  mode  loc  force  
152 26 Dennis Box
	   
153 26 Dennis Box
	 change file permissions
154 26 Dennis Box
155 26 Dennis Box
* *ifdh* *rename*  loc  loc2  force  
156 26 Dennis Box
	   
157 26 Dennis Box
	 atomic rename items in same directory/fs
158 26 Dennis Box
159 26 Dennis Box
*  *ifdh* *ll*  loc  recursion_depth  force  
160 26 Dennis Box
	   
161 26 Dennis Box
	 list files with long listing 
162 26 Dennis Box
	 use recursion_depth== 0 to check directory without contents
163 26 Dennis Box
164 26 Dennis Box
*  *ifdh* *lss*  loc  recursion_depth  force  
165 26 Dennis Box
	   
166 26 Dennis Box
	 list files with sizes 
167 26 Dennis Box
	 use recursion_depth== 0 to check directory without contents
168 26 Dennis Box
169 26 Dennis Box
*	*ifdh* *findMatchingFiles*  path  glob 
170 26 Dennis Box
	   
171 26 Dennis Box
	 find filenames and sizes matching pattern
172 26 Dennis Box
173 26 Dennis Box
*	*ifdh* *fetchSharedFiles* args
174 26 Dennis Box
	   
175 26 Dennis Box
	 filenames and sizes matching pattern moved locally enough to be seen
176 26 Dennis Box
177 26 Dennis Box
*	*ifdh* *locateFiles* args
178 26 Dennis Box
	   
179 26 Dennis Box
	 locate multiple files
180 26 Dennis Box
181 26 Dennis Box
*	*ifdh* *checksum*  loc 
182 26 Dennis Box
	   
183 26 Dennis Box
	 cheksum file