Running Jobs » History » Version 15
Gianluca Petrillo, 01/22/2016 11:03 AM
1 | 1 | Erica Snider | {{toc}} |
---|---|---|---|
2 | 1 | Erica Snider | |
3 | 1 | Erica Snider | Under construction... NOT YET A SOURCE OF INFORMATION!! |
4 | 1 | Erica Snider | |
5 | 3 | Gianluca Petrillo | |
6 | 1 | Erica Snider | h1. Running Jobs |
7 | 1 | Erica Snider | |
8 | 9 | Gianluca Petrillo | This page describes the job configuration file (often, in jargon: _FHiCL file_) and how to run a job using one. |
9 | 1 | Erica Snider | |
10 | 3 | Gianluca Petrillo | It is assumed that you have previously read the [[ Quick-start guide to using and developing LArSoft code ]] and the information on [[Using LArSoft on the GPVM nodes]]. |
11 | 1 | Erica Snider | |
12 | 11 | Gianluca Petrillo | h2. The Job Configuration |
13 | 1 | Erica Snider | |
14 | 9 | Gianluca Petrillo | Once a base release is set up, it is easy to run a job. The basic unit for running a job is the job-control script, written in the FHICL language. The FHICL language provides a simple mechanism for including parameter set configurations from different files such that many job configuration files can use the same configuration for a module or service. |
15 | 1 | Erica Snider | |
16 | 10 | Gianluca Petrillo | There is also a nice "FHiCL quick start guide":https://cdcvs.fnal.gov/redmine/attachments/16021/quick_start.pdf available for more details. |
17 | 2 | Lynn Garren | |
18 | 3 | Gianluca Petrillo | |
19 | 10 | Gianluca Petrillo | h3. Key Concepts in FHiCL |
20 | 1 | Erica Snider | |
21 | 1 | Erica Snider | There are a few key concepts to writing a FHICL job control script. In order, they are |
22 | 1 | Erica Snider | |
23 | 1 | Erica Snider | # Including previously defined configurations for services and modules from other files. This is done using @#include@ statements. *Be sure you don't have any trailing space or tab characters* on the @#include@ line. |
24 | 9 | Gianluca Petrillo | # Service block, denoted by @services: { }@ This block will contain configurations for ART specific services such as the @TFileService@ and the @RandomNumberGenerator@. It also contains the configuration of LArSoft specific services[1]. |
25 | 9 | Gianluca Petrillo | # Source block, denoted by @source: { }@. This block tells the job what kind of source to expect (@EmptyEvent@ in the case of Monte Carlo generation, @RootInput@ in the case of anything downstream of a Monte Carlo generator or reconstruction, and specific modules for data from the detector), the file name for the input source if appropriate, and how many events to process. Both the file name and number of events to process can be specified on the command line. |
26 | 9 | Gianluca Petrillo | # Output block, denoted by @outputs: { }@ This block tells the job what kind of output to make, i.e. @RootOutput@, and what the name of the output file should be. The output file name can be specified on the command line. It is possible to define more than one output file if one wanted to run a job that produced different output files based on filter criteria - i.e. empty events are put in one file and events with neutrinos in them are put in another. Multiple output files can only be specified in the job configuration file, not from the command line. |
27 | 9 | Gianluca Petrillo | # Physics block, denoted by @physics: { }@ This block is where all producer, analyzer, and filter modules are configured. Sequences of producer and filter modules to run is defined in user-named _paths_ in this block. The list of analyzers and output modules to run is defined in a separate user-named path. The block also defines two keyword parameters, @trigger_paths@ and @end_paths@. @trigger_paths@ contains all producer and filter paths to run, and @end_paths@ contains the analyzer and output path. |
28 | 3 | Gianluca Petrillo | |
29 | 10 | Gianluca Petrillo | Comments may be included in FHiCL configuration files using the "#" character. The @#include@ is a keyword so that the parser knows not to ignore what comes after "#include[2]". |
30 | 9 | Gianluca Petrillo | |
31 | 9 | Gianluca Petrillo | fn1. In old FHiCL files you will notice that LArSoft and in general non-art service configuration is enclosed in a @user@ block, that is now deprecated. |
32 | 9 | Gianluca Petrillo | |
33 | 9 | Gianluca Petrillo | fn2. Note that the FHiCL parser can't recognize a comment on the same line as a @#include@ directive. |
34 | 1 | Erica Snider | |
35 | 3 | Gianluca Petrillo | |
36 | 10 | Gianluca Petrillo | h3. FHiCL Rules |
37 | 1 | Erica Snider | |
38 | 10 | Gianluca Petrillo | There are a few of rules to keep in mind about FHiCL: |
39 | 10 | Gianluca Petrillo | * The value of the @process_name@ parameter may not contain underscores as the process name is used in the ROOT file branch name. Module labels may not contain underscores either, for the same reason. |
40 | 1 | Erica Snider | * Parameter set names may not contain numbers, periods, backslashes, stars, etc. They may contain underscores. |
41 | 3 | Gianluca Petrillo | * Put the values for all string parameters in double quotes, @"..."@ |
42 | 3 | Gianluca Petrillo | * Specify input vectors using @[ , , ]@, i.e. if you want a vector of doubles do @MyVector: [1.0, 3e-9, -900.]@ |
43 | 10 | Gianluca Petrillo | * You pick out configurations from the @PROLOG@ section(s), usually defined in the @#include@ files, using the @ @local::@ syntax. The value after the "@::@" is the name of the configuration specified in the @PROLOG@ (see the next bullet) |
44 | 3 | Gianluca Petrillo | * You can override the value of an included configuration. For example, imagine there is a configuration specified in a included file called @mymoduleconfig@ and it contains the value @-5@ for the parameter named @myint@. One can load the configuration and then change the value of @myint@ by doing the following: |
45 | 3 | Gianluca Petrillo | *# inside the producers block: |
46 | 1 | Erica Snider | <pre> |
47 | 3 | Gianluca Petrillo | physics: { |
48 | 3 | Gianluca Petrillo | producers: { |
49 | 3 | Gianluca Petrillo | # ... |
50 | 3 | Gianluca Petrillo | mymod: @local::mymoduleconfig |
51 | 3 | Gianluca Petrillo | } |
52 | 3 | Gianluca Petrillo | } |
53 | 3 | Gianluca Petrillo | </pre> |
54 | 3 | Gianluca Petrillo | *# out of the physics block |
55 | 3 | Gianluca Petrillo | <pre> |
56 | 1 | Erica Snider | physics.producer.mymod.myint: 1 |
57 | 1 | Erica Snider | </pre> |
58 | 3 | Gianluca Petrillo | The last value for a parameter always wins. If the second line was repeated with the value @2@ instead of @1@, the job would run with @myint@ as @2@. |
59 | 3 | Gianluca Petrillo | Also note that in the example the original content of @mymoduleconfig@ is not changed when the content of @mymod@ is. |
60 | 3 | Gianluca Petrillo | |
61 | 1 | Erica Snider | |
62 | 6 | Gianluca Petrillo | h3. Example configuration file: @detsimmodules.fcl@ |
63 | 6 | Gianluca Petrillo | |
64 | 6 | Gianluca Petrillo | An example of a file with predefined configurations for modules is in the "larsim/DetSim/detsimmodules.fcl":https://cdcvs.fnal.gov/redmine/projects/larsim/repository/revisions/develop/entry/DetSim/detsimmodules.fcl file. |
65 | 6 | Gianluca Petrillo | All the definitions are inside a prologue block. |
66 | 6 | Gianluca Petrillo | The following is taken from LArSoft @v02_03_01@ (experiments now use their specific @detsimmodule_Xxxx.fcl@ configuration file though): |
67 | 6 | Gianluca Petrillo | |
68 | 6 | Gianluca Petrillo | <pre> |
69 | 6 | Gianluca Petrillo | ###### All files that are parameter set definitions must contain BEGIN_PROLOG as their first line ###### |
70 | 6 | Gianluca Petrillo | ###### This tag tells the FHICL parser that parameter set definitions are coming ###### |
71 | 6 | Gianluca Petrillo | BEGIN_PROLOG |
72 | 6 | Gianluca Petrillo | |
73 | 6 | Gianluca Petrillo | |
74 | 6 | Gianluca Petrillo | ###### Generic configuration of the analyser for SimWireXxxx producer output ###### |
75 | 6 | Gianluca Petrillo | standard_simwireana: |
76 | 6 | Gianluca Petrillo | { |
77 | 6 | Gianluca Petrillo | module_type: "SimWireAna" |
78 | 6 | Gianluca Petrillo | DetSimModuleLabel: "daq" |
79 | 6 | Gianluca Petrillo | } |
80 | 6 | Gianluca Petrillo | |
81 | 6 | Gianluca Petrillo | ###### Configuration of the SimWire module (digitization) for ArgoNeuT ###### |
82 | 6 | Gianluca Petrillo | # extra comment to test check-in |
83 | 6 | Gianluca Petrillo | argoneut_simwire: |
84 | 6 | Gianluca Petrillo | { |
85 | 6 | Gianluca Petrillo | module_type: "SimWireT962" |
86 | 6 | Gianluca Petrillo | DriftEModuleLabel: "largeant" |
87 | 6 | Gianluca Petrillo | ResponseFile: "shape-argo.root" |
88 | 6 | Gianluca Petrillo | NoiseFact: 0.132 #Noise Scale |
89 | 6 | Gianluca Petrillo | NoiseWidth: 62.4 #Exponential Noise width (kHz) |
90 | 6 | Gianluca Petrillo | LowCutoff: 7.5 #Low frequency filter cutoff (kHz) |
91 | 6 | Gianluca Petrillo | FieldBins: 75 |
92 | 6 | Gianluca Petrillo | Col3DCorrection: 2.5 |
93 | 6 | Gianluca Petrillo | Ind3DCorrection: 1.5 |
94 | 6 | Gianluca Petrillo | ColFieldRespAmp: 0.0354 |
95 | 6 | Gianluca Petrillo | IndFieldRespAmp: 0.018 |
96 | 6 | Gianluca Petrillo | ShapeTimeConst: [ 3000., 900. ] |
97 | 6 | Gianluca Petrillo | CompressionType: "none" #could also be Huffman |
98 | 6 | Gianluca Petrillo | } |
99 | 6 | Gianluca Petrillo | ###### Configuration of the analyser for SimWireXxxx output for ArgoNeuT (copy of generic) ###### |
100 | 6 | Gianluca Petrillo | argoneut_simwireana: @local::standard_simwireana |
101 | 6 | Gianluca Petrillo | |
102 | 6 | Gianluca Petrillo | ###### Configuration of the SimWire module (digitization) for MicroBooNE ###### |
103 | 6 | Gianluca Petrillo | microboone_simwire: |
104 | 6 | Gianluca Petrillo | { |
105 | 6 | Gianluca Petrillo | module_type: "SimWireMicroBooNE" |
106 | 6 | Gianluca Petrillo | DriftEModuleLabel: "largeant" |
107 | 6 | Gianluca Petrillo | NoiseFact: 0.0132 #Noise Scale |
108 | 6 | Gianluca Petrillo | #NoiseFact: 0.15 #Noise Scale to use with histogram |
109 | 6 | Gianluca Petrillo | NoiseWidth: 62.4 #Exponential Noise width (kHz) |
110 | 6 | Gianluca Petrillo | LowCutoff: 7.5 #Low frequency filter cutoff (kHz) |
111 | 6 | Gianluca Petrillo | CompressionType: "none" #could also be none |
112 | 6 | Gianluca Petrillo | GetNoiseFromHisto: false |
113 | 6 | Gianluca Petrillo | NoiseFileFname: "uboone_noise_v0.1.root" |
114 | 6 | Gianluca Petrillo | NoiseHistoName: "NoiseFreq" |
115 | 6 | Gianluca Petrillo | |
116 | 6 | Gianluca Petrillo | } |
117 | 6 | Gianluca Petrillo | ###### Configuration of the analyser for SimWireXxxx output for MicroBooNE (copy of generic) ###### |
118 | 6 | Gianluca Petrillo | microboone_simwireana: @local::standard_simwireana |
119 | 6 | Gianluca Petrillo | |
120 | 6 | Gianluca Petrillo | |
121 | 6 | Gianluca Petrillo | |
122 | 6 | Gianluca Petrillo | |
123 | 6 | Gianluca Petrillo | |
124 | 6 | Gianluca Petrillo | ###### Configuration of a different analyser for SimWireXxxx output, generic and experiment-specific ###### |
125 | 6 | Gianluca Petrillo | standard_wienerfilterana: |
126 | 6 | Gianluca Petrillo | { |
127 | 6 | Gianluca Petrillo | module_type: "WienerFilterAna" |
128 | 6 | Gianluca Petrillo | DetSimModuleLabel: "daq" |
129 | 6 | Gianluca Petrillo | } |
130 | 6 | Gianluca Petrillo | |
131 | 6 | Gianluca Petrillo | bo_wienerfilterana: @local::standard_wienerfilterana |
132 | 6 | Gianluca Petrillo | argoneut_wienerfilterana: @local::standard_wienerfilterana |
133 | 6 | Gianluca Petrillo | microboone_wienerfilterana: @local::standard_wienerfilterana |
134 | 6 | Gianluca Petrillo | |
135 | 6 | Gianluca Petrillo | |
136 | 6 | Gianluca Petrillo | ###### All files that are parameter set definitions must contain END_PROLOG as their last line ###### |
137 | 6 | Gianluca Petrillo | ###### This tag tells the FHICL parser that parameter set definitions are ended ###### |
138 | 6 | Gianluca Petrillo | END_PROLOG |
139 | 6 | Gianluca Petrillo | </pre> |
140 | 6 | Gianluca Petrillo | |
141 | 5 | Gianluca Petrillo | h3. Configuring the "message service":https://cdcvs.fnal.gov/redmine/projects/messagefacility/wiki/Using_MessageFacility#Using-MessageFacility |
142 | 1 | Erica Snider | |
143 | 4 | Gianluca Petrillo | Several standard configurations for the message service are in "lardata/Utilities/messageservice.fcl":https://cdcvs.fnal.gov/redmine/projects/lardata/repository/revisions/develop/entry/Utilities/messageservice.fcl. There is one configuration for each level of message output - Debug, Info, Warning, and Error. These configurations will be applied to each message level that is specified and those of higher priority. For example, the Info configuration will print out Info, Warning and Error level messages while the Warning configuration only print outs Warning and Error level messages. The "standard" debug configurations will cause the messages to go to a specified output file, @debug.log@. The Error configuration redirect to standard error stream (like @std::cerr@), while the others print into the standard output (@std::cout@). All impose some limits on the repetition of some frequent messages. |
144 | 5 | Gianluca Petrillo | Remember that to use one of these "standard" configurations you need to include it in your FCL file: they are standard, not default. |
145 | 4 | Gianluca Petrillo | If you want to define your own configuration, please take a look at the comments in "lardata/Utilities/messageservice.fcl":https://cdcvs.fnal.gov/redmine/projects/lardata/repository/revisions/develop/entry/Utilities/messageservice.fcl file to determine how to do so. |
146 | 1 | Erica Snider | |
147 | 1 | Erica Snider | Examples of how to include the usual use of the message service configurations are in the example files below. |
148 | 1 | Erica Snider | |
149 | 5 | Gianluca Petrillo | To get a different level of output from just one module (say @DBSCAN@) one would do: |
150 | 1 | Erica Snider | |
151 | 1 | Erica Snider | <pre> |
152 | 1 | Erica Snider | services: |
153 | 1 | Erica Snider | { |
154 | 1 | Erica Snider | # Load the service that manages root files for histograms. |
155 | 1 | Erica Snider | TFileService: { fileName: "reco_hist.root" } |
156 | 1 | Erica Snider | Timing: {} |
157 | 1 | Erica Snider | RandomNumberGenerator: {} #ART native random number generator |
158 | 1 | Erica Snider | |
159 | 1 | Erica Snider | # configure the message service with the INFO for DBSCAN |
160 | 1 | Erica Snider | # and WARNING level for everything else |
161 | 1 | Erica Snider | message: { |
162 | 1 | Erica Snider | destinations: { |
163 | 1 | Erica Snider | infomsg: { |
164 | 1 | Erica Snider | type: "cout" |
165 | 1 | Erica Snider | threshold: "INFO" |
166 | 1 | Erica Snider | append: true |
167 | 1 | Erica Snider | category: { |
168 | 1 | Erica Snider | DBSCAN: { |
169 | 1 | Erica Snider | reportEvery: 1 |
170 | 1 | Erica Snider | } |
171 | 1 | Erica Snider | } |
172 | 1 | Erica Snider | } |
173 | 1 | Erica Snider | warningmsg: { |
174 | 1 | Erica Snider | type: "cout" |
175 | 1 | Erica Snider | threshold: "WARNING" |
176 | 1 | Erica Snider | append: true |
177 | 1 | Erica Snider | categories: { |
178 | 1 | Erica Snider | default: { |
179 | 1 | Erica Snider | limit: 1000 |
180 | 1 | Erica Snider | timespan: 60 |
181 | 1 | Erica Snider | } |
182 | 1 | Erica Snider | } # end categories |
183 | 1 | Erica Snider | } # end warningmsg |
184 | 1 | Erica Snider | } # end destinations |
185 | 1 | Erica Snider | } # end standard_warning |
186 | 1 | Erica Snider | |
187 | 1 | Erica Snider | user: @local::argoneut_services |
188 | 1 | Erica Snider | } |
189 | 1 | Erica Snider | </pre> |
190 | 1 | Erica Snider | |
191 | 12 | Gianluca Petrillo | h4. Debug messages |
192 | 12 | Gianluca Petrillo | |
193 | 12 | Gianluca Petrillo | Note that debugging messages are treated in a special way: |
194 | 12 | Gianluca Petrillo | # there are two ways to print a debug message in the code: by using @mf::LogDebug@, and by using @LOG_DEBUG@ macro; the messages using the latter will not be present when a non-debug qualifier (@prof@ and @opt@) is used. In fact, no code is generated at all from a @LOG_DEBUG@ call unless using a @debug@ qualified build |
195 | 12 | Gianluca Petrillo | # debug output from modules is selectively enabled by the @debugModules@ list; to enable all the messages, use @debugModules: [ "*" ]@ |
196 | 12 | Gianluca Petrillo | |
197 | 12 | Gianluca Petrillo | Also remember that, depending on the configuration, the debug output could be not shown on screen, but only stored in a file (e.g. @debug.log@). |
198 | 1 | Erica Snider | |
199 | 5 | Gianluca Petrillo | h3. Example job script: @prodgenie.fcl@ |
200 | 1 | Erica Snider | |
201 | 5 | Gianluca Petrillo | An example job script to produce Monte Carlo events is "larsim/EventGenerator/GENIE/prodgenie.fcl":https://cdcvs.fnal.gov/redmine/projects/larsim/repository/revisions/develop/entry/EventGenerator/GENIE/prodgenie.fcl . The job defined by this script will generate neutrino interactions using GENIE, run them through Geant4, do the electron transport and then simulate the electronics. |
202 | 1 | Erica Snider | |
203 | 1 | Erica Snider | Comments on the form of the file are included as ###### Commment ###### |
204 | 1 | Erica Snider | |
205 | 1 | Erica Snider | <pre> |
206 | 1 | Erica Snider | ###### This is how to include configurations from other files ###### |
207 | 5 | Gianluca Petrillo | #include "services.fcl" |
208 | 5 | Gianluca Petrillo | #include "genie.fcl" |
209 | 5 | Gianluca Petrillo | #include "largeantmodules.fcl" |
210 | 5 | Gianluca Petrillo | #include "detsimmodules.fcl" |
211 | 1 | Erica Snider | |
212 | 1 | Erica Snider | ###### give the process a name ###### |
213 | 1 | Erica Snider | process_name: GenieGen |
214 | 1 | Erica Snider | |
215 | 5 | Gianluca Petrillo | ###### Please note the convention of defining detector specific configurations ###### |
216 | 1 | Erica Snider | ###### Pick out the configurations from the #include files using the @local:: syntax ###### |
217 | 1 | Erica Snider | ###### for services from LArSoft, in the user{} block - see definitions for configurations in ###### |
218 | 1 | Erica Snider | ###### job/geometry.fcl ###### |
219 | 1 | Erica Snider | ###### job/services.fcl ###### |
220 | 5 | Gianluca Petrillo | ###### job/simulationservices.fcl ###### |
221 | 5 | Gianluca Petrillo | |
222 | 5 | Gianluca Petrillo | services: |
223 | 1 | Erica Snider | { |
224 | 5 | Gianluca Petrillo | # Load the service that manages root files for histograms. |
225 | 1 | Erica Snider | TFileService: { fileName: "genie_hist.root" } |
226 | 1 | Erica Snider | Timing: {} |
227 | 5 | Gianluca Petrillo | SimpleMemoryCheck: { ignoreTotal: 1 } # default is one |
228 | 5 | Gianluca Petrillo | RandomNumberGenerator: {} #ART native random number generator |
229 | 5 | Gianluca Petrillo | user: @local::argoneut_simulation_services |
230 | 1 | Erica Snider | } |
231 | 1 | Erica Snider | |
232 | 1 | Erica Snider | ###### source is where you get events from - can also be RootInput ###### |
233 | 1 | Erica Snider | #Start each new event with an empty event. |
234 | 1 | Erica Snider | source: |
235 | 1 | Erica Snider | { |
236 | 1 | Erica Snider | module_type: EmptyEvent |
237 | 1 | Erica Snider | maxEvents: 10 # Number of events to create |
238 | 1 | Erica Snider | firstRun: 1 # Run number to use for this file |
239 | 1 | Erica Snider | firstEvent: 1 # number of first event in the file |
240 | 5 | Gianluca Petrillo | } |
241 | 1 | Erica Snider | |
242 | 1 | Erica Snider | ###### physics is the block that controls configuration of modules ###### |
243 | 5 | Gianluca Petrillo | # Define and configure some modules to do work on each event. |
244 | 1 | Erica Snider | # First modules are defined; they are scheduled later. |
245 | 5 | Gianluca Petrillo | # Modules are grouped by type. |
246 | 5 | Gianluca Petrillo | physics: |
247 | 1 | Erica Snider | { |
248 | 1 | Erica Snider | |
249 | 1 | Erica Snider | ###### the module labels in the output file will be generator, largeant, and daq ###### |
250 | 1 | Erica Snider | ###### their configuration is taken from ArgoNeuT defaults ###### |
251 | 1 | Erica Snider | producers: |
252 | 1 | Erica Snider | { |
253 | 1 | Erica Snider | generator: @local::argoneut_genie_simple_neutrino |
254 | 5 | Gianluca Petrillo | largeant: @local::argoneut_largeant |
255 | 1 | Erica Snider | daq: @local::argoneut_simwire |
256 | 1 | Erica Snider | rns: { module_type: "RandomNumberSaver" } |
257 | 1 | Erica Snider | } |
258 | 1 | Erica Snider | |
259 | 1 | Erica Snider | #define the producer and filter modules for this path, order matters, |
260 | 1 | Erica Snider | #filters reject all following items. see lines starting physics.producers below |
261 | 1 | Erica Snider | simulate: [ rns, generator, largeant, daq, ] |
262 | 1 | Erica Snider | |
263 | 1 | Erica Snider | #define the output stream, there could be more than one if using filters |
264 | 1 | Erica Snider | stream1: [ out1 ] |
265 | 1 | Erica Snider | |
266 | 1 | Erica Snider | #trigger_paths is a keyword and contains the paths that modify the art::event, |
267 | 1 | Erica Snider | #ie filters and producers |
268 | 1 | Erica Snider | trigger_paths: [simulate] |
269 | 1 | Erica Snider | |
270 | 5 | Gianluca Petrillo | #end_paths is a keyword and contains the paths that do not modify the art::Event, |
271 | 1 | Erica Snider | #ie analyzers and output streams. these all run simultaneously |
272 | 5 | Gianluca Petrillo | end_paths: [stream1] |
273 | 5 | Gianluca Petrillo | } |
274 | 1 | Erica Snider | |
275 | 1 | Erica Snider | #block to define where the output goes. if you defined a filter in the physics |
276 | 1 | Erica Snider | #block and put it in the trigger_paths then you need to put a SelectEvents: {SelectEvents: [XXX]} |
277 | 1 | Erica Snider | #entry in the output stream you want those to go to, where XXX is the label of the filter module(s) |
278 | 1 | Erica Snider | outputs: |
279 | 1 | Erica Snider | { |
280 | 1 | Erica Snider | out1: |
281 | 1 | Erica Snider | { |
282 | 1 | Erica Snider | module_type: RootOutput |
283 | 1 | Erica Snider | fileName: "genie_gen.root" #default file name, can override from command line with -o or --output |
284 | 1 | Erica Snider | } |
285 | 1 | Erica Snider | } |
286 | 1 | Erica Snider | |
287 | 1 | Erica Snider | </pre> |
288 | 1 | Erica Snider | |
289 | 1 | Erica Snider | Notice that you have not specified which libraries to load anywhere. That is because the build system compiles the plugin shared library files (@.so@) against the ones they depend upon. |
290 | 1 | Erica Snider | |
291 | 1 | Erica Snider | |
292 | 1 | Erica Snider | h3. Example job script: @standard_reco.fcl@ |
293 | 1 | Erica Snider | |
294 | 6 | Gianluca Petrillo | There is an example reconstruction job script available for people to use, "lardata/Utilities/standard_reco":https://cdcvs.fnal.gov/redmine/projects/lardata/repository/revisions/develop/entry/Utilities/standard_reco.fcl . |
295 | 6 | Gianluca Petrillo | This script takes the output of either raw data or MC that has produced simulated raw digits and performs a list of reconstruction tasks. |
296 | 6 | Gianluca Petrillo | The version from LArSoft @v02_03_01@ is copied here: |
297 | 1 | Erica Snider | |
298 | 1 | Erica Snider | <pre> |
299 | 6 | Gianluca Petrillo | #include "services.fcl" |
300 | 6 | Gianluca Petrillo | #include "caldata.fcl" |
301 | 6 | Gianluca Petrillo | #include "hitfindermodules.fcl" |
302 | 6 | Gianluca Petrillo | #include "clustermodules.fcl" |
303 | 6 | Gianluca Petrillo | #include "trackfindermodules.fcl" |
304 | 1 | Erica Snider | |
305 | 1 | Erica Snider | process_name: Reco |
306 | 1 | Erica Snider | |
307 | 1 | Erica Snider | services: |
308 | 1 | Erica Snider | { |
309 | 1 | Erica Snider | # Load the service that manages root files for histograms. |
310 | 1 | Erica Snider | TFileService: { fileName: "reco_hist.root" } |
311 | 1 | Erica Snider | Timing: {} |
312 | 1 | Erica Snider | RandomNumberGenerator: {} #ART native random number generator |
313 | 1 | Erica Snider | message: @local::standard_warning |
314 | 6 | Gianluca Petrillo | user: @local::argoneut_services |
315 | 1 | Erica Snider | } |
316 | 1 | Erica Snider | |
317 | 6 | Gianluca Petrillo | |
318 | 1 | Erica Snider | #source is now a root file |
319 | 1 | Erica Snider | source: |
320 | 1 | Erica Snider | { |
321 | 1 | Erica Snider | module_type: RootInput |
322 | 1 | Erica Snider | maxEvents: 10 # Number of events to create |
323 | 1 | Erica Snider | } |
324 | 1 | Erica Snider | |
325 | 1 | Erica Snider | # Define and configure some modules to do work on each event. |
326 | 1 | Erica Snider | # First modules are defined; they are scheduled later. |
327 | 1 | Erica Snider | # Modules are grouped by type. |
328 | 1 | Erica Snider | physics: |
329 | 1 | Erica Snider | { |
330 | 1 | Erica Snider | |
331 | 1 | Erica Snider | producers: |
332 | 1 | Erica Snider | { |
333 | 6 | Gianluca Petrillo | caldata: @local::argoneut_calwire |
334 | 6 | Gianluca Petrillo | ffthit: @local::argoneut_hitfinder # note if job is for MC, use argoneut_mc_hitfinder |
335 | 6 | Gianluca Petrillo | dbcluster: @local::argoneut_dbcluster |
336 | 1 | Erica Snider | hough: @local::argoneut_houghlinefinder |
337 | 6 | Gianluca Petrillo | linemerger: @local::argoneut_linemerger |
338 | 6 | Gianluca Petrillo | track: @local::argoneut_track |
339 | 6 | Gianluca Petrillo | harris: @local::argoneut_endpointmod |
340 | 1 | Erica Snider | } |
341 | 1 | Erica Snider | |
342 | 6 | Gianluca Petrillo | analyzers: |
343 | 6 | Gianluca Petrillo | { |
344 | 6 | Gianluca Petrillo | dbclusterana: @local::argoneut_dbclusterana |
345 | 6 | Gianluca Petrillo | } |
346 | 6 | Gianluca Petrillo | |
347 | 1 | Erica Snider | #define the producer and filter modules for this path, order matters, |
348 | 1 | Erica Snider | #filters reject all following items. see lines starting physics.producers below |
349 | 6 | Gianluca Petrillo | reco: [ caldata, ffthit, dbcluster, hough, linemerger, track, harris ] |
350 | 6 | Gianluca Petrillo | ana: [ dbclusterana ] |
351 | 6 | Gianluca Petrillo | |
352 | 1 | Erica Snider | #define the output stream, there could be more than one if using filters |
353 | 1 | Erica Snider | stream1: [ out1 ] |
354 | 1 | Erica Snider | |
355 | 1 | Erica Snider | #trigger_paths is a keyword and contains the paths that modify the art::event, |
356 | 1 | Erica Snider | #ie filters and producers |
357 | 1 | Erica Snider | trigger_paths: [reco] |
358 | 1 | Erica Snider | |
359 | 1 | Erica Snider | #end_paths is a keyword and contains the paths that do not modify the art::Event, |
360 | 1 | Erica Snider | #ie analyzers and output streams. these all run simultaneously |
361 | 6 | Gianluca Petrillo | end_paths: [ ana, stream1] |
362 | 1 | Erica Snider | } |
363 | 1 | Erica Snider | |
364 | 1 | Erica Snider | #block to define where the output goes. if you defined a filter in the physics |
365 | 1 | Erica Snider | #block and put it in the trigger_paths then you need to put a SelectEvents: {SelectEvents: [XXX]} |
366 | 1 | Erica Snider | #entry in the output stream you want those to go to, where XXX is the label of the filter module(s) |
367 | 1 | Erica Snider | outputs: |
368 | 1 | Erica Snider | { |
369 | 1 | Erica Snider | out1: |
370 | 1 | Erica Snider | { |
371 | 1 | Erica Snider | module_type: RootOutput |
372 | 1 | Erica Snider | fileName: "standard_reco.root" #default file name, can override from command line with -o or --output |
373 | 1 | Erica Snider | } |
374 | 1 | Erica Snider | } |
375 | 1 | Erica Snider | |
376 | 1 | Erica Snider | </pre> |
377 | 1 | Erica Snider | |
378 | 1 | Erica Snider | h3. How to override a default parameter |
379 | 1 | Erica Snider | |
380 | 1 | Erica Snider | If you want to override a default parameter that has been included from a predefined parameter set, you must specify which parameter and its value as |
381 | 1 | Erica Snider | |
382 | 7 | Gianluca Petrillo | <pre> |
383 | 1 | Erica Snider | mainBlock.subBlock.label.parameterName: newValue |
384 | 7 | Gianluca Petrillo | </pre> |
385 | 1 | Erica Snider | |
386 | 1 | Erica Snider | where |
387 | 1 | Erica Snider | |
388 | 13 | Gianluca Petrillo | * @mainBlock@ can be @services@ or @physics@ |
389 | 13 | Gianluca Petrillo | * @subBlock@ can be @producers@, @filters@, or @analyzers@, skipped in case of services, and @user@ for old service configuration |
390 | 13 | Gianluca Petrillo | * @label@ is the name of the desired service or module instance in a producers, filters, or analyzers block |
391 | 1 | Erica Snider | * @parameterName@ is the name of the desired parameter |
392 | 13 | Gianluca Petrillo | * @newValue@ is the desired new value; a list or entire block (that is, a brace-enclosed sequence of @key: value@ pairs) can be specified |
393 | 1 | Erica Snider | |
394 | 1 | Erica Snider | These lines must go after the @mainBlock@ and be outside of any other block. |
395 | 1 | Erica Snider | |
396 | 13 | Gianluca Petrillo | For example, if one wanted to change the default value of the @HitsModuleLabel@ parameter in the @dbcluster@ module instance in the previous section, one would put |
397 | 1 | Erica Snider | <pre> |
398 | 13 | Gianluca Petrillo | physics.producers.dbcluster.HitsModuleLabel: "differentHitModuleLabel" |
399 | 1 | Erica Snider | </pre> |
400 | 13 | Gianluca Petrillo | Note that FHiCL allows for completely replacing a list but not for changing its size (i.e., it's not possible to add an element to an existing list). |
401 | 1 | Erica Snider | |
402 | 13 | Gianluca Petrillo | This behavior is general: FHiCL only allows to replace elements. For example, if one wants to change both @eps@ and @epstwo@ parameters, |
403 | 13 | Gianluca Petrillo | <pre> |
404 | 13 | Gianluca Petrillo | physics.producers.dbcluster.DBScanAlg: { # WRONG!!! |
405 | 13 | Gianluca Petrillo | eps: 1.1 |
406 | 13 | Gianluca Petrillo | epstwo: 1.6 |
407 | 13 | Gianluca Petrillo | } |
408 | 14 | Gianluca Petrillo | </pre> will not work, because it is replacing the whole @physics.producers.dbcluster.DBScanAlg@ parameter set, thus removing e.g. the parameter @minPts@. |
409 | 13 | Gianluca Petrillo | The correct way to change parameters is to think of that as _replacing_ them, and to point exactly at each of them: |
410 | 13 | Gianluca Petrillo | <pre> |
411 | 1 | Erica Snider | physics.producers.dbcluster.DBScanAlg.eps: 1.1 |
412 | 13 | Gianluca Petrillo | physics.producers.dbcluster.DBScanAlg.epstwo: 1.6 |
413 | 14 | Gianluca Petrillo | </pre> will leave the other parameters (e.g., @minPts@) unchanged. |
414 | 7 | Gianluca Petrillo | |
415 | 1 | Erica Snider | h3. fhicl Emacs syntax highlighting |
416 | 1 | Erica Snider | |
417 | 7 | Gianluca Petrillo | If you use Emacs as your editor, you can put the following into your @.emacs@ file in your home directory to cause it to display @.fcl@ files with syntax highlighting |
418 | 1 | Erica Snider | |
419 | 1 | Erica Snider | <pre> |
420 | 1 | Erica Snider | (setq fclKeywords |
421 | 1 | Erica Snider | '( |
422 | 1 | Erica Snider | ;; This, due to poor language design, conflicts with comments and fails |
423 | 1 | Erica Snider | ("#include" . font-lock-keyword-face) |
424 | 1 | Erica Snider | ("@local" . font-lock-keyword-face) |
425 | 1 | Erica Snider | ;; All these names are magic, I think |
426 | 1 | Erica Snider | |
427 | 1 | Erica Snider | |
428 | 1 | Erica Snider | ("process_name:\\|services:\\|source:\\|outputs:\\|physics\\|producers:\\|filters:\\|analyzers:" . font-lock-builtin-face) |
429 | 1 | Erica Snider | ("true\\|false" . font-lock-builtin-face) |
430 | 1 | Erica Snider | ;; Variable definitions are followed by colons |
431 | 1 | Erica Snider | |
432 | 1 | Erica Snider | ("[a-zA-Z0-9_]*:" . font-lock-variable-name-face) |
433 | 1 | Erica Snider | ) |
434 | 1 | Erica Snider | ) |
435 | 1 | Erica Snider | |
436 | 1 | Erica Snider | ;; Python mode gets us comment handling and indentation at colons |
437 | 1 | Erica Snider | |
438 | 1 | Erica Snider | (define-derived-mode fcl-mode python-mode |
439 | 1 | Erica Snider | (setq mode-name "FHICL") |
440 | 1 | Erica Snider | (setq font-lock-defaults '(fclKeywords)) |
441 | 1 | Erica Snider | ;; (setq tab-width 2) ;; Doesn't seem to work |
442 | 1 | Erica Snider | |
443 | 1 | Erica Snider | ) |
444 | 1 | Erica Snider | |
445 | 1 | Erica Snider | (add-to-list 'auto-mode-alist '("\\.fcl\\'" . fcl-mode)) |
446 | 1 | Erica Snider | </pre> |
447 | 1 | Erica Snider | |
448 | 15 | Gianluca Petrillo | bq. Please use _only spaces_ for alignment: it's not the perfect solution, but it makes people using different editors see the same code alignment. |
449 | 7 | Gianluca Petrillo | |
450 | 1 | Erica Snider | h2. Executable and command line options |
451 | 1 | Erica Snider | |
452 | 7 | Gianluca Petrillo | Currently there is one executable to run in LArSoft. The executable to run a typical reconstruction or analysis job is @lar@ which is placed in the user's path by the setup script. To see what options are available do |
453 | 7 | Gianluca Petrillo | <pre> |
454 | 7 | Gianluca Petrillo | $ lar -h |
455 | 7 | Gianluca Petrillo | </pre> |
456 | 7 | Gianluca Petrillo | The output for @art@ version @v1_09_03@ is: |
457 | 7 | Gianluca Petrillo | <pre> |
458 | 7 | Gianluca Petrillo | Usage: lar <-c <config-file>> <other-options> [<source-file>]+ |
459 | 1 | Erica Snider | |
460 | 7 | Gianluca Petrillo | Allowed options: |
461 | 7 | Gianluca Petrillo | -c [ --config ] arg Configuration file. |
462 | 7 | Gianluca Petrillo | -h [ --help ] produce help message |
463 | 7 | Gianluca Petrillo | --process-name arg art process name. |
464 | 1 | Erica Snider | |
465 | 7 | Gianluca Petrillo | -s [ --source ] arg Source data file (multiple OK). |
466 | 7 | Gianluca Petrillo | -S [ --source-list ] arg file containing a list of source files to read, |
467 | 7 | Gianluca Petrillo | one per line. |
468 | 7 | Gianluca Petrillo | -e [ --estart ] arg Event # of first event to process. |
469 | 7 | Gianluca Petrillo | -n [ --nevts ] arg Number of events to process. |
470 | 7 | Gianluca Petrillo | --nskip arg Number of events to skip. |
471 | 7 | Gianluca Petrillo | -T [ --TFileName ] arg File name for TFileService. |
472 | 7 | Gianluca Petrillo | -o [ --output ] arg Event output stream file. |
473 | 7 | Gianluca Petrillo | --trace Activate tracing. |
474 | 7 | Gianluca Petrillo | --notrace Deactivate tracing. |
475 | 7 | Gianluca Petrillo | --memcheck Activate monitoring of memory use. |
476 | 7 | Gianluca Petrillo | --nomemcheck Deactivate monitoring of memory use. |
477 | 7 | Gianluca Petrillo | --default-exceptions some exceptions may be handled differently by |
478 | 7 | Gianluca Petrillo | default (e.g. ProductNotFound). |
479 | 7 | Gianluca Petrillo | --rethrow-default all exceptions default to rethrow. |
480 | 7 | Gianluca Petrillo | --rethrow-all all exceptions overridden to rethrow (cf |
481 | 7 | Gianluca Petrillo | rethrow-default). |
482 | 7 | Gianluca Petrillo | --sam-web-uri arg URI for SAM web service. |
483 | 7 | Gianluca Petrillo | --sam-process-id arg SAM process ID. |
484 | 7 | Gianluca Petrillo | --sam-application-family arg SAM application family. |
485 | 7 | Gianluca Petrillo | --sam-app-family arg SAM application family. |
486 | 7 | Gianluca Petrillo | --sam-application-version arg SAM application version. |
487 | 7 | Gianluca Petrillo | --sam-app-version arg SAM application version. |
488 | 7 | Gianluca Petrillo | --sam-file-type arg File type for SAM metadata. |
489 | 7 | Gianluca Petrillo | --sam-data-tier arg SAM data tier spec (<module-label>:<tier-spec>) |
490 | 7 | Gianluca Petrillo | . |
491 | 7 | Gianluca Petrillo | --sam-stream-name arg SAM stream name (<module-label>:<stream-name>). |
492 | 1 | Erica Snider | |
493 | 7 | Gianluca Petrillo | Art has completed and will exit with status 1. |
494 | 7 | Gianluca Petrillo | </pre> |
495 | 1 | Erica Snider | |
496 | 7 | Gianluca Petrillo | |
497 | 1 | Erica Snider | h2. Running a Job |
498 | 1 | Erica Snider | |
499 | 1 | Erica Snider | To run the job defined by the script above, do |
500 | 7 | Gianluca Petrillo | <pre> |
501 | 7 | Gianluca Petrillo | $ lar -c job/prodgenie.fcl |
502 | 7 | Gianluca Petrillo | </pre> |
503 | 1 | Erica Snider | |
504 | 7 | Gianluca Petrillo | One can stop a job in two ways: |
505 | 7 | Gianluca Petrillo | # press <Ctrl>+<C> once: the job will complete at the end of the current module. If the job is running in the background type @kill -9 %jobID@ on the command line. |
506 | 7 | Gianluca Petrillo | # press <Ctrl>+<C> twice: the job should stop immediately and, depending on the shall settings, might produce a core dump. |
507 | 1 | Erica Snider | |
508 | 7 | Gianluca Petrillo | If you want to have your job keep running even if you get disconnected from a remote session, depending on your shell you might need to start it with: |
509 | 7 | Gianluca Petrillo | <pre> |
510 | 7 | Gianluca Petrillo | $ nohup lar job/prodgenie.fcl >& pg.out |
511 | 7 | Gianluca Petrillo | </pre> |
512 | 1 | Erica Snider | |
513 | 1 | Erica Snider | To stop such a job, then do |
514 | 7 | Gianluca Petrillo | <pre> |
515 | 7 | Gianluca Petrillo | $ pgrep lar # to find the job ID (use ps to disambiguate if more than one lar process are running) |
516 | 7 | Gianluca Petrillo | $ kill -INT jobID |
517 | 7 | Gianluca Petrillo | </pre> |
518 | 1 | Erica Snider | |
519 | 7 | Gianluca Petrillo | One can print out the configuration of the job without starting the executable by: |
520 | 7 | Gianluca Petrillo | <pre> |
521 | 7 | Gianluca Petrillo | $ ART_DEBUG_CONFIG=1 lar -c prodgenie.fcl |
522 | 7 | Gianluca Petrillo | </pre> |
523 | 1 | Erica Snider | in @bash@, or |
524 | 7 | Gianluca Petrillo | <pre> |
525 | 7 | Gianluca Petrillo | > env ART_DEBUG_CONFIG=1 lar -c prodgenie.fcl |
526 | 7 | Gianluca Petrillo | </pre> |
527 | 7 | Gianluca Petrillo | in C-shell, which produces the output |
528 | 7 | Gianluca Petrillo | <pre> |
529 | 7 | Gianluca Petrillo | ** ART_DEBUG_CONFIG is defined: config debug output follows ** |
530 | 1 | Erica Snider | all_modules: [ "out1" |
531 | 1 | Erica Snider | , "daq" |
532 | 1 | Erica Snider | , "generator" |
533 | 1 | Erica Snider | , "largeant" |
534 | 1 | Erica Snider | , "rns" |
535 | 1 | Erica Snider | ] |
536 | 1 | Erica Snider | outputs: { out1: { fileName: "genie_gen.root" |
537 | 1 | Erica Snider | module_label: "out1" |
538 | 1 | Erica Snider | module_type: "RootOutput" |
539 | 1 | Erica Snider | } |
540 | 1 | Erica Snider | } |
541 | 1 | Erica Snider | physics: { end_paths: [ "stream1" ] |
542 | 1 | Erica Snider | producers: { daq: { Col3DCorrection: 2.5 |
543 | 1 | Erica Snider | ColFieldRespAmp: 3.54e-2 |
544 | 1 | Erica Snider | CompressionType: "none" |
545 | 1 | Erica Snider | DriftEModuleLabel: "largeant" |
546 | 1 | Erica Snider | FieldBins: 75 |
547 | 1 | Erica Snider | Ind3DCorrection: 1.5 |
548 | 1 | Erica Snider | IndFieldRespAmp: 1.8e-2 |
549 | 1 | Erica Snider | LowCutoff: 7.5 |
550 | 1 | Erica Snider | NoiseFact: 1.32e-1 |
551 | 1 | Erica Snider | NoiseWidth: 6.24e1 |
552 | 1 | Erica Snider | ResponseFile: "shape-argo.root" |
553 | 1 | Erica Snider | ShapeTimeConst: [ 3000 |
554 | 1 | Erica Snider | , 900 |
555 | 1 | Erica Snider | ] |
556 | 1 | Erica Snider | module_label: "daq" |
557 | 1 | Erica Snider | module_type: "SimWireT962" |
558 | 1 | Erica Snider | } |
559 | 7 | Gianluca Petrillo | generator: { BeamCenter: [ -1400 |
560 | 7 | Gianluca Petrillo | , -350 |
561 | 1 | Erica Snider | , 0 |
562 | 1 | Erica Snider | ] |
563 | 7 | Gianluca Petrillo | [...] |
564 | 1 | Erica Snider | </pre> |
565 | 1 | Erica Snider | |
566 | 1 | Erica Snider | This functionality is particularly helpful when trying to debug what input parameters were passed to the job. |
567 | 1 | Erica Snider | |
568 | 7 | Gianluca Petrillo | |
569 | 1 | Erica Snider | h2. Why did my job fail? |
570 | 1 | Erica Snider | |
571 | 8 | Gianluca Petrillo | In case of failure, @art@ usually provides some information about the failure. |
572 | 1 | Erica Snider | |
573 | 8 | Gianluca Petrillo | If a job fails, look at the warnings printed to the screen or any output log files. |
574 | 1 | Erica Snider | |
575 | 8 | Gianluca Petrillo | Also check in the [[Breaking_Changes|breaking changes]] page for information about problems (and solutions) caused by changes to the code. |
576 | 1 | Erica Snider | |
577 | 8 | Gianluca Petrillo | You can ask help to the larsoft@fnal.gov mailing list, or open a issue in the appropriate tracker (usually "the LArSoft one":https://cdcvs.fnal.gov/redmine/projects/larsoft/issues), or to artists@fnal.gov . |
578 | 8 | Gianluca Petrillo | The usual bug report rules apply: include at least |
579 | 8 | Gianluca Petrillo | * the full command line of the failing command |
580 | 8 | Gianluca Petrillo | * a full path to the input files |
581 | 8 | Gianluca Petrillo | * which LArSoft release you are using |
582 | 8 | Gianluca Petrillo | * the complete error message. |
583 | 8 | Gianluca Petrillo | |
584 | 8 | Gianluca Petrillo | If a bug should be reported to the artists@fnal.gov list, attach the _complete output of the job_ to the email. |
585 | 8 | Gianluca Petrillo | |
586 | 8 | Gianluca Petrillo | |
587 | 1 | Erica Snider | h2. Submitting Jobs to the compute farms |
588 | 1 | Erica Snider | |
589 | 8 | Gianluca Petrillo | We have to find out where the page with the submission instructions went (*TODO*!). |