Project

General

Profile

Running Jobs » History » Version 7

Gianluca Petrillo, 07/28/2014 06:03 PM

1 1 Erica Snider
{{toc}}
2 1 Erica Snider
3 1 Erica Snider
Under construction... NOT YET A SOURCE OF INFORMATION!!
4 1 Erica Snider
5 3 Gianluca Petrillo
6 1 Erica Snider
h1. Running Jobs
7 1 Erica Snider
8 1 Erica Snider
This page describes the job configuration script and how to run a job using one.
9 1 Erica Snider
10 3 Gianluca Petrillo
It is assumed that you have previously read the [[ Quick-start guide to using and developing LArSoft code ]] and the information on [[Using LArSoft on the GPVM nodes]].
11 1 Erica Snider
12 1 Erica Snider
13 1 Erica Snider
h2. The Job Configuration Script
14 1 Erica Snider
15 1 Erica Snider
Once a base and test release are set up, it is easy to run a job. The basic unit for running a job is the job-control script, written in the FHICL language. The FHICL language provides a simple mechanism for including parameter set configurations from different files such that many job configuration files can use the same configuration for a module or service.  
16 1 Erica Snider
17 1 Erica Snider
There is also a nice "FHICL quick start guide":https://cdcvs.fnal.gov/redmine/attachments/16021/quick_start.pdf available for more details. 
18 2 Lynn Garren
19 3 Gianluca Petrillo
20 1 Erica Snider
h3. Key Concepts in FHICL
21 1 Erica Snider
22 1 Erica Snider
There are a few key concepts to writing a FHICL job control script.  In order, they are
23 1 Erica Snider
24 3 Gianluca Petrillo
# Including previously defined configurations for services and modules from other files.  This is done using @#include@ statements. *Be sure you don't have any trailing space or tab characters* on the @#include@ line.
25 3 Gianluca Petrillo
# Services block, denoted by @services: { }@  This block will contain configurations for ART specific services such as the @TFileService@ and the @RandomNumberGenerator@.  It also contains a @user: {}@ sub-block where LArSoft specific services are configured.  
26 3 Gianluca Petrillo
# Source block, denoted by @source: { }@.  This block tells the job what kind of source to expect (@EmptyEvent@ in the case of Monte Carlo generation, @RootInput@ in the case of anything downstream of a Monte Carlo generator or reconstruction), the file name for the input source if appropriate, and how many events to process.  Both the file name and number of events to process can be specified on the command line.
27 3 Gianluca Petrillo
# Outputs block, denoted by @outputs: { }@  This block tells the job what kind of output to make, i.e. @RootOutput@, and what the name of the output file should be.  The output file name can be specified on the command line.  It is possible to define more than one output file if one wanted to run a job that produced different output files based on filter criteria - i.e. empty events are put in one file and events with neutrinos in them are put in another. Multiple output files can only be specified in the job configuration file, not from the command line.
28 3 Gianluca Petrillo
# Physics block, denoted by @physics: { }@  This block is where all producer, analyzer, and filter modules are configured.  The sequence of producer and filter modules to run is defined in a user-named path in this block.  The list of analyzers to run is defined in separate user-named path.  The block also defines two keyword parameters, @trigger_paths@ and @end_paths@.  @trigger_paths@ contains all producer and filter paths to run, and @end_paths@ contains the analyzer paths and output streams.
29 1 Erica Snider
30 3 Gianluca Petrillo
Comments may be included in FHICL configuration files using the "#" character.  The @#include@ is a keyword so that the parser knows not to ignore what comes after "#include".
31 1 Erica Snider
32 3 Gianluca Petrillo
33 1 Erica Snider
h3. FHICL Rules
34 1 Erica Snider
35 1 Erica Snider
There are a couple of rules to keep in mind about FHICL:
36 1 Erica Snider
* The value of the process_name parameter may not contain underscores as the process name is used in the ROOT file branch name.  Module labels may not contain underscores either, for the same reason.
37 1 Erica Snider
* Parameter set names may not contain numbers, periods, backslashes, stars, etc. They may contain underscores.
38 3 Gianluca Petrillo
* Put the values for all string parameters in double quotes, @"..."@
39 3 Gianluca Petrillo
* Specify input vectors using @[ , , ]@, i.e. if you want a vector of doubles do @MyVector: [1.0, 3e-9, -900.]@
40 3 Gianluca Petrillo
* You pick out configurations from the @PROLOG@ section(s), usually defined in the @#include@ files, using the @local::@ syntax.  The value after the "::" is the name of the configuration specified in the @PROLOG@ (see the next bullet)
41 3 Gianluca Petrillo
* You can override the value of an included configuration. For example, imagine there is a configuration specified in a included file called @mymoduleconfig@ and it contains the value @-5@ for the parameter named @myint@. One can load the configuration and then change the value of @myint@ by doing the following:
42 3 Gianluca Petrillo
*# inside the producers block:
43 1 Erica Snider
<pre>
44 3 Gianluca Petrillo
physics: {
45 3 Gianluca Petrillo
  producers: {
46 3 Gianluca Petrillo
    # ...
47 3 Gianluca Petrillo
    mymod: @local::mymoduleconfig
48 3 Gianluca Petrillo
  }
49 3 Gianluca Petrillo
}
50 3 Gianluca Petrillo
</pre>
51 3 Gianluca Petrillo
*# out of the physics block 
52 3 Gianluca Petrillo
<pre>
53 1 Erica Snider
physics.producer.mymod.myint: 1
54 1 Erica Snider
</pre>
55 3 Gianluca Petrillo
The last value for a parameter always wins. If the second line was repeated with the value @2@ instead of @1@, the job would run with @myint@ as @2@.
56 3 Gianluca Petrillo
Also note that in the example the original content of @mymoduleconfig@ is not changed when the content of @mymod@ is.
57 3 Gianluca Petrillo
58 1 Erica Snider
59 6 Gianluca Petrillo
h3. Example configuration file: @detsimmodules.fcl@
60 6 Gianluca Petrillo
61 6 Gianluca Petrillo
An example of a file with predefined configurations for modules is in the "larsim/DetSim/detsimmodules.fcl":https://cdcvs.fnal.gov/redmine/projects/larsim/repository/revisions/develop/entry/DetSim/detsimmodules.fcl file.
62 6 Gianluca Petrillo
All the definitions are inside a prologue block.
63 6 Gianluca Petrillo
The following is taken from LArSoft @v02_03_01@ (experiments now use their specific @detsimmodule_Xxxx.fcl@ configuration file though):
64 6 Gianluca Petrillo
65 6 Gianluca Petrillo
<pre>
66 6 Gianluca Petrillo
###### All files that are parameter set definitions must contain BEGIN_PROLOG as their first line ######
67 6 Gianluca Petrillo
###### This tag tells the FHICL parser that parameter set definitions are coming                  ######
68 6 Gianluca Petrillo
BEGIN_PROLOG
69 6 Gianluca Petrillo
70 6 Gianluca Petrillo
71 6 Gianluca Petrillo
###### Generic configuration of the analyser for SimWireXxxx producer output ######
72 6 Gianluca Petrillo
standard_simwireana:
73 6 Gianluca Petrillo
{ 
74 6 Gianluca Petrillo
 module_type:       "SimWireAna"
75 6 Gianluca Petrillo
 DetSimModuleLabel: "daq"
76 6 Gianluca Petrillo
}
77 6 Gianluca Petrillo
78 6 Gianluca Petrillo
###### Configuration of the SimWire module (digitization) for ArgoNeuT ######
79 6 Gianluca Petrillo
# extra comment to test check-in
80 6 Gianluca Petrillo
argoneut_simwire:
81 6 Gianluca Petrillo
{ 
82 6 Gianluca Petrillo
 module_type:        "SimWireT962"
83 6 Gianluca Petrillo
 DriftEModuleLabel:  "largeant"
84 6 Gianluca Petrillo
 ResponseFile:       "shape-argo.root"
85 6 Gianluca Petrillo
 NoiseFact:          0.132        #Noise Scale
86 6 Gianluca Petrillo
 NoiseWidth:         62.4         #Exponential Noise width (kHz)
87 6 Gianluca Petrillo
 LowCutoff:          7.5          #Low frequency filter cutoff (kHz)
88 6 Gianluca Petrillo
 FieldBins:          75
89 6 Gianluca Petrillo
 Col3DCorrection:    2.5
90 6 Gianluca Petrillo
 Ind3DCorrection:    1.5
91 6 Gianluca Petrillo
 ColFieldRespAmp:    0.0354
92 6 Gianluca Petrillo
 IndFieldRespAmp:    0.018
93 6 Gianluca Petrillo
 ShapeTimeConst:     [ 3000., 900. ]
94 6 Gianluca Petrillo
 CompressionType:    "none"        #could also be Huffman
95 6 Gianluca Petrillo
}
96 6 Gianluca Petrillo
###### Configuration of the analyser for SimWireXxxx output for ArgoNeuT (copy of generic) ######
97 6 Gianluca Petrillo
argoneut_simwireana: @local::standard_simwireana
98 6 Gianluca Petrillo
99 6 Gianluca Petrillo
###### Configuration of the SimWire module (digitization) for MicroBooNE ######
100 6 Gianluca Petrillo
microboone_simwire:  
101 6 Gianluca Petrillo
{ 
102 6 Gianluca Petrillo
 module_type:        "SimWireMicroBooNE"
103 6 Gianluca Petrillo
 DriftEModuleLabel:  "largeant"
104 6 Gianluca Petrillo
 NoiseFact:          0.0132       #Noise Scale
105 6 Gianluca Petrillo
 #NoiseFact:          0.15       #Noise Scale to use with histogram
106 6 Gianluca Petrillo
 NoiseWidth:         62.4         #Exponential Noise width (kHz)
107 6 Gianluca Petrillo
 LowCutoff:          7.5          #Low frequency filter cutoff (kHz)
108 6 Gianluca Petrillo
 CompressionType:    "none"       #could also be none
109 6 Gianluca Petrillo
 GetNoiseFromHisto:   false  
110 6 Gianluca Petrillo
 NoiseFileFname:      "uboone_noise_v0.1.root"
111 6 Gianluca Petrillo
 NoiseHistoName:      "NoiseFreq"    
112 6 Gianluca Petrillo
113 6 Gianluca Petrillo
}
114 6 Gianluca Petrillo
###### Configuration of the analyser for SimWireXxxx output for MicroBooNE (copy of generic) ######
115 6 Gianluca Petrillo
microboone_simwireana: @local::standard_simwireana
116 6 Gianluca Petrillo
117 6 Gianluca Petrillo
118 6 Gianluca Petrillo
119 6 Gianluca Petrillo
120 6 Gianluca Petrillo
121 6 Gianluca Petrillo
###### Configuration of a different analyser for SimWireXxxx output, generic and experiment-specific ######
122 6 Gianluca Petrillo
standard_wienerfilterana: 
123 6 Gianluca Petrillo
{
124 6 Gianluca Petrillo
 module_type:        "WienerFilterAna"
125 6 Gianluca Petrillo
 DetSimModuleLabel:  "daq"
126 6 Gianluca Petrillo
}
127 6 Gianluca Petrillo
128 6 Gianluca Petrillo
bo_wienerfilterana:         @local::standard_wienerfilterana
129 6 Gianluca Petrillo
argoneut_wienerfilterana:   @local::standard_wienerfilterana
130 6 Gianluca Petrillo
microboone_wienerfilterana: @local::standard_wienerfilterana
131 6 Gianluca Petrillo
132 6 Gianluca Petrillo
133 6 Gianluca Petrillo
###### All files that are parameter set definitions must contain END_PROLOG as their last line ######
134 6 Gianluca Petrillo
###### This tag tells the FHICL parser that parameter set definitions are ended                ######
135 6 Gianluca Petrillo
END_PROLOG
136 6 Gianluca Petrillo
</pre>
137 6 Gianluca Petrillo
138 6 Gianluca Petrillo
139 5 Gianluca Petrillo
h3. Configuring the "message service":https://cdcvs.fnal.gov/redmine/projects/messagefacility/wiki/Using_MessageFacility#Using-MessageFacility
140 1 Erica Snider
141 4 Gianluca Petrillo
Several standard configurations for the message service are in "lardata/Utilities/messageservice.fcl":https://cdcvs.fnal.gov/redmine/projects/lardata/repository/revisions/develop/entry/Utilities/messageservice.fcl.  There is one configuration for each level of message output - Debug, Info, Warning, and Error.  These configurations will be applied to each message level that is specified and those of higher priority.  For example, the Info configuration will print out Info, Warning and Error level messages while the Warning configuration only print outs Warning and Error level messages. The "standard" debug configurations will cause the messages to go to a specified output file, @debug.log@. The Error configuration redirect to standard error stream (like @std::cerr@), while the others print into the standard output (@std::cout@). All impose some limits on the repetition of some frequent messages.
142 5 Gianluca Petrillo
Remember that to use one of these "standard" configurations you need to include it in your FCL file: they are standard, not default.
143 4 Gianluca Petrillo
If you want to define your own configuration, please take a look at the comments in "lardata/Utilities/messageservice.fcl":https://cdcvs.fnal.gov/redmine/projects/lardata/repository/revisions/develop/entry/Utilities/messageservice.fcl file to determine how to do so.  
144 1 Erica Snider
145 1 Erica Snider
Examples of how to include the usual use of the message service configurations are in the example files below.
146 1 Erica Snider
147 5 Gianluca Petrillo
To get a different level of output from just one module (say @DBSCAN@) one would do:
148 1 Erica Snider
149 1 Erica Snider
<pre>
150 1 Erica Snider
services:
151 1 Erica Snider
{
152 1 Erica Snider
  # Load the service that manages root files for histograms.
153 1 Erica Snider
  TFileService: { fileName: "reco_hist.root" }
154 1 Erica Snider
  Timing:       {}
155 1 Erica Snider
  RandomNumberGenerator: {} #ART native random number generator
156 1 Erica Snider
157 1 Erica Snider
  # configure the message service with the INFO for DBSCAN
158 1 Erica Snider
  # and WARNING level for everything else
159 1 Erica Snider
  message: { 
160 1 Erica Snider
    destinations: {  
161 1 Erica Snider
      infomsg: {
162 1 Erica Snider
        type: "cout"
163 1 Erica Snider
        threshold: "INFO"
164 1 Erica Snider
        append: true
165 1 Erica Snider
        category: {
166 1 Erica Snider
           DBSCAN: {
167 1 Erica Snider
             reportEvery: 1
168 1 Erica Snider
           }
169 1 Erica Snider
        }
170 1 Erica Snider
      }
171 1 Erica Snider
      warningmsg: {
172 1 Erica Snider
        type:      "cout"	
173 1 Erica Snider
        threshold: "WARNING"
174 1 Erica Snider
        append:    true        
175 1 Erica Snider
        categories: {
176 1 Erica Snider
          default: {
177 1 Erica Snider
            limit:       1000   
178 1 Erica Snider
            timespan:    60    
179 1 Erica Snider
          }
180 1 Erica Snider
        } # end categories
181 1 Erica Snider
      } # end warningmsg
182 1 Erica Snider
    } # end destinations
183 1 Erica Snider
  } # end standard_warning
184 1 Erica Snider
185 1 Erica Snider
  user:         @local::argoneut_services 	     
186 1 Erica Snider
}
187 1 Erica Snider
</pre>
188 1 Erica Snider
189 1 Erica Snider
190 5 Gianluca Petrillo
h3. Example job script: @prodgenie.fcl@
191 1 Erica Snider
192 5 Gianluca Petrillo
An example job script to produce Monte Carlo events is "larsim/EventGenerator/GENIE/prodgenie.fcl":https://cdcvs.fnal.gov/redmine/projects/larsim/repository/revisions/develop/entry/EventGenerator/GENIE/prodgenie.fcl .  The job defined by this script will generate neutrino interactions using GENIE, run them through Geant4, do the electron transport and then simulate the electronics.
193 1 Erica Snider
194 1 Erica Snider
Comments on the form of the file are included as ###### Commment ######
195 1 Erica Snider
196 1 Erica Snider
<pre>
197 1 Erica Snider
###### This is how to include configurations from other files ######
198 5 Gianluca Petrillo
#include "services.fcl"
199 5 Gianluca Petrillo
#include "genie.fcl"
200 5 Gianluca Petrillo
#include "largeantmodules.fcl"
201 5 Gianluca Petrillo
#include "detsimmodules.fcl"
202 1 Erica Snider
203 1 Erica Snider
###### give the process a name ######
204 1 Erica Snider
process_name: GenieGen
205 1 Erica Snider
206 5 Gianluca Petrillo
###### Please note the convention of defining detector specific configurations                ######
207 1 Erica Snider
###### Pick out the configurations from the #include files using the @local:: syntax          ######
208 1 Erica Snider
###### for services from LArSoft, in the user{} block - see definitions for configurations in ######
209 1 Erica Snider
###### job/geometry.fcl                                                                       ######
210 1 Erica Snider
###### job/services.fcl                                                                       ######
211 5 Gianluca Petrillo
###### job/simulationservices.fcl                                                             ######
212 5 Gianluca Petrillo
213 5 Gianluca Petrillo
services:
214 1 Erica Snider
{
215 5 Gianluca Petrillo
  # Load the service that manages root files for histograms.
216 1 Erica Snider
  TFileService: { fileName: "genie_hist.root" }
217 1 Erica Snider
  Timing:       {}
218 5 Gianluca Petrillo
  SimpleMemoryCheck:     { ignoreTotal: 1 } # default is one
219 5 Gianluca Petrillo
  RandomNumberGenerator: {} #ART native random number generator
220 5 Gianluca Petrillo
  user:         @local::argoneut_simulation_services
221 1 Erica Snider
}
222 1 Erica Snider
223 1 Erica Snider
###### source is where you get events from - can also be RootInput ######
224 1 Erica Snider
#Start each new event with an empty event.
225 1 Erica Snider
source:
226 1 Erica Snider
{
227 1 Erica Snider
  module_type: EmptyEvent
228 1 Erica Snider
  maxEvents:   10          # Number of events to create
229 1 Erica Snider
  firstRun:    1           # Run number to use for this file
230 1 Erica Snider
  firstEvent:  1           # number of first event in the file
231 5 Gianluca Petrillo
}
232 1 Erica Snider
233 1 Erica Snider
###### physics is the block that controls configuration of modules ######
234 5 Gianluca Petrillo
# Define and configure some modules to do work on each event.
235 1 Erica Snider
# First modules are defined; they are scheduled later.
236 5 Gianluca Petrillo
# Modules are grouped by type.
237 5 Gianluca Petrillo
physics:
238 1 Erica Snider
{
239 1 Erica Snider
240 1 Erica Snider
 ###### the module labels in the output file will be generator, largeant, and daq ######
241 1 Erica Snider
 ###### their configuration is taken from ArgoNeuT defaults                       ######
242 1 Erica Snider
 producers:
243 1 Erica Snider
 {
244 1 Erica Snider
   generator: @local::argoneut_genie_simple_neutrino
245 5 Gianluca Petrillo
   largeant:  @local::argoneut_largeant	       
246 1 Erica Snider
   daq:       @local::argoneut_simwire         
247 1 Erica Snider
   rns:       { module_type: "RandomNumberSaver" }
248 1 Erica Snider
 }
249 1 Erica Snider
250 1 Erica Snider
 #define the producer and filter modules for this path, order matters, 
251 1 Erica Snider
 #filters reject all following items.  see lines starting physics.producers below
252 1 Erica Snider
 simulate: [ rns, generator, largeant, daq, ] 
253 1 Erica Snider
 
254 1 Erica Snider
 #define the output stream, there could be more than one if using filters 
255 1 Erica Snider
 stream1:  [ out1 ]
256 1 Erica Snider
257 1 Erica Snider
 #trigger_paths is a keyword and contains the paths that modify the art::event, 
258 1 Erica Snider
 #ie filters and producers
259 1 Erica Snider
 trigger_paths: [simulate] 
260 1 Erica Snider
261 5 Gianluca Petrillo
 #end_paths is a keyword and contains the paths that do not modify the art::Event, 
262 1 Erica Snider
 #ie analyzers and output streams.  these all run simultaneously
263 5 Gianluca Petrillo
 end_paths:     [stream1]  
264 5 Gianluca Petrillo
}
265 1 Erica Snider
266 1 Erica Snider
#block to define where the output goes.  if you defined a filter in the physics
267 1 Erica Snider
#block and put it in the trigger_paths then you need to put a SelectEvents: {SelectEvents: [XXX]}
268 1 Erica Snider
#entry in the output stream you want those to go to, where XXX is the label of the filter module(s)
269 1 Erica Snider
outputs:
270 1 Erica Snider
{
271 1 Erica Snider
 out1:
272 1 Erica Snider
 {
273 1 Erica Snider
   module_type: RootOutput
274 1 Erica Snider
   fileName:    "genie_gen.root" #default file name, can override from command line with -o or --output
275 1 Erica Snider
 }
276 1 Erica Snider
}
277 1 Erica Snider
278 1 Erica Snider
</pre>
279 1 Erica Snider
280 1 Erica Snider
Notice that you have not specified which libraries to load anywhere. That is because the build system compiles the plugin shared library files (@.so@) against the ones they depend upon.  
281 1 Erica Snider
282 1 Erica Snider
283 1 Erica Snider
h3. Example job script: @standard_reco.fcl@
284 1 Erica Snider
285 6 Gianluca Petrillo
There is an example reconstruction job script available for people to use, "lardata/Utilities/standard_reco":https://cdcvs.fnal.gov/redmine/projects/lardata/repository/revisions/develop/entry/Utilities/standard_reco.fcl .
286 6 Gianluca Petrillo
This script takes the output of either raw data or MC that has produced simulated raw digits and performs a list of reconstruction tasks.
287 6 Gianluca Petrillo
The version from LArSoft @v02_03_01@ is copied here:
288 1 Erica Snider
289 1 Erica Snider
<pre>
290 6 Gianluca Petrillo
#include "services.fcl"
291 6 Gianluca Petrillo
#include "caldata.fcl"
292 6 Gianluca Petrillo
#include "hitfindermodules.fcl"
293 6 Gianluca Petrillo
#include "clustermodules.fcl"
294 6 Gianluca Petrillo
#include "trackfindermodules.fcl"
295 1 Erica Snider
296 1 Erica Snider
process_name: Reco
297 1 Erica Snider
298 1 Erica Snider
services:
299 1 Erica Snider
{
300 1 Erica Snider
  # Load the service that manages root files for histograms.
301 1 Erica Snider
  TFileService: { fileName: "reco_hist.root" }
302 1 Erica Snider
  Timing:       {}
303 1 Erica Snider
  RandomNumberGenerator: {} #ART native random number generator
304 1 Erica Snider
  message:      @local::standard_warning
305 6 Gianluca Petrillo
  user:         @local::argoneut_services 	     
306 1 Erica Snider
}
307 1 Erica Snider
308 6 Gianluca Petrillo
309 1 Erica Snider
#source is now a root file
310 1 Erica Snider
source:
311 1 Erica Snider
{
312 1 Erica Snider
  module_type: RootInput
313 1 Erica Snider
  maxEvents:  10        # Number of events to create
314 1 Erica Snider
}
315 1 Erica Snider
316 1 Erica Snider
# Define and configure some modules to do work on each event.
317 1 Erica Snider
# First modules are defined; they are scheduled later.
318 1 Erica Snider
# Modules are grouped by type.
319 1 Erica Snider
physics:
320 1 Erica Snider
{
321 1 Erica Snider
322 1 Erica Snider
 producers:
323 1 Erica Snider
 {
324 6 Gianluca Petrillo
  caldata:    @local::argoneut_calwire	    
325 6 Gianluca Petrillo
  ffthit:     @local::argoneut_hitfinder # note if job is for MC, use argoneut_mc_hitfinder
326 6 Gianluca Petrillo
  dbcluster:  @local::argoneut_dbcluster	    
327 1 Erica Snider
  hough:      @local::argoneut_houghlinefinder
328 6 Gianluca Petrillo
  linemerger: @local::argoneut_linemerger	    
329 6 Gianluca Petrillo
  track:      @local::argoneut_track	    
330 6 Gianluca Petrillo
  harris:     @local::argoneut_endpointmod    
331 1 Erica Snider
 }
332 1 Erica Snider
333 6 Gianluca Petrillo
 analyzers:
334 6 Gianluca Petrillo
 {
335 6 Gianluca Petrillo
  dbclusterana: @local::argoneut_dbclusterana
336 6 Gianluca Petrillo
 }
337 6 Gianluca Petrillo
338 1 Erica Snider
 #define the producer and filter modules for this path, order matters, 
339 1 Erica Snider
 #filters reject all following items.  see lines starting physics.producers below
340 6 Gianluca Petrillo
 reco: [ caldata, ffthit, dbcluster, hough, linemerger, track, harris ] 
341 6 Gianluca Petrillo
 ana:  [ dbclusterana ]
342 6 Gianluca Petrillo
343 1 Erica Snider
 #define the output stream, there could be more than one if using filters 
344 1 Erica Snider
 stream1:  [ out1 ]
345 1 Erica Snider
346 1 Erica Snider
 #trigger_paths is a keyword and contains the paths that modify the art::event, 
347 1 Erica Snider
 #ie filters and producers
348 1 Erica Snider
 trigger_paths: [reco] 
349 1 Erica Snider
350 1 Erica Snider
 #end_paths is a keyword and contains the paths that do not modify the art::Event, 
351 1 Erica Snider
 #ie analyzers and output streams.  these all run simultaneously
352 6 Gianluca Petrillo
 end_paths:     [ ana, stream1]  
353 1 Erica Snider
}
354 1 Erica Snider
355 1 Erica Snider
#block to define where the output goes.  if you defined a filter in the physics
356 1 Erica Snider
#block and put it in the trigger_paths then you need to put a SelectEvents: {SelectEvents: [XXX]}
357 1 Erica Snider
#entry in the output stream you want those to go to, where XXX is the label of the filter module(s)
358 1 Erica Snider
outputs:
359 1 Erica Snider
{
360 1 Erica Snider
 out1:
361 1 Erica Snider
 {
362 1 Erica Snider
   module_type: RootOutput
363 1 Erica Snider
   fileName:    "standard_reco.root" #default file name, can override from command line with -o or --output
364 1 Erica Snider
 }
365 1 Erica Snider
}
366 1 Erica Snider
367 1 Erica Snider
</pre>
368 1 Erica Snider
369 7 Gianluca Petrillo
370 1 Erica Snider
h3. How to override a default parameter
371 1 Erica Snider
372 1 Erica Snider
If you want to override a default parameter that has been included from a predefined parameter set, you must specify which parameter and its value as
373 1 Erica Snider
374 7 Gianluca Petrillo
<pre>
375 1 Erica Snider
mainBlock.subBlock.label.parameterName: newValue
376 7 Gianluca Petrillo
</pre>
377 1 Erica Snider
378 1 Erica Snider
where 
379 1 Erica Snider
380 7 Gianluca Petrillo
* @mainBlock@ can be services or physics 
381 7 Gianluca Petrillo
* @subBlock@ can be user, producers, filters, or analyzers
382 7 Gianluca Petrillo
* @label@ is the name of the desired service or module in a producers, filters, or analyzers block
383 7 Gianluca Petrillo
* @parameterName@ is the name of the desired parameter
384 7 Gianluca Petrillo
* @newValue@ is the desired new value; lists or entire blocks (that is, a comma-separated list of @key: value@ pairs in braces) can be specified
385 1 Erica Snider
386 7 Gianluca Petrillo
These lines must go after the @mainBlock@ and be outside of any other block.
387 1 Erica Snider
388 7 Gianluca Petrillo
For example, if one wanted to change the default value of the @fhitsModuleLabel@ parameter in the @DBcluster@ module in the previous section, one would put
389 1 Erica Snider
390 7 Gianluca Petrillo
<pre>
391 1 Erica Snider
physics.producers.cluster.fhitsModuleLabel: "differentHitModuleLabel"
392 7 Gianluca Petrillo
</pre>
393 1 Erica Snider
394 7 Gianluca Petrillo
Note that FHiCL allows for completely replacing a value but not for changing it (e.g., it's not possible to add an element to an existing list).
395 1 Erica Snider
396 7 Gianluca Petrillo
397 1 Erica Snider
h3. fhicl Emacs syntax highlighting
398 1 Erica Snider
399 7 Gianluca Petrillo
If you use Emacs as your editor, you can put the following into your @.emacs@ file in your home directory to cause it to display @.fcl@ files with syntax highlighting
400 1 Erica Snider
401 1 Erica Snider
<pre>
402 1 Erica Snider
(setq fclKeywords
403 1 Erica Snider
'(
404 1 Erica Snider
  ;; This, due to poor language design, conflicts with comments and fails
405 1 Erica Snider
  ("#include" . font-lock-keyword-face)
406 1 Erica Snider
  ("@local" . font-lock-keyword-face)
407 1 Erica Snider
  ;; All these names are magic, I think
408 1 Erica Snider
409 1 Erica Snider
410 1 Erica Snider
("process_name:\\|services:\\|source:\\|outputs:\\|physics\\|producers:\\|filters:\\|analyzers:" . font-lock-builtin-face)
411 1 Erica Snider
  ("true\\|false" . font-lock-builtin-face)
412 1 Erica Snider
  ;; Variable definitions are followed by colons
413 1 Erica Snider
414 1 Erica Snider
  ("[a-zA-Z0-9_]*:" . font-lock-variable-name-face)
415 1 Erica Snider
 )
416 1 Erica Snider
)
417 1 Erica Snider
418 1 Erica Snider
;; Python mode gets us comment handling and indentation at colons
419 1 Erica Snider
420 1 Erica Snider
(define-derived-mode fcl-mode python-mode
421 1 Erica Snider
 (setq mode-name "FHICL")
422 1 Erica Snider
 (setq font-lock-defaults '(fclKeywords))
423 1 Erica Snider
;;  (setq tab-width 2) ;; Doesn't seem to work
424 1 Erica Snider
425 1 Erica Snider
)
426 1 Erica Snider
427 1 Erica Snider
(add-to-list 'auto-mode-alist '("\\.fcl\\'" . fcl-mode))
428 1 Erica Snider
</pre>
429 1 Erica Snider
430 7 Gianluca Petrillo
Also, please use _only spaces_ for alignment: it's not the perfect solution, but it makes people using different editors see the same code alignment.
431 7 Gianluca Petrillo
432 7 Gianluca Petrillo
433 1 Erica Snider
h2. Executable and command line options
434 1 Erica Snider
435 7 Gianluca Petrillo
Currently there is one executable to run in LArSoft. The executable to run a typical reconstruction or analysis job is @lar@ which is placed in the user's path by the setup script. To see what options are available do
436 7 Gianluca Petrillo
<pre>
437 7 Gianluca Petrillo
$ lar -h
438 7 Gianluca Petrillo
</pre>
439 7 Gianluca Petrillo
The output for @art@ version @v1_09_03@ is:
440 7 Gianluca Petrillo
<pre>
441 7 Gianluca Petrillo
Usage: lar <-c <config-file>> <other-options> [<source-file>]+
442 1 Erica Snider
443 7 Gianluca Petrillo
Allowed options:
444 7 Gianluca Petrillo
  -c [ --config ] arg           Configuration file.
445 7 Gianluca Petrillo
  -h [ --help ]                 produce help message
446 7 Gianluca Petrillo
  --process-name arg            art process name.
447 1 Erica Snider
448 7 Gianluca Petrillo
  -s [ --source ] arg           Source data file (multiple OK).
449 7 Gianluca Petrillo
  -S [ --source-list ] arg      file containing a list of source files to read,
450 7 Gianluca Petrillo
                                one per line.
451 7 Gianluca Petrillo
  -e [ --estart ] arg           Event # of first event to process.
452 7 Gianluca Petrillo
  -n [ --nevts ] arg            Number of events to process.
453 7 Gianluca Petrillo
  --nskip arg                   Number of events to skip.
454 7 Gianluca Petrillo
  -T [ --TFileName ] arg        File name for TFileService.
455 7 Gianluca Petrillo
  -o [ --output ] arg           Event output stream file.
456 7 Gianluca Petrillo
  --trace                       Activate tracing.
457 7 Gianluca Petrillo
  --notrace                     Deactivate tracing.
458 7 Gianluca Petrillo
  --memcheck                    Activate monitoring of memory use.
459 7 Gianluca Petrillo
  --nomemcheck                  Deactivate monitoring of memory use.
460 7 Gianluca Petrillo
  --default-exceptions          some exceptions may be handled differently by 
461 7 Gianluca Petrillo
                                default (e.g. ProductNotFound).
462 7 Gianluca Petrillo
  --rethrow-default             all exceptions default to rethrow.
463 7 Gianluca Petrillo
  --rethrow-all                 all exceptions overridden to rethrow (cf 
464 7 Gianluca Petrillo
                                rethrow-default).
465 7 Gianluca Petrillo
  --sam-web-uri arg             URI for SAM web service.
466 7 Gianluca Petrillo
  --sam-process-id arg          SAM process ID.
467 7 Gianluca Petrillo
  --sam-application-family arg  SAM application family.
468 7 Gianluca Petrillo
  --sam-app-family arg          SAM application family.
469 7 Gianluca Petrillo
  --sam-application-version arg SAM application version.
470 7 Gianluca Petrillo
  --sam-app-version arg         SAM application version.
471 7 Gianluca Petrillo
  --sam-file-type arg           File type for SAM metadata.
472 7 Gianluca Petrillo
  --sam-data-tier arg           SAM data tier spec (<module-label>:<tier-spec>)
473 7 Gianluca Petrillo
                                .
474 7 Gianluca Petrillo
  --sam-stream-name arg         SAM stream name (<module-label>:<stream-name>).
475 1 Erica Snider
476 7 Gianluca Petrillo
Art has completed and will exit with status 1.
477 7 Gianluca Petrillo
</pre>
478 1 Erica Snider
479 7 Gianluca Petrillo
480 1 Erica Snider
h2. Running a Job
481 1 Erica Snider
482 1 Erica Snider
To run the job defined by the script above, do
483 7 Gianluca Petrillo
<pre>
484 7 Gianluca Petrillo
$ lar -c job/prodgenie.fcl
485 7 Gianluca Petrillo
</pre>
486 1 Erica Snider
487 7 Gianluca Petrillo
One can stop a job in two ways:
488 7 Gianluca Petrillo
# press <Ctrl>+<C> once: the job will complete at the end of the current module. If the job is running in the background type @kill -9 %jobID@ on the command line.
489 7 Gianluca Petrillo
# press <Ctrl>+<C> twice: the job should stop immediately and, depending on the shall settings, might produce a core dump.
490 1 Erica Snider
491 7 Gianluca Petrillo
If you want to have your job keep running even if you get disconnected from a remote session, depending on your shell you might need to start it with:
492 7 Gianluca Petrillo
<pre>
493 7 Gianluca Petrillo
$ nohup lar job/prodgenie.fcl >& pg.out
494 7 Gianluca Petrillo
</pre>
495 1 Erica Snider
496 1 Erica Snider
To stop such a job, then do
497 7 Gianluca Petrillo
<pre>
498 7 Gianluca Petrillo
$ pgrep lar # to find the job ID (use ps to disambiguate if more than one lar process are running)
499 7 Gianluca Petrillo
$ kill -INT jobID
500 7 Gianluca Petrillo
</pre>
501 1 Erica Snider
502 7 Gianluca Petrillo
One can print out the configuration of the job without starting the executable by:
503 7 Gianluca Petrillo
<pre>
504 7 Gianluca Petrillo
$ ART_DEBUG_CONFIG=1 lar -c prodgenie.fcl
505 7 Gianluca Petrillo
</pre>
506 1 Erica Snider
in @bash@, or
507 7 Gianluca Petrillo
<pre>
508 7 Gianluca Petrillo
> env ART_DEBUG_CONFIG=1 lar -c prodgenie.fcl
509 7 Gianluca Petrillo
</pre>
510 7 Gianluca Petrillo
in C-shell, which produces the output
511 7 Gianluca Petrillo
<pre>
512 7 Gianluca Petrillo
** ART_DEBUG_CONFIG is defined: config debug output follows **
513 1 Erica Snider
all_modules: [ "out1"
514 1 Erica Snider
             , "daq"
515 1 Erica Snider
             , "generator"
516 1 Erica Snider
             , "largeant"
517 1 Erica Snider
             , "rns"
518 1 Erica Snider
             ]
519 1 Erica Snider
outputs: { out1: { fileName: "genie_gen.root"
520 1 Erica Snider
                   module_label: "out1"
521 1 Erica Snider
                   module_type: "RootOutput"
522 1 Erica Snider
                 }
523 1 Erica Snider
         }
524 1 Erica Snider
physics: { end_paths: [ "stream1" ]
525 1 Erica Snider
           producers: { daq: { Col3DCorrection: 2.5
526 1 Erica Snider
                               ColFieldRespAmp: 3.54e-2
527 1 Erica Snider
                               CompressionType: "none"
528 1 Erica Snider
                               DriftEModuleLabel: "largeant"
529 1 Erica Snider
                               FieldBins: 75
530 1 Erica Snider
                               Ind3DCorrection: 1.5
531 1 Erica Snider
                               IndFieldRespAmp: 1.8e-2
532 1 Erica Snider
                               LowCutoff: 7.5
533 1 Erica Snider
                               NoiseFact: 1.32e-1
534 1 Erica Snider
                               NoiseWidth: 6.24e1
535 1 Erica Snider
                               ResponseFile: "shape-argo.root"
536 1 Erica Snider
                               ShapeTimeConst: [ 3000
537 1 Erica Snider
                                               , 900
538 1 Erica Snider
                                               ]
539 1 Erica Snider
                               module_label: "daq"
540 1 Erica Snider
                               module_type: "SimWireT962"
541 1 Erica Snider
                             }
542 7 Gianluca Petrillo
                        generator: { BeamCenter: [ -1400
543 7 Gianluca Petrillo
                                                 , -350
544 1 Erica Snider
                                                 , 0
545 1 Erica Snider
                                                 ]
546 7 Gianluca Petrillo
[...]
547 1 Erica Snider
</pre>
548 1 Erica Snider
549 1 Erica Snider
This functionality is particularly helpful when trying to debug what input parameters were passed to the job.
550 1 Erica Snider
551 7 Gianluca Petrillo
552 1 Erica Snider
h2. Why did my job fail?
553 1 Erica Snider
554 1 Erica Snider
If a job fails with a seg fault look at the warnings printed to the screen or any output log files.
555 1 Erica Snider
556 1 Erica Snider
If a bug should be reported to the artists@fnal.gov list, attach the complete output of the job to the email.
557 7 Gianluca Petrillo
558 1 Erica Snider
559 1 Erica Snider
h2. Submitting Jobs to the compute farms
560 1 Erica Snider
561 1 Erica Snider
Instructions are on [[Batch_job_submission|this page]].