Project

General

Profile

Running Jobs » History » Version 4

Gianluca Petrillo, 07/22/2014 07:17 PM

1 1 Erica Snider
{{toc}}
2 1 Erica Snider
3 1 Erica Snider
Under construction... NOT YET A SOURCE OF INFORMATION!!
4 1 Erica Snider
5 3 Gianluca Petrillo
6 1 Erica Snider
h1. Running Jobs
7 1 Erica Snider
8 1 Erica Snider
This page describes the job configuration script and how to run a job using one.
9 1 Erica Snider
10 3 Gianluca Petrillo
It is assumed that you have previously read the [[ Quick-start guide to using and developing LArSoft code ]] and the information on [[Using LArSoft on the GPVM nodes]].
11 1 Erica Snider
12 1 Erica Snider
13 1 Erica Snider
h2. The Job Configuration Script
14 1 Erica Snider
15 1 Erica Snider
Once a base and test release are set up, it is easy to run a job. The basic unit for running a job is the job-control script, written in the FHICL language. The FHICL language provides a simple mechanism for including parameter set configurations from different files such that many job configuration files can use the same configuration for a module or service.  
16 1 Erica Snider
17 1 Erica Snider
There is also a nice "FHICL quick start guide":https://cdcvs.fnal.gov/redmine/attachments/16021/quick_start.pdf available for more details. 
18 2 Lynn Garren
19 3 Gianluca Petrillo
20 1 Erica Snider
h3. Key Concepts in FHICL
21 1 Erica Snider
22 1 Erica Snider
There are a few key concepts to writing a FHICL job control script.  In order, they are
23 1 Erica Snider
24 3 Gianluca Petrillo
# Including previously defined configurations for services and modules from other files.  This is done using @#include@ statements. *Be sure you don't have any trailing space or tab characters* on the @#include@ line.
25 3 Gianluca Petrillo
# Services block, denoted by @services: { }@  This block will contain configurations for ART specific services such as the @TFileService@ and the @RandomNumberGenerator@.  It also contains a @user: {}@ sub-block where LArSoft specific services are configured.  
26 3 Gianluca Petrillo
# Source block, denoted by @source: { }@.  This block tells the job what kind of source to expect (@EmptyEvent@ in the case of Monte Carlo generation, @RootInput@ in the case of anything downstream of a Monte Carlo generator or reconstruction), the file name for the input source if appropriate, and how many events to process.  Both the file name and number of events to process can be specified on the command line.
27 3 Gianluca Petrillo
# Outputs block, denoted by @outputs: { }@  This block tells the job what kind of output to make, i.e. @RootOutput@, and what the name of the output file should be.  The output file name can be specified on the command line.  It is possible to define more than one output file if one wanted to run a job that produced different output files based on filter criteria - i.e. empty events are put in one file and events with neutrinos in them are put in another. Multiple output files can only be specified in the job configuration file, not from the command line.
28 3 Gianluca Petrillo
# Physics block, denoted by @physics: { }@  This block is where all producer, analyzer, and filter modules are configured.  The sequence of producer and filter modules to run is defined in a user-named path in this block.  The list of analyzers to run is defined in separate user-named path.  The block also defines two keyword parameters, @trigger_paths@ and @end_paths@.  @trigger_paths@ contains all producer and filter paths to run, and @end_paths@ contains the analyzer paths and output streams.
29 1 Erica Snider
30 3 Gianluca Petrillo
Comments may be included in FHICL configuration files using the "#" character.  The @#include@ is a keyword so that the parser knows not to ignore what comes after "#include".
31 1 Erica Snider
32 3 Gianluca Petrillo
33 1 Erica Snider
h3. FHICL Rules
34 1 Erica Snider
35 1 Erica Snider
There are a couple of rules to keep in mind about FHICL:
36 1 Erica Snider
* The value of the process_name parameter may not contain underscores as the process name is used in the ROOT file branch name.  Module labels may not contain underscores either, for the same reason.
37 1 Erica Snider
* Parameter set names may not contain numbers, periods, backslashes, stars, etc. They may contain underscores.
38 3 Gianluca Petrillo
* Put the values for all string parameters in double quotes, @"..."@
39 3 Gianluca Petrillo
* Specify input vectors using @[ , , ]@, i.e. if you want a vector of doubles do @MyVector: [1.0, 3e-9, -900.]@
40 3 Gianluca Petrillo
* You pick out configurations from the @PROLOG@ section(s), usually defined in the @#include@ files, using the @local::@ syntax.  The value after the "::" is the name of the configuration specified in the @PROLOG@ (see the next bullet)
41 3 Gianluca Petrillo
* You can override the value of an included configuration. For example, imagine there is a configuration specified in a included file called @mymoduleconfig@ and it contains the value @-5@ for the parameter named @myint@. One can load the configuration and then change the value of @myint@ by doing the following:
42 3 Gianluca Petrillo
*# inside the producers block:
43 1 Erica Snider
<pre>
44 3 Gianluca Petrillo
physics: {
45 3 Gianluca Petrillo
  producers: {
46 3 Gianluca Petrillo
    # ...
47 3 Gianluca Petrillo
    mymod: @local::mymoduleconfig
48 3 Gianluca Petrillo
  }
49 3 Gianluca Petrillo
}
50 3 Gianluca Petrillo
</pre>
51 3 Gianluca Petrillo
*# out of the physics block 
52 3 Gianluca Petrillo
<pre>
53 1 Erica Snider
physics.producer.mymod.myint: 1
54 1 Erica Snider
</pre>
55 3 Gianluca Petrillo
The last value for a parameter always wins. If the second line was repeated with the value @2@ instead of @1@, the job would run with @myint@ as @2@.
56 3 Gianluca Petrillo
Also note that in the example the original content of @mymoduleconfig@ is not changed when the content of @mymod@ is.
57 3 Gianluca Petrillo
58 1 Erica Snider
59 1 Erica Snider
h3. Configuring the [[Using_the_Framework#Message-Facility-and-MessageLogger|message service]]
60 1 Erica Snider
61 4 Gianluca Petrillo
Several standard configurations for the message service are in "lardata/Utilities/messageservice.fcl":https://cdcvs.fnal.gov/redmine/projects/lardata/repository/revisions/develop/entry/Utilities/messageservice.fcl.  There is one configuration for each level of message output - Debug, Info, Warning, and Error.  These configurations will be applied to each message level that is specified and those of higher priority.  For example, the Info configuration will print out Info, Warning and Error level messages while the Warning configuration only print outs Warning and Error level messages. The "standard" debug configurations will cause the messages to go to a specified output file, @debug.log@. The Error configuration redirect to standard error stream (like @std::cerr@), while the others print into the standard output (@std::cout@). All impose some limits on the repetition of some frequent messages.
62 4 Gianluca Petrillo
Note that these "standard" configurations need to be included in your FCL file to be used: they are standard, not default.
63 4 Gianluca Petrillo
If you want to define your own configuration, please take a look at the comments in "lardata/Utilities/messageservice.fcl":https://cdcvs.fnal.gov/redmine/projects/lardata/repository/revisions/develop/entry/Utilities/messageservice.fcl file to determine how to do so.  
64 1 Erica Snider
65 1 Erica Snider
Examples of how to include the usual use of the message service configurations are in the example files below.
66 1 Erica Snider
67 1 Erica Snider
To get a different level of output from just one module (say DBSCAN) one would do:
68 1 Erica Snider
69 1 Erica Snider
<pre>
70 1 Erica Snider
services:
71 1 Erica Snider
{
72 1 Erica Snider
  # Load the service that manages root files for histograms.
73 1 Erica Snider
  TFileService: { fileName: "reco_hist.root" }
74 1 Erica Snider
  Timing:       {}
75 1 Erica Snider
  RandomNumberGenerator: {} #ART native random number generator
76 1 Erica Snider
77 1 Erica Snider
  # configure the message service with the INFO for DBSCAN
78 1 Erica Snider
  # and WARNING level for everything else
79 1 Erica Snider
  message: { 
80 1 Erica Snider
    destinations: {  
81 1 Erica Snider
      infomsg: {
82 1 Erica Snider
        type: "cout"
83 1 Erica Snider
        threshold: "INFO"
84 1 Erica Snider
        append: true
85 1 Erica Snider
        category: {
86 1 Erica Snider
           DBSCAN: {
87 1 Erica Snider
             reportEvery: 1
88 1 Erica Snider
           }
89 1 Erica Snider
        }
90 1 Erica Snider
      }
91 1 Erica Snider
      warningmsg: {
92 1 Erica Snider
        type:      "cout"	
93 1 Erica Snider
        threshold: "WARNING"
94 1 Erica Snider
        append:    true        
95 1 Erica Snider
        categories: {
96 1 Erica Snider
          default: {
97 1 Erica Snider
            limit:       1000   
98 1 Erica Snider
            timespan:    60    
99 1 Erica Snider
          }
100 1 Erica Snider
        } # end categories
101 1 Erica Snider
      } # end warningmsg
102 1 Erica Snider
    } # end destinations
103 1 Erica Snider
  } # end standard_warning
104 1 Erica Snider
105 1 Erica Snider
  user:         @local::argoneut_services 	     
106 1 Erica Snider
}
107 1 Erica Snider
</pre>
108 1 Erica Snider
109 1 Erica Snider
110 1 Erica Snider
h3. Example job script: prodgenie.fcl
111 1 Erica Snider
112 1 Erica Snider
An example job script to produce Monte Carlo events is source:trunk/EventGenerator/prodgenie.fcl.  The job defined by this script will generate neutrino interactions using GENIE, run them through Geant4, do the electron transport and then simulate the electronics.
113 1 Erica Snider
114 1 Erica Snider
Comments on the form of the file are included as ###### Commment ######
115 1 Erica Snider
116 1 Erica Snider
<pre>
117 1 Erica Snider
###### This is how to include configurations from other files ######
118 1 Erica Snider
#include "job/services.fcl"
119 1 Erica Snider
#include "job/simulationservices.fcl"
120 1 Erica Snider
#include "job/genie.fcl"
121 1 Erica Snider
#include "job/largeantmodules.fcl"
122 1 Erica Snider
#include "job/detsimmodules.fcl"
123 1 Erica Snider
#include "job/driftmodules.fcl"
124 1 Erica Snider
125 1 Erica Snider
###### give the process a name ######
126 1 Erica Snider
process_name: GenieGen
127 1 Erica Snider
128 1 Erica Snider
###### Please note the convention of defining detector specific configurations                ######
129 1 Erica Snider
###### Pick out the configurations from the #include files using the @local:: syntanx         ######
130 1 Erica Snider
###### for services from LArSoft, in the user{} block - see definitions for configurations in ###### 
131 1 Erica Snider
###### job/geometry.fcl                                                                       ######
132 1 Erica Snider
###### job/services.fcl                                                                       ######
133 1 Erica Snider
###### job/simulationservices.fcl                                                             ######
134 1 Erica Snider
135 1 Erica Snider
services:
136 1 Erica Snider
{
137 1 Erica Snider
  # Load the service that manages root files for histograms.
138 1 Erica Snider
  TFileService: { fileName: "genie_hist.root" }
139 1 Erica Snider
  # Timing records the time spent in each module for each event
140 1 Erica Snider
  Timing:       {}
141 1 Erica Snider
  # message configures the message service
142 1 Erica Snider
  message:      @local::standard_warning
143 1 Erica Snider
  # the ART native random number generator
144 1 Erica Snider
  RandomNumberGenerator: {} 
145 1 Erica Snider
  # LArSoft specific services
146 1 Erica Snider
  user:         @local::argoneut_services
147 1 Erica Snider
}
148 1 Erica Snider
149 1 Erica Snider
150 1 Erica Snider
###### source is where you get events from - can also be RootInput ######
151 1 Erica Snider
#Start each new event from scratch
152 1 Erica Snider
source:
153 1 Erica Snider
{
154 1 Erica Snider
  module_type: EmptyEvent
155 1 Erica Snider
  maxEvents:  10        # Number of events to create
156 1 Erica Snider
}
157 1 Erica Snider
158 1 Erica Snider
###### physics is the block that controls configuration of modules ######
159 1 Erica Snider
# Define and configure some modules to do work on each event.
160 1 Erica Snider
# First modules are defined; they are scheduled later.
161 1 Erica Snider
# Modules are grouped by type.
162 1 Erica Snider
physics:
163 1 Erica Snider
{
164 1 Erica Snider
165 1 Erica Snider
 ###### the module labels in the output file will be generator, largeant, and daq ######
166 1 Erica Snider
 producers:
167 1 Erica Snider
 {
168 1 Erica Snider
  generator: @local::argoneut_genie
169 1 Erica Snider
  largeant:  @local::argoneut_largeant
170 1 Erica Snider
  daq:       @local::argoneut_simwire
171 1 Erica Snider
 }
172 1 Erica Snider
173 1 Erica Snider
 analyzers:
174 1 Erica Snider
 {
175 1 Erica Snider
  detsimana: @local::argoneut_simwireana
176 1 Erica Snider
 }
177 1 Erica Snider
178 1 Erica Snider
 #define the producer and filter modules for this path, order matters, 
179 1 Erica Snider
 #filters reject all following items.  see lines starting physics.producers below
180 1 Erica Snider
 simulate: [ generator, largeant, daq ] 
181 1 Erica Snider
182 1 Erica Snider
 #define a path for any analyzers to use
183 1 Erica Snider
 anapath:  [ detsimana ]
184 1 Erica Snider
185 1 Erica Snider
 #define the output stream, there could be more than one if using filters 
186 1 Erica Snider
 stream1:  [ out1 ]
187 1 Erica Snider
188 1 Erica Snider
 #trigger_paths is a keyword and contains the paths that modify the art::event, 
189 1 Erica Snider
 #ie filters and producers
190 1 Erica Snider
 trigger_paths: [simulate] 
191 1 Erica Snider
192 1 Erica Snider
 #end_paths is a keyword and contains the paths that do not modify the art::Event, 
193 1 Erica Snider
 #ie analyzers and output streams.  these all run simultaneously
194 1 Erica Snider
 end_paths:     [anapath, stream1]  
195 1 Erica Snider
}
196 1 Erica Snider
197 1 Erica Snider
#block to define where the output goes.  if you defined a filter in the physics
198 1 Erica Snider
#block and put it in the trigger_paths then you need to put a SelectEvents: {SelectEvents: [XXX]}
199 1 Erica Snider
#entry in the output stream you want those to go to, where XXX is the label of the filter module(s)
200 1 Erica Snider
outputs:
201 1 Erica Snider
{
202 1 Erica Snider
 out1:
203 1 Erica Snider
 {
204 1 Erica Snider
   module_type: RootOutput
205 1 Erica Snider
   fileName:    "genie_gen.root" #default file name, can override from command line with -o or --output
206 1 Erica Snider
 }
207 1 Erica Snider
}
208 1 Erica Snider
209 1 Erica Snider
</pre>
210 1 Erica Snider
211 1 Erica Snider
Notice that you have not specified which libraries to load anywhere. That is because the SRT build compiles the plugin .so files against the .so's they depend upon.  
212 1 Erica Snider
213 1 Erica Snider
h3. Example job script: standard_reco.fcl
214 1 Erica Snider
215 1 Erica Snider
There is an example reconstruction job script available for people to use, source:trunk/Utilities/standard_reco.fcl.  This script takes the output of either raw data or MC that has produced simulated raw digits and performs a list of reconstruction tasks.
216 1 Erica Snider
217 1 Erica Snider
<pre>
218 1 Erica Snider
#include "job/services.fcl"
219 1 Erica Snider
#include "job/caldata.fcl"
220 1 Erica Snider
#include "job/fftfinder.fcl"
221 1 Erica Snider
#include "job/clustermodules.fcl"
222 1 Erica Snider
#include "job/trackfindermodules.fcl"
223 1 Erica Snider
#include "job/vertexfindermodules.fcl"
224 1 Erica Snider
225 1 Erica Snider
process_name: Reco
226 1 Erica Snider
227 1 Erica Snider
services:
228 1 Erica Snider
{
229 1 Erica Snider
  # Load the service that manages root files for histograms.
230 1 Erica Snider
  TFileService: { fileName: "reco_hist.root" }
231 1 Erica Snider
  scheduler:    { wantTracer: true wantSummary: true }
232 1 Erica Snider
  message:      {}
233 1 Erica Snider
  Timing:       {}
234 1 Erica Snider
  RandomNumberGenerator: {} #ART native random number generator
235 1 Erica Snider
  message:      @local::standard_warning
236 1 Erica Snider
  user:         @local::argoneut_services  
237 1 Erica Snider
}
238 1 Erica Snider
239 1 Erica Snider
#source is now a root file
240 1 Erica Snider
source:
241 1 Erica Snider
{
242 1 Erica Snider
  module_type: RootInput
243 1 Erica Snider
  maxEvents:  10        # Number of events to create
244 1 Erica Snider
}
245 1 Erica Snider
246 1 Erica Snider
# Define and configure some modules to do work on each event.
247 1 Erica Snider
# First modules are defined; they are scheduled later.
248 1 Erica Snider
# Modules are grouped by type.
249 1 Erica Snider
physics:
250 1 Erica Snider
{
251 1 Erica Snider
252 1 Erica Snider
 producers:
253 1 Erica Snider
 {
254 1 Erica Snider
  caldata:    @local::argoneut_calwire	
255 1 Erica Snider
  ffthit:     @local::argoneut_hitfinder	
256 1 Erica Snider
  cluster:    @local::argoneut_dbcluster	
257 1 Erica Snider
  hough:      @local::argoneut_houghlinefinder
258 1 Erica Snider
  linemerger: @local::argoneut_linemerger
259 1 Erica Snider
  track:      @local::argoneut_track	
260 1 Erica Snider
  harris:     @local::argoneut_harris	
261 1 Erica Snider
 }
262 1 Erica Snider
263 1 Erica Snider
 #define the producer and filter modules for this path, order matters, 
264 1 Erica Snider
 #filters reject all following items.  see lines starting physics.producers below
265 1 Erica Snider
 reco: [ caldata, ffthit, cluster, hough, linemerger, track, harris ] 
266 1 Erica Snider
 
267 1 Erica Snider
 #define the output stream, there could be more than one if using filters 
268 1 Erica Snider
 stream1:  [ out1 ]
269 1 Erica Snider
270 1 Erica Snider
 #trigger_paths is a keyword and contains the paths that modify the art::event, 
271 1 Erica Snider
 #ie filters and producers
272 1 Erica Snider
 trigger_paths: [reco] 
273 1 Erica Snider
274 1 Erica Snider
 #end_paths is a keyword and contains the paths that do not modify the art::Event, 
275 1 Erica Snider
 #ie analyzers and output streams.  these all run simultaneously
276 1 Erica Snider
 end_paths:     [stream1]  
277 1 Erica Snider
}
278 1 Erica Snider
279 1 Erica Snider
#block to define where the output goes.  if you defined a filter in the physics
280 1 Erica Snider
#block and put it in the trigger_paths then you need to put a SelectEvents: {SelectEvents: [XXX]}
281 1 Erica Snider
#entry in the output stream you want those to go to, where XXX is the label of the filter module(s)
282 1 Erica Snider
outputs:
283 1 Erica Snider
{
284 1 Erica Snider
 out1:
285 1 Erica Snider
 {
286 1 Erica Snider
   module_type: RootOutput
287 1 Erica Snider
   fileName:    "standard_reco.root" #default file name, can override from command line with -o or --output
288 1 Erica Snider
 }
289 1 Erica Snider
}
290 1 Erica Snider
291 1 Erica Snider
</pre>
292 1 Erica Snider
293 1 Erica Snider
h3. How to override a default parameter
294 1 Erica Snider
295 1 Erica Snider
If you want to override a default parameter that has been included from a predefined parameter set, you must specify which parameter and its value as
296 1 Erica Snider
297 1 Erica Snider
<pre><code class="c">
298 1 Erica Snider
mainBlock.subBlock.label.parameterName: newValue
299 1 Erica Snider
</code></pre>
300 1 Erica Snider
301 1 Erica Snider
where 
302 1 Erica Snider
303 1 Erica Snider
* mainBlock can be services or physics 
304 1 Erica Snider
* subBlock can be user, producers, filters, or analyzers
305 1 Erica Snider
* label is the name of the desired service or module in a producers, filters, or analyzers block
306 1 Erica Snider
* parameterName is the name of the desired parameter
307 1 Erica Snider
* newValue is the desired new value
308 1 Erica Snider
309 1 Erica Snider
These lines must go after the mainBlock and be outside of any other mainBlocks.
310 1 Erica Snider
311 1 Erica Snider
For example, if one wanted to change the default value of the fhitsModuleLabel parameter in the DBcluster module in the previous section, one would put
312 1 Erica Snider
313 1 Erica Snider
<pre><code class="c">
314 1 Erica Snider
physics.producers.cluster.fhitsModuleLabel: "differentHitModuleLabel"
315 1 Erica Snider
</code></pre>
316 1 Erica Snider
317 1 Erica Snider
318 1 Erica Snider
h3. Example configuration file: geometry.fcl
319 1 Erica Snider
320 1 Erica Snider
An example of a file with predefined configurations for a service is in the source:trunk/Geometry/geometry.fcl file:
321 1 Erica Snider
322 1 Erica Snider
<pre>
323 1 Erica Snider
###### All files that are parameter set definitions must contain BEGIN_PROLOG as their first line ######
324 1 Erica Snider
###### This tag tells the FHICL parser that parameter set definitions are coming                  ######
325 1 Erica Snider
BEGIN_PROLOG
326 1 Erica Snider
327 1 Erica Snider
###### The argoneut geometry definition ######
328 1 Erica Snider
argoneut_geo:
329 1 Erica Snider
{
330 1 Erica Snider
 SurfaceY: 130.0e2                      #in cm, vertical distance to the surface
331 1 Erica Snider
 Name:     "argoneut"
332 1 Erica Snider
 GDML:     "Geometry/gdml/argoneut.gdml"
333 1 Erica Snider
 ROOT:     "Geometry/gdml/argoneut.root"
334 1 Erica Snider
}
335 1 Erica Snider
336 1 Erica Snider
###### The microboone geometry definition ######
337 1 Erica Snider
microboone_geo:
338 1 Erica Snider
{
339 1 Erica Snider
 SurfaceY: 2.0e2                          #in cm, vertical distance to the surface
340 1 Erica Snider
 Name:     "microboone"
341 1 Erica Snider
 GDML:     "Geometry/gdml/microboone.gdml"
342 1 Erica Snider
 ROOT:     "Geometry/gdml/microboone.root"
343 1 Erica Snider
}
344 1 Erica Snider
345 1 Erica Snider
###### The two lbne geometry definitions ######
346 1 Erica Snider
lbne10kt_geo:
347 1 Erica Snider
{
348 1 Erica Snider
 SurfaceY: 0.0e2                         #in cm, vertical distance to the surface
349 1 Erica Snider
 Name:     "lbne10kT"
350 1 Erica Snider
 GDML:     "Geometry/gdml/lbne10kT.gdml"
351 1 Erica Snider
 ROOT:     "Geometry/gdml/lbne10kT.root"
352 1 Erica Snider
 DisableWiresInG4: true
353 1 Erica Snider
}
354 1 Erica Snider
 
355 1 Erica Snider
lbne35t_geo:
356 1 Erica Snider
{
357 1 Erica Snider
 SurfaceY: 0.0e2                         #in cm, vertical distance to the surface
358 1 Erica Snider
 Name:     "lbne35t"
359 1 Erica Snider
 GDML:     "Geometry/gdml/lbne35t.gdml"
360 1 Erica Snider
 ROOT:     "Geometry/gdml/lbne35t.root"
361 1 Erica Snider
 DisableWiresInG4: true
362 1 Erica Snider
}
363 1 Erica Snider
364 1 Erica Snider
365 1 Erica Snider
###### All files that are parameter set definitions must contain END_PROLOG as their last line ######
366 1 Erica Snider
###### This tag tells the FHICL parser that parameter set definitions are ended                ######
367 1 Erica Snider
END_PROLOG
368 1 Erica Snider
</pre>
369 1 Erica Snider
370 1 Erica Snider
h3. fhicl Emacs syntax highlighting
371 1 Erica Snider
372 1 Erica Snider
If you use Emacs as your editor, you can put the following into your .emacs file in your home directory to cause it to display .fcl files with syntax highlighting
373 1 Erica Snider
374 1 Erica Snider
<pre>
375 1 Erica Snider
(setq fclKeywords
376 1 Erica Snider
'(
377 1 Erica Snider
  ;; This, due to poor language design, conflicts with comments and fails
378 1 Erica Snider
  ("#include" . font-lock-keyword-face)
379 1 Erica Snider
  ("@local" . font-lock-keyword-face)
380 1 Erica Snider
  ;; All these names are magic, I think
381 1 Erica Snider
382 1 Erica Snider
383 1 Erica Snider
("process_name:\\|services:\\|source:\\|outputs:\\|physics\\|producers:\\|filters:\\|analyzers:" . font-lock-builtin-face)
384 1 Erica Snider
  ("true\\|false" . font-lock-builtin-face)
385 1 Erica Snider
  ;; Variable definitions are followed by colons
386 1 Erica Snider
387 1 Erica Snider
  ("[a-zA-Z0-9_]*:" . font-lock-variable-name-face)
388 1 Erica Snider
 )
389 1 Erica Snider
)
390 1 Erica Snider
391 1 Erica Snider
;; Python mode gets us comment handling and indentation at colons
392 1 Erica Snider
393 1 Erica Snider
(define-derived-mode fcl-mode python-mode
394 1 Erica Snider
 (setq mode-name "FHICL")
395 1 Erica Snider
 (setq font-lock-defaults '(fclKeywords))
396 1 Erica Snider
;;  (setq tab-width 2) ;; Doesn't seem to work
397 1 Erica Snider
398 1 Erica Snider
)
399 1 Erica Snider
400 1 Erica Snider
(add-to-list 'auto-mode-alist '("\\.fcl\\'" . fcl-mode))
401 1 Erica Snider
</pre>
402 1 Erica Snider
403 1 Erica Snider
h2. Executable and command line options
404 1 Erica Snider
405 1 Erica Snider
Currently there is one executable to run in LArSoft. The executable to run a typical reconstruction or analysis job is lar which is placed in the user's path by the setup script. To see what options are available do
406 1 Erica Snider
407 1 Erica Snider
@$lar -h@
408 1 Erica Snider
409 1 Erica Snider
The output is 
410 1 Erica Snider
411 1 Erica Snider
@lar <options> [config-file]:
412 1 Erica Snider
  -T [ --TFileName ] arg File name for TFileService.
413 1 Erica Snider
  -c [ --config ]    arg Configuration file.
414 1 Erica Snider
  -e [ --estart ]    arg Event # of first event to process.
415 1 Erica Snider
  -h [ --help ]          produce help message
416 1 Erica Snider
  -n [ --nevts ]     arg Number of events to process.
417 1 Erica Snider
  --nskip            arg Number of events to skip.
418 1 Erica Snider
  -o [ --output ]    arg Event output stream file.
419 1 Erica Snider
  -s [ --source ]    arg Source data file (multiple OK).
420 1 Erica Snider
  --trace                Activate tracing.
421 1 Erica Snider
  --notrace              Dectivate tracing.@
422 1 Erica Snider
423 1 Erica Snider
h2. Running a Job
424 1 Erica Snider
425 1 Erica Snider
To run the job defined by the script above, do
426 1 Erica Snider
427 1 Erica Snider
@$ lar -c job/prodgenie.fcl@
428 1 Erica Snider
429 1 Erica Snider
One can stop a job in two ways, 
430 1 Erica Snider
431 1 Erica Snider
# type ctrl-c once the job will complete at the end of the current module. If the job is running in the background type @kill -9 %jobID@ on the command line.
432 1 Erica Snider
# If you type ctrl-c, ctrl-c the job will stop immediately and produce a core dump.
433 1 Erica Snider
434 1 Erica Snider
If you want to have your job keep running even if you get disconnected from a remote session you can do
435 1 Erica Snider
436 1 Erica Snider
@$ nohup lar job/prodgenie.fcl >& pg.out @
437 1 Erica Snider
438 1 Erica Snider
To stop such a job, then do
439 1 Erica Snider
@$ps aux@ to find the job ID
440 1 Erica Snider
@kill -INT jobID@
441 1 Erica Snider
442 1 Erica Snider
One can print out the configuration of the job without starting the executable by doing 
443 1 Erica Snider
444 1 Erica Snider
@$ ART_DEBUG_CONFIG=1 lar -c job/prodgenie.fcl@
445 1 Erica Snider
446 1 Erica Snider
in @bash@, or
447 1 Erica Snider
448 1 Erica Snider
@> env ART_DEBUG_CONFIG=1 lar -c job/prodgenie.fcl@
449 1 Erica Snider
450 1 Erica Snider
in cshell which produces the output
451 1 Erica Snider
452 1 Erica Snider
<pre>** ART_DEBUG_CONFIG is defined: config debug output follows **
453 1 Erica Snider
all_modules: [ "out1"
454 1 Erica Snider
             , "daq"
455 1 Erica Snider
             , "generator"
456 1 Erica Snider
             , "largeant"
457 1 Erica Snider
             , "rns"
458 1 Erica Snider
             ]
459 1 Erica Snider
outputs: { out1: { fileName: "genie_gen.root"
460 1 Erica Snider
                   module_label: "out1"
461 1 Erica Snider
                   module_type: "RootOutput"
462 1 Erica Snider
                 }
463 1 Erica Snider
         }
464 1 Erica Snider
physics: { end_paths: [ "stream1" ]
465 1 Erica Snider
           producers: { daq: { Col3DCorrection: 2.5
466 1 Erica Snider
                               ColFieldRespAmp: 3.54e-2
467 1 Erica Snider
                               CompressionType: "none"
468 1 Erica Snider
                               DriftEModuleLabel: "largeant"
469 1 Erica Snider
                               FieldBins: 75
470 1 Erica Snider
                               Ind3DCorrection: 1.5
471 1 Erica Snider
                               IndFieldRespAmp: 1.8e-2
472 1 Erica Snider
                               LowCutoff: 7.5
473 1 Erica Snider
                               NoiseFact: 1.32e-1
474 1 Erica Snider
                               NoiseWidth: 6.24e1
475 1 Erica Snider
                               ResponseFile: "shape-argo.root"
476 1 Erica Snider
                               ShapeTimeConst: [ 3000
477 1 Erica Snider
                                               , 900
478 1 Erica Snider
                                               ]
479 1 Erica Snider
                               module_label: "daq"
480 1 Erica Snider
                               module_type: "SimWireT962"
481 1 Erica Snider
                             }
482 1 Erica Snider
                        generator: { BeamCenter: [ 2.5e-1
483 1 Erica Snider
                                                 , 0
484 1 Erica Snider
                                                 , 0
485 1 Erica Snider
                                                 ]
486 1 Erica Snider
                                     BeamDirection: [ 0
487 1 Erica Snider
                                                    , 0
488 1 Erica Snider
                                                    , 1
489 1 Erica Snider
                                                    ]
490 1 Erica Snider
                                     BeamName: "numi"
491 1 Erica Snider
                                     BeamRadius: 3
492 1 Erica Snider
                                     DebugFlags: 0
493 1 Erica Snider
                                     DetectorLocation: "MINOS-NearDet"
494 1 Erica Snider
                                     Environment: [ "GSPLOAD"
495 1 Erica Snider
                                                  , "gxspl-NUMIsmall-R2.6.0.xml"
496 1 Erica Snider
                                                  , "GPRODMODE"
497 1 Erica Snider
                                                  , "YES"
498 1 Erica Snider
                                                  , "GEVGL"
499 1 Erica Snider
                                                  , "Default"
500 1 Erica Snider
                                                  ]
501 1 Erica Snider
                                     EventsPerSpill: 0
502 1 Erica Snider
                                     FiducialCut: "none"
503 1 Erica Snider
                                     FluxFiles: [ "argoneut/gsimple_ArgoNeuT_le010z185i_run3_38l0-9r_00001.root" ]
504 1 Erica Snider
                                     FluxType: "simple_flux"
505 1 Erica Snider
                                     GenFlavors: [ 12
506 1 Erica Snider
                                                 , 14
507 1 Erica Snider
                                                 , -12
508 1 Erica Snider
                                                 , -14
509 1 Erica Snider
                                                 ]
510 1 Erica Snider
                                     GlobalTimeOffset: 10000
511 1 Erica Snider
                                     MixerBaseline: 0
512 1 Erica Snider
                                     MixerConfig: "none"
513 1 Erica Snider
                                     MonoEnergy: 2
514 1 Erica Snider
                                     POTPerSpill: 5e13
515 1 Erica Snider
                                     PassEmptySpills: false
516 1 Erica Snider
                                     RandomTimeOffset: 10000
517 1 Erica Snider
                                     SurroundingMass: 0
518 1 Erica Snider
                                     TopVolume: "volTPCActive"
519 1 Erica Snider
                                     module_label: "generator"
520 1 Erica Snider
                                     module_type: "GENIEGen"
521 1 Erica Snider
                                   }
522 1 Erica Snider
                        largeant: { DebugVoxelAccumulation: 0
523 1 Erica Snider
                                    DisableWireplanes: false
524 1 Erica Snider
                                    DumpLArVoxelList: false
525 1 Erica Snider
                                    DumpParticleList: false
526 1 Erica Snider
                                    GeantCommandFile: "LArG4/LArG4.mac"
527 1 Erica Snider
                                    SmartStacking: 0
528 1 Erica Snider
                                    VisualizeEvents: false
529 1 Erica Snider
                                    module_label: "largeant"
530 1 Erica Snider
                                    module_type: "LArG4"
531 1 Erica Snider
                                  }
532 1 Erica Snider
                        rns: { module_label: "rns"
533 1 Erica Snider
                               module_type: "RandomNumberSaver"
534 1 Erica Snider
                             }
535 1 Erica Snider
                      }
536 1 Erica Snider
           simulate: [ "generator"
537 1 Erica Snider
                     , "largeant"
538 1 Erica Snider
                     , "daq"
539 1 Erica Snider
                     ]
540 1 Erica Snider
           stream1: [ "out1" ]
541 1 Erica Snider
           trigger_paths: [ "simulate" ]
542 1 Erica Snider
         }
543 1 Erica Snider
process_name: "GenieGen"
544 1 Erica Snider
services: { RandomNumberGenerator: {}
545 1 Erica Snider
            SimpleMemoryCheck: { ignoreTotal: 1
546 1 Erica Snider
                               }
547 1 Erica Snider
            TFileService: { fileName: "genie_hist.root"
548 1 Erica Snider
                          }
549 1 Erica Snider
            Timing: {}
550 1 Erica Snider
            message: { destinations: { STDOUT: { categories: { ArtReport: { limit: 100
551 1 Erica Snider
                                                                          }
552 1 Erica Snider
                                                               default: { limit: -1
553 1 Erica Snider
                                                                        }
554 1 Erica Snider
                                                             }
555 1 Erica Snider
                                                 threshold: "INFO"
556 1 Erica Snider
                                                 type: "cout"
557 1 Erica Snider
                                               }
558 1 Erica Snider
                                     }
559 1 Erica Snider
                     }
560 1 Erica Snider
            user: { BackTracker: { G4ModuleLabel: "largeant"
561 1 Erica Snider
                                 }
562 1 Erica Snider
                    CatalogInterface: { service_provider: "TrivialFileDelivery"
563 1 Erica Snider
                                      }
564 1 Erica Snider
                    DatabaseUtil: { DBHostName: "fnalpgsdev.fnal.gov"
565 1 Erica Snider
                                    DBName: "argoneut_dev"
566 1 Erica Snider
                                    DBUser: "argoneut_reader"
567 1 Erica Snider
                                    PassFileName: ".apswd"
568 1 Erica Snider
                                    Port: 5457
569 1 Erica Snider
                                    ShouldConnect: true
570 1 Erica Snider
                                    ToughErrorTreatment: false
571 1 Erica Snider
                                  }
572 1 Erica Snider
                    DetectorProperties: { ElectronsToADC: 1.208041e-3
573 1 Erica Snider
                                          NumberTimeSamples: 2048
574 1 Erica Snider
                                          ReadOutWindowSize: 2048
575 1 Erica Snider
                                          SamplingRate: 198
576 1 Erica Snider
                                          TimeOffsetU: -5.193
577 1 Erica Snider
                                          TimeOffsetV: 5.85e-1
578 1 Erica Snider
                                          TimeOffsetW: 0
579 1 Erica Snider
                                          TriggerOffset: 60
580 1 Erica Snider
                                        }
581 1 Erica Snider
                    FileTransfer: { service_provider: "TrivialFileTransfer"
582 1 Erica Snider
                                  }
583 1 Erica Snider
                    Geometry: { GDML: "Geometry/gdml/argoneut.gdml"
584 1 Erica Snider
                                Name: "argoneut"
585 1 Erica Snider
                                ROOT: "Geometry/gdml/argoneut.root"
586 1 Erica Snider
                                SurfaceY: 13000
587 1 Erica Snider
                              }
588 1 Erica Snider
                    LArFFT: { FFTOption: "P"
589 1 Erica Snider
                              FitBins: 20
590 1 Erica Snider
                            }
591 1 Erica Snider
                    LArG4Parameters: { CosmogenicK0Bias: 0
592 1 Erica Snider
                                       CosmogenicXSMNBiasFactor: 1
593 1 Erica Snider
                                       CosmogenicXSMNBiasOn: 0
594 1 Erica Snider
                                       DisableWireplanes: false
595 1 Erica Snider
                                       ElectronClusterSize: 600
596 1 Erica Snider
                                       EnabledPhysics: [ "Em"
597 1 Erica Snider
                                                       , "SynchrotronAndGN"
598 1 Erica Snider
                                                       , "Ion"
599 1 Erica Snider
                                                       , "Hadron"
600 1 Erica Snider
                                                       , "Decay"
601 1 Erica Snider
                                                       , "HadronElastic"
602 1 Erica Snider
                                                       , "Stopping"
603 1 Erica Snider
                                                       , "NeutronTrackingCut"
604 1 Erica Snider
                                                       ]
605 1 Erica Snider
                                       KeepEMShowerDaughters: false
606 1 Erica Snider
                                       LongitudinalDiffusion: 6.2e-9
607 1 Erica Snider
                                       OpticalSimVerbosity: 0
608 1 Erica Snider
                                       ParticleKineticEnergyCut: 1e-5
609 1 Erica Snider
                                       StoreTrajectories: true
610 1 Erica Snider
                                       TransverseDiffusion: 1.63e-8
611 1 Erica Snider
                                       UseCustomPhysics: false
612 1 Erica Snider
                                       VisualizationEnergyCut: 1e-2
613 1 Erica Snider
                                       VisualizeNeutrals: false
614 1 Erica Snider
                                     }
615 1 Erica Snider
                    LArProperties: { AbsLengthEnergies: [ 9.5
616 1 Erica Snider
                                                        , 9.7
617 1 Erica Snider
                                                        , 9.9
618 1 Erica Snider
                                                        ]
619 1 Erica Snider
                                     AbsLengthSpectrum: [ 2000
620 1 Erica Snider
                                                        , 2000
621 1 Erica Snider
                                                        , 2000
622 1 Erica Snider
                                                        ]
623 1 Erica Snider
                                     AtomicMass: 3.9948e1
624 1 Erica Snider
                                     AtomicNumber: 18
625 1 Erica Snider
                                     Efield: [ 4.81e-1
626 1 Erica Snider
                                             , 7e-1
627 1 Erica Snider
                                             , 8.9e-1
628 1 Erica Snider
                                             ]
629 1 Erica Snider
                                     Electronlifetime: 750
630 1 Erica Snider
                                     ExcitationEnergy: 188
631 1 Erica Snider
                                     FastScintEnergies: [ 9.5
632 1 Erica Snider
                                                        , 9.7
633 1 Erica Snider
                                                        , 9.9
634 1 Erica Snider
                                                        ]
635 1 Erica Snider
                                     FastScintSpectrum: [ 5e-1
636 1 Erica Snider
                                                        , 1
637 1 Erica Snider
                                                        , 5e-1
638 1 Erica Snider
                                                        ]
639 1 Erica Snider
                                     RIndexEnergies: [ 9.5
640 1 Erica Snider
                                                     , 9.7
641 1 Erica Snider
                                                     , 9.9
642 1 Erica Snider
                                                     ]
643 1 Erica Snider
                                     RIndexSpectrum: [ 1.38
644 1 Erica Snider
                                                     , 1.38
645 1 Erica Snider
                                                     , 1.38
646 1 Erica Snider
                                                     ]
647 1 Erica Snider
                                     RadiationLength: 1.955e1
648 1 Erica Snider
                                     RayleighEnergies: [ 9.5
649 1 Erica Snider
                                                       , 9.7
650 1 Erica Snider
                                                       , 9.9
651 1 Erica Snider
                                                       ]
652 1 Erica Snider
                                     RayleighSpectrum: [ 90
653 1 Erica Snider
                                                       , 90
654 1 Erica Snider
                                                       , 90
655 1 Erica Snider
                                                       ]
656 1 Erica Snider
                                     ReflectiveSurfaceDiffuseFractions: [ [ 5e-1
657 1 Erica Snider
                                                                          , 5e-1
658 1 Erica Snider
                                                                          , 5e-1
659 1 Erica Snider
                                                                          ] ]
660 1 Erica Snider
                                     ReflectiveSurfaceEnergies: [ 9.5
661 1 Erica Snider
                                                                , 9.7
662 1 Erica Snider
                                                                , 9.9
663 1 Erica Snider
                                                                ]
664 1 Erica Snider
                                     ReflectiveSurfaceNames: [ "STEEL_STAINLESS_Fe7Cr2Ni" ]
665 1 Erica Snider
                                     ReflectiveSurfaceReflectances: [ [ 2.5e-1
666 1 Erica Snider
                                                                      , 2.5e-1
667 1 Erica Snider
                                                                      , 2.5e-1
668 1 Erica Snider
                                                                      ] ]
669 1 Erica Snider
                                     ScintBirksConstant: 3.22e-3
670 1 Erica Snider
                                     ScintFastTimeConst: 6
671 1 Erica Snider
                                     ScintResolutionScale: 5e-3
672 1 Erica Snider
                                     ScintSlowTimeConst: 1590
673 1 Erica Snider
                                     ScintYield: 24000
674 1 Erica Snider
                                     ScintYieldRatio: 3e-1
675 1 Erica Snider
                                     SlowScintEnergies: [ 9.5
676 1 Erica Snider
                                                        , 9.7
677 1 Erica Snider
                                                        , 9.9
678 1 Erica Snider
                                                        ]
679 1 Erica Snider
                                     SlowScintSpectrum: [ 5e-1
680 1 Erica Snider
                                                        , 1
681 1 Erica Snider
                                                        , 5e-1
682 1 Erica Snider
                                                        ]
683 1 Erica Snider
                                     SternheimerA: 1.956e-1
684 1 Erica Snider
                                     SternheimerCbar: 5.2146
685 1 Erica Snider
                                     SternheimerK: 3
686 1 Erica Snider
                                     SternheimerX0: 2e-1
687 1 Erica Snider
                                     SternheimerX1: 3
688 1 Erica Snider
                                     Temperature: 8.84e1
689 1 Erica Snider
                                   }
690 1 Erica Snider
                    LArVoxelCalculator: { VoxelEnergyCut: 1e-6
691 1 Erica Snider
                                          VoxelOffsetT: -2500
692 1 Erica Snider
                                          VoxelOffsetX: 0
693 1 Erica Snider
                                          VoxelOffsetY: 0
694 1 Erica Snider
                                          VoxelOffsetZ: 0
695 1 Erica Snider
                                          VoxelSizeT: 5000
696 1 Erica Snider
                                          VoxelSizeX: 3e-2
697 1 Erica Snider
                                          VoxelSizeY: 3e-2
698 1 Erica Snider
                                          VoxelSizeZ: 3e-2
699 1 Erica Snider
                                        }
700 1 Erica Snider
                    MagneticField: { ConstantField: [ 0
701 1 Erica Snider
                                                    , 0
702 1 Erica Snider
                                                    , 0
703 1 Erica Snider
                                                    ]
704 1 Erica Snider
                                     MagnetizedVolume: "vWorld"
705 1 Erica Snider
                                     UseField: false
706 1 Erica Snider
                                   }
707 1 Erica Snider
                  }
708 1 Erica Snider
          }
709 1 Erica Snider
source: { firstEvent: 1
710 1 Erica Snider
          firstRun: 1
711 1 Erica Snider
          maxEvents: 10
712 1 Erica Snider
          module_label: "source"
713 1 Erica Snider
          module_type: "EmptyEvent"
714 1 Erica Snider
        }
715 1 Erica Snider
trigger_paths: { trigger_paths: [ "simulate" ]
716 1 Erica Snider
</pre>
717 1 Erica Snider
718 1 Erica Snider
This functionality is particularly helpful when trying to debug what input parameters were passed to the job.
719 1 Erica Snider
720 1 Erica Snider
h2. Why did my job fail?
721 1 Erica Snider
722 1 Erica Snider
If a job fails with a seg fault look at the warnings printed to the screen or any output log files.
723 1 Erica Snider
724 1 Erica Snider
If a bug should be reported to the artists@fnal.gov list, attach the complete output of the job to the email.
725 1 Erica Snider
726 1 Erica Snider
h2. Submitting Jobs to the compute farms
727 1 Erica Snider
728 1 Erica Snider
Instructions are on [[Batch_job_submission|this page]].