Project

General

Profile

So You Want to Make Some Monte Carlo (A Qwik Start Guide for Production) » History » Version 21

« Previous - Version 21/35 (diff) - Next » - Current version
Adam Moren, 12/02/2016 08:21 AM


So You Want to Make Some Monte Carlo (A Qwik Start Guide for Production)

Not running production jobs? Look here: So You Want to Make Some Monte Carlo (A Qwik Start Guide for non-Productioners)

So you need to make some Monte Carlo and perhaps don't have a month's time to devote learning all software properly. This guide is for you. The goal of this page is to get you started on generating some MC with minimal thought and effort.

There are two scripts you need to use to generate MC:
1. make_sim_fcl
- This script generates the fcls you need to process MC. This script resides in the "novaproduction" package. It is setup automatically to run without needing to locally install the package. You can verify this by using, which make_sim_fcl
2. submit_nova_art.py
- This script submits jobs to GRID for you based on an input configuration file that contains the SAM dataset definition generated from make_sim_fcl. Much like make_sim_fcl, this script works without you needing to setup the NovaGridUtils package.

Though you don't need to, I would still suggest setting up local copies of the both novaproduction and NovaGridUtils (addpkg_svn -h novaproduction) in a test release.

make_sim_fcl

Now let's go through some examples of how you'll be using the scripts above. The examples below were used in the Second Analysis Campaign and pulled directly from the ECL
When you're ready to run your make_sim_fcl command, you'll need to run it as novapro. Log into a machine as novapro and then setup the nova environment with setup_nova.

Standard production make_sim_fcl settings for beam MC

Standard settings:

-fts --iteration 1 --batchnumber 1 --timespec full --generator genie

Real conditions, variable intensity:

# ND
--randomsubruns <nfiles> /nova/app/users/projas/runlists/nd_grl_v0_epoch1-3c.txt --potperspill
# FD
--randomsubruns <nfiles> /nova/app/users/projas/runlists/FD_GRL_v3_020816.txt

Events or POT per/file:

# ND
--pot 5e16
# FD
-n 1000

Detector:

#ND
-det nd
#FD
-det fd

Horn current:

-hp fhc 
# or
-hp rhc

Flux swap:

-fs nonswap
# or
-fs swap
# or ( swaps both nue and numu for tau)
-fs tau
# or ( swaps only numu for tau)
-fs numutau 

Overlay:

#ND rock FHC
--mix rock g4_secondaries_nd_genie_nonswap_fhc_R16-02-11-prod2genie.a_nova_v08_rocksingles_v1_rock_secondaries
# ND rock RHC
--mix rock g4_secondaries_nd_genie_nonswap_rhc_nova_v08_ideal_batch1_v1_ideal-gain100-rock_secondaries
# FD cosmic
--mix cosmic prod_artdaq_S15-03-11_fd_cosmic_epochs1-3c_v1_goodruns

Important note: the overlay definition you give needs to have an available snapshot. If you're not sure, run:

samweb take-snapshot <definition>

You will also want to make sure that this dataset has also been pre-staged from tape. You can check if it needs to be prestaged by listing a few filenames and checking them with:

cache_state.py -d <definition>

If the files are on tape you will need to follow the instructions for pre-staging the files in the SAM web cookbook.

examples using make_sim_fcl for ND

These are the three most requested samples for the ND

ND Genie non-swap no-RW (no-reweight) w/Rock Overlays:

make_sim_fcl -rsr 20000 /nova/app/users/projas/runlists/nd_grl_v0_epoch1-3c.txt -n 2000 -det nd -gen genie -hp fhc -fs nonswap -i 1 -fts -bn 3 -ts full -nogenierw

ND Genie Fluxswap (no overlays):

make_sim_fcl -rsr 20000 /nova/app/users/projas/runlists/nd_grl_v0_epoch1-3c.txt -n 2000 -det nd -gen genie -hp fhc -fs swap -i 1 -fts -bn 1 -ts full -nogenierw

ND Genie nonswap w/RW (reweight) w/Rock Overlays:

make_sim_fcl -rsr 100000 /nova/app/users/projas/runlists/nd_grl_v0_epoch1-3c.txt -n 400 -det nd -gen genie -hp fhc -fs nonswap -i 1 -fts -bn 4 -ts full

If you notice, this sample has less spills per file than the other two (2000 vs 400), this was to shorten processing time as overlays already take more time, plus the genie reweighting adds even more time.

ND CRY (gainmode 100):

The following command produces 5k files with 1M events a piece:
make_sim_fcl -gm 100 -lr 1000049 -ns 100 -n 1000000 -gen cry -fs all -det nd -i 1 -bn 1 -ts ideal -fts

examples using make_sim_fcl for FD

FD Genie nonswap:

make_sim_fcl -rsr 6000 /nova/app/users/projas/runlists/FD_GRL_v3_020816.txt -n 1000 -det fd -gen genie -hp fhc -fs nonswap -i 1 -bn 1 -ts full -nogenierw -fts

FD Genie fluxswap:

make_sim_fcl -rsr 6000 /nova/app/users/projas/runlists/FD_GRL_v3_020816.txt -n 1000 -det fd -gen genie -hp fhc -fs swap -i 1 -bn 1 -ts full -nogenierw -fts

FD Genie tauswap:

make_sim_fcl -rsr 6000 /nova/app/users/projas/runlists/FD_GRL_v3_020816.txt -n 1000 -det fd -gen genie -hp fhc -fs tau -i 1 -bn 1 -ts full -nogenierw -fts

FD CRY gainmode 100:

make_sim_fcl -gm 100 -lr 1000049 -ns 100 -n 200 -gen cry -fs all -det fd -i 1 -bn 1 -ts ideal -fts

FD CRY gainmode 140:

make_sim_fcl -gm 140 -lr 2000049 -ns 100 -n 200 -gen cry -fs all -det fd -i 1 -bn 1 -ts ideal -fts

submit_nova_art.py

I suggest setting up a local copy of NovaGridUtils for this if you haven't already. In the configs directory of NovaGridUtils there are two configuration templates ready to use, prod_mcgen_template.cfg and prod_nd_genie_overlay_template.cfg. If you don't want download the package, you can just copy them from the latest NGU package at $NOVAGRIDUTILS_DIR/configs/

The syntax for running jobs is, submit_nova_art.py -f /<path_to>/<config_file>.cfg

Standard configurations (non-overlays)

I refer to non-overlay MC jobs as standard jobs, this is typically anything made for FD as well as certain ND samples (Fluxswap, CRY)
For these standard jobs, you'll want to use $NOVAGRIDUTILS_DIR/configs/prod_mcgen_template.cfg. This template has instructions on how to fill it out that are hopefully very straightforward. Much of the boiler plate has now been moved into includes so only the options you need to change are seen in the template.

#This is a template to help you configure you mc generation jobs on the grid.

# Include standard MC generation configuration
-f mcgen.inc

########################################
# General options that you need to set #
########################################

#Type the name of your job here
--jobname <job_name>

#Type the definition that contains the fcl you want to process
--defname <dataset_definition_name>

#This is where you place which tag you want to generate in
--tag <novasoft_release>

#Number of jobs you'd like to submit
--njobs <num_jobs>

####################################
# Default options you might change #
####################################

# By default, run MC generation off-site
-f everywhere.inc
#-f onsite.inc OR offsite.inc

# Official production job?
-f production.inc

# Normally produce artdaq but...
--outTier=out1:artdaq

# ...switch to g4 for making rock MC and other overlay pieces
#--outTier=out1:g4

Overlay configurations

Overlays are more complex jobs to run than standard configurations. Once again the template below should help get you started. Again, much of the boiler plate has now been moved into includes so only the options you need to change are seen in the template.

Important note:

#This is a template to help you configure you mc generation jobs on the grid.

# Include standard MC generation configuration
-f mcgen.inc

########################################
# General options that you need to set #
########################################

#Type the name of your job here
--jobname <job_name>

#Type the definition that contains the fcl you want to process
--defname <dataset_definition_name>

#This is where you place which tag you want to generate in
--tag <novasoft_release>

#Number of jobs you'd like to submit
--njobs <num_jobs>

####################################
# Default options you might change #
####################################

# By default, run MC generation off-site
-f everywhere.inc
#-f onsite.inc OR offsite.inc

# Official production job?
-f production.inc

#########################
# Overlay configuration #
#########################

# Choose your overlay procedure. 
# The script(s) you choose here should
# match the settings used when making
# your fhicl files

#--mix rock_overlay.sh
#--mix cosmic_overlay.sh
#--mix rock_overlay.sh:cosmic_overlay.sh
#--mix singles_overlay.sh

# Overlay jobs always produce artdaq files and can
# only be run one at a time. Always set these for
# overlay jobs.
--outTier=out1:artdaq
--files_per_job 1

If you need to run with a non-standard overlay rate, you should copy the contents of $NOVAGRIDUTILS_DIR/configs/base/fhc_overlay.inc into your configuration file and modify as needed. This should also include the base overlay boilerplate via -f rock_overlay.cfg which you are very unlikely to need to modify.

Final Notes

You should now have everything you need to run jobs. The output of submit_nova_art.py should have given you a jobsubID and a samweb monitoring page. I suggest saving them somewhere so you can access them at a later date if needed. Additionally, don't forget to fill out an ELOG entry in the ECL, so that other may learn from your work and Conveners can track your progress.

If you need to make something different from the above, such as a systematics sample or some other special sample, you'll probably using the --syst option.

For a list of stand/basic options plus more examples, use:
make_sim_fcl -h
For a full list of options and brief explanations, use:
make_sim_fcl -ah

If you know you will be running lots of MC, it would behoove you to learn more about how make_sim_fcl works: Running make_sim_fcl as well as just setup up the head version (described near the top) and just reading the code.