Project

General

Profile

Generating GSimpleNtpFlux files

What is a "gsimple" flux?

The "gsimple" flux file format is a simplified ntuple format for storing flux rays for use by the GSimpleNtpFlux GENIE flux driver. Generally the file is generated by taking a beam simulation output (such as flugg/g4numi/g4lbne) which stores the decay of particles that generate neutrinos (generally, nowadays in "dk2nu" format), and evaluating those files (taking care of all the importance weightings and position weightings) to generate the flux of a neutrino rays (represented by each entry in the "gsimple" tree) passing through a designated "flux window".

The purpose of "gsimple" files is to trade off compute time (deweighting decays) at the cost of file space and details of the decay.

The particular coordinate transformation from beam to detector coordinates (and relative placements) and flux window size are thus all "frozen" into the file.

Translating a flux file using GNuMIFlux

Files suitable for use by GSimpleNtpFlux are generally created by sampling a flugg or g4numi (soon to include dk2nu) file at a particular location using the flux driver for those file and writing the result into the few format.

See the attached gnumi2simple_basic.C script attached to this page.

Doing it on your own

One can write one's own infrastructure for file handing and such or use the FNAL setup.

NOTE: when using gsimple flux files it is important that one not intermix files that represent different POTs/file when handing them to the flux driver. At this time (2016-08-05) the flux driver incorrectly calculates the POTs to assign to GENIE events when using intermixed gsimple files. So if you create some different sets of files in the process of generating a larger sample (i.e. early test files), do not mix them in together.

using the FNAL setup

This is by far the easier option. The user makes up a "request" file that contains a header section with common information:

  • loc the parameter set to use from GNuMIFlux.xml (location)
  • inpath the common path to the input files (can use wildcards because files will be copied to the local disk first)
  • outpath where the output files should go (if run on the grid, be sure it is group writable)
  • xml [optional] path+filename for alternative XML configuration file

These lines must start with a '#' character to prevent them from looking like those used for individual jobs.

Then there are a series of lines that serve as the basis for individual jobs. Those lines have 4 elements:

  • pattern a wildcard pattern for the input files (should not include any path info, just the filename pattern)
  • seed the random # seed to use (the same input files can be used repeatedly as long as the GENIE seed is changed)
  • pots the proton-on-target limit to use in generation
  • nentries an upper limit on the number of entries in a output file (usually one wants to be POT limited, so use 0 here)

creating a "request" file

# loc = NOvA-ND
# inpath = /nova/data/flux/flugg/flugg_mn000z200i_20101117.gpcfgrid_lowth/Run???
# outpath = /minos/data/users/rhatcher/gsimple/nova_v03/nd/mn/fhc
# xml = /path/to/alternative/MyGNuMIFlux.xml 
# 
# pattern           seed   pots  nentries 
flugg_mn000z200i_20101117.gpcfgrid_lowth_?00.root 1 1.0e8 0
flugg_mn000z200i_20101117.gpcfgrid_lowth_?01.root 1 1.0e8 0
flugg_mn000z200i_20101117.gpcfgrid_lowth_?02.root 1 1.0e8 0
flugg_mn000z200i_20101117.gpcfgrid_lowth_?03.root 1 1.0e8 0
flugg_mn000z200i_20101117.gpcfgrid_lowth_?04.root 1 1.0e8 0
flugg_mn000z200i_20101117.gpcfgrid_lowth_?05.root 1 1.0e8 0
[...]

or

# loc = NOvA-FD-modified
# inpath = /nusoft/data/flux/blackbird-numix/flugg_mn000z200i_rp11_lowth_pnut_f11f093bbird/dk2nu
# outpath = /nova/ana/users/rhatcher/test_dk2nu2simple/process_request
# xml     = /nova/ana/users/rhatcher/test_dk2nu2simple/MyDk2NuFlux.xml
#
# pattern           seed   pots  nentries 
flugg_mn000z200i_rp11_bs1.1_pnut_lowth_f11f093bbird_target_?001.dk2nu.root 1 1.0e20 100
flugg_mn000z200i_rp11_bs1.1_pnut_lowth_f11f093bbird_target_?002.dk2nu.root 1 1.0e20 100
flugg_mn000z200i_rp11_bs1.1_pnut_lowth_f11f093bbird_target_?003.dk2nu.root 1 1.0e20 100
flugg_mn000z200i_rp11_bs1.1_pnut_lowth_f11f093bbird_target_?004.dk2nu.root 1 1.0e20 100
flugg_mn000z200i_rp11_bs1.1_pnut_lowth_f11f093bbird_target_?005.dk2nu.root 1 1.0e20 100

gsimple scripts as a ups product

NOTE: 2018-01-29 /nusoft/app/alt/setup is no longer available on the grid nodes. Use the new CVMFS installation.

The scripts for this process are now packaged as UPS products. One can see what versions are installed using:

  source /cvmfs/fermilab.opensciencegrid.org/products/genie//bootstrap_genie_ups.sh
  ups list -aK+ gsimple

To see what existing (example) request files are available in the package use:

  setup gsimple v2_8_6d -q e9:debug
  ls ${GSIMPLE}/requests

Command flags (given below) allow you to supply your own text file.

submitting a job

If v2_8_6d (v2_8_6c or R286) is the chosen version one can use commands similar to:

export PRODUCTS=${PRODUCTS}:/grid/fermiapp/products/common/db
setup jobsub_client
export JOBSUB_GROUP=nova  # or whatever

jobsub_submit -g --group $GROUP -N 100 file://${GSIMPLE_DIR}/scripts/process_request.sh \
   -r mysetup.request -R /path/to/requestfile -V v2_8_6d -q debug:e9 -x 25

This example submits, to the grid, 100 jobs each of which will use one line out of the requests file mysetup.request found at /path/to/requestfile. By using the -x 25 flag the first job will use the 26th entry, 2nd the 27th, etc. So in this case you must have 125 entries beyond the header and/or comment (#) lines. With the new jobsub_client one needs to explicitly supply the -V and -q flags so it knows what to setup.

Look for a line of the form:

JobsubJobId of first job: 407672.0@fifebatch1.fnal.gov

Check on status and fetch condor logs using:

jobsub_q --user $USER
jobsub_fetchlog --group $GROUP --jobid  407672.0@fifebatch1.fnal.gov
tar xvzf 407672.0\@fifebatch1.fnal.gov.tgz

while the .root and .log files should end up in the directory specified as the outpath in the request file.

Script options and usage help will be shown with the -h option:

$ /nusoft/app/alt/gsimple/xyzzy/scripts/process_request.sh -h
PRODUCTS=/nusoft/app/alt:/nusoft/app/externals:/grid/fermiapp/products/nova/externals
  Purpose:  Generate GSimpleNtpFlux file from a FLUGG, G4NuMI or Dk2Nu flux file
      Script is driven by basic text files describing the configuration;
      condor variable PROCESS is used to select entry to process
  Usage:  process_request.sh [options] <request.list>
    -h                          # show help and quit
    -r <request.list>           # [no-request.list] (-r flag optional)
    -R </path/to/requestlist/>  # []
    -e <entry>                  # if not set use PROCESS+1       []
    -x <offset>                 # use PROCESS+1+<offset>         [0]

    -V <version>                # version of gsimple UPS package [v2_8_6d]
    -q <qual>                   # qualifier for gsimple UPS pkg  [debug:e9]

    --dk2nu | --notdk2nu        # force assumption that input files
                                # are/aren't of dk2nu form (otherwise infer
                                # from presence of "dk2nu" in name)

  if given the following override values in the <request.list>
    -l <location>               # XML param_set with transformation info
    -i <inpath>                 # path to input files
    -f <filepatt>               # file pattern
    -o <outpath>                # output path
    -G <altgxml>                # alt XML file for flux driver []
    -P <pots>                   # number of pots
    -N <nrays>                  # additional limit on number of rays

    -v                          # increase verbosity

  experimental features:
    -s                          # random # seed offset [0]
    -M                          # use alternative genie w/ support for 
                                # Minerva variant g4numi flux files
    -a                          # (experimental) alternative genie version
    -w                          # generate weighted entries

running older release

  setup_condor
  nova_jobsub -g -N 100 /grid/fermiapp/minos/sim/gsimple/requests/process_request.sh \
        -r mysetup.request -R /path/to/requestfile -x 25

Resulting files

The resulting files are written back to the outpath along with their accompanying log file.

The input line

flugg_mn000z200i_20101117.gpcfgrid_lowth_?00.root 1 1.0e8 0

results in files with the base name:
gsimple_NOvA-ND_mn000z200i_20101117.gpcfgrid_lowth_q00_00001

One sees that the location is embedded in the name; the file pattern is carried over with substitutions for wildcarding characters; and the seed # is tacked onto the end. The wildcard substitutions are "?" becomes "q", "*" becomes "s" and "[" "]" become "l" and "r".

What locations are there?

The values one can use for loc are those "parameter sets" that GNuMIFlux can call upon to do the coordinate transformations. By default these are what is in the GNuMIFlux.xml file for the installed version of GENIE:

$ source /nusoft/app/alt/setup
$ setup gsimple R286 -q debug:e6
$ grep "param_set name" $GENIE/src/FluxDrivers/GNuMINtuple/GNuMIFlux.xml
  <param_set name="MINOS-NearDet">
  <param_set name="MINOS-FarDet">
  <param_set name="MINOS-NearRock">
  <param_set name="MINOS-FarRock">
  <param_set name="NOvA-NDOS">
  <param_set name="NOvA-NDOSRock">
  <param_set name="NOvA-ND">
  <param_set name="NOvA-NDRock">
  <param_set name="NOvA-FD">
  <param_set name="NOvA-FDRock">
  <param_set name="NOvA-NDOS-BNB">
  <param_set name="BNB-NDOS">
  <param_set name="microboone-booster">
  <param_set name="microboone-numi">
  <param_set name="MINERVA">
  <param_set name="ArgoNeuT">
  <param_set name="LBNE-NearDet_01">
  <param_set name="LBNE-FarDet_01">

The form of these parameter sets is describe on the wiki page GNuMIFlux_code_and_config

If the "requst" file contain a line:

#  xml = /path/subdir/myGNuMIFlux.xml" 

as part of the header then processing will use that file (myGNuMIFlux.xml in this case) for the parameter set. This is useful if the values in the installed file are outdated or a new location is desired. The loc name must exist in the (implicit or explicitly selected) XML file.

scripts