Creating cross-section spline files¶
Generally users should generate events using the splines that exist in the
nusoft UPS area; these in conjunction with the GENIE installation in the same area ensures a consistent, reproducible setup. Personal installations are strongly discouraged and certainly should not be used for any general production. If a special setup is needed then a steps should be taken to get it installed in the
nusoft UPS area for reference. If the spline files are missing needed isotopes please contact Robert Hatcher so they can be generated.
This page describes a set of scripts written to wrapper the details of how
gxspl files are created. These scripts are tuned for use at FNAL (file paths) and with condor (batch job processing). They are meant for general use by the FNAL neutrino community; they are intended to be user and experiment agnostic. Users should not copy the scripts to alternative locations, but use them as they are -- this will avoid version skew and confusion about modified versions. If there are problems running them (due to path issues or such) then please contact Robert Hatcher to have them modified to be compatible with your setup. Copies in alternative locations will not be supported.
Currently the process is broken into six steps:
prepare.sh: initialize the script and output directories
- stage 1: generate individual neutrino on free nucleon splines
- stage 2: combine all free nucleon splines
- stage 3: generate spines for all neutrinos on a specific isotope
- stage 4: combine all isotope files
- stage 5: reduce file to only the (current) minimum needed isotopes
While the 5 stages of processing could be joined together with a DAG (Directed Acyclical Graph) they currently need manual user intervention to get started. Jobs for later stages should not be submitted until the prior steps are complete.
The end result should be a file named
gxspl-NuMIsmall.xml. Care must be taken to ensure that information about how this file was created is not lost. A
gxspl file is only valid for use by
GENIE for the exact same release and configuration of
GENIE as was used to create it. If you use a cross-section spline file in a different release or one with a different configuration your results may be inconsistent.
GENIE has no way of validating whether there is an inconsistent usage, so it is important to maintain awareness. It is recommended that users rename the generic
gxspl-NuMI*.xml files to embed relevant information into the filename to at least indicate a non-standard file.
Preparing a setup:¶
The script to initialize the process is located at:
Setting a user script and output location¶
These scripts are written to allow them to be run on the (FNAL) "grid" under the condor batch system. This imposes restrictions on where scripts can be run from and files written to. The user must supply a subpath using the
-o flag to designate the directory structure.
/is used for scripts
/is used for output XML files and logs (for NOvA only:
The subpath needn't be a single name but can be something like:
my_genie_xsec/modified_MA/MA1.0. The expt and user are inferred from the
id of the user running the script. The
id group to BlueArc mounting point has been verified for MINOS and NOvA but may not be correct for other experiments; please let me know what it needs to be if it doesn't work as-is.
Telling the scripts how to get the
These scripts need the
gspladd to do the work. Thus a working version of
GENIE needs to be available on the grid, with all the ancillary packages (e.g.
LHAPDF, etc.) configured. For MINOS and NOvA (currently) this is built into the scripts by setting up their respective offline environments using
-s minos or
-s nova. Specification of a release can be done using the
-S flag, eg.
-S "-r development"; quotes are required for cases where there are multiple arguments. For other experiments or non-standard installations one can use the
-s flag to identify a
bash script that, when
sourced, will setup the desired
GENIE release. If such a script is written to accept arguments to differentiate between releases it too can use the
-S flag to pass those arguments to it.
Examples of prepare.sh¶
$ /grid/fermiapp/nusoft/generate_genie_xsec/prepare.sh -o my_genie_xsec/modified -s /path/to/genie_setup.sh -S "-r myversion"
generates the screen output:
#========================================================================= ### setup and conditions script for my_genie_xsec/modified # # how to setup GENIE SETUPCMD="source /path/to/genie_setup.sh -r myversion" # # set the environment export APPPATH=/grid/fermiapp/nusoft/generate_genie_xsec export USERAPPPATH=/nova/app/users/rhatcher/my_genie_xsec/modified export USERDATAPATH=/nova/ana/users/rhatcher/my_genie_xsec/modified export STAGE1RESULTS=/nova/ana/users/rhatcher/my_genie_xsec/modified/work/freenuc export STAGE3RESULTS=/nova/ana/users/rhatcher/my_genie_xsec/modified/work/element export GXMLPATH=.:/nova/app/users/rhatcher/my_genie_xsec/modified:/nova/data/users/rhatcher/my_genie_xsec/modified export GMSGLAYOUT=SIMPLE #[BASIC] or timestamp-less SIMPLE export JOBSUB=jobsub # export NKNOTS=500 export EMIN=0.01 export EMAX=200.00 # # end-of-setup script #========================================================================= to run: $ cd /nova/app/users/rhatcher/my_genie_xsec/modified $./driver_script.sh follow the instructions
The first part of the output displays the selected configuration. The
to run: then directs the user to the configured location and the script to run.
On systems where setting up genie is part of the offline framework, one can simply supply the experiment name (i.e. nova or minos) for the
Using NOvA is odd in two ways. First the setup requires more than sourcing a single script, so we make a wrapper script in the new location and source that. Secondly, their user-writable output disk is not
$ /grid/fermiapp/nusoft/generate_genie_xsec/prepare.sh -o my_genie_xsec/modified -s nova -S "-r S12.05.31"
SETUPCMD="source /nova/app/users/rhatcher/my_genie_xsec/modified/nova_setup_wrapper.sh "
Running the stages¶
Initial state after
$ ls /nova/app/users/rhatcher/my_genie_xsec/modified driver_script.sh setup_script.sh $ ls /nova/data/users/rhatcher/my_genie_xsec/modified work
If you are planning on running
GENIE with a modified set of parameters then at this point copy the
UserPhysicsOptions.xml to the corresponding
data directory The
GXMLPATH variable will be set to include the
data directory, so this will get picked up automatically if the file is placed there.
driver_script.sh in the
app directory without args will check the current state of processing and suggest what step to take to proceed:
$ cd /nova/app/users/rhatcher/my_genie_xsec/modified $ ./driver_script.sh checking status /grid/fermiapp/nusoft/generate_genie_xsec/check_progress.sh checking stage 1 in /nova/data/users/rhatcher/my_genie_xsec/modified/work/freenuc: missing: [-P 0] gxspl-nu_tau_bar-freep.xml missing: [-P 1] gxspl-nu_tau_bar-freen.xml missing: [-P 2] gxspl-nu_mu_bar-freep.xml missing: [-P 3] gxspl-nu_mu_bar-freen.xml missing: [-P 4] gxspl-nu_e_bar-freep.xml missing: [-P 5] gxspl-nu_e_bar-freen.xml missing: [-P 6] gxspl-nu_e-freep.xml missing: [-P 7] gxspl-nu_e-freen.xml missing: [-P 8] gxspl-nu_mu-freep.xml missing: [-P 9] gxspl-nu_mu-freen.xml missing: [-P 10] gxspl-nu_tau-freep.xml missing: [-P 11] gxspl-nu_tau-freen.xml stage 1: have 0, missing 12 to start processing this stage: cd /nova/app/users/rhatcher/my_genie_xsec/modified jobsub -g -N 12 driver_script.sh -S 1
At any time, running
./driver_script.sh will check the current status of processing. Only proceed with the
jobsub after checking that all the jobs from the previous submission are complete. If in a multi-job stage some of the jobs failed to complete (or are still running) then the
driver_script.sh will indicate how to run individual make-up jobs (substitute the # following the
#). This check
does not know how to check whether condor jobs are still running so it might suggest running make-up jobs when a stage is incomplete; it is thus important that the user know the status of their jobs.
When all steps are complete it will look like:
$ ./driver_script.sh checking status /grid/fermiapp/nusoft/generate_genie_xsec/check_progress.sh checking stage 1 in /minos/data/users/rhatcher/my_genie_xsec/modified/work/freenuc: stage 1: have 12, missing 0 SUCCESS: stage 1 checking stage 2 in /minos/data/users/rhatcher/my_genie_xsec/modified/work/element: stage 2: have gxspl-freenuc.xml SUCCESS: stage 2 checking stage 3 in /minos/data/users/rhatcher/my_genie_xsec/modified/work/element: stage 3: have 30, missing 0 SUCCESS: stage 3 checking stage 4 in /minos/data/users/rhatcher/my_genie_xsec/modified: stage 4: have gxspl-NuMIbig.xml SUCCESS: stage 4 checking stage 5 in /minos/data/users/rhatcher/my_genie_xsec/modified: stage 5: have gxspl-NuMIsmall.xml SUCCESS: stage 5
Use in the ART framework¶
To use under the ART framework one must tell GENIE the name of the spline file and where to find it (and any
UserPhysicsOptions.xml file used to make the spline files). In the job
.fcl file one can override the defaults for
physics.producers.generator along the lines of:
physics.producers.generator.Environment: [ "GSPLOAD", "gxspl-NuMIsmall.xml", # default name of newly created spline file # set GXMLPATH so that GENIE knows alternative locations where to find XML files # this will allow GENIE to find the new xsec file AND any modified UserPhysicsOptions.xml # self-consistency between the two is IMPORTANT "GXMLPATH", "/minos/data/users/rhatcher/my_genie_xsec/modified", "GEVGL", "Default", "GPRODMODE", "YES", # supress lots of GENIE messages # "GMSGCONF", "EventGenerator/MyGenieMessengerVerbose.xml", # override log mesg levels "GMSGLAYOUT", "SIMPLE" # [BASIC] or timestamp-less SIMPLE, GENIE log4cpp message format # last entry doesn't have trailing comma! ]
$ /grid/fermiapp/nusoft/generate_genie_xsec/prepare.sh -h /grid/fermiapp/nusoft/generate_genie_xsec/prepare.sh -o outpath -s genie_setup [-S gsetup_args] Required Args: -o <out/path> output directory for scripts and results: relative to /<expt>/[app|data|ana]/users/<user> subdirectory used for intermediate work products NOTE: it must be *only* the subdir portion of the path -s <genie_setup> script, when sourced, that will setup genie and auxillary packages (e.g. ROOT, log4cpp, LHAPDF) must define and add to PATH & LD_LIBRARY_PATH special case: "nova" or "minos" -S <gsetup_args> arguments to genie_setup script (e.g. "-r S12.05.31") needs to be quoted if contains a space Optional Args: -k <nknots> number of knots to use in splines -e <emin> lowest neutrino energy (not supported by gmkspl) -E <emax> maximum neutrino energy -v increase verbosity level -X <XMLpath> GXMLPATH for GENIE to use for alternative XML files if you are using this to override GENIE parameters then it might be wise to put a copy of the files in the -o directory for reference, and list the same path for this flag. -A <apppath> where this code is installed -J <jobsub> name of condor job submission command -G <group> expt ("minos", "nova", etc) for BlueArc -U <user> override "whoami" results