Project

General

Profile

GENIE and Geant4 neutrons from rock propagation

This documentation (and underlying code) is very very preliminary

Existing code

/annie/app/users/rhatcher/
  /bin
    setup_annie.sh      # setup UPS products
    gevgen_numi_rock    # modified version of GENIE gevgen_numi for ANNIE
    run_annie_genie.sh  # grid run-able script for running GENIE w/ ANNIE geometry + flux
    run_annie_g4dirt.sh # run g4annie_dirt_flux in grid environment
    ana_annie_genie.sh  # script for making gntp.?xx.ntuple.root from gntp.???.ghep.root files
    g4annie_dirt_flux   # Geant4 app reading GENIE files, writing annie_tank_flux.???.root
    draw3.C             # simple display of gntp.?xx.ntuple.root file
    draw_nsource.C      # complex script for generating proposal plots from annie_tank_flux.???.root files

  /gevgen_fixed/ # source, build area for gevgen_numi_rock
  /tank_flux/    # source, build area for g4annie_dirt_flux
  source /annie/app/users/rhatcher/setup_annie.sh
  setup_annie

Running GENIE

$ ./run_annie_genie.sh -h

./run_annie_genie.sh -h
run_annie_genie.sh VERSION 2015-08-25d

run_annie_genie.sh: you must use either the -n or -e flag to limit run
run_annie_genie.sh: you must set the starting run number using the -r flag

run_annie_genie.sh -r RUNBASE (-n NEVT | -e POTS) [ -F FIDCUT | -E EFACTORS | -R ROCKBOX ] [ other flags ]

Run a GENIE job
   -r | --run=RUNNUM       run number [+${PROCESS}]
   -n | --nevt=NEVENTS     limit by number of events []
   -e | --exposure=POTS    limit by exposure []

   -s | --seed=RNDMSEED    random # seed
   -g | --gdml=GDMLFILE    geometry file [annie_v01.gdml]
   -t | --topvol=TOPVOL    geometry top volume []
   -F | --fid=FIDCUT       fiducial cut string []
   -E | --expand=EFACTORS  create fid cut string by expanding from BLDG
                             extent; if one value then add equally in
                             all directions, if 3 values {-z,transverse,+z}
                             in geometry units (cm) []
   -R | --rockbox=RFACTORS create "rock box" style fid cut expanding from BLDG
                             extent.  e.g. 1,100,
                                [0|1]=rockonly
                                100 min size (geom units, eg. cm)
                                0.00425 dE/dx
                                1.05 fudge
                                0 efromwall
                             [] only "rockonly" (0|1) required
   --rock                  generate in greater rock
                              change TOPVOVL to WORLD_LV
                              use wider flux window
                              push back starting Z to -20m

   -v                      increase verbosity
   -h | --help             this output
        --helpfid          expanded FIDCUT help

Running the Geant4 application

The grid aware wrapper for the actual application is:

un_annie_g4dirt.sh -r RUNBASE [ -n NEVT ] [ --repeat NREPEAT ] -i INDIR -o OUTDIR

  Assumes input file name form:  gntp.${RUNNUM}.ghep.root

Run a g4annie_dirt_flux job
   -i | --indir=IDIR       input directory  (usually /pnfs/annie/...)
   -o | --outdir=ODIR      output directory (usually /pnfs/annie/..)
   -r | --run=RUNNUM       run number [+${PROCESS}]
   -n | --nevt=NEVENTS     limit by number of events [-1] (-1=all)
   -R | --repeat=NREPEAT   re-run NEVENTS, NREPEAT times [1]
   -g | --gdml=GDMLFILE    geometry file [annie_v01.gdml]
   -p | --physics=PHYLIST  PhysicsList name [FTFP_BERT_HP]

   -v                      increase verbosity
   -h | --help             this output

The compiled Geant4 application is:

$ ./g4annie_dirt_flux -h
g4annie_dirt_flux:  a simplified stand-alone Geant4 application 
  g4annie_dirt_flux [options]  
  -h --help           this output
  -b --batch          run in batch mode 
  -v --verbose        increase verbosity [0]
  -n --nevents        limit # of events [-1]
  -r --repeat         repeat each GENIE event in G4 [1]
  -g --gdml=GFILE     input geometry file [../../annie.gdml]
  -p --physics=PNAME  PhysicsList name [FTFP_BERT_HP]
                         (unknown names print the list)
  -i --input=GNTP     GENIE file [gntp.42.ghep.root]
  -o --output=OFILE   output file [annie_flux.root]

Existing data files used for Proposal (and TSW)

2015-08-30 current files were written using ROOT version v5.34.25.

GENIE output files and simple summary

Files of the form: gntp. ??? .ghep.root are standard format GENIE event files as generated by an application such as gevgen_fermi. These contain the full event record of the GENIE record and the corresponding flux info (if derived from a ntuple based driver).

Files of the form gntp. ?xx .ntuple.root are a simple sumarization of the GENIE event files, with a ROOT ntuple with the interaction positions {x,y,z}, neutrino energy E and flavor pdg, for all the GENIE files in the same directory.

Geant4 output files

Files of the form annie_tank_flux. xyz .root contain two trees. The tankflux tree contains information about particles that reach the surface of (or are created within) the water of within the tank. The tankmeta tree, for a singe original output file, contains a single entry about the metadata associated with the tankflux tree. This two tree structure allows the files to easily be "chained" together.

root [] tankflux->Show(1)
======> EVENT:1
 run             = 201         # info about GENIE event file usage
 entry           = 7
 iter            = 0
 niter           = 1
 nupdg           = 14          # neutrino flavor
 nuvtxx          = -0.0763673  # interaction vertex
 nuvtxy          = -1.90518
 nuvtxz          = 3.0304
 nuvtxt          = 6.68201e-06
 intank          = 1           # generated in the tank 1=true, 0=false
 inhall          = -1
 vtxvol          = TWATER_PV   # volume nu interaction vertex was within
 vtxmat          = TankWater   # material ...
 ntank           = 2           # particles (generated) in water, here 2
 pdgtank         = 22,         #   one gamma, one neutron 
                  2112
 primary         = 1,          # 1=primary out of GENIE
                  1
 vx              = -7.63673,   # point where particle is (generated) in water
                  -7.63673
 vy              = -190.518, 
                  -190.518
 vz              = 303.04, 
                  303.04
 vt              = 6682.01, 
                  6682.01
 px              = -0.00455339, # momentum as particle starts in water
                  -0.197694
 py              = 0.002426, 
                  0.198147
 pz              = 0.00340199, 
                  0.115703
 E               = 0.00618, 
                  0.98718
 kE              = 0.00618, 
                  0.0476098

root [] tankflux->Show(3)
======> EVENT:3
 run             = 201
 entry           = 15
 iter            = 0
 niter           = 1
 nupdg           = 14
 nuvtxx          = 3.43085
 nuvtxy          = 0.504457
 nuvtxz          = 0.151479
 nuvtxt          = 6.67606e-06
 intank          = 0            # nu interaction was NOT in tank's water
 inhall          = -1
 vtxvol          = BLDG_PV      # was within building physical volume
 vtxmat          = G4_CONCRETE
 ntank           = 1            # 1 particle reached the water
 pdgtank         = 22           # it was a gamma
 primary         = 0            # not primary out of GENIE
 vx              = 101.528
 vy              = 32.2721
 vz              = 259.777
 vt              = 6693.01
 px              = -0.000168244
 py              = 6.0351e-05
 pz              = -0.000103384
 E               = 0.000206486
 kE              = 0.000206486

root [] tankmeta->Show(0)
======> EVENT:0
 inputFluxName   = ../genie_2xx/gntp.201.ghep.root
 inputNEntries   = 20000
 inputTotalPOTs  = 6.35061e+17
 nG4Repeat       = 1
 nG4BeamOn       = 20000
 nG4Events       = 20000
 POTsPerPass     = 6.35061e+17
 POTs            = 6.35061e+17

Selecting neutrons internal and external to the tank:

string sel_internal_n = "(primary==1 && abs(pdgtank)==2112)";
string sel_external_n = "(primary==0 && abs(pdgtank)==2112)";

$ root
  TChain* tf = new TChain("tankflux");
  tf->Add("annie_tank_flux.*.root");
  tf->Draw("vz:vy:vx","(primary==0 && abs(pdgtank)==2112)")

Variants

All "sets" of GENIE files were generated with run_annie_genie.sh --rock -n 20000 -r <run#>

Series additional flags Description
1?? (no cuts) consider entire world rock volume
2?? -E 250,50,50 expanding from BLDG extent {-z,transverse,+z}
3?? -R 1,200 use GENIE 'rockbox' fid cut: include only outside EXP_HALL_PV;
minimum 2m beyond BLDG?(HALL?) + Enu dependent expansion

Directory structure

/annie/data/users/rhatcher/
   genie_1xx/  
      gntp.1??.ghep.root
      gntp.1xx.ntuple.root
   genie_2xx/
      gntp.2??.ghep.root
      gntp.1xx.ntuple.root
   g4_1xx/
      annie_tank_flux.1??.root
   g4_2xx/
      annie_tank_flux_flux.2??.root

Things to try ... running on the Grid as ANNIE

ANNIEPATH=/annie/app/users/rhatcher/bin
source $ANNIEPATH/setup_annie.sh
setup_annie
setup jobsub_client
export GROUP=annie
# --OS=SL6 --resource-provides=usage_model=DEDICATED,OPPORTUNISTIC

mkdir -p /pnfs/annie/persistent/users/${USER}/genie
mkdir -p /pnfs/annie/persistent/users/${USER}/g4dirt

# export VOLCUT="-E 900,500,500" 
# 20K GENIE events w/ this cut take 12-25hrs per file
# g4 step takes < ~2.5hr per file

export VOLCUT="-E 1100,500,500" 
export FIRSTRUN=2000
export NEVENTS=10000

export NJOBS=1000

# was 20000 evt, 250 jobs

jobsub_submit -g --group $GROUP -N ${NJOBS} file://$ANNIEPATH/run_annie_genie.sh \
   --rock -r ${FIRSTRUN} -n ${NEVENTS} -v ${VOLCUT} \
   -o /pnfs/annie/persistent/users/${USER}/genie

# when those are complete, the run the next stage
jobsub_submit -g --group $GROUP -N ${NJOBS} file://$ANNIEPATH/run_annie_g4dirt.sh \
   -r ${FIRSTRUN} -v -i /pnfs/annie/persistent/users/${USER}/genie \
   -o /pnfs/annie/persistent/users/${USER}/g4dirt

# when those are complete, make (and save pdf's) plots
#
# processing a large number of small files can be slow ...
# so make a file to combine the trees in the multiple files

cd /annie/data/users/${USER}/   # somewhere not pnfs
hadd -n 0 annie_tank_flux.2xxx.root /pnfs/annie/persistent/users/${USER}/g4dirt/annie_tank_flux.2???.root

root $ANNIEPATH/draw_nsource.C\(\"./annie_tank_flux.2xxx.root\",true\)