Using ANNIESoft in the ANNIE Interactive Node (Under construction)


Users must have a valid kerberos ticket to access Fermilab computing at the time an attempt to log into a Fermilab machine. The ticket is obtained by executing the following command at a terminal prompt:

$ kinit principal@FNAL.GOV
where principal is the user's kerberos principal.
Enter your Kerberos password (not to be confused with your FNAL Services password). No output is good here! Ticket lifetime is 1 day. Move on to the next step.

Logging into Fermilab machines

ssh -Y -l principal

WARNING - SLF6 systems will be shut down in November 2020.
anniegpvm01 and anniegpvm02 interactive systems are running SLF6.
anniesl7gpvm01 is an interactive system running SLF7.

Setting up ANNIESoft

Copy the following function into your .bashrc or .bash_profile

# put in your .bashrc
  source /grid/fermiapp/products/common/etc/setup
  export PRODUCTS=${PRODUCTS}:/grid/fermiapp/products/larsoft
  setup genie v2_8_6d -q e9:debug
  setup genie_xsec v2_8_6 -q default
  setup genie_phyopt v2_8_6 -q dkcharmtau
  setup geant4 v4_10_1_p02a -q e9:debug:qt

Whenever you login, run the setup function
# type at the command line (but NOT in .bashrc!)

Disk Space

The user space for those logging into these nodes is

This is where users should store their test releases as well as any analysis files. This area is backed up daily, and snapshots are saved for up to 1 week in /annie/app/.snapshot/

Files that are generated by the analysis groups, as well as processed data and Monte Carlo samples that are used for general consumption should be kept in


The /annie/data area is completely inaccessible to condor batch jobs. Such job must use the /pnfs/annie/persistent (or /pnfs/annie/scratch) areas. On the interactive node this is directly accessible (though with some write limitations, e.g. no appending to files). In grid jobs files need to be fetched or written using ifdh cp (part of the ifdhc package, see BestPractices ).

Running in the Grid

To access the link above you need a valid KCA certificate

  • Setup anniesoft and jobsub client
    setup jobsub_client
  • Lines below should work. Substitute 99999 for the jobid printed after submission . From
    jobsub_submit --resource-provides=usage_model=OPPORTUNISTIC --OS=SL5,SL6 --group=annie file:///usr/bin/id
    jobsub_q --group=annie
    jobsub_history --group=annie
    jobsub_fetchlog --group=annie