Setting up keytabs and proxies for jobsub in shared accounts¶

In certain cases we use shared accounts for production tasks:

• production grid job submission
• management of SAM metadata
• data handling tasks

Because Fermigrid jobs and other tasks require Grid proxies tied to an individual,
we need to make that individual's proxies available to the shared account.

The recommended technique is to copy a designated individual's keytab file into the shared account.
We place it in the path normally used by kcron, so that scripts can be written to run in both shared and individual accounts.
The method of generating and copying the keytab seems arcane and complicated.
There are reasons for each of the steps, so do everything described here !

In the following discussion we describe copying the kreymer keytab to minospro on gpsn01.fnal.gov .

We want to simply copy the kreymer kcron keytab file into the corresponding place in minospro.

• The name of each account's kcron keytab file under /var/adm/krb5 is given by kcron -f .
• File permissions prohibit the direct creation of the minospro keytab file.
• We run kcroninit under the minospro account to create a kcron keytab file.
• This creates a new kreymer keytab, which invalidates the former kreymer keytab on this host.
• So we run kcroninit in the kreymer account, which invalidates the keytab under minospro
• So we copy the new kreymer account keytab over the now stale keytab in minospro.

1. kreymer opens two windows
   • MINOSPRO ssh minospro@gpsn01.fnal.gov
   • KREYMER ssh kreymer@gpsn01.fnal.gov
2. MINOSPRO
   • Create a kcron keytab with kreymer's kcron identity.
     

     kcroninit
     

     
     [we do this so that the kcron keytab file for MINOSPRO gets created with the right permissions and we can update it later]
3. KREYMER
   • Make a fresh kreymer kcron keytab
   • It is best to clear out the existing kcron keytab before making a new one.
     

     kcrondestroy
     kcroninit
     

     
     [We do this because running kcroninit for our kerberos principal in the other account invalidated this copy]
4. KREYMER
   • Verify that the keytab works with kcron
     

     kcron klist
     
     Ticket cache: FILE:/tmp/krb5cc_1060_IVnxb30674
     Default principal: kreymer/cron/gpsn01.fnal.gov@FNAL.GOV
     
     Valid starting     Expires            Service principal
     10/11/13 09:09:09  10/11/13 19:09:09  krbtgt/FNAL.GOV@FNAL.GOV
     ...
     

     
     The important bits here are that you
     • don't get an error message
     • the Default principal: line has your username/cron/hostname in it
     • the ticket expiration date is in the future
5. KREYMER
   • Generate a command to be entered in the MINOSPRO window
     

     echo KEYUSE=/var/adm/krb5/`kcron -f`
     KEYUSE=/var/adm/krb5/PbQYe9_Fl093H0sO6CRM1Q
     

     
     [Here we're locating your kcron keytab file]
6. MINOSPRO
   • enter the KEYUSE definition command echoed above
     

     KEYUSE=/var/adm/krb5/PbQYe9_Fl093H0sO6CRM1Q
     

7. MINOSPRO Copy the kreymer key on top of the now obsolete minospro keytab.
   

   KEYSHARE=/var/adm/krb5/`kcron -f`
   scp kreymer@gpsn01.fnal.gov:${KEYUSE} ${KEYSHARE}
   

   
   [Here we locate MINOSPRO's keytab, and copy ours over it. We use scp rather than cp so we have permission to do so]
8. MINOSPRO
   • Test the keytab, see that we have kreymer's cron principal
     

     KEYTAB=/var/adm/krb5/`kcron -f`
     KEYUSER=`klist -k ${KEYTAB} | grep FNAL.GOV | cut -c 5- | cut -f 1 -d /`
     kinit -5 -A  -kt ${KEYTAB} ${KEYUSER}/cron/`hostname`@FNAL.GOV
     klist -f
     Ticket cache: FILE:/tmp/krb5cc_42411_ZneQA30945
     Default principal: kreymer/cron/gpsn01.fnal.gov@FNAL.GOV
     
     Valid starting     Expires            Service principal
     10/11/13 09:13:56  10/11/13 19:13:56  krbtgt/FNAL.GOV@FNAL.GOV
     ...
     

     
     Once again, we are checking that
     • we don't get an error message
     • the Default principal: line has your username/cron/hostname in it
     • the ticket expiration date is in the future
       [Here we're using kinit instead of kcron, because kcron will only look for MINOSPRO/cron/...@FNAL.GOV and not kreymer/cron/...@FNAL.GOV]
9. MINOSPRO
   • Verify that we can get a production proxy for grid submission
     

     /scratch/grid/kproxy minos Production  # 
     /scratch/grid/kproxy -i
     ...
     lrwxrwxrwx 1 minospro gpcf   65 Oct 11 09:14 /scratch/minospro/grid/minospro.minos.production.proxy -> /scratch/minospro/grid/minospro.minos.production.proxy.2013101109
     -rw------- 1 minospro gpcf 7004 Oct 11 09:14 /scratch/minospro/grid/minospro.minos.production.proxy.2013101109
     lrwxrwxrwx 1 minospro gpcf   54 Oct 11 09:14 /scratch/minospro/grid/minospro.production.proxy -> /scratch/minospro/grid/minospro.minos.production.proxy
     ...
     /scratch/minospro/grid/minospro.minos.production.proxy
     attribute : /fermilab/minos/Role=Production/Capability=NULL
     Valid proxy expires in 43198 seconds (11 hours)
     Valid proxy expires at Fri Oct 11 21:14:42 CDT 2013
     
     

10. MINOSPRO
    • use cron to automate the process --
      Run crontab -e to edit the crontab for your shared account, and add lines:
      

      1 1-23/2 * * * /scratch/grid/kproxy minos Production
      1 1-23/2 * * * /scratch/grid/kproxy minos Analysis
      

      
      (one for each role needed in the account)

You should now have a proxy refreshed automatically.