Project

General

Profile

SHAREDJOBSUBSETUP » History » Version 6

Arthur Kreymer, 10/29/2014 03:09 PM
KEYUSER needs head -1 filter for keytabs created since the 2014/10/11 Fermilab KDC upgrade

1 1 Katherine Lato
h1. SHAREDJOBSUBSETUP
2 1 Katherine Lato
3 1 Katherine Lato
h1. Setting up keytabs and proxies for jobsub in shared accounts
4 1 Katherine Lato
5 1 Katherine Lato
In certain cases we use shared accounts for production tasks:
6 1 Katherine Lato
* production grid job submission
7 1 Katherine Lato
* management of SAM metadata
8 1 Katherine Lato
* data handling tasks
9 1 Katherine Lato
10 1 Katherine Lato
Because Fermigrid jobs and other tasks require Grid proxies tied to an individual,
11 1 Katherine Lato
we need to make that individual's proxies available to the shared account.
12 1 Katherine Lato
13 1 Katherine Lato
The recommended technique is to copy a designated individual's keytab file into the shared account.
14 1 Katherine Lato
We place it in the path normally used by kcron, so that scripts can be written to run in both shared and individual accounts.
15 1 Katherine Lato
The method of generating and copying the keytab seems arcane and complicated.
16 1 Katherine Lato
There are reasons for each of the steps, so do everything described here !
17 1 Katherine Lato
18 1 Katherine Lato
*In the following discussion we describe copying the kreymer keytab to minospro on gpsn01.fnal.gov .*
19 1 Katherine Lato
20 5 Arthur Kreymer
We want to copy the kreymer kcron keytab file into the corresponding place in minospro.
21 1 Katherine Lato
* The name of each account's kcron keytab file under /var/adm/krb5 is given by   kcron -f .
22 1 Katherine Lato
* File permissions prohibit the direct creation of the minospro keytab file.
23 5 Arthur Kreymer
* Run kcroninit under the minospro account to create a minospro@FNAL.GOV keytab (give a fake password)
24 5 Arthur Kreymer
* Copy the kreymer account keytab into the minospro keytab file
25 1 Katherine Lato
26 1 Katherine Lato
# kreymer opens two windows 
27 1 Katherine Lato
** *_MINOSPRO_* ssh minospro@gpsn01.fnal.gov
28 1 Katherine Lato
** *_KREYMER_* ssh kreymer@gpsn01.fnal.gov
29 1 Katherine Lato
# *_MINOSPRO_*
30 5 Arthur Kreymer
** Create an invalid kcron keytab to be overwritten.
31 1 Katherine Lato
<pre>
32 1 Katherine Lato
kcroninit
33 5 Arthur Kreymer
...
34 5 Arthur Kreymer
Are you on a secure channel?  (default = y): y
35 5 Arthur Kreymer
What is your kerberos principal (default = kreymer@FNAL.GOV): 
36 5 Arthur Kreymer
Enter the password for minospro@FNAL.GOV: junq
37 1 Katherine Lato
</pre>
38 1 Katherine Lato
_(info) [we do this so that the kcron keytab file for MINOSPRO gets created with the right permissions and we can update it later]_
39 1 Katherine Lato
# *_KREYMER_*
40 5 Arthur Kreymer
** Verify that the existing kreymer keytab works with kcron
41 1 Katherine Lato
<pre>
42 1 Katherine Lato
kcron klist
43 1 Katherine Lato
44 1 Katherine Lato
Ticket cache: FILE:/tmp/krb5cc_1060_IVnxb30674
45 1 Katherine Lato
Default principal: kreymer/cron/gpsn01.fnal.gov@FNAL.GOV
46 1 Katherine Lato
47 1 Katherine Lato
Valid starting     Expires            Service principal
48 1 Katherine Lato
10/11/13 09:09:09  10/11/13 19:09:09  krbtgt/FNAL.GOV@FNAL.GOV
49 1 Katherine Lato
...
50 1 Katherine Lato
</pre>
51 1 Katherine Lato
The important bits here are that you
52 1 Katherine Lato
#** don't get an error message
53 1 Katherine Lato
#** the Default principal: line has your username/cron/hostname in it
54 1 Katherine Lato
#** the ticket expiration date is in the future
55 1 Katherine Lato
# *_KREYMER_*
56 1 Katherine Lato
** Generate a command to be entered in the  *_MINOSPRO_* window
57 1 Katherine Lato
<pre>
58 1 Katherine Lato
echo KEYUSE=/var/adm/krb5/`kcron -f`
59 1 Katherine Lato
KEYUSE=/var/adm/krb5/PbQYe9_Fl093H0sO6CRM1Q
60 1 Katherine Lato
</pre>
61 1 Katherine Lato
_(info) [Here we're locating your kcron keytab file]_
62 5 Arthur Kreymer
# *_MINOSPRO_* Copy the kreymer keytab into of the invalid minospro keytab.
63 1 Katherine Lato
<pre>
64 1 Katherine Lato
KEYUSE=/var/adm/krb5/PbQYe9_Fl093H0sO6CRM1Q
65 1 Katherine Lato
KEYSHARE=/var/adm/krb5/`kcron -f`
66 1 Katherine Lato
scp kreymer@gpsn01.fnal.gov:${KEYUSE} ${KEYSHARE}
67 1 Katherine Lato
</pre>
68 1 Katherine Lato
_(info) [Here we locate MINOSPRO's keytab, and copy ours over it. We use scp rather than cp so we have permission to do so]_
69 1 Katherine Lato
# *_MINOSPRO_* 
70 1 Katherine Lato
** Test the keytab, see that we have kreymer's cron principal
71 1 Katherine Lato
<pre>
72 1 Katherine Lato
KEYTAB=/var/adm/krb5/`kcron -f`
73 6 Arthur Kreymer
KEYUSER=`klist -k ${KEYTAB} | grep FNAL.GOV | cut -c 5- | cut -f 1 -d / | head -1`
74 1 Katherine Lato
kinit -5 -A  -kt ${KEYTAB} ${KEYUSER}/cron/`hostname`@FNAL.GOV
75 1 Katherine Lato
klist -f
76 1 Katherine Lato
Ticket cache: FILE:/tmp/krb5cc_42411_ZneQA30945
77 1 Katherine Lato
Default principal: kreymer/cron/gpsn01.fnal.gov@FNAL.GOV
78 1 Katherine Lato
79 1 Katherine Lato
Valid starting     Expires            Service principal
80 1 Katherine Lato
10/11/13 09:13:56  10/11/13 19:13:56  krbtgt/FNAL.GOV@FNAL.GOV
81 1 Katherine Lato
...
82 1 Katherine Lato
</pre>
83 1 Katherine Lato
Once again, we are checking that
84 1 Katherine Lato
#** we don't get an error message
85 1 Katherine Lato
#** the Default principal: line has your username/cron/hostname in it
86 1 Katherine Lato
#** the ticket expiration date is in the future
87 1 Katherine Lato
_(info) [Here we're using kinit instead of kcron, because kcron will only look for MINOSPRO/cron/...@FNAL.GOV and not kreymer/cron/...@FNAL.GOV]_
88 1 Katherine Lato
# *_MINOSPRO_* 
89 1 Katherine Lato
** Verify that we can get a production proxy for grid submission
90 1 Katherine Lato
<pre>
91 1 Katherine Lato
/scratch/grid/kproxy minos Production  # 
92 1 Katherine Lato
/scratch/grid/kproxy -i
93 1 Katherine Lato
...
94 1 Katherine Lato
lrwxrwxrwx 1 minospro gpcf   65 Oct 11 09:14 /scratch/minospro/grid/minospro.minos.production.proxy -> /scratch/minospro/grid/minospro.minos.production.proxy.2013101109
95 1 Katherine Lato
-rw------- 1 minospro gpcf 7004 Oct 11 09:14 /scratch/minospro/grid/minospro.minos.production.proxy.2013101109
96 1 Katherine Lato
lrwxrwxrwx 1 minospro gpcf   54 Oct 11 09:14 /scratch/minospro/grid/minospro.production.proxy -> /scratch/minospro/grid/minospro.minos.production.proxy
97 1 Katherine Lato
...
98 1 Katherine Lato
/scratch/minospro/grid/minospro.minos.production.proxy
99 1 Katherine Lato
attribute : /fermilab/minos/Role=Production/Capability=NULL
100 1 Katherine Lato
Valid proxy expires in 43198 seconds (11 hours)
101 1 Katherine Lato
Valid proxy expires at Fri Oct 11 21:14:42 CDT 2013
102 1 Katherine Lato
103 1 Katherine Lato
104 1 Katherine Lato
</pre>
105 1 Katherine Lato
# *_MINOSPRO_* 
106 1 Katherine Lato
** use cron to automate the process -- 
107 1 Katherine Lato
Run @crontab -e@ to edit the crontab for  your shared account, and add lines:
108 1 Katherine Lato
<pre>
109 1 Katherine Lato
1 1-23/2 * * * /scratch/grid/kproxy minos Production
110 1 Katherine Lato
1 1-23/2 * * * /scratch/grid/kproxy minos Analysis
111 1 Katherine Lato
</pre>
112 1 Katherine Lato
(one for each role needed in the account) 
113 1 Katherine Lato
You should now have a proxy refreshed automatically.
114 4 Arthur Kreymer
# *_JOBSUB CLIENT_*
115 2 Arthur Kreymer
When using the new jobsub_client in 2014, you do not run kproxy.
116 2 Arthur Kreymer
Instead, you need to have an active kerberos ticket.
117 3 Arthur Kreymer
Get this when needed within your cron scripts with
118 2 Arthur Kreymer
<pre>
119 2 Arthur Kreymer
export KRB5CCNAME="FILE:/tmp/krb5cc_jobsub_`whoami`_${$}"
120 2 Arthur Kreymer
KEYTAB=/var/adm/krb5/`kcron -f`
121 6 Arthur Kreymer
KEYUSER=`klist -k ${KEYTAB} | grep FNAL.GOV | cut -c 5- | cut -f 1 -d / | head -1`
122 2 Arthur Kreymer
kinit -5 -A  -kt ${KEYTAB} ${KEYUSER}/cron/`hostname`@FNAL.GOV
123 2 Arthur Kreymer
124 2 Arthur Kreymer
</pre>