Project

General

Profile

SHAREDJOBSUBSETUP » History » Version 8

Arthur Kreymer, 12/03/2014 11:13 AM

1 1 Katherine Lato
h1. SHAREDJOBSUBSETUP
2 1 Katherine Lato
3 1 Katherine Lato
h1. Setting up keytabs and proxies for jobsub in shared accounts
4 1 Katherine Lato
5 1 Katherine Lato
In certain cases we use shared accounts for production tasks:
6 1 Katherine Lato
* production grid job submission
7 1 Katherine Lato
* management of SAM metadata
8 1 Katherine Lato
* data handling tasks
9 1 Katherine Lato
10 1 Katherine Lato
Because Fermigrid jobs and other tasks require Grid proxies tied to an individual,
11 1 Katherine Lato
we need to make that individual's proxies available to the shared account.
12 1 Katherine Lato
13 1 Katherine Lato
The recommended technique is to copy a designated individual's keytab file into the shared account.
14 1 Katherine Lato
We place it in the path normally used by kcron, so that scripts can be written to run in both shared and individual accounts.
15 1 Katherine Lato
The method of generating and copying the keytab seems arcane and complicated.
16 1 Katherine Lato
There are reasons for each of the steps, so do everything described here !
17 1 Katherine Lato
18 1 Katherine Lato
*In the following discussion we describe copying the kreymer keytab to minospro on gpsn01.fnal.gov .*
19 1 Katherine Lato
20 5 Arthur Kreymer
We want to copy the kreymer kcron keytab file into the corresponding place in minospro.
21 1 Katherine Lato
* The name of each account's kcron keytab file under /var/adm/krb5 is given by   kcron -f .
22 1 Katherine Lato
* File permissions prohibit the direct creation of the minospro keytab file.
23 5 Arthur Kreymer
* Run kcroninit under the minospro account to create a minospro@FNAL.GOV keytab (give a fake password)
24 5 Arthur Kreymer
* Copy the kreymer account keytab into the minospro keytab file
25 1 Katherine Lato
26 1 Katherine Lato
# kreymer opens two windows 
27 1 Katherine Lato
** *_MINOSPRO_* ssh minospro@gpsn01.fnal.gov
28 1 Katherine Lato
** *_KREYMER_* ssh kreymer@gpsn01.fnal.gov
29 1 Katherine Lato
# *_MINOSPRO_*
30 5 Arthur Kreymer
** Create an invalid kcron keytab to be overwritten.
31 1 Katherine Lato
<pre>
32 1 Katherine Lato
kcroninit
33 5 Arthur Kreymer
...
34 5 Arthur Kreymer
Are you on a secure channel?  (default = y): y
35 5 Arthur Kreymer
What is your kerberos principal (default = kreymer@FNAL.GOV): 
36 5 Arthur Kreymer
Enter the password for minospro@FNAL.GOV: junq
37 1 Katherine Lato
</pre>
38 1 Katherine Lato
_(info) [we do this so that the kcron keytab file for MINOSPRO gets created with the right permissions and we can update it later]_
39 8 Arthur Kreymer
** Check that the keytab was created
40 8 Arthur Kreymer
<pre>
41 8 Arthur Kreymer
ls -l /var/adm/krb5/`kcron -f`
42 8 Arthur Kreymer
</pre>
43 1 Katherine Lato
# *_KREYMER_*
44 5 Arthur Kreymer
** Verify that the existing kreymer keytab works with kcron
45 1 Katherine Lato
<pre>
46 1 Katherine Lato
kcron klist
47 1 Katherine Lato
48 1 Katherine Lato
Ticket cache: FILE:/tmp/krb5cc_1060_IVnxb30674
49 1 Katherine Lato
Default principal: kreymer/cron/gpsn01.fnal.gov@FNAL.GOV
50 1 Katherine Lato
51 1 Katherine Lato
Valid starting     Expires            Service principal
52 1 Katherine Lato
10/11/13 09:09:09  10/11/13 19:09:09  krbtgt/FNAL.GOV@FNAL.GOV
53 1 Katherine Lato
...
54 1 Katherine Lato
</pre>
55 1 Katherine Lato
The important bits here are that you
56 1 Katherine Lato
#** don't get an error message
57 1 Katherine Lato
#** the Default principal: line has your username/cron/hostname in it
58 1 Katherine Lato
#** the ticket expiration date is in the future
59 1 Katherine Lato
# *_KREYMER_*
60 1 Katherine Lato
** Generate a command to be entered in the  *_MINOSPRO_* window
61 1 Katherine Lato
<pre>
62 1 Katherine Lato
echo KEYUSE=/var/adm/krb5/`kcron -f`
63 1 Katherine Lato
KEYUSE=/var/adm/krb5/PbQYe9_Fl093H0sO6CRM1Q
64 1 Katherine Lato
</pre>
65 1 Katherine Lato
_(info) [Here we're locating your kcron keytab file]_
66 5 Arthur Kreymer
# *_MINOSPRO_* Copy the kreymer keytab into of the invalid minospro keytab.
67 1 Katherine Lato
<pre>
68 1 Katherine Lato
KEYUSE=/var/adm/krb5/PbQYe9_Fl093H0sO6CRM1Q
69 1 Katherine Lato
KEYSHARE=/var/adm/krb5/`kcron -f`
70 1 Katherine Lato
scp kreymer@gpsn01.fnal.gov:${KEYUSE} ${KEYSHARE}
71 1 Katherine Lato
</pre>
72 1 Katherine Lato
_(info) [Here we locate MINOSPRO's keytab, and copy ours over it. We use scp rather than cp so we have permission to do so]_
73 1 Katherine Lato
# *_MINOSPRO_* 
74 1 Katherine Lato
** Test the keytab, see that we have kreymer's cron principal
75 1 Katherine Lato
<pre>
76 1 Katherine Lato
KEYTAB=/var/adm/krb5/`kcron -f`
77 6 Arthur Kreymer
KEYUSER=`klist -k ${KEYTAB} | grep FNAL.GOV | cut -c 5- | cut -f 1 -d / | head -1`
78 1 Katherine Lato
kinit -5 -A  -kt ${KEYTAB} ${KEYUSER}/cron/`hostname`@FNAL.GOV
79 1 Katherine Lato
klist -f
80 1 Katherine Lato
Ticket cache: FILE:/tmp/krb5cc_42411_ZneQA30945
81 1 Katherine Lato
Default principal: kreymer/cron/gpsn01.fnal.gov@FNAL.GOV
82 1 Katherine Lato
83 1 Katherine Lato
Valid starting     Expires            Service principal
84 1 Katherine Lato
10/11/13 09:13:56  10/11/13 19:13:56  krbtgt/FNAL.GOV@FNAL.GOV
85 1 Katherine Lato
...
86 1 Katherine Lato
</pre>
87 1 Katherine Lato
Once again, we are checking that
88 1 Katherine Lato
#** we don't get an error message
89 1 Katherine Lato
#** the Default principal: line has your username/cron/hostname in it
90 1 Katherine Lato
#** the ticket expiration date is in the future
91 1 Katherine Lato
_(info) [Here we're using kinit instead of kcron, because kcron will only look for MINOSPRO/cron/...@FNAL.GOV and not kreymer/cron/...@FNAL.GOV]_
92 1 Katherine Lato
# *_MINOSPRO_* 
93 1 Katherine Lato
** Verify that we can get a production proxy for grid submission
94 1 Katherine Lato
<pre>
95 1 Katherine Lato
/scratch/grid/kproxy minos Production  # 
96 1 Katherine Lato
/scratch/grid/kproxy -i
97 1 Katherine Lato
...
98 1 Katherine Lato
lrwxrwxrwx 1 minospro gpcf   65 Oct 11 09:14 /scratch/minospro/grid/minospro.minos.production.proxy -> /scratch/minospro/grid/minospro.minos.production.proxy.2013101109
99 1 Katherine Lato
-rw------- 1 minospro gpcf 7004 Oct 11 09:14 /scratch/minospro/grid/minospro.minos.production.proxy.2013101109
100 1 Katherine Lato
lrwxrwxrwx 1 minospro gpcf   54 Oct 11 09:14 /scratch/minospro/grid/minospro.production.proxy -> /scratch/minospro/grid/minospro.minos.production.proxy
101 1 Katherine Lato
...
102 1 Katherine Lato
/scratch/minospro/grid/minospro.minos.production.proxy
103 1 Katherine Lato
attribute : /fermilab/minos/Role=Production/Capability=NULL
104 1 Katherine Lato
Valid proxy expires in 43198 seconds (11 hours)
105 1 Katherine Lato
Valid proxy expires at Fri Oct 11 21:14:42 CDT 2013
106 1 Katherine Lato
107 1 Katherine Lato
108 1 Katherine Lato
</pre>
109 1 Katherine Lato
# *_MINOSPRO_* 
110 1 Katherine Lato
** use cron to automate the process -- 
111 1 Katherine Lato
Run @crontab -e@ to edit the crontab for  your shared account, and add lines:
112 1 Katherine Lato
<pre>
113 1 Katherine Lato
1 1-23/2 * * * /scratch/grid/kproxy minos Production
114 1 Katherine Lato
1 1-23/2 * * * /scratch/grid/kproxy minos Analysis
115 1 Katherine Lato
</pre>
116 1 Katherine Lato
(one for each role needed in the account) 
117 1 Katherine Lato
You should now have a proxy refreshed automatically.
118 4 Arthur Kreymer
# *_JOBSUB CLIENT_*
119 2 Arthur Kreymer
When using the new jobsub_client in 2014, you do not run kproxy.
120 2 Arthur Kreymer
Instead, you need to have an active kerberos ticket.
121 3 Arthur Kreymer
Get this when needed within your cron scripts with
122 2 Arthur Kreymer
<pre>
123 2 Arthur Kreymer
export KRB5CCNAME="FILE:/tmp/krb5cc_jobsub_`whoami`_${$}"
124 2 Arthur Kreymer
KEYTAB=/var/adm/krb5/`kcron -f`
125 6 Arthur Kreymer
KEYUSER=`klist -k ${KEYTAB} | grep FNAL.GOV | cut -c 5- | cut -f 1 -d / | head -1`
126 2 Arthur Kreymer
kinit -5 -A  -kt ${KEYTAB} ${KEYUSER}/cron/`hostname`@FNAL.GOV
127 2 Arthur Kreymer
128 2 Arthur Kreymer
</pre>
129 7 Arthur Kreymer
130 7 Arthur Kreymer
h2. Using the shared account for samweb access
131 7 Arthur Kreymer
132 7 Arthur Kreymer
Using the same procedure to get a shared ticket,
133 7 Arthur Kreymer
we can also access samweb from a shared account.
134 7 Arthur Kreymer
135 7 Arthur Kreymer
# Register the shared idenity with samweb, mapping to an appropriate account
136 7 Arthur Kreymer
<pre>
137 7 Arthur Kreymer
https://samweb.fnal.gov:8483/sam/minos/admin/users
138 7 Arthur Kreymer
    Added the account
139 7 Arthur Kreymer
    Added an appropriate Grid subject to that account, like
140 7 Arthur Kreymer
/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=minos27.fnal.gov/CN=cron/CN=Arthur E. Kreymer/CN=UID:kreymer
141 7 Arthur Kreymer
</pre>
142 7 Arthur Kreymer
# Get a ticket as with jobsub, with a unique ticket file
143 7 Arthur Kreymer
<pre>
144 7 Arthur Kreymer
export KRB5CCNAME="FILE:/tmp/krb5cc_sam_`whoami`_${$}" 
145 7 Arthur Kreymer
KEYTAB=/var/adm/krb5/`kcron -f`
146 7 Arthur Kreymer
KEYUSER=`klist -k ${KEYTAB} | grep FNAL.GOV | cut -c 5- | cut -f 1 -d / | head -1`
147 7 Arthur Kreymer
kinit -5 -A  -kt ${KEYTAB} ${KEYUSER}/cron/`hostname`@FNAL.GOV
148 7 Arthur Kreymer
getcert -s
149 7 Arthur Kreymer
</pre>