Bug #10581
Vulcan is constantly updating the mapping files
0%
Description
This is moved from SNOW ticket.
On Jun 12, 2015, at 5:02 PM, Fermilab Service Desk <fermi@service-now.com<mailto:fermi@service-now.com>> wrote:
Incident INC000000557763<https://fermi.service-now.com/nav_to.do?uri=incident.do%3Fsys_id=d86b596a6f2c0240000131012e3ee409%26sysparm_stack=incident_list.do%3Fsysparm_query=active=true> comments have been updated.
Last update:
2015-06-12 17:02:14 CDT - Gerard Bernabeu Altayo (Additional comments)
reply from: gerard1@fnal.gov<mailto:gerard1@fnal.gov>
Hi Chih-Hao,
there is an actual problem, it's not causing a widespread issue but there
is a problem, at a minimum the user below has a changing user mapping:
-mapping "/DC=ch/DC=cern/OU=Organic
Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool" uscms6227
+mapping "/DC=ch/DC=cern/OU=Organic
Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool" uscms6224
There are a couple more users like this, this is also triggering puppet
alarms considering some nodes as tangled.
This is not good and may point to further issues (with GUMS?).
thanks,
Gerard
2015-06-12 10:57:50 Gerard Bernabeu Altayo Changed: Additional comments (customer communication)
Here are the changes that are probably going back and forth at each run:
Info: Caching catalog for cmssrmdisk.fnal.gov
Info: Applying configuration version '1434124533'
Notice: /Stage[main]/P_krb5::Rootusers/K5login[/root/.k5login]/principals: principals changed ['bobyshev@FNAL.GOV', 'burt@FNAL.GOV', 'catalind@FNAL.GOV', 'dmason@FNAL.GOV', 'fagan@FNAL.GOV', 'gerard1@FNAL.GOV', 'host/cmssrv28.fnal.gov@FNAL.GOV', 'huangch@FNAL.GOV', 'klarson1@FNAL.GOV', 'lisa@FNAL.GOV', 'litvinse@FNAL.GOV', 'merina@FNAL.GOV', 'natasha@FNAL.GOV', 'njp@FNAL.GOV', 'richt@FNAL.GOV', 'tiradani@FNAL.GOV', 'tskirvin@FNAL.GOV'] to 'bobyshev@FNAL.GOV burt@FNAL.GOV catalind@FNAL.GOV dmason@FNAL.GOV fagan@FNAL.GOV gerard1@FNAL.GOV host/cmssrv28.fnal.gov@FNAL.GOV huangch@FNAL.GOV klarson1@FNAL.GOV lisa@FNAL.GOV merina@FNAL.GOV natasha@FNAL.GOV njp@FNAL.GOV richt@FNAL.GOV tiradani@FNAL.GOV tskirvin@FNAL.GOV'
Notice: /Stage[main]/Vulcan::Dcache/File[/etc/dcache/dcache.kpwd]/content:
--- /etc/dcache/dcache.kpwd 2015-06-12 09:48:43.235849093 -0500
+++ /tmp/puppet-file20150612-32744-jp7o9i-0 2015-06-12 10:56:03.479370997 -0500@ -2777,7 +2777,7
@
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahasan/CN=760824/CN=Areeg Hasan" uscms5966
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahassan/CN=727546/CN=Asmaa Fawzi Ali Hassan" uscms3718
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahawkins/CN=683912/CN=Anthony Hawkins" atlas
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" uscms6231
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" uscms6223
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=aheintz/CN=760077/CN=Aneesh M Heintz" uscms5889
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahernand/CN=718787/CN=Alberto Hernandez Almada" uscms3630
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahlinist/CN=739241/CN=Anton Hlinisty" uscms4732@ -2984,7 +2984,7
@
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=battilan/CN=644924/CN=Carlo Battilana" uscms2636
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbarnett/CN=372728/CN=Bruce M Barnett" atlas
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbarrefo/CN=752354/CN=Bjorn Peter Barrefors" uscms5468
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool" uscms6227
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool" uscms6224
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbetchar/CN=671708/CN=Burt Betchart" bbetchar
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbilin/CN=705521/CN=Bugra Bilin" uscms4764
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbockelm/CN=659869/CN=Brian Paul Bockelman" uscms3859@ -3892,7 +3892,7
@
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmejiagu/CN=738553/CN=Jhovanny Andres Mejia Guisao" uscms4599
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhernan/CN=763451/CN=Jonathan Matthew Hernandez" uscms6206
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Hogan" jmanagan
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" uscms6223
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" uscms6230
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmolinai/CN=671301/CN=Jorge Andres Molina Insfran" uscms5608
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmonk/CN=664162/CN=James William Monk" atlas
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmonkevi/CN=759961/CN=Jonas Monkevicius" uscms5876@ -4297,7 +4297,7
@
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=mendez/CN=490587/CN=Hector Mendez" mendez
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=mendo/CN=711096/CN=Masaki Endo" atlas
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meneguz/CN=394388/CN=Anna Meneguzzo" uscms2810
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" uscms6224
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" uscms6228
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meridian/CN=555986/CN=Paolo Meridiani" meridian
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=merina/CN=673555/CN=Merina Albert" uscms5650
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=merkelp/CN=631815/CN=Petra Merkel" uscms1570@ -4716,7 +4716,7
@
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=rharris/CN=514003/CN=Robert Harris" uscms5602
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=rhawking/CN=399511/CN=Richard Hawkings" atlas
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=rheller/CN=743534/CN=Ryan Edward Heller" uscms5977
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=riahi/CN=672488/CN=Hassen Riahi" uscms6225
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=riahi/CN=672488/CN=Hassen Riahi" uscms6229
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ribon/CN=571572/CN=Alberto Ribon" geant4
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ric/CN=477544/CN=Riccardo Paramatti" uscms646
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ricab/CN=683129/CN=Ricardo Lince Amaral Farto E Abreu" atlas@ -54174,26 +54174,28
@
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=sicheng/CN=748935/CN=Sicheng Wang
login uscms6223 read-only 1123 1999 / /dcache /dcache
- /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan
+ /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi
login uscms6224 read-only 1123 1999 / /dcache /dcache
- /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola
+ /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool
login uscms6225 read-only 1123 1999 / /dcache /dcache
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai
- /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=riahi/CN=672488/CN=Hassen Riahi
login uscms6226 read-only 1123 1999 / /dcache /dcache
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=strolog/CN=679949/CN=John Strologas
login uscms6227 read-only 1123 1999 / /dcache /dcache /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool
login uscms6228 read-only 1123 1999 / /dcache /dcache
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola
+login uscms6229 read-only 1123 1999 / /dcache /dcache
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=riahi/CN=672488/CN=Hassen Riahi
login uscms623 read-only 1123 1999 / /dcache /dcache
/DC=es/DC=irisgrid/O=ciemat/CN=isabel.josa
login uscms6231 read-only 1123 1999 / /dcache /dcache /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi
login uscms6230 read-only 1123 1999 / /dcache /dcache
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan
login uscms6234 read-only 1123 1999 / /dcache /dcache
/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Gerard Bernabeu Altayo/CN=UID:gerard1
Info: /Stage[main]/Vulcan::Dcache/File[/etc/dcache/dcache.kpwd]: Filebucketed /etc/dcache/dcache.kpwd to main with sum b444663d26cbfed3997ee192f3c33e1a
Notice: /Stage[main]/Vulcan::Dcache/File[/etc/dcache/dcache.kpwd]/content: content changed '{md5}b444663d26cbfed3997ee192f3c33e1a' to '{md5}3a3b60952b73a01c21d15891dca4e8e2'
Info: FileBucket got a duplicate file {md5}3692a554bef3dc154a87a0a54a6e97b3
Info: /Stage[main]/P_fact::Puppet/P_fact::String[puppetenvironment]/File[/etc/facter/facts.d/puppetenvironment.txt]: Filebucketed /etc/facter/facts.d/puppetenvironment.txt to main with sum 3692a554bef3dc154a87a0a54a6e97b3
Notice: /Stage[main]/P_fact::Puppet/P_fact::String[puppetenvironment]/File[/etc/facter/facts.d/puppetenvironment.txt]/content: content changed '{md5}3692a554bef3dc154a87a0a54a6e97b3' to '{md5}4b109ca6e8287d4d28211b8e268bbf05'
Info: /Stage[main]/Motd::Warning/Motd[/etc/motd]/File[/etc/motd]: Filebucketed /etc/motd to main with sum 9a6487e55473560a7e9286c21abe5b8b
Notice: /Stage[main]/Motd::Warning/Motd[/etc/motd]/File[/etc/motd]/content: content changed '{md5}9a6487e55473560a7e9286c21abe5b8b' to '{md5}80e7b70c53600a1f9c94e4685dd5ebdc'
Notice: /Stage[main]/Vulcan::Dcache/File[/etc/grid-security/grid-vorolemap]/content:
--- /etc/grid-security/grid-vorolemap 2015-06-12 09:48:53.147488552 0500"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" "/cms" uscms6223
++ /tmp/puppet-file20150612-32744-1vebshr-0 2015-06-12 10:56:15.991913336 -0500@ -3955,8 +3955,8
@
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahassan/CN=727546/CN=Asmaa Fawzi Ali Hassan" "/cms" uscms3718
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahassan/CN=727546/CN=Asmaa Fawzi Ali Hassan" uscms3718
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahawkins/CN=683912/CN=Anthony Hawkins" "/atlas" atlas"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" uscms6223"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai" "/cms" uscms6232
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" "/cms" uscms6231
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" uscms6231
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=aheintz/CN=760077/CN=Aneesh M Heintz" "/cms" uscms5889
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=aheintz/CN=760077/CN=Aneesh M Heintz" "/cms/Role=cmsuser" uscms5889
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=aheintz/CN=760077/CN=Aneesh M Heintz" "/cms/uscms/Role=cmsuser" uscms5889@ -5546,8 +5546,8
@
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fanzago/CN=610896/CN=Federica Fanzago" uscms706
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=farida/CN=518177/CN=Farida Fassi" "/atlas" atlas
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fassoul/CN=390926/CN=Dimitris Fassouliotis" "/atlas" atlas"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai" uscms6232"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" "/cms" uscms6230
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai" "/cms" uscms6225
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai" uscms6225
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=favaro/CN=698706/CN=Carlotta Favaro" "/cms" uscms2711
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=favaro/CN=698706/CN=Carlotta Favaro" "/cms/Role=cmsuser" uscms2711
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=favaro/CN=698706/CN=Carlotta Favaro" uscms2711@ -6563,8 +6563,8
@
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhernan/CN=763451/CN=Jonathan Matthew Hernandez" uscms6206
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Hogan" "/cms" jmanagan
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Hogan" jmanagan"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" uscms6230"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" "/cms" uscms6228
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" "/cms" uscms6223
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" uscms6223
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmolinai/CN=671301/CN=Jorge Andres Molina Insfran" "/cms" uscms5608
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmolinai/CN=671301/CN=Jorge Andres Molina Insfran" uscms5608
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmonk/CN=664162/CN=James William Monk" "/atlas" atlas@ -7474,8 +7474,8
@
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=mendo/CN=711096/CN=Masaki Endo" "/atlas" atlas
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meneguz/CN=394388/CN=Anna Meneguzzo" "/cms" uscms2810
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meneguz/CN=394388/CN=Anna Meneguzzo" uscms2810
-"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" uscms6228
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" "/cms" uscms6224
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" uscms6224
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meridian/CN=555986/CN=Paolo Meridiani" "/cms" meridian
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meridian/CN=555986/CN=Paolo Meridiani" meridian
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=merina/CN=673555/CN=Merina Albert" "/cms" uscms5650@ -16496,6 +16496,8
@
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/nova" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/nova/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/numix" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/numix/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/uboone" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/uboone/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Dalath rachitha A. Mendis/CN=UID:dmendis" "/fermilab" fermilab@ -33625,6 +33627,8
@
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/nova" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/nova/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/numix" fermilab
+"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/numix/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/uboone" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/uboone/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Dalath rachitha A. Mendis/CN=UID:dmendis" "/fermilab" fermilab
Info: /Stage[main]/Vulcan::Dcache/File[/etc/grid-security/grid-vorolemap]: Filebucketed /etc/grid-security/grid-vorolemap to main with sum 584c5bc1a3ac6aa2cec42dc2fb5d4bc9
Notice: /Stage[main]/Vulcan::Dcache/File[/etc/grid-security/grid-vorolemap]/content: content changed '{md5}584c5bc1a3ac6aa2cec42dc2fb5d4bc9' to '{md5}31392937823d1fdfa23e6736a5e33815'