Project

General

Profile

Bug #10581

Vulcan is constantly updating the mapping files

Added by Chih-Hao Huang about 5 years ago.

Status:
Assigned
Priority:
Normal
Assignee:
Chih-Hao Huang
Start date:
10/19/2015
Due date:
% Done:

0%

Estimated time:
Duration:

Description

This is moved from SNOW ticket.

On Jun 12, 2015, at 5:02 PM, Fermilab Service Desk <<mailto:>> wrote:

Incident INC000000557763<https://fermi.service-now.com/nav_to.do?uri=incident.do%3Fsys_id=d86b596a6f2c0240000131012e3ee409%26sysparm_stack=incident_list.do%3Fsysparm_query=active=true> comments have been updated.

Last update:
2015-06-12 17:02:14 CDT - Gerard Bernabeu Altayo (Additional comments)
reply from: <mailto:>

Hi Chih-Hao,

there is an actual problem, it's not causing a widespread issue but there
is a problem, at a minimum the user below has a changing user mapping:

-mapping "/DC=ch/DC=cern/OU=Organic
Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool" uscms6227
+mapping "/DC=ch/DC=cern/OU=Organic
Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool" uscms6224

There are a couple more users like this, this is also triggering puppet
alarms considering some nodes as tangled.

This is not good and may point to further issues (with GUMS?).

thanks,
Gerard

2015-06-12 10:57:50 Gerard Bernabeu Altayo Changed: Additional comments (customer communication)
Here are the changes that are probably going back and forth at each run:

Info: Caching catalog for cmssrmdisk.fnal.gov
Info: Applying configuration version '1434124533'
Notice: /Stage[main]/P_krb5::Rootusers/K5login[/root/.k5login]/principals: principals changed ['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', ''] to ' '
Notice: /Stage[main]/Vulcan::Dcache/File[/etc/dcache/dcache.kpwd]/content:
--- /etc/dcache/dcache.kpwd 2015-06-12 09:48:43.235849093 -0500
+++ /tmp/puppet-file20150612-32744-jp7o9i-0 2015-06-12 10:56:03.479370997 -0500
@ -2777,7 +2777,7 @
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahasan/CN=760824/CN=Areeg Hasan" uscms5966
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahassan/CN=727546/CN=Asmaa Fawzi Ali Hassan" uscms3718
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahawkins/CN=683912/CN=Anthony Hawkins" atlas
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" uscms6231
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" uscms6223
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=aheintz/CN=760077/CN=Aneesh M Heintz" uscms5889
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahernand/CN=718787/CN=Alberto Hernandez Almada" uscms3630
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahlinist/CN=739241/CN=Anton Hlinisty" uscms4732
@ -2984,7 +2984,7 @
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=battilan/CN=644924/CN=Carlo Battilana" uscms2636
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbarnett/CN=372728/CN=Bruce M Barnett" atlas
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbarrefo/CN=752354/CN=Bjorn Peter Barrefors" uscms5468
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool" uscms6227
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool" uscms6224
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbetchar/CN=671708/CN=Burt Betchart" bbetchar
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbilin/CN=705521/CN=Bugra Bilin" uscms4764
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbockelm/CN=659869/CN=Brian Paul Bockelman" uscms3859
@ -3892,7 +3892,7 @
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmejiagu/CN=738553/CN=Jhovanny Andres Mejia Guisao" uscms4599
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhernan/CN=763451/CN=Jonathan Matthew Hernandez" uscms6206
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Hogan" jmanagan
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" uscms6223
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" uscms6230
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmolinai/CN=671301/CN=Jorge Andres Molina Insfran" uscms5608
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmonk/CN=664162/CN=James William Monk" atlas
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmonkevi/CN=759961/CN=Jonas Monkevicius" uscms5876
@ -4297,7 +4297,7 @
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=mendez/CN=490587/CN=Hector Mendez" mendez
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=mendo/CN=711096/CN=Masaki Endo" atlas
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meneguz/CN=394388/CN=Anna Meneguzzo" uscms2810
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" uscms6224
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" uscms6228
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meridian/CN=555986/CN=Paolo Meridiani" meridian
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=merina/CN=673555/CN=Merina Albert" uscms5650
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=merkelp/CN=631815/CN=Petra Merkel" uscms1570
@ -4716,7 +4716,7 @
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=rharris/CN=514003/CN=Robert Harris" uscms5602
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=rhawking/CN=399511/CN=Richard Hawkings" atlas
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=rheller/CN=743534/CN=Ryan Edward Heller" uscms5977
-mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=riahi/CN=672488/CN=Hassen Riahi" uscms6225
+mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=riahi/CN=672488/CN=Hassen Riahi" uscms6229
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ribon/CN=571572/CN=Alberto Ribon" geant4
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ric/CN=477544/CN=Riccardo Paramatti" uscms646
mapping "/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ricab/CN=683129/CN=Ricardo Lince Amaral Farto E Abreu" atlas
@ -54174,26 +54174,28 @
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=sicheng/CN=748935/CN=Sicheng Wang

login uscms6223 read-only 1123 1999 / /dcache /dcache
- /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan
+ /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi

login uscms6224 read-only 1123 1999 / /dcache /dcache
- /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola
+ /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool

login uscms6225 read-only 1123 1999 / /dcache /dcache
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai
- /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=riahi/CN=672488/CN=Hassen Riahi

login uscms6226 read-only 1123 1999 / /dcache /dcache
/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=strolog/CN=679949/CN=John Strologas

login uscms6227 read-only 1123 1999 / /dcache /dcache
 /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=bbatool/CN=766833/CN=Binish Batool
login uscms6228 read-only 1123 1999 / /dcache /dcache
 /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola

+login uscms6229 read-only 1123 1999 / /dcache /dcache
 /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=riahi/CN=672488/CN=Hassen Riahi

login uscms623 read-only 1123 1999 / /dcache /dcache
/DC=es/DC=irisgrid/O=ciemat/CN=isabel.josa

login uscms6231 read-only 1123 1999 / /dcache /dcache
 /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi
login uscms6230 read-only 1123 1999 / /dcache /dcache
 /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan

login uscms6234 read-only 1123 1999 / /dcache /dcache
/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Gerard Bernabeu Altayo/CN=UID:gerard1

Info: /Stage[main]/Vulcan::Dcache/File[/etc/dcache/dcache.kpwd]: Filebucketed /etc/dcache/dcache.kpwd to main with sum b444663d26cbfed3997ee192f3c33e1a
Notice: /Stage[main]/Vulcan::Dcache/File[/etc/dcache/dcache.kpwd]/content: content changed '{md5}b444663d26cbfed3997ee192f3c33e1a' to '{md5}3a3b60952b73a01c21d15891dca4e8e2'
Info: FileBucket got a duplicate file {md5}3692a554bef3dc154a87a0a54a6e97b3
Info: /Stage[main]/P_fact::Puppet/P_fact::String[puppetenvironment]/File[/etc/facter/facts.d/puppetenvironment.txt]: Filebucketed /etc/facter/facts.d/puppetenvironment.txt to main with sum 3692a554bef3dc154a87a0a54a6e97b3
Notice: /Stage[main]/P_fact::Puppet/P_fact::String[puppetenvironment]/File[/etc/facter/facts.d/puppetenvironment.txt]/content: content changed '{md5}3692a554bef3dc154a87a0a54a6e97b3' to '{md5}4b109ca6e8287d4d28211b8e268bbf05'
Info: /Stage[main]/Motd::Warning/Motd[/etc/motd]/File[/etc/motd]: Filebucketed /etc/motd to main with sum 9a6487e55473560a7e9286c21abe5b8b
Notice: /Stage[main]/Motd::Warning/Motd[/etc/motd]/File[/etc/motd]/content: content changed '{md5}9a6487e55473560a7e9286c21abe5b8b' to '{md5}80e7b70c53600a1f9c94e4685dd5ebdc'
Notice: /Stage[main]/Vulcan::Dcache/File[/etc/grid-security/grid-vorolemap]/content:
--- /etc/grid-security/grid-vorolemap 2015-06-12 09:48:53.147488552 0500
++ /tmp/puppet-file20150612-32744-1vebshr-0 2015-06-12 10:56:15.991913336 -0500
@ -3955,8 +3955,8 @
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahassan/CN=727546/CN=Asmaa Fawzi Ali Hassan" "/cms" uscms3718
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahassan/CN=727546/CN=Asmaa Fawzi Ali Hassan" uscms3718
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahawkins/CN=683912/CN=Anthony Hawkins" "/atlas" atlas
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" "/cms" uscms6223
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" uscms6223
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" "/cms" uscms6231
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=ahazi/CN=676282/CN=Andras Hazi" uscms6231
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=aheintz/CN=760077/CN=Aneesh M Heintz" "/cms" uscms5889
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=aheintz/CN=760077/CN=Aneesh M Heintz" "/cms/Role=cmsuser" uscms5889
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=aheintz/CN=760077/CN=Aneesh M Heintz" "/cms/uscms/Role=cmsuser" uscms5889
@ -5546,8 +5546,8 @
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fanzago/CN=610896/CN=Federica Fanzago" uscms706
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=farida/CN=518177/CN=Farida Fassi" "/atlas" atlas
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fassoul/CN=390926/CN=Dimitris Fassouliotis" "/atlas" atlas
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai" "/cms" uscms6232
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai" uscms6232
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai" "/cms" uscms6225
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=fatsai/CN=711765/CN=Fang-Ying Tsai" uscms6225
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=favaro/CN=698706/CN=Carlotta Favaro" "/cms" uscms2711
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=favaro/CN=698706/CN=Carlotta Favaro" "/cms/Role=cmsuser" uscms2711
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=favaro/CN=698706/CN=Carlotta Favaro" uscms2711
@ -6563,8 +6563,8 @
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhernan/CN=763451/CN=Jonathan Matthew Hernandez" uscms6206
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Hogan" "/cms" jmanagan
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Hogan" jmanagan
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" "/cms" uscms6230
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" uscms6230
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" "/cms" uscms6223
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmhogan/CN=676292/CN=Julie Managan Hogan" uscms6223
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmolinai/CN=671301/CN=Jorge Andres Molina Insfran" "/cms" uscms5608
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmolinai/CN=671301/CN=Jorge Andres Molina Insfran" uscms5608
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=jmonk/CN=664162/CN=James William Monk" "/atlas" atlas
@ -7474,8 +7474,8 @
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=mendo/CN=711096/CN=Masaki Endo" "/atlas" atlas
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meneguz/CN=394388/CN=Anna Meneguzzo" "/cms" uscms2810
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meneguz/CN=394388/CN=Anna Meneguzzo" uscms2810
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" "/cms" uscms6228
-"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" uscms6228
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" "/cms" uscms6224
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meola/CN=665823/CN=Sabino Meola" uscms6224
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meridian/CN=555986/CN=Paolo Meridiani" "/cms" meridian
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=meridian/CN=555986/CN=Paolo Meridiani" meridian
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=merina/CN=673555/CN=Merina Albert" "/cms" uscms5650
@ -16496,6 +16496,8 @
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/nova" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/nova/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/numix" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/numix/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/uboone" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/uboone/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=People/CN=Dalath rachitha A. Mendis/CN=UID:dmendis" "/fermilab" fermilab
@ -33625,6 +33627,8 @
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/nova" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/nova/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/numix" fermilab
+"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/numix/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/uboone" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Daisy Kalra/CN=UID:kalra" "/fermilab/uboone/Role=Analysis" fermilab
"/DC=gov/DC=fnal/O=Fermilab/OU=Robots/CN=fifegrid/CN=batch/CN=Dalath rachitha A. Mendis/CN=UID:dmendis" "/fermilab" fermilab

Info: /Stage[main]/Vulcan::Dcache/File[/etc/grid-security/grid-vorolemap]: Filebucketed /etc/grid-security/grid-vorolemap to main with sum 584c5bc1a3ac6aa2cec42dc2fb5d4bc9
Notice: /Stage[main]/Vulcan::Dcache/File[/etc/grid-security/grid-vorolemap]/content: content changed '{md5}584c5bc1a3ac6aa2cec42dc2fb5d4bc9' to '{md5}31392937823d1fdfa23e6736a5e33815'

Also available in: Atom PDF