Project

General

Profile

Bug #23187

Active entries not corresponding with the ones defined in the glideinWMS.xml

Added by Lorena Lobato Pardavila 4 months ago. Updated 2 months ago.

Status:
Closed
Priority:
Normal
Category:
-
Target version:
Start date:
08/28/2019
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

Investigating #22877 (No error when there are no enabled entries)on one of my environments, I saw during the Factory upgrade/reconfig and having all the entries disabled (only had 2 entries : ITB_FC_CE2 and ITB_FC_HTC_CE2 ), that the active entries were more and different ones than the expected (none):

[root@fermicloud116 ~]# /usr/sbin/gwms-factory upgrade
Using default factory config file: /etc/gwms-factory/glideinWMS.xml
...Updated the glidein_startup.sh and local_start.sh scripts
...Updated the glidein_startup.sh file in the staging area
...Updated the factory_startup script
...Reconfigured glidein 'gfactory_instance' is complete
...Active entries are:
     el6_osg33
     el6_osg34
     el7_osg33
     el7_osg34
...Verifying rrd schema
...Submit files are in /var/lib/gwms-factory/work-dir
Upgrading the factory       

Any of those entries indicated exist in the glideinWMS.xml but they do exist in the glidein.descript:

# File: glidein.descript
#
CondorTarballDirMap     {u'/var/lib/gwms-factory/condor/condor-8.6.11-x86_64_RedHat7-stripped': u'/var/lib/gwms-factory/web-area/stage/condor_bin_4.j3ejv5.tgz', u'/var/lib/gwms-factory/condor/condor-8.8.0-x86_64_RedHat6-stripped': u'/var/lib/gwms-factory/web-area/stage/condor_bin_2.j3ejv5.tgz', u'/var/lib/gwms-factory/condor/condor-8.8.1-x86_64_RedHat6-stripped': u'/var/lib/gwms-factory/web-area/stage/condor_bin_2.j43fuC.tgz', u'/usr': u'/var/lib/gwms-factory/web-area/stage/condor_bin_0.j3ejv5.tgz'}
FactoryCollector        None
FactoryName     gfactory_service
GlideinName     gfactory_instance
WebURL  http://fermicloud116.fnal.gov/factory/stage
PubKeyType      RSA
OldPubKeyGraceTime      900
MonitorUpdateThreadCount        1
RemoveOldCredFreq       24
RemoveOldCredAge        30
Entries         el6_osg33,el6_osg34,el7_osg33,el7_osg34
AdvertiseWithTCP        False
AdvertiseWithMultiple   False
LoopDelay       60
AdvertisePilotAccounting        False
AdvertiseDelay  5
RestartAttempts         3
RestartInterval         1800
EntryParallelWorkers    0
LogDir  /var/log/gwms-factory/server
ClientLogBaseDir        /var/log/gwms-factory/client
ClientProxiesBaseDir    /var/lib/gwms-factory/client-proxies
DowntimesFile   /var/lib/gwms-factory/work-dir/glideinWMS.downtimes
MonitorDisplayText
MonitorLink
JobLogRetentionMaxDays  7.0
JobLogRetentionMinDays  3.0
JobLogRetentionMaxMBs   100.0
SummaryLogRetentionMaxDays      31.0
SummaryLogRetentionMinDays      3.0
SummaryLogRetentionMaxMBs       100.0
CondorLogRetentionMaxDays       14.0
CondorLogRetentionMinDays       3.0
CondorLogRetentionMaxMBs        100.0
ProcessLogs     [{u'compression': u'', u'extension': u'info', u'min_days': u'3.0', u'msg_types': u'INFO', u'max_days': u'7.0', u'max_mbytes': u'100.0', u'backup_count': u'5'}, {u'compression': u'', u'extension': u'err', u'min_days': u'3.0', u'msg_types': u'DEBUG,ERR,WARN,EXCEPTION', u'max_days': u'7.0', u'max_mbytes': u'100.0', u'backup_count': u'5'}]
~

Note: If I enable one of the existing ones, I keep seeing these ones:

[root@fermicloud116 glideinwms]# /usr/sbin/gwms-factory upgrade
Using default factory config file: /etc/gwms-factory/glideinWMS.xml
...Updated the glidein_startup.sh and local_start.sh scripts
...Updated the glidein_startup.sh file in the staging area
...Updated the factory_startup script
...Reconfigured glidein 'gfactory_instance' is complete
...Active entries are:
     ITB_FC_HTC_SIN_CE2
     el6_osg33
     el6_osg34
     el7_osg33
     el7_osg34
...Verifying rrd schema
...Submit files are in /var/lib/gwms-factory/work-dir
Upgrading the factory                                      [  OK  ]
[root@fermicloud116 glideinwms]#

And thus, the glidein.descript is updated:

# File: glidein.descript
#
CondorTarballDirMap     {u'/var/lib/gwms-factory/condor/condor-8.6.11-x86_64_RedHat7-stripped': u'/var/lib/gwms-factory/web-area/stage/condor_bin_4.j3ejv5.tgz', u'/var/lib/gwms-factory/condor/condor-8.8.0-x86_64_RedHat6-stripped': u'/var/lib/gwms-factory/web-area/stage/condor_bin_2.j3ejv5.tgz', u'/var/lib/gwms-factory/condor/condor-8.8.1-x86_64_RedHat6-stripped': u'/var/lib/gwms-factory/web-area/stage/condor_bin_2.j43fuC.tgz', u'/usr': u'/var/lib/gwms-factory/web-area/stage/condor_bin_0.j3ejv5.tgz'}
FactoryCollector        None
FactoryName     gfactory_service
GlideinName     gfactory_instance
WebURL  http://fermicloud116.fnal.gov/factory/stage
PubKeyType      RSA
OldPubKeyGraceTime      900
MonitorUpdateThreadCount        1
RemoveOldCredFreq       24
RemoveOldCredAge        30
Entries         ITB_FC_HTC_SIN_CE2,el6_osg33,el6_osg34,el7_osg33,el7_osg34
...

TODO: Investigate if it's either a malformed XML or an error in our parsing code.


Related issues

Related to GlideinWMS - Bug #22877: No error when there are no enabled entriesClosed07/09/2019

History

#1 Updated by Lorena Lobato Pardavila 4 months ago

  • Related to Bug #22877: No error when there are no enabled entries added

#2 Updated by Lorena Lobato Pardavila 4 months ago

  • Assignee set to Lorena Lobato Pardavila

#3 Updated by Marco Mascheroni 4 months ago

Do you confirm the /etc/gwms-factory/config.d/ dir is empty?

#4 Updated by Lorena Lobato Pardavila 3 months ago

It's not empty

[root@fermicloud116 ~]# ll  /etc/gwms-factory/config.d/
total 28
-rw-r--r--. 1 root root  348 Jul 22 10:49 Condor_Tarballs.xml.not.used
-rw-r--r--. 1 root root 7162 Jul 22 11:01 Dev_Sites.xml
-rw-r--r--. 1 root root 5126 Jul 22 11:01 Dev_Sites2.xml
-rw-r--r--. 1 root root 5127 Jul 22 11:01 Dev_Sites3.xml

And these 4 entries are coming from Dev_Sites2.xml which is used for fixtures in the unittests. What I don't understand is that these files seem to have been modified on July 22 when I was on holidays during that time. Have to investigate how they ended being "active"

#5 Updated by Lorena Lobato Pardavila 3 months ago

  • Status changed from New to Closed

There is a reasonable explanation about why the entries were appearing when they were not defined in glideinwms.xml.After investigation, I figured out there is a config.d directory somewhere down in glideinwms/unittests/fixtures that does the same behavior that the entries in glideinwms.xml.
The fact that those files were created during a period that I was on holidays was confusing me. I just knew Dennis was doing several tests to run the code, and put those files there and forgot about them.

Closing ticket

#6 Updated by Marco Mambelli 2 months ago

  • Target version changed from v3_5_x to v3_6_x


Also available in: Atom PDF