Project

General

Profile

Bug #4588

Misleading error messages for misconfigured credentials

Added by Brian Bockelman over 7 years ago. Updated about 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Frontend
Target version:
Start date:
08/27/2013
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

Right now, if there is no proxy / certificate configured, this is the only error message in the logs:

[2013-08-26 22:06:41,545] ERROR: Unknown error advertising glidein requests
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendElement.py", line 872, in iterate_one
advertizer.do_advertize()
File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 711, in do_advertize
advertizeWorkFromFile(factory_pool, filename, remove_file=True, is_multi=frontendConfig.advertise_use_multi)
File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 527, in advertizeWorkFromFile
os.remove(fname)

Basically, the ad file is written inside a for-loop over the configured certificates. If there's no proxy configured (or, in my case, none configured correctly :), then the loop is not run and the ad file is not created.

The fact that os.remove has an error also masks the underlying exception from condor_advertise failing.

History

#1 Updated by Parag Mhashilkar over 7 years ago

  • Target version set to v3_2_x

#2 Updated by Burt Holzman over 7 years ago

Can you be more specific about the error that caused this? If I leave out the <credential> line, I get:

[2013-08-27 20:16:23,590] ERROR: Exception writing advertisement file: 
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 746, in createAdvertizeWorkFile
    if x509_proxies_data is not None:
UnboundLocalError: local variable 'x509_proxies_data' referenced before assignment
[2013-08-27 20:16:23,591] ERROR: Unknown error advertising glidein requests
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendElement.py", line 872, in iterate_one
    advertizer.do_advertize()
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 700, in do_advertize
    filename_arr_el=self.createAdvertizeWorkFile(factory_pool,params_obj,key_obj)
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 746, in createAdvertizeWorkFile
    if x509_proxies_data is not None:
UnboundLocalError: local variable 'x509_proxies_data' referenced before assignment

which is also opaque, but not the same error.

#3 Updated by Burt Holzman over 7 years ago

Ok, this is probably what you did - here's what you get if you have a credential that doesn't exist (or if the file isn't a proxy, the errors are similar):

[2013-08-27 20:19:25,965] ERROR: Could not read credential file '/tmp/vo_proxydingus'

Followed by

[2013-08-27 20:20:23,631] ERROR: Advertising global credential /tmp/vo_proxydingus failed
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 616, in do_global_advertize
    if (not os.path.exists(cred_el.filename)) and (cred_el.creation_script is not None):
AttributeError: Credential instance has no attribute 'creation_script'

and

[2013-08-27 20:20:23,682] ERROR: Unknown error advertising glidein requests
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendElement.py", line 872, in iterate_one
    advertizer.do_advertize()
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 711, in do_advertize
    advertizeWorkFromFile(factory_pool, filename, remove_file=True, is_multi=frontendConfig.advertise_use_multi)
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 527, in advertizeWorkFromFile
    os.remove(fname)
OSError: [Errno 2] No such file or directory: '/tmp/gfi_ad_gc_1_327634823_877'

#4 Updated by Brian Bockelman over 7 years ago

Actually - try putting a valid proxy into an invalid trust domain. You don't get either of the first two errors.

#5 Updated by Burt Holzman over 7 years ago

  • Subject changed from Warn / Error early if there is no security configured to Misleading error messages for misconfigured credentials

#6 Updated by Parag Mhashilkar over 7 years ago

Brian Bockelman wrote:

Actually - try putting a valid proxy into an invalid trust domain. You don't get either of the first two errors.

There is nothing called invalid trust_domain as a possible value is just string that you can set to anything. As far as you don't get any error in this case I think that's ok, because what you are saying is, for some trust_domain="foo" use the proxy. Indirectly your test case implies that for the trust domains supported by the factory, there is no valid credential.

Does this make sense or are you seeing completely different error? If so what is it?

#7 Updated by Brian Bockelman over 7 years ago

Hi Parag,

If I set trust_domain="osg" instead of trust_domain="grid", I get something like:

[2013-08-27 20:20:23,682] ERROR: Unknown error advertising glidein requests
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendElement.py", line 872, in iterate_one
    advertizer.do_advertize()
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 711, in do_advertize
    advertizeWorkFromFile(factory_pool, filename, remove_file=True, is_multi=frontendConfig.advertise_use_multi)
  File "/usr/lib/python2.6/site-packages/glideinwms/frontend/glideinFrontendInterface.py", line 527, in advertizeWorkFromFile
    os.remove(fname)
OSError: [Errno 2] No such file or directory: '/tmp/gfi_ad_gc_1_327634823_877'

That's not good. At the least, we should have a loud WARNING that we are not advertising any glideclients to the factory. I can't think of a legit use case where a user would add a factory, but use any compatible credentials; it's very likely to be a spelling mistake, and glideinWMS should warn accordingly.

#8 Updated by Parag Mhashilkar over 7 years ago

Brian Bockelman wrote:

Hi Parag,

If I set trust_domain="osg" instead of trust_domain="grid", I get something like:

[...]

That's not good. At the least, we should have a loud WARNING that we are not advertising any glideclients to the factory. I can't think of a legit use case where a user would add a factory, but use any compatible credentials; it's very likely to be a spelling mistake, and glideinWMS should warn accordingly.

I completely agree on warning user when we do not advertise glideclient classads.
However its possible that

  • admin configured extra credentials with unused trust domains * entry with a given trust domain is down * trust domain is mistyped in frontend.xml * there is a temporary failure querying factory classads for certain trust domains

Its not easy to distinguish between any (maybe except 2nd). So does it make sense to warn the frontend admin or will s/he be confused more?

#9 Updated by Brian Bockelman over 7 years ago

Hi Parag,

For this ticket, I think it is sufficient to just warning the user that there is no security credentials which match the current factory under consideration (so there is no advertising).

Further work diagnostic can happen later.

Brian

#10 Updated by Burt Holzman over 7 years ago

  • Category set to Frontend
  • Status changed from New to Assigned
  • Assignee set to Burt Holzman

Before I left the office today I wrote a patch that basically did this, but I needed to test it before pushing.
I did some basic testing on commit:6474c2d1274, seems to work.

#11 Updated by Burt Holzman over 7 years ago

  • Status changed from Assigned to Feedback
  • Assignee changed from Burt Holzman to Parag Mhashilkar

#12 Updated by Parag Mhashilkar about 7 years ago

  • Target version changed from v3_2_x to v3_2_1

Looks ok. Waiting for v3_2 to merge. Bumping the release version.

#13 Updated by Parag Mhashilkar about 7 years ago

  • Status changed from Feedback to Closed
  • Assignee changed from Parag Mhashilkar to Burt Holzman

Also available in: Atom PDF