Project

General

Profile

Idea #3389

Add a Collector for glidein monitoring to the factory

Added by Igor Sfiligoi over 6 years ago. Updated about 4 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Factory
Target version:
Start date:
09/08/2014
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Stakeholders:

OSG, CMS

Duration:

Description

Currently the factory does not have a good idea of which glideins are running and which are not;
it only relies on Condor-G information, which we all know is not very reliable.

So the proposal for this ticket is to add a factory-specific collector (tree) that all the glideins would report to.

This should be separate from the one used for classad exchange between the GF and the FEs.
I would also recommend installing it on a completely different node.
And it should be optional.

One of the reasons why we never did it in the past was due to authentication issues;
we do not want to track all the DNs that the FEs use for their pilots.
So the proposal for this ticket is to use the shared secret authentication for this collector.

Now, the shared secret will be pretty much impossible to get in a secure way to the WNs (e.g. GT2 does not provide any such guarantee);
the best we can hope for is making it non trivial to discover.

However, the information in the Collector are not really that sensitive.... it is just for monitoring, and as long people/services using this information understand that, it is still better than the current situation.
The reason to have any authentication at all is mostly to avoid random scanning services/script kiddies to compromise it.


Subtasks

Idea #6770: Add a Collector for glidein monitoring to the factory - Add support in the glideinsClosedIgor Sfiligoi

Bug #6943: Changes to #6770 broke branch_v3_2 and masterClosedParag Mhashilkar


Related issues

Related to glideinWMS - Feature #2454: Advertise classad in case of glidein failureClosed2013-04-18

Related to glideinWMS - Feature #5309: Need more prompt fake running glidein detectionNew2014-01-31

Related to glideinWMS - Feature #6317: Add periodic test glidein submissionsNew2014-05-22

Related to glideinWMS - Feature #6319: Propagate and/or validate factory attributes from running glideinsNew2014-05-22

History

#1 Updated by Igor Sfiligoi over 6 years ago

The exact nature of advertising to this collector should be carefully thought through.

Just pointing the current startd to it is likely not a good idea.
The collector is normally the most trusted service in a condor pool, so there are/may be some information the startd is sending that are not appropriate for a low security collector I am proposing.

#2 Updated by Burt Holzman over 6 years ago

  • Assignee set to Burt Holzman

#3 Updated by Burt Holzman over 6 years ago

  • Target version set to v3_1

#4 Updated by Burt Holzman over 6 years ago

  • Target version changed from v3_1 to v3_x

#5 Updated by Igor Sfiligoi over 5 years ago

Another option is to dynamically add all the pilot DNs as the factory sees them.
We indeed have all the information.

I am a bit uncomfortable with automatic changes to the security configs, but it is still better than no security.

#6 Updated by Igor Sfiligoi over 5 years ago

  • Assignee changed from Burt Holzman to Igor Sfiligoi
  • Target version changed from v3_x to v3_2_6
  • Stakeholders updated (diff)

Align with #5309.

#7 Updated by Parag Mhashilkar about 5 years ago

  • Target version changed from v3_2_6 to v3_2_7

#8 Updated by Igor Sfiligoi about 5 years ago

I have created a sub-task ( #6770 ) that will only deal with the glidein configuration part.
Once that's done, the GF admins can start using it by manually maintaining the security of the GF collector.

Of course we do want automatic management of the security, but the two do not need to be implemented at the same time.

#9 Updated by Parag Mhashilkar almost 5 years ago

  • Target version changed from v3_2_7 to v3_x

#10 Updated by Parag Mhashilkar about 4 years ago

  • Assignee deleted (Igor Sfiligoi)

Igor Sfiligoi's tickets. These were some ideas brewing up in his mind but they never materialized or got priority.



Also available in: Atom PDF