Project

General

Profile

Feature #21741

Improve monitoring stats in glidefactory and glidefactoryclient classads

Added by Marco Mambelli 9 months ago. Updated 14 days ago.

Status:
Work in progress
Priority:
High
Category:
-
Target version:
Start date:
01/22/2019
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

glidefactory and glidefactorystatus ClassAds contain monitoring information coming from the Factory startd (condor_q query).
This is stored as:
GlideinMonitorStatus... (in GFC for the specific client)
GlideinMonitorTotalStatus... (in GF, summary for the entry)

When there is interaction w/ clients the partial stats are calculated via subquery and the total is calculated doing the sum.
When there is no interaction w/clients the total is calculated from the list of jobs returned by condor_q for the entry (condorQ), see [#21525]

It may be convenient to calculate all the partial and total stats running once through the list of all the glidiens in the entry, doing all the calculations once.
This way all the monitoring info will be fresh and evaluated the same way.
Furthermore, the current method may leave some stale info if only some clients are interacting w/ one entry.

Some considerations before implementing:
- consider if the client name is all in the job (glidein) classad, without the need to check glidefactoryclient classads
- consider if the information is used within the same process
- evaluate the use of parallel workers
- think about the memory footprint
- do a benchmark to compare performance:
- trigger 1000 or more glideins, store the list of classads (will be useful also for unittests)
- calculate the stats w/ subqueries + total
- claculate all the stats in the new way
- compare memory usage and time
- evaluate the checks on the client names
- pay attention to the 2 stats dictionaries: client_stats (w/ client_int_name) and qc_stats (w/ client_log_name)


Related issues

Related to GlideinWMS - Feature #22163: Check if there are load changes in Factory and solve TODOs added in #21880New03/19/2019

History

#1 Updated by Marco Mambelli 7 months ago

Check changes done in [#21880] and scheduled for [#21741]

#2 Updated by Marco Mambelli 7 months ago

  • Related to Feature #22163: Check if there are load changes in Factory and solve TODOs added in #21880 added

#3 Updated by Marco Mambelli 6 months ago

  • Target version changed from v3_5 to v3_5_1

#4 Updated by Marco Mambelli 2 months ago

  • Target version changed from v3_5_1 to v3_6_1

#5 Updated by Marco Mambelli about 1 month ago

  • Assignee changed from Marco Mambelli to Lorena Lobato Pardavila

#6 Updated by Marco Mambelli about 1 month ago

  • Priority changed from Normal to High

#7 Updated by Lorena Lobato Pardavila 21 days ago

  • Status changed from New to Work in progress
  • Subject changed from Improve monitoring stats in glidefactory and glidefactorystatus classads to Improve monitoring stats in glidefactory and glidefactoryclient classads

#8 Updated by Lorena Lobato Pardavila 14 days ago

  • Assignee changed from Lorena Lobato Pardavila to Marco Mambelli


Also available in: Atom PDF