Project

General

Profile

Bug #5249

GLIDEIN_MaxMemMBs_Estimate not taking GLIDEIN_CPUS into consideration

Added by Igor Sfiligoi about 6 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Parag Mhashilkar
Category:
-
Target version:
Start date:
01/23/2014
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:

CMS

Duration:

Description

The automatic detection of the available memory does not work with multi-core glideins
since it assumes a single core in all cases.

It should be using the GLIDEIN_CPUS attribute, if defined.

History

#1 Updated by Parag Mhashilkar almost 6 years ago

  • Target version set to v3_2_5

#2 Updated by Parag Mhashilkar almost 6 years ago

  • Target version changed from v3_2_5 to v3_2_6

#3 Updated by Krista Larson over 5 years ago

Just to note that CMS is also running into this issue (from an update from Antonio's testing). I am curious what the effect of over provisioning this attr will do.

-Multicore at PIC: The issue with the multicore pilots pulling just 7
single corejobs instead of 8 is solved. It was a matter of taking into
account that production jobs request by default 2300 MB of memory in their
classads. So the glidein factory entry for multicore pilots must include
"GLIDEIN_MaxMemMBs" of 2300 x Number of cores (we even used 2300 x 8 x 1.1,
adding a 10% extra to avoid any possible rounding effects). This has
nothing to do with the real memory usage by the jobs, which in reality is
lower than 2300 MB, but it's needed so that pilot can partition its
resources into 8 slots, not 7 due to the memory being "exhausted".

#4 Updated by Parag Mhashilkar over 5 years ago

  • Target version changed from v3_2_6 to v3_2_7

#5 Updated by Parag Mhashilkar over 5 years ago

  • Status changed from New to Feedback
  • Assignee changed from Parag Mhashilkar to Marco Mambelli

Made and tested the changes. They are in branch v3/5249 for feedback.

#6 Updated by Marco Mambelli over 5 years ago

  • Assignee changed from Marco Mambelli to Parag Mhashilkar

I confirm that it works on both real and VM hosts.
It can be committed

An optional change (do it if you think it's the case.
There should be no error but if for some reason $mem ends up having a letter, then the result of bc is 0
To catch also that you could use (I also used -z):
if [[ -z "$GLIDEIN_MaxMemMBs" || "$GLIDEIN_MaxMemMBs" = "0" ]]; then

Thank you,
Marco

#7 Updated by Parag Mhashilkar over 5 years ago

  • Status changed from Feedback to Resolved

Made required changes and merged to branch_v3_2 and master

#8 Updated by Parag Mhashilkar over 5 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF