Project

General

Profile

Bug #3446

Installer not setting SHARED_PORT_MAX_WORKERS

Added by Igor Sfiligoi almost 8 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
John Weigand
Category:
Ini-Installer
Target version:
Start date:
02/07/2013
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

The glideinWMS installation does not set SHARED_PORT_MAX_WORKERS when enabling the shared port daemon.

The default is only 10, which is ridiculously low for anything but toy installations.

We should set this to
SHARED_PORT_MAX_WORKERS=1000

History

#1 Updated by Burt Holzman almost 8 years ago

  • Status changed from New to Assigned
  • Assignee changed from Burt Holzman to John Weigand
  • Priority changed from High to Normal

#2 Updated by John Weigand over 7 years ago

  • Status changed from Assigned to Feedback
  • Assignee changed from John Weigand to Igor Sfiligoi
  • Target version set to v2_7_x

Igor,

I have a couple questions on this.

1. In looking at this Condor ticket...
https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=1681,56
.. it appears this may have been resolved in v7.5
.. yes / no?

2. Then I found this...
http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/GLIDEINWMS/condor_config/generic/80_cms_schedd_tunning.config?view=markup&revision=1.1
.. should we be setting the SCHEDD_QUERY_WORKERS=20? (default I think is 3)
.. and increasing our MAX_JOBS_RUNNING from something greater than the 6000
we currently set it to?
.. or is it just sufficient to add the single attribute you suggest?

John Weigand

#3 Updated by Igor Sfiligoi over 7 years ago

Hi John.

The large number of SHARED_PORT_WORKERS is still needed for performance reasons;
the fix you point out just prevent s Condor from dying if the number is too low.

Aboth the other attributes:

MAX_JOBS_RUNNING=6k is quite high for many of our VOs. It is not good enough for CMS, but may be for other users.
I have no strong feeling about it... but anyhow would not pair it with SHARED_PORT_MAX_WORKERS change.

Finally, the SCHEDD_QUERY_WORKERS is a completely different beast... has nothing to do with how the system schedules jobs... it is more how users use it.
Raising it may indeed be a good idea... but once again, I would not pair it with this ticket.

#4 Updated by John Weigand over 7 years ago

  • Status changed from Feedback to Assigned
  • Assignee changed from Igor Sfiligoi to John Weigand

#5 Updated by John Weigand over 7 years ago

Igor,

One more question as I am not that versed in Condor.

Should the condor config attribute be qualified with

SCHEDD.SHARED_PORT_MAX_WORKERS=1000

... since we qualify the shared port currently with
SHADOW.USE_SHARED_PORT = True
SCHEDD.USE_SHARED_PORT = True

John Weigand

#6 Updated by Igor Sfiligoi over 7 years ago

I would keep it at the global level;
should not hurt the daemons that do not use the shared port.

#7 Updated by John Weigand over 7 years ago

  • Status changed from Assigned to Feedback
  • Assignee changed from John Weigand to Igor Sfiligoi

Igor,

Committed to branch_v2plus_3446
hash: 67c2126db78dd411b9b791f119b433c6996728d2

For your review/feedback.

Added SHARED_PORT_MAX_WORKERS = 1000 to the condor config when the
shared port option is in effect.

Files changed:
- doc/components/condor.html
- doc/components/condor_config.html
- install/services/Condor.py
- install/templates/02_gwms_factory_schedds.config
- install/templates/02_gwms_schedds.config

John Weigand

#8 Updated by Igor Sfiligoi over 7 years ago

Can you patch the Q&A installer, too?

#9 Updated by John Weigand over 7 years ago

Igor,

install/glideinWMS_install changed.
branch_v2plus_3446
hash 946418ff3495c90027797dcff5060beaccda07c7

We good now?

John Weigand

#10 Updated by Igor Sfiligoi over 7 years ago

Looks good.

#11 Updated by John Weigand over 7 years ago

  • Status changed from Feedback to Assigned
  • Assignee changed from Igor Sfiligoi to John Weigand

Merged into branch v2plus
commits:
67c2126db78dd411b9b791f119b433c6996728d2
946418ff3495c90027797dcff5060beaccda07c7

John Weigand

#12 Updated by John Weigand over 7 years ago

  • Status changed from Assigned to Resolved

cherry-pick'd into master
ONLY commit 67e361255119850a4bd620a109853b5d1bb3a9a3
since the ./install/glideinWMS_install module is not
supported in master.

John Weigand

#13 Updated by Parag Mhashilkar over 7 years ago

  • Target version changed from v2_7_x to v2_7

#14 Updated by Parag Mhashilkar over 7 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF