Project

General

Profile

Feature #22855

Document and expand multi-node Glidein

Added by Marco Mambelli 3 months ago. Updated about 1 month ago.

Status:
Closed
Priority:
Normal
Category:
-
Target version:
Start date:
07/01/2019
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

An undocumented feature making a Glidein fork N copies of itself was added in [#15176] to test multi-node submission parameters:

attribute GLIDEIN_MULTIGLIDEIN (in frontend or factory) forks multiple glideins all w/ the same parameters (similar to multinode submission, but is all on one node)
e.g. <attr name="GLIDEIN_MULTIGLIDEIN" glidein_publish="True" job_publish="True" parameter="True" type="int" value="3"/>

This feature should be documented and made more useful by adding Glidein launchers prefixing the Glidein invocation:
- GLIDEIN_MULTIGLIDEIN_LAUNCHALL: single prefix handling also the spawning/forking
- GLIDEIN_MULTIGLIDEIN_LAUNCHER: prefixing each forked submission

glidein_startup.sh (66.6 KB) glidein_startup.sh glidein_startup Marco Mambelli, 07/11/2019 02:09 PM
glidein_startup.sh (67.9 KB) glidein_startup.sh Marco Mambelli, 08/02/2019 11:09 PM

History

#1 Updated by Marco Mambelli 3 months ago

  • Assignee changed from Marco Mambelli to Dennis Box
  • Status changed from New to Feedback

Changes are in branch v34/22855

#2 Updated by Marco Mambelli 2 months ago

Attaching the new glidein_startup.sh file.
To patch a GWMS 3.4.5 factory to have these feature replace the file /var/lib/gwms-factory/web-base/glidein_startup.sh and run a factory upgrade.

Here a summary of the instructions.
These are all attributes (attrs section) that can be set in the Factory and/or Frontend. Usual rules apply.

GLIDEIN_MULTIGLIDEIN
Int
Forks multiple glideins, each in its own subdirectory, all w/ the same parameters
(similar to multinode submission, but is all on one node).
Unlsess GLIDEIN_MULTIGLIDEIN_LAUNCHALL is set, then a single invocation
is used, expecting the system to handle the spawning.

GLIDEIN_MULTIGLIDEIN_LAUNCHER
String
Used only if GLIDEIN_MULTIGLIDEIN is set and > 1, and GLIDEIN_MULTIGLIDEIN_LAUNCHALL is not set.
Defines a launcher program that is added at the
beginning of the command line when each multiple Glidein instance is started.
NOTE: parameter and job_publish should be both True for this attribute.

GLIDEIN_MULTIGLIDEIN_LAUNCHALL
String
Used only if GLIDEIN_MULTIGLIDEIN is set and > 1. When this is set, a command line with this string
followed by the glidein invocation is used to start all Glideins.
NOTE: parameter and job_publish should be both True for this attribute.

GLIDEIN_MULTIGLIDEIN is available since 3.4.4 (even if it was undocumented).
GLIDEIN_MULTIGLIDEIN_LAUNCHER and GLIDEIN_MULTIGLIDEIN_LAUNCHALL will be in 3.4.6 and are enabled with this patch

#3 Updated by Dennis Box 2 months ago

  • Assignee changed from Dennis Box to Marco Mambelli

feedback sent

#4 Updated by Marco Mambelli about 2 months ago

To patch update glidein_startup.sh with the one attached and run a factory upgrade command.

#5 Updated by Marco Mambelli about 2 months ago

  • Status changed from Feedback to Resolved

#6 Updated by Marco Mambelli about 1 month ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF