Project

General

Profile

Feature #24227

Reorganize setup of module and spack in Singularity

Added by Marco Mambelli 2 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
High
Category:
-
Target version:
Start date:
03/25/2020
Due date:
% Done:

0%

Estimated time:
Stakeholders:

CMS

Duration:

Description

Glideins are currently setting up by default module and spack off CVMFS for all VOs except LIGO.
This behavior is inherited from the implementation of the OSG singularity wrapper and uses many variables some of which obsolete (LMOD_BETA, MODULE_USE, InitializeModulesEnv)
To make module's setup more straightforward there will be 2 variables (actually 1 in 2 places):
  • MODULE_USE, settable in Factory and/or Frontend, defaulting to True (GWMS_MODULE_USE_DEFAULT=1)
  • MODULE_USE, settable in the job submit file (+MODULE_USE), and overriding the value from Factory, Frontend, and default

During the transition (3.6.x?), InitializeModulesEnv will be a synonym of MODULE_USE only in the job classad and
the default for LIGO will be False/0

LMOD_BETA is removed (module-beta-init.sh is never used)

Here the email thread discussion this

So I will remove LMOD_BETA
This means that we'll always use . /cvmfs/oasis.opensciencegrid.org/osg/sw/module-init.sh
and never . /cvmfs/oasis.opensciencegrid.org/osg/sw/module-beta-init.sh

MODULE_USE may have been a remanence from earlier or other VO wrappers
I kind of like that the name is more similar to what is used for other things (singularity, glexec, gpu, ...)

I agree on the VOs having full control of the module setup.
But as it is now the control comes from the default for everyone or a job attribute. I'd like to add also a per VO default, something controllable in the Frontend, e.g. MODULE_USE_DEFAULT
Would that be OK w/ you?

Marco

On Jan 31, 2020, at 6:34 PM, Mats Rynge <rynge@isi.edu> wrote:

LMOD_BETA should be removed - I will do this next time I spend time on the OSG VO wrapper.

I don't have a MODULE_USE.

InitializeModulesEnv was used for users opting out of modules. OSG VO has taken the approach that modules will be loaded by default if available (basically mimicing the submit node). LIGO asked us to change this behavior for the (see https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_opensciencegrid_osg-2Dflock_blob_master_job-2Dwrappers_user-2Djob-2Dwrapper.sh-23L222&d=DwICaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=EF06-Wh4L9CNLgD8bnIjNQ&m=RsosJ56QjLIK8qQTGQ2AVX3S99yP0f9yKcVEkrTsyxI&s=olb9oJYpMP0PAsA0FvtJcsz7RsrXWLcWovcxiuhz1jc&e= ). 

In general, I think modules should be a VO function and not something GWMS should worry about. GWMS should provide all the hooks for the VO to implement whatever environment they want.

I also suspect we will use modules less and less. One reason is that the software stack is very difficult to maintain (yes, even with Spack) and containers are solving some of the problems better than modules ever did.

/Mats

On 1/31/20 4:27 PM, Marco Mambelli wrote:
Mats, Edgar,
I'd like to know which is the expected way to control the setup of module for jobs running on glideins.
In the GWMS singularity wrapper I copied OSG behavior (or what I understood).
Now that CMS is starting to use the wrapper and is not interested in module I'd like to understand or define a clear way to control the setup of modules.
I think there are 3 variables that can enable the setup of module:
LMOD_BETA, MODULE_USE, InitializeModulesEnv
If either one is 1, then module is set up.
LMOD_BETA checks the condor ad (_CONDOR_JOB_AD) and defaults to 0 if there is nothing
MODULE_USE checks the condor ad (_CONDOR_JOB_AD) and defaults to 1 if there is nothing
InitializeModulesEnv checks the condor ad and defaults to 1 for everyone except ligo (defaults to 0). The ligo VO is checked in GLIDEIN_Client
Is this correct?
What's the difference between these variables?
Do you expect the job to set these variables?
I'd like the glidein (i.e. Factory or Frontend configuration) to have a say on the default at least.
Then I'd like to reduce the different variables unless there is a reason to have so many.
Would that be OK with you?
Thank you,
Marco

History

#1 Updated by Marco Mambelli 2 months ago

  • Priority changed from Normal to High
  • Assignee changed from Marco Mambelli to Marco Mascheroni
  • Status changed from New to Feedback
  • Description updated (diff)

#2 Updated by Marco Mascheroni 2 months ago

  • Assignee changed from Marco Mascheroni to Marco Mambelli

LGTM

#3 Updated by Marco Mambelli 2 months ago

  • Status changed from Feedback to Resolved

#4 Updated by Marco Mambelli about 2 months ago

  • Stakeholders updated (diff)

#5 Updated by Marco Mambelli about 2 months ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF