Project

General

Profile

Feature #25440

Fail, no fall-back when the user-provided singularity image is missing

Added by Marco Mambelli about 1 month ago. Updated 23 days ago.

Status:
Resolved
Priority:
High
Category:
Glidein
Target version:
Start date:
01/26/2021
Due date:
% Done:

0%

Estimated time:
Stakeholders:

OSG

Duration:

Description

The singularity wrapper is failing if singularity is REQUIRED and falling back to running w/o singularity if it is only PREFERRED or OPTIONAL.
User jobs specifying an image run only ono Glideins advertising HAS_SINGULARITY=True
Singularity can still fail for a number of reasons:
  • misspelled user image path (or missing file)
  • CVMFS hosting singularity or the image broke
    The current behavior is to fall-back if the initial option allows

This should change in an error (consider singularity becoming REQUIRED if there is a user image specified).
The two options are a job failure (preferred by Marco and Mats because the most likely error is a wrong image path in the user job)
or a wrapper failure (this will cause the job to retry somewhere else).

History

#1 Updated by Marco Mambelli 27 days ago

  • Assignee changed from Marco Mambelli to Marco Mascheroni
  • Status changed from New to Feedback
  • Category set to Glidein

Changes are in branch v37/25440

#2 Updated by Marco Mascheroni 27 days ago

  • Status changed from Feedback to Resolved

Do i understand it correctly that GWMS_SINGULARITY_STATUS_EFFECTIVE cannot be set in the factory (or frontend)? If that's the case it is ok to merge.

#3 Updated by Marco Mascheroni 27 days ago

  • Assignee changed from Marco Mascheroni to Marco Mambelli

#4 Updated by Marco Mambelli 23 days ago

Yes, GWMS_SINGULARITY_STATUS_EFFECTIVE (same as GWMS_SINGULARITY_STATUS_EFFECTIVE and other variables marked as "Dynamic" in https://glideinwms.fnal.gov/doc.prd/factory/custom_vars.html) are set and used within the scripts as a result of other variables set in Factory and frontend).
Merged to branch_v3_7.

Also available in: Atom PDF