Project

General

Profile

Feature #25510

Allow also a PATH with multiple directories, not only a single file path, to override the singularity binary

Added by Marco Mambelli 2 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Normal
Category:
Glidein
Target version:
Start date:
02/11/2021
Due date:
% Done:

0%

Estimated time:
Stakeholders:

OSG

Duration:

Description

Currently GLIDEIN_SINGULARITY_BINARY_OVERRIDE is a file path that allows at multiple levels (Fectory, Frontend, Worker Node environment) to override which Singularity binary is used.

The desire for OSG is to have also a path to search for the binary.
The suggestion is to add a new vatiable: OSG_SINGULARITY_PREPEND_PATH

diff --git a/creation/web_base/singularity_lib.sh b/creation/web_base/singularity_lib.sh
index d054a9c8..b7bc6668 100644
--- a/creation/web_base/singularity_lib.sh
+++ b/creation/web_base/singularity_lib.sh
@@ -1264,7 +1264,7 @@ singularity_locate_bin() {
     # In:
     #   1 - s_location, suggested Singularity directory, will be added first in PATH before searching for Singularity
     #   2 - s_image, if provided will be used to test Singularity (as additional test)
-    #   OSG_SINGULARITY_BINARY, OSG_SINGULARITY_BINARY_DEFAULT, LMOD_CMD, optional if in the environment
+    #   OSG_SINGULARITY_BINARY, OSG_SINGULARITY_BINARY_DEFAULT, OSG_SINGULARITY_PREPEND_PATH, LMOD_CMD, optional if in the environment
     # Out (E - exported):
     #   E GWMS_SINGULARITY_MODE - unprivileged, privileged, fakeroot or unknown (no image to test)
     #   E GWMS_SINGULARITY_VERSION
@@ -1282,6 +1282,7 @@ singularity_locate_bin() {
     local s_test test_out
     HAS_SINGULARITY=False
     local osg_singularity_binary="${OSG_SINGULARITY_BINARY:-${OSG_SINGULARITY_BINARY_DEFAULT}}" 
+    local osg_singularity_prepend_path="${OSG_SINGULARITY_PREPEND_PATH}" 
     local singularity_binary_override="${GLIDEIN_SINGULARITY_BINARY_OVERRIDE}" 

     if [[ -n "$singularity_binary_override" ]]; then
@@ -1320,6 +1321,11 @@ singularity_locate_bin() {
         # 5. Invoke module singularitypro
         # 6. Invoke module singularity
         #    some sites requires us to do a module load first - not sure if we always want to do that
+        # first, prepend to the path if requested
+        old_path="$PATH" 
+        if [ -n "$osg_singularity_prepend_path" ]; then
+            export PATH="$osg_singularity_prepend_path:$PATH" 
+        fi
         for attempt in "OSG,${osg_singularity_binary}" "PATH,singularity" "module,singularitypro" "module,singularity"; do
             if test_out=$(singularity_test_bin "$attempt" "$s_image"); then
                 HAS_SINGULARITY=True
@@ -1328,6 +1334,7 @@ singularity_locate_bin() {
             bread_crumbs+="${test_out##*@}" 
         done
         bread_crumbs+="${test_out##*@}" 
+        export PATH="$old_path" 
     fi
     # Execution test done w/ default image
     info_dbg "Has singularity $HAS_SINGULARITY. Tests: $bread_crumbs" 
diff --git a/doc/factory/custom_vars.html b/doc/factory/custom_vars.html
index 425a2a55..fa6f33bd 100644
--- a/doc/factory/custom_vars.html
+++ b/doc/factory/custom_vars.html
@@ -1585,6 +1585,7 @@ space (eg EGI's $TMPDIR policy), then this value refers to that space.</P>
     <li>If defined (in environment, Frontend or Factory): Try first the override path (GLIDEIN_SINGULARITY_BINARY_OVERRIDE)</li>
     <li>If defined (in environment, Frontend or Factory): Try a singularity binary in the directory suggested via SINGULARITY_BIN (keyword OSG -> OSG_SINGULARITY_BINARY)</li>
     <li>Try the default OSG location (OSG_SINGULARITY_BINARY)</li>
+    <li>Try the singularity binary in GLIDEIN_SINGULARITY_PREPEND_PATH</li>
     <li>Try the singularity binary in PATH</li>
     <li>Try the singularity binary in PATH after invoking module singularitypro</li>
     <li>Try the singularity binary in PATH after invoking module singularity</li>

My proposal is to change instead GLIDEIN_SINGULARITY_BINARY_OVERRIDE, which will be either:
  • the full path of an executable (named as desired)
  • a PATH like variable (one or more directories, colon-separated) where a binary named 'singularity' is searched

History

#1 Updated by Marco Mambelli 2 months ago

  • Assignee changed from Marco Mambelli to Dennis Box
  • Status changed from New to Feedback

changes are in v37/25510

#2 Updated by Dennis Box 2 months ago

  • Assignee changed from Dennis Box to Marco Mambelli

Feedback sent,OK to merge

#3 Updated by Marco Mambelli 2 months ago

  • Status changed from Feedback to Resolved

#4 Updated by Marco Mambelli 2 months ago

To patch a v3.7.x Factory, please replace web_base/singularity_lib.sh w/ a copy of creation/web_base/singularity_lib.sh from the branch_v3_7 branch in git/GitHub. Then run a factory upgrade command.

Also available in: Atom PDF