Project

General

Profile

Bug #24282

singularity command identification failed on some worker nodes

Added by Marco Mambelli 6 months ago. Updated 19 days ago.

Status:
Closed
Priority:
High
Category:
Glidein
Target version:
Start date:
04/07/2020
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:

OSG

Duration:

Description

There are some nodes where the OSG script seems to be able to find singularity in /usr/bin/singularity and
the GWMS script (singularity_setup.sh using functions in singularity_lib.sh) is not:

=== validation OK in /local/execute/dir_1870049/glide_65oDt6/main/gcb_setup.sh (0) ===
singularity_setup.k46bWy.sh: OK
Signature OK for main:singularity_setup.k46bWy.sh.
INFO  Sourcing add config line: /local/execute/dir_1870049/glide_65oDt6/add_config_line.source
INFO  Checking for singularity...
ERROR  : Failed to create user namespace: Invalid argument
INFO  Singularity at /cvmfs/oasis.opensciencegrid.org/mis/singularity/bin/singularity failed 
INFO  Singularity at '/cvmfs/oasis.opensciencegrid.org/mis/singularity/bin/singularity' failed w/ unexpected output
which: no singularity in ((null))
which: no singularity in ((null))
Mon Apr 6 18:02:37 EDT 2020 Singularity not found in PATH, OSG_SINGULARITY_BINARY_DEFAULT and module

The problem seems to be an empty path found by the command which (sin_path=$(which singularity)).
It seems strange that which was found in the path, which is not a builtin in bash and the script is invoked as "#!/bin/bash".

History

#1 Updated by Marco Mambelli 5 months ago

  • Stakeholders updated (diff)
  • Assignee changed from Marco Mambelli to Bruno Coimbra
  • Status changed from New to Feedback

Unable to reproduce the problem.
I added some extra message as a warning so that it will be in the logs also when the debug messages are not.
Changes are in v36/24282

#2 Updated by Marco Mambelli 3 months ago

  • Assignee changed from Bruno Coimbra to Dennis Box

#3 Updated by Dennis Box 3 months ago

  • Assignee changed from Dennis Box to Marco Mambelli

changes look helpful. OK to merge

#4 Updated by Marco Mambelli 3 months ago

  • Status changed from Feedback to Resolved

#5 Updated by Marco Mambelli 19 days ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF