Project

General

Profile

Bug #21264

--dataset DAG start job ignores -E IFDH_BASE_URI=...

Added by Marc Mengel 7 months ago. Updated 7 days ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
10/30/2018
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

Example: https://jobsub02.fnal.gov:8443/jobsub/acctgroups/uboone/sandboxes/yuyi/12781074.0@jobsub02.fnal.gov/

dagbegin.cmd has:

environment = CLUSTER=$(Cluster);PROCESS=$(Process);CONDOR_TMP=/fife/local/scratch/uploads/uboone/yuyi/2018-10-30_081725.568583_3370;CONDOR_EXEC=/tmp;DAGMANJOBID=$(DAGManJobId);POMS_CAMPAIGN_ID=2082;POMS_TASK_ID=401083;IFDH_DEBUG=1;IFDH_BASE_URI=http://samweb.fnal.gov:8480/sam/samdev/api;SAM_EXPERIMENT=samdev;EXPERIMENT=samdev;SAM_GROUP=fermilab;SAM_STATION=samdev;GRID_USER=yuyi;JOBSUBJOBID=$(CLUSTER).$(PROCESS)@jobsub02.fnal.gov;JOBSUBPARENTJOBID=$(DAGManJobId).0@jobsub02.fnal.gov;SAM_USER=yuyi;SAM_PROJECT=yuyi-fife_wrap_20181030_081726_3986553;SAM_PROJECT_NAME=yuyi-fife_wrap_20181030_081726_3986553;SAM_DATASET=poms_depends_400687_1;JOBSUBJOBSECTION=0

but the actual sambegin.sh script does:


if [ "$SAM_STATION" = "" ]; then
SAM_STATION=$1
fi
if [ "$SAM_GROUP" = "" ]; then
SAM_GROUP=$1
fi
if [ "$SAM_DATASET" = "" ]; then
SAM_DATASET=$2
fi
if [ "$SAM_PROJECT" = "" ]; then
SAM_PROJECT=$3
fi
if [ "$SAM_USER" = "" ]; then
SAM_USER=$4
fi
export IFDH_BASE_URI=http://samweb.fnal.gov:8480/sam/uboone/api

the last line of which just stomps all over teh IFDH_BASE_URI I'm passing in, thus forcing the SAM
calls to an instance that doesn't even have the dataset I want...

This looks to come from JobSettings.py around line 860 where we do:

                f.write("""SAM_PROJECT=$3\n""")
                f.write("fi\n")
                f.write("""if [ "$SAM_USER" = "" ]; then\n""")
                f.write("""SAM_USER=$4\n""")
                f.write("fi\n")
                f.write("""export IFDH_BASE_URI=%s\n"""%settings['ifdh_base_uri'])

Ah.. but :

Jobsettings.py line 96:

                self.settings['ifdh_base_uri'] = os.environ.get("IFDH_BASE_URI")
                if self.settings['ifdh_base_uri'] == None:
                        group=os.environ.get("GROUP","common")
                        self.settings['ifdh_base_uri'] = "http://samweb.fnal.gov:8480/sam/"+group+"/api" 

History

#1 Updated by Marc Mengel 7 months ago

So possibly only if it's not set when you do the launch...

#2 Updated by Dennis Box 6 months ago

I think this issue can be closed, it was discussed extensively in an email thread. Does anyone object?

#3 Updated by Dennis Box 7 days ago

  • Status changed from New to Closed


Also available in: Atom PDF