Project

General

Profile

Feature #5272

Milestone #5761: Release JobSub v0.2

Support user defined job attributes that can be validated against existing JobSub + Glideinwms deployment

Added by Parag Mhashilkar about 7 years ago. Updated about 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
JobSub Tools
Target version:
Start date:
01/27/2014
Due date:
% Done:

100%

Estimated time:
Stakeholders:
Duration:

Description

JobSub supports user defined attributes that get added to the HTCondor JDF. However these attributes and their values can not be validated against existing configuration. For example, users can say something like following, but the info like 'cvmfs' can not be validated.

jobsub -L IFOS=SL5

We require a better means to support this type of user-defined attributes that can change over time and can be accounted for in the JobSub and/or Glideinwms configuration.

Proposal:

JobSub tools should support a new command line option like following:

--resource-provides=IFOS=SL5,SL6,SL7 --resource-provides=CVMFS=OSG

Semantics:

  • Variables IFOS and CVMFS and their values in above example need to be configured in the JobSub tools experiment settings.
  • When Jobsub finds such an options, the variable name and their values are validated against the settings. For example, for following jobsub-tools ini configuration
[submit_host]
IFOS = SL5,SL6
CVMFS=FNAL,OSG,EGI
  • If the variable names or their values don't match, JobSub will not submit the job and exit with appropriate error message. So from the example above,
    --resource-provides=CVMFS=OSG, CVMFS should be a valid attribute with OSG as one of the values in the ini file.
  • JobSub will convert IFOS=SL5,SL6,SL7 to appropriate requirements in the job. In this example, Glideinwms should be configured by the operations so that HTCondor startds that glideins start should report back classad with HAS_IFOS set correctly.
+DESIRED_IFOS=SL5
requirements = stringListIMember(HAS_IFOS, DESIRED_IFOS, ",") [... Other requirements ...]

Or a better way to support for multiple HAS_* is to use stringListsIntersect instead of stringListIMember following this way a worker node for example can have multiple CVMFS configured

stringListsIntersect(toUpper(HAS_CVMFS), toUpper(DESIRED_IFOS), ",")

History

#1 Updated by Parag Mhashilkar about 7 years ago

  • Description updated (diff)

#2 Updated by Parag Mhashilkar about 7 years ago

  • Description updated (diff)

#3 Updated by Dennis Box about 7 years ago

  • Status changed from New to Resolved

resolved with jobsub_tools v1_2s

#4 Updated by Parag Mhashilkar about 7 years ago

  • % Done changed from 0 to 100

#5 Updated by Parag Mhashilkar about 7 years ago

  • Parent task set to #5761

#6 Updated by Parag Mhashilkar about 7 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF