Project

General

Profile

Feature #12001

Make Jobs X509 related info available in startds classad during the lifetime of the job

Added by Parag Mhashilkar over 3 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
03/21/2016
Due date:
% Done:

0%

Estimated time:
Stakeholders:

OSG

Duration:

Description

Hi parag,

Coming back to this -

Can you include the STARTD_JOB_ATTRS below as part of the defaults?

Brian

On Mar 11, 2016, at 10:44 AM, Parag A Mhashilkar wrote:

Hi Brian,

SYSTEM_JOB_MACHINE_ATTRS works other way round and puts machines attrs in the job’s classad. You want to use STARTD_JOB_ATTRS instead.

Here is a sample frontend config for getting payloads, x509 info into the machine's classad

<attr name="STARTD_JOB_ATTRS" glidein_publish="True" job_publish="False" parameter="True" type="expr" value="$(STARTD_JOB_ATTRS),x509userproxysubject,x509UserProxyFQAN,x509UserProxyVOName,x509UserProxyEmail,x509UserProxyExpiration"/>

And this is what you see in the startd’s classad for above setting

[testuser@fermicloud385 testjobs]$ condor_status -af Name x509userproxysubject x509UserProxyFQAN x509UserProxyVOName x509UserProxyEmail x509UserProxyExpiration

/DC=com/DC=DigiCert-Grid/O=Open Science Grid/OU=People/CN=Parag Mhashilkar 209917 /DC=com/DC=DigiCert-Grid/O=Open Science Grid/OU=People/CN=Parag Mhashilkar 209917,/dzero/users/Role=NULL/Capability=NULL,/dzero/Role=NULL/Capability=NULL dzero 1458334591

For the ProjectName and VOName, you may have to depend on the configuration settings in frontend or Job. There is no way of getting VO info in case of non-vomsproxy payloads. Said that you can somehow infer it from the AuthenticatedIdentity & GLIDEIN_* in the startd’s classad. You may need to come up with a map of these settings to conclude anything. Hmmm, more I think about it, can you have cron type custom script that populates this info from the map? At least custom scripts will keep this solution more generic.

Using a very minimalistic factory & frontend configuration, you will get following info from job’s classad

You may already know about the Memory consumption, Network activity, Disk usage, Committed times, Request Mem/Cpu/Disk related attributes so I am excluding them here.

  1. Some of the job related attributes if they are present and can be useful
    DESIRED_Sites = "Test_Site_2"
    GlobalJobId = "fermicloud385.fnal.gov#18.0#1457711403"
  1. Following are expanded in the equivalent MATCH
    JOB_GLIDEIN_ClusterId= "$$(GLIDEIN_ClusterId:Unknown)"
    JOB_GLIDEIN_Entry_Name = "$$(GLIDEIN_Entry_Name:Unknown)"
    JOB_GLIDEIN_Factory = "$$(GLIDEIN_Factory:Unknown)"
    JOB_GLIDEIN_Name = "$$(GLIDEIN_Name:Unknown)"
    JOB_GLIDEIN_ProcId = "$$(GLIDEIN_ProcId:Unknown)"
    JOB_GLIDEIN_Schedd = "$$(GLIDEIN_Schedd:Unknown)"
    JOB_GLIDEIN_Site = "$$(GLIDEIN_Site:Unknown)"
    JOB_Site = "$$(GLIDEIN_Site:Unknown)"

MATCH_EXP_JOB_GLIDEIN_ClusterId = "105"
MATCH_EXP_JOB_GLIDEIN_Entry_Name = "TEST_SITE_2"
MATCH_EXP_JOB_GLIDEIN_Factory = "GlideinFactory-master"
MATCH_EXP_JOB_GLIDEIN_Name = "v1_0"
MATCH_EXP_JOB_GLIDEIN_ProcId = "0"
MATCH_EXP_JOB_GLIDEIN_Schedd = "fermicloud385.fnal.gov"
MATCH_EXP_JOB_GLIDEIN_Site = "Test_Site_2"
MATCH_EXP_JOB_Site = "Test_Site_2"

  1. Info about the glidein
    MATCH_GLIDEIN_ClusterId = 105
    MATCH_GLIDEIN_Entry_Name = "TEST_SITE_2"
    MATCH_GLIDEIN_Factory = "GlideinFactory-master"
    MATCH_GLIDEIN_Name = "v1_0"
    MATCH_GLIDEIN_ProcId = 0
    MATCH_GLIDEIN_Schedd = "fermicloud385.fnal.gov"
    MATCH_GLIDEIN_Site = "Test_Site_2"
  1. x509 info
    x509UserProxyEmail = "xxxxxx"
    x509UserProxyExpiration = 1458334591
    x509UserProxyFirstFQAN = "/dzero/users/Role=NULL/Capability=NULL"
    x509UserProxyFQAN = "/DC=com/DC=DigiCert-Grid/O=Open Science Grid/OU=People/CN=Parag Mhashilkar 209917,/dzero/users/Role=NULL/Capability=NULL,/dzero/Role=NULL/Capability=NULL"
    x509userproxy = "/local/home/testuser/security/grid_proxy"
    x509userproxysubject = "/DC=com/DC=DigiCert-Grid/O=Open Science Grid/OU=People/CN=Parag Mhashilkar 209917"
    x509UserProxyVOName = "dzero"

On Mar 10, 2016, at 9:28 PM, Brian Bockelman wrote:

Hi Parag,

In OSG, we’d like to increase our use of the ClassAds which the GlideinWMS pilot forwards to the condor-ce’s collector.

I think we need:
- Payload DN and FQAN (if present).
- Any way we could have a VO name (even if there’s no payload proxy)? I’m a bit stumped on this actually.
- ProjectName.

IIRC, this can be done with:

SYSTEM_JOB_MACHINE_ATTRS

What else is relevant to forward from the job ad to the machine ad?

Thanks,

Brian

History

#1 Updated by Parag Mhashilkar over 3 years ago

  • Target version changed from v3_2_14 to v3_2_15

#2 Updated by HyunWoo Kim over 3 years ago

  • Status changed from New to Feedback
  • Assignee changed from HyunWoo Kim to Parag Mhashilkar

One line is added in
creation/web_base/condor_config

# This will create the following new attributes to Machine ClassAd and fetch the values from the corresponding attributes of Job ClassAd
STARTD_JOB_ATTRS = $(STARTD_JOB_ATTRS),x509userproxysubject,x509UserProxyFQAN,x509UserProxyVOName,x509UserProxyEmail,x509UserProxyExpiration

#3 Updated by Parag Mhashilkar over 3 years ago

  • Status changed from Feedback to Resolved
  • Assignee changed from Parag Mhashilkar to HyunWoo Kim

Looks ok. Merged.

#4 Updated by Parag Mhashilkar over 3 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF