Project

General

Profile

Bug #12908

Error when submitting jobs with HTConodor 8.4.7 or later

Added by Marco Mambelli over 4 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Urgent
Category:
-
Target version:
Start date:
06/10/2016
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

When submitting a job form a GWMS schedd with HTCondor 8.4.7 or later the following error happens:

ERROR: Parse error in expression:
    JobAdInformationAttrs = JOB_Site, JOB_GLIDEIN_Entry_Name, JOB_GLIDEIN_Name, JOB_GLIDEIN_Factory, JOB_GLIDEIN_Schedd, JOB_GLIDEIN_ClusterId, JOB_GLIDEIN_ProcId, JOB_GLIDEIN_Site, JOB_GLIDEIN_SiteWMS, JOB_GLIDEIN_SiteWMS_Slot, JOB_GLIDEIN_SiteWMS_JobId, JOB_GLIDEIN_SiteWMS_Queue
    ^^^
Error in submit file

This is due to JobAdInformationAttrs in the condor configuration being in SUBMIT_EXPRS and not being a classad but a list (allowed in the macro language but not in classads)

History

#1 Updated by Marco Mambelli over 4 years ago

The changes in 8.4.7 are not compatible with 8.4.6: there is workaround and the condor team will keep us posted about a fix.

This works with 8.4.7:

JobAdInformationAttrs = "JOB_GLIDEIN_SiteWMS, JOB_GLIDEIN_SiteWMS_Slot" 
SUBMIT_EXPRS = +JobAdInformationAttrs
—>gives the correct classad (condor_q -l):
JobAdInformationAttrs = "JOB_GLIDEIN_SiteWMS, JOB_GLIDEIN_SiteWMS_Slot" 

So the lines towards the end of /etc/condor/config.d/02_gwms_schedds.config must be fixed by adding quotes and the "+":

JobAdInformationAttrs = "$(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)" 
SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) +JobAdInformationAttrs

Note that both the following result incorrect:

JobAdInformationAttrs = "JOB_GLIDEIN_SiteWMS, JOB_GLIDEIN_SiteWMS_Slot" 
SUBMIT_EXPRS = JobAdInformationAttrs
—> causes a wrong expression in the classad (condor_q -l):
JobAdInformationAttrs = "\"JOB_GLIDEIN_SiteWMS, JOB_GLIDEIN_SiteWMS_Slot\"" 

JobAdInformationAttrs = JOB_GLIDEIN_SiteWMS, JOB_GLIDEIN_SiteWMS_Slot
SUBMIT_EXPRS = JobAdInformationAttrs
—> gives the error

#2 Updated by Marco Mambelli over 4 years ago

This affects GWMS submit nodes (no factories and frontends only if they are used also to submit jobs) with HTCondor 8.4.7 (currently in osg-testing, to be released tomorrow) and 8.5.5 (latest development version).

HTCondor team is working on it: https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=5720

The workaround mentioned fixes the problem but is not compatible w/ 8.4.6 (the classad is ignored - tested) so there is no solution that can be implemented in GWMS.
The current agreement w/ OSG is that the issue will be handled via documentation, suggesting what to change in the configuration. Below the email sent to OSG software.

Greetings,
the current GWMS submit host configuration of the HTCondor schedd causes a parse error (Parse error in expression).
See ticket https://cdcvs.fnal.gov/redmine/issues/12908

This will affect all GWMS submit host running HTCondor 8.4.7 or 8.5.5
It will not affect the factory, it will affect the fronted only if it is used also to submit jobs.

* POSSIBLE CAUSE:
JobAdInformationAttrs (both submit file macro and classad atribute) it is first checked for correctness and then quoted.

* WORKAROUND
In order to work with HTCondor 8.4.7
The following 2 lines (below I reported also the comment before the lines in question) should be changed from:
# Add attributes to Job's event log
JobAdInformationAttrs = $(JOB_GLIDEIN_ATTRS), $(JOB_GLIDEIN_SITEWMS_ATTRS)

SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) JobAdInformationAttrs
— to (added quotes on the first and a plus “+” on the last):
# Add attributes to Job's event log
JobAdInformationAttrs = "$(JOB_GLIDEIN_ATTRS), $(JOB_GLIDEIN_SITEWMS_ATTRS)" 

SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) +JobAdInformationAttrs

* NOTE 
that these changes break compatibility with HTCondor 8.4.6 or earlier. The attribute JobAdInformationAttrs is not in the job classad and the extra attributes are not logged

Cheers,
Marco

PS 
I opened a condor ticket but was before understanding the exact problem: https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=5720
The ticket was closed, a new one may follow

#3 Updated by Marco Mambelli over 4 years ago

Here the code suggested by the condor team.
It will account for most versions (some 8.5 may have problems)

if version >= 8.4.7
      JobAdInformationAttrs = "$(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)" 
      SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) +JobAdInformationAttrs
else
      JobAdInformationAttrs = $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)
      SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) JobAdInformationAttrs
endif

#4 Updated by Marco Mambelli over 4 years ago

  • Status changed from New to Feedback
  • Assignee changed from Marco Mambelli to Parag Mhashilkar

#5 Updated by Parag Mhashilkar over 4 years ago

  • Target version set to v3_2_14_1

#6 Updated by Marco Mambelli over 4 years ago

Added the new changes taking into account also HTC 8.5

#7 Updated by Parag Mhashilkar over 4 years ago

  • Status changed from Feedback to Resolved

Looks ok. Merged to branch_v3_2

#8 Updated by Parag Mhashilkar over 4 years ago

  • Assignee changed from Parag Mhashilkar to Marco Mambelli

#9 Updated by Parag Mhashilkar over 4 years ago

  • Status changed from Resolved to Closed

#10 Updated by Marco Mambelli over 4 years ago

Just for reference here is the conditional code added in the configuration and taking into account of all condor versions (prod and devel):

if version <= 8.4.6
    JobAdInformationAttrs = $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)
    SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) JobAdInformationAttrs
elif version >= 8.5.0
    if version <= 8.5.4
        JobAdInformationAttrs = $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)
        SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) JobAdInformationAttrs
    elif version == 8.5.5
        JobAdInformationAttrs = "$(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)" 
        SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) +JobAdInformationAttrs
    else
        JobAdInformationAttrs = "$(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)" 
        SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) JobAdInformationAttrs
    endif
elif version == 8.4.7
    JobAdInformationAttrs = "$(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)" 
    SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) +JobAdInformationAttrs
else
    JobAdInformationAttrs = "$(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS)" 
    SUBMIT_EXPRS = $(SUBMIT_EXPRS) $(JOB_GLIDEIN_ATTRS) $(JOB_GLIDEIN_SITEWMS_ATTRS) JobAdInformationAttrs
endif

Also available in: Atom PDF