Project

General

Profile

Bug #10152

jobsub_submit_dag parses submit file incorrectly in some cases

Added by Dennis Box over 4 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Parag Mhashilkar
Category:
-
Target version:
v1.1.7
Start date:
09/15/2015
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

see RITM0270806.
a jobsub_submit_dag input file like so:

<parallel>
jobsub -n job1
jobsub -n job2
jobsub -n job3
<serial>
jobsub -n job4
jobsub -n job5
</serial>
</parallel>

should produce a DAG like so:

(begin dag)
/   |   |    \
     job1 job2 job3  job4
      |    |    |     |
      |    |    |    job5
      \    |    |    /
(end dag)

Instead, it produces garbage that condor_submit rightly rejects

dag2.jpg (142 KB) dag2.jpg Dennis Box, 10/13/2015 02:54 PM

History

#1 Updated by Dennis Box over 4 years ago

Dennis Box wrote:

see RITM0270806.
a jobsub_submit_dag input file like so:

<parallel>
jobsub -n job1
jobsub -n job2
jobsub -n job3
<serial>
jobsub -n job4
jobsub -n job5
</serial>
</parallel>

> should produce a DAG like so:
> 
>          (begin dag)
> 
>        /   |   |    \
>      job1 job2 job3  job4
>       |    |    |     |
>       |    |    |    job5
>       \    |    |    /
> 
>            (end dag)
> 
> Instead, it produces garbage that condor_submit rightly rejects

my diagram of the dag in the ticket seems to contains redimine wiki directives
trying to fix it with pre quotation

#2 Updated by Philip Rodrigues over 4 years ago

This is what I was after, thanks. The case that we'll want for minerva will be:


<parallel>
<serial>
jobsub -n job1
jobsub -n job2
</serial>
<serial>
jobsub -n job3
jobsub -n job4
</serial>
<serial>
jobsub -n job5
jobsub -n job6
</serial>
</parallel>

producing


       (begin dag)

        /    |    \
      job1  job3  job5
       |     |     |
      job2  job4  job6
       \     |     /

          (end dag)

#3 Updated by Dennis Box over 4 years ago

  • Target version set to v1.1.7

#4 Updated by Dennis Box over 4 years ago

  • Status changed from New to Feedback
  • Assignee changed from Dennis Box to Parag Mhashilkar

#5 Updated by Dennis Box over 4 years ago

Hi Phil,

Jobsub_tools v1.1.7.rc1, which has this fix, has been installed on fifebatch-dev.fnal.gov.

I said somewhere in this ticket that DAGs can be arbitrarily complicated, which is not quite true. Generated DAGs have the limitation that they must start and end with a '<serial>' section, so the input file needs to look like this:

<serial>
jobsub job1.sh
<serial>

arbitrarily complicated DAG can go in here

<serial>
jobsub jobN.sh
</serial>

I have attached a picture of one of my test DAGs that I think is similar to what Minerva wants to submit. If you want to try a test dag on fifebatch-dev I would like to know the result.

Dennis

#6 Updated by Philip Rodrigues over 4 years ago

I did a quick test and it seems to be working. Thanks!

#7 Updated by Dennis Box about 4 years ago

  • Status changed from Feedback to Closed

Also available in: Atom PDF