Add support to submit jobs for a workflow splitted in n DAGs
In the usual experiment workflow there are N jobs submitted for each exp stage.
The exp stages are sequential.
Then there is a special stage where the files of the last exp stage are merged and some code is execute to analyze data in the merged file to produce plots.
This kind of workflow can be represented by a DAG with independent branches, where the nodes in each branch are jobs for each exp stage.
Then another node depending on the jobs of the last exp stage can be added to the DAG.
In this configuration, if a job fails, the jobs in the other branches are not affected, but the job associated to the last node of the DAG will not be executed as there is a dependency that failed.
In a configuration where the exp workflow can be split in n DAGs where each DAG consist of independent branches, a failure in one of the job in one of the branches doesn't affect the subsequent DAG.
In this configuration, the first job that is executed for the last stage of a DAG, it will submit a monitor job that monitors the status of the jobs of the DAG and when all the jobs of the DAG are completed, the monitor job will submit the next DAG