Campaign Stage Types Proposal¶
In recent POMS releases we've added a field for stage types, but it is so far largely unused.
In upcoming releases I would like to use this field, and back it up with some code. Possible
campaign stage types are:
- Test -- means this is a stage designed to test a Job Type and may not be part of a campaign, etc.
- Data Transfer -- means this stage is used to move data around, and it's output dataset is the same as its input dataset
- Data Merge -- means that multiple dependencies entering this stage all wait for all of them and do one submission
- Resplit -- means that any split types on this stage should be applied to the input dataset; and also it acts as a data merge stage
- Generator -- means that
- there is no input dataset and that
- the submissions will generate poms_depends_${POMS_TASK_ID}_${i} datasets for i in 1..${POMS_NUM_DEPENDENCIES}
- Any split_type info is really a parameter list rather than a dataset list, so it should set %(parameter)s (as well as %dataset)s for backwards compatability)
These should be reflected in the editor with suitable