Services Being Integrated¶
glideinWMS Frontend and Factory¶
OSG site prioritization¶
Resource tiers: OSC, OSG dedicated resources (Harvard, SMU, Prague), OSG opportunistic, Fermilab, Commercial Clouds
Requirement: submit as many jobs as possible to OSC, then OSG dedicated, then opportunistic, and finally FermiGrid.
Within a tier, pilots are distributed evenly, even if one of the resources is busy.
- 1,000 jobs in the queue
- Submit as many jobs as possible to OSC
- FrontEnd looks for jobs of type X (defined by the user) e.g. idle and submitted for more than 6 hours. Pilots are submitted proportionally to the number of jobs (typically 1 to 1).
- After more than 12 hours, a new resource tier is provisioned; etc.
- If all the jobs are run and there are remaining pilots, they timeout after 20 mins when they run.
Extreme case: to keep a resource busy, submit to it a lot of jobs and wait, even if using other tiers would take less time. In reality, we want to get jobs done as fast as possible.
The first timeline should be 2h more than the average job duration, so that only the left-over jobs are submitted to the 2nd tier.