Project

General

Profile

NOvA Integration

Contacts

FIFE Integration Lead - Gabriele Garzoglio
Experiment Integration Lead - Andrew John Norman
Experiment CS Liaison - Andrew John Norman

Services Being Integrated

CVMFS

Job Submission

glideinWMS Frontend and Factory

jobsub

OSG site prioritization

Resource tiers: OSC, OSG dedicated resources (Harvard, SMU, Prague), OSG opportunistic, Fermilab, Commercial Clouds

Requirement: submit as many jobs as possible to OSC, then OSG dedicated, then opportunistic, and finally FermiGrid.

Within a tier, pilots are distributed evenly, even if one of the resources is busy.

Time-based provisioning:

- 1,000 jobs in the queue
- Submit as many jobs as possible to OSC
- FrontEnd looks for jobs of type X (defined by the user) e.g. idle and submitted for more than 6 hours. Pilots are submitted proportionally to the number of jobs (typically 1 to 1).
- After more than 12 hours, a new resource tier is provisioned; etc.
- If all the jobs are run and there are remaining pilots, they timeout after 20 mins when they run.

Extreme case: to keep a resource busy, submit to it a lot of jobs and wait, even if using other tiers would take less time. In reality, we want to get jobs done as fast as possible.

The first timeline should be 2h more than the average job duration, so that only the left-over jobs are submitted to the 2nd tier.

CVMFS in scripts

ifdh

Lessons Learned