Project

General

Profile

Idea #7888

Job start staggering to reduce instaneous resource demand

Added by Daniel Ruterbories over 4 years ago. Updated 3 months ago.

Status:
New
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
02/17/2015
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

I was wondering if there is currently the capability to stagger the start of certain types of jobs. Currently, for instance, I can submit a large number of calibration jobs which each access the database. I can stagger my submissions, but that doesn't guarantee they will start at different times due to the particular queue climate at the time of submission. What sometimes ends up happening is several hundred of these jobs hit the slots at the same time and end up waiting for their turn to access the DB. This causes huge efficiency losses if enough of these jobs start at the same time.

A solution is of course limit the number of jobs allowed to run at the same time. For my jobs in particular the DB access time is short in comparison to the processing time and happens all at once at run time. By limiting the number of jobs I can run at a time for fear of locking down a shared resource I'm increasing my full job set processing time by large amounts.

Could something like this be made available? It would greatly benefit job classes which use shared resources.

History

#1 Updated by Dennis Box 3 months ago

  • Target version set to v1.3.2
  • Assignee set to Shreyas Bhat


Also available in: Atom PDF