Project

General

Profile

Feature #12085

Experiment-Defined Virtual Queues

Added by Alexander Himmel over 3 years ago. Updated 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
03/28/2016
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

We've been trying, with some difficulty, to coordinate our work around an external service which can only handle so much simultaneous load. --maxConcurrent helps, but it's been difficult to coordinate since there are multiple jobs, done by multiple people, which are all trying to use this same external resources. What we would really like is the ability to tell a job on submission that it belongs in some virtual queue, and we only allow X (ideally an easy to adjust number) of jobs in that queue to run simultaneously. This would allow us to limit the demand on this external resource while also allowing users to submit at any time without needing to coordinate with one another how many jobs they would run at the same time.

History

#1 Updated by Joe Boyd over 2 years ago

  • Priority changed from Normal to High

#2 Updated by Alexander Himmel over 2 years ago

We no longer use this workflow in production, so I would classify this as low priority from NOvA's perspective.

#3 Updated by Dennis Box about 2 years ago

  • Priority changed from High to Normal

#4 Updated by Dennis Box 2 months ago

  • Target version set to v1.4

#5 Updated by Alexander Himmel 2 months ago

For what it's worth, this is not a need for us anymore. We no longer run this problematic algorithm.

It's still a feature you might consider, though, since workflows like this might come back into vogue for LAr Experiments.



Also available in: Atom PDF