Balancing glidein pressure to sites that are aliases or Meta-Sites
The pressure to a site is difficult to balance when there is not a one-to-one setup: one site, one gatekeeper (CEs) in front of one cluster.
In actual sites there can be:
- multiple gatekeepers in front from one single cluster, all having its own entry in the factory configuration.
- a single alias and a single entry in the factory configuration that refers to multiple gatekeepers (via DNS or load balancer) that give access to the same cluster
- a single alias and a single entry in the factory configuration that refers to multiple gatekeepers (via DNS or load balancer) that give access to distinct clusters
Here I use the term cluster to refer to a set of worker nodes in the queue of a batch system (aka job manager or local resource manager). This is a bit of a simplification since some worker nodes may actually be shared across multiple queues (that compete for them) or may be acquired dynamically, e.g. via cloud submission or other elastic systems.
Factory administrators prefer to have a reduced number of entries:
- to shorten configuration
- to avoid to have to keep multiple entries consistent
It is important to keep the desired pressure to the site no matter which the configuration
#11 Updated by Marco Mascheroni almost 2 years ago
I pusched today a first version of this.
Here is the feedback I got after a chat with Marco Mambelli:
- Add the possibility of st limits for each entry in the entryset (currently entry could only be empty)
- Improve how load is divided between two different gatekeepers of the same entry set
- Verify that configuration generation works with merged files as used by ops
- My comment: move auth_method and trust_domain to the entry_set level
#12 Updated by Marco Mascheroni almost 2 years ago
Today ad the GWMS meeting I presented my first implementation (https://docs.google.com/presentation/d/140DOV4E_5VmJyxns_tfcPGYmzohy4pyKMFfI4ASt-j4/edit?usp=sharing)
There were no major comment besides the need of a monitor breakdown per/entry (currently it is per entry_set). I think this requires some work though, So I would like to get a first version out ASAP since Jeff kindly voolounteered to try it out on a test factory at UCSD once we have something to try out. Also, the possibility of setting limits for each entry in the entryset is something we can add later.
Since I just finished improving how load is divided between two different gatekeepers of the same entry set, I would like to do a full validation, and then check out the configuration merge. Then I think we can include this first version in the next release.