Have the --OS flag to the jobsub_submit command drive DesiredOS AND RequiredOS
Currently, the classad DesiredOS is populated using the --OS flag to jobsub_submit. This is fine for bare-metal running of jobs, since slots will advertise back IFOS_Installed, and we match the two.
However, for Singularity containers, the classad that has to be considered from the job side is called RequiredOS. Currently, the jobsub client has no method of natively setting that. The issue, then, is that if we submit a job like this:
jobsub_submit -G nova --OS SL6
The job will not match, for example, to a slot that is running SL7, but can launch an SL6 singularity container. Furthermore, the following will fail:
jobsub_submit -G nova --OS SL6 --lines='+RequiredOS=?="rhel6"'
because the OS flag will prevent the job from ever being matched to our example slot running SL7.
What we need to do is to have the --OS flag drive both DesiredOS and RequiredOS. The algorithm should be something like the following:
- --OS should set both DesiredOS and RequiredOS. RequiredOS should be mapped appropriately from SL* to rhel* as specified by GlideinWMS. The jobsub config file is a good place for this. Both would then be advertised in the job classad.
- Then, our job requirements expression should contain logic like the following: (IFOS_Installed in DesiredOS) OR HAS_SINGULARITY == true. If it's the latter, we're OK with just trusting whatever container is specified by RequiredOS, since that's how we launch containers currently.
In the future, we'd like to add a special singularity flag (maybe --singularity-container?) where you can specify a cvmfs location or URL of a container to run. In this case, you shouldn't specify the --OS flag at all, and we'll check for that.