Allow multiple remote directories for BOSCO submissions
When a job is using BOSCO (HTCondor remote batch gahp) it uses a directory on the remote resource submit host to store some software and as spool. This is by default $HOME/bosco. It would be nice to chose this, e.g. for clusters that share the same remote user home directory (like NERSC's Cori and Edison) so that different entries can have a different one.
HTCondor batch gahp allows this as parameter, so GWMS development will suffice to add this feature
#1 Updated by Dirk Hufnagel almost 4 years ago
I see this is listed with normal priority and target release v3_x. From the CMS side this isnn't neccessarily high priority, but from the hepcloud side it might be. I think this would make debugging new Bosco wrapper settings much easier while in parallel have other ongoing activity that isn't affected.
We also haven't really determined what happened with the pilots that ended up on the wrong NERSC cluster. With this feature in place we could either confirm this was something going wrong with the Bosco spool at NERSC (and have a way to work around it). Or we could point back at this likely being a problem the condor-g bosco submissions in the factory. Either way, being able to use multiple factory entries in parallel that give us access to different types of resources at NERSC IMO is critical to our ability to ramp up scale for NERSC.
As such, I think this is very much in the critical path for hepcloud, at least as far as ramping up scale in terms of number of jobs and cores is concerned.
Given this, what priority and target release would be appropriate for this ?