Project

General

Profile

Feature #15171

Allow multiple remote directories for BOSCO submissions

Added by Marco Mambelli almost 4 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
High
Category:
-
Target version:
Start date:
01/17/2017
Due date:
% Done:

0%

Estimated time:
Stakeholders:

HEPCloud CMS

Duration:

Description

When a job is using BOSCO (HTCondor remote batch gahp) it uses a directory on the remote resource submit host to store some software and as spool. This is by default $HOME/bosco. It would be nice to chose this, e.g. for clusters that share the same remote user home directory (like NERSC's Cori and Edison) so that different entries can have a different one.
HTCondor batch gahp allows this as parameter, so GWMS development will suffice to add this feature

History

#1 Updated by Dirk Hufnagel almost 4 years ago

Hi,

I see this is listed with normal priority and target release v3_x. From the CMS side this isnn't neccessarily high priority, but from the hepcloud side it might be. I think this would make debugging new Bosco wrapper settings much easier while in parallel have other ongoing activity that isn't affected.

We also haven't really determined what happened with the pilots that ended up on the wrong NERSC cluster. With this feature in place we could either confirm this was something going wrong with the Bosco spool at NERSC (and have a way to work around it). Or we could point back at this likely being a problem the condor-g bosco submissions in the factory. Either way, being able to use multiple factory entries in parallel that give us access to different types of resources at NERSC IMO is critical to our ability to ramp up scale for NERSC.

As such, I think this is very much in the critical path for hepcloud, at least as far as ramping up scale in terms of number of jobs and cores is concerned.

Given this, what priority and target release would be appropriate for this ?

Cheers

Dirk

#2 Updated by Parag Mhashilkar almost 4 years ago

  • Target version changed from v3_x to v3_3_2
  • Stakeholders updated (diff)

#3 Updated by Parag Mhashilkar almost 4 years ago

  • Assignee set to Marco Mambelli

#4 Updated by Parag Mhashilkar over 3 years ago

  • Priority changed from Normal to High

#5 Updated by Marco Mambelli over 3 years ago

  • Status changed from New to Feedback
  • Assignee changed from Marco Mambelli to Parag Mhashilkar

#6 Updated by Parag Mhashilkar over 3 years ago

  • Status changed from Feedback to Resolved
  • Assignee changed from Parag Mhashilkar to Marco Mambelli

Looks ok. Merged to master

#7 Updated by Parag Mhashilkar over 3 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF