Project

General

Profile

Milestone #5333

Incorporate functionality of cdfcaf into JobSub

Added by Parag Mhashilkar almost 6 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Parag Mhashilkar
Category:
-
Target version:
Start date:
06/23/2014
Due date:
07/30/2014
% Done:

100%

Estimated time:
(Total: 0.00 h)
Spent time:
(Total: 9.00 h)
Stakeholders:

Run2 - CDF

Duration: 38

Description

Support for CDF CAF is growing thin and the functionality needs to be supported in case some one needs to analyze the CDF data in future. Some of the functionality provided by CDF CAF is same as JobSub. We need to identify the missing functionality and map common functionality to that provided in JobSub. Bo from CDF will help with testing and integration with CDF scripts. Current plan is to port the missing functionality to JobSub and Bo/Joe (?) would modify the the cdf caf submit to can JobSub internally.


Subtasks

Milestone #6689: Incorporate initial Run II Functionality for v0.4ClosedParag Mhashilkar

Bug #6562: dropbox:///some_file needs to put servers /path/to/some_file in the transfer_input_files listClosedDennis Box

Feature #6541: Sending KRBCC to the jobsClosedDennis Box

Bug #6551: Rename --unzipdir option in jobsub_fetchlog to something meaningfulClosedDennis Box

Bug #6586: make sure SAM projects work for CDFClosedDennis Box

Bug #6659: add -rfc flag to voms-proxy-init in auth.pyClosedDennis Box

Bug #6676: dropbox location and submit location end up on different hosts in DNS round robin setupClosedDennis Box

Bug #6657: need an equivalent of MAX_PARALLEL_SECTIONS for jobsubClosedDennis Box

Bug #6789: Make SAM_DATASET available in the job's environmentClosedDennis Box

History

#1 Updated by Parag Mhashilkar almost 6 years ago

  • Due date set to 05/30/2014

Meeting notes from March 07, 2014. Related tasks should be in there own ticket.

Present: Dennis Box, Parag Mhashilkar, Tanya Levshina, Joe Boyd, Mike Kirby, Stephan Lammel

(M) = Mandatory
(P) = Preferred
(N) = Not Required
(*) = higher in each category

CAFExe:

--tarFile(M): Input to the job. This needs to be extracted before executing the job. Jobsub supports it or can be made to change.

--outLocation(P): Tar ed up output directory. --depot in jobsub? If output is bigger than certain amount then only dirs are shipped back.
Need to come back to this one later for implementation details and available info

--procType (M): Is job long or short. We need to support in jobsub

--start, --end, --section (M): This is to user's job and for section and dag generation.

--dhaccess(N): How to handle the data handling.

--maxParallelSection(R): Run upto this dag nodes at a time.

--email(P*): Email for job notification. Needs to be passed by wrapper

--farm(N): JobSub equivalent --jobsub-server

--group(R): Already there in JobSub

--os: Already supported in JobSub

--cdfsoft: Moot with with cvmfs

--site: Already supported in Jobsub

--donotdrain(N)

command(R)

experiment wrapper : CAFExe or equivalent wrapper provided by JobSub

#2 Updated by Parag Mhashilkar over 5 years ago

  • Parent task set to #6025

#3 Updated by Parag Mhashilkar over 5 years ago

  • Target version changed from v0.3 to v0.3.1

#4 Updated by Parag Mhashilkar over 5 years ago

  • Parent task deleted (#6025)

#5 Updated by Parag Mhashilkar over 5 years ago

  • Target version changed from v0.3.1 to v0.4

#6 Updated by Parag Mhashilkar over 5 years ago

Pass Kerberos ticket securely #6541 High Priority
Concurrency limit #6657 High Priority
experiment wrapper No Ticket. CAFExe or equivalent wrapper provided by JobSub High Priority
--tarFile(M) #6523 Done
--outLocation(P) #6551 Done
--procType (M) No ticket -- Not required Very low priority
--start, --end, --section (M) #6697 Low priority
--maxParallelSection(M) #6657 This is required when we start supporting DAGs
--email(P*) #6697 Low Priority

#7 Updated by Parag Mhashilkar over 5 years ago

  • Target version changed from v0.4 to v1.0

#8 Updated by Parag Mhashilkar over 5 years ago

  • Status changed from New to Closed


Also available in: Atom PDF