Project

General

Profile

Feature #17617

Check that files given to jobsub_submit -f will be transferrable with ifdh

Added by Kevin Retzke almost 2 years ago. Updated 9 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
JobSub Client
Target version:
Start date:
08/30/2017
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

From Jeremy Wolcott:

It seems like (now that BA has been unmounted from grid nodes) that
jobsub itself should be screening for 'impossible-to-access' files given
as '-f' arguments to jobsub_submit. (We've implemented a similar check
in the wrapper we use for most jobsub_submit invocations on NOvA, but
there are always cases where a user needs to use jobsub_submit directly,
and human-nature type mistakes will continue to happen.) Is there any
chance something like that could be deployed?

History

#1 Updated by Kevin Retzke almost 2 years ago

For clarification, this doesn't mean running ifdh to try accessing the file or such, since as Ken pointed out it's possible the files won't be accessible on the submit node. This probably entails comparing the file path to a whitelist or blacklist of (un)allowable locations.

#2 Updated by Dennis Box over 1 year ago

  • Target version set to v1.2.7

related issues: #17828 #16775 #12871 #13875

#3 Updated by Dennis Box over 1 year ago

In general: do the right thing with -f dropbox:// file:// and /pnfs

These are all related tickets ( and should be merged into one maybe?) #17617 #17828 #16775 #12871 #13785

#4 Updated by Dennis Box about 1 year ago

  • Assignee changed from Parag Mhashilkar to Dennis Box

#5 Updated by Dennis Box about 1 year ago

  • Target version changed from v1.2.7 to v1.2.8

#6 Updated by Dennis Box 9 months ago

  • Status changed from New to Closed

jobsub_submit now fails if it cant transfer a file to pnfs with -f and if the file doesnt exist there. closing ticket



Also available in: Atom PDF