Project

General

Profile

Feature #25453

Mechanism for file selection in Glidein files

Added by Marco Mambelli 3 months ago. Updated 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
01/31/2021
Due date:
% Done:

0%

Estimated time:
Stakeholders:

internship

Duration:

Description

A Glidein will download arbitrary files from Factory and Frontend.
Would be useful to have a generic mechanism similar to the HTCondor tarball download selection, where multiple files are made available in the stage area but only the appropriate one is downloaded.
A first to benefit would be the cvmfsexec extension (requested by CMS, OSG).

Some ideas:
  • file names have a base name and end with a set of selectors separated by a special character (dash?)
  • add an attribute to the file basename:selectors_number ()
  • selectors are dash (or dot? may confuse w/ extension/s .tgz .tar.gz) separated, last part of the name before extension
  • no dots in the name except for extensions, no dashes in the selectors' names
  • basename-setup - to run before the files
  • basename-selector - to run to select instead of the default
  • basename - used in the download list to trigger the proper download (may be replaced by -selector or -setup)
Default selector
  • use the variable if available (maybe variable before selector file? )
  • use basename-selector if available
  • add a default?
Allow macros that can be used in the selection variable (values from the system that are replaced w/o the need to modify the variable):
  • OS - e.g. rhelX debX ubuntuX
  • ARCH - architecture x86 x86_64 ppc64 ppc64le
    • (control the current names used in other parts, e.g. condor tarball selection)
  • CONDOR_VERSION?
  • ENV_VARIABLES?
    what else could be useful and generic?
    just name or % or other char for macro substitution

This is self-contained, bash development, could be an internship project:
Flexible, system-dependent pilot extensions

Also available in: Atom PDF