Project

General

Profile

Feature #25535

Create a Glidein bundle that includes all the files downloaded from Factory and Frontend to be able to start without network connectivity

Added by Marco Mambelli 2 months ago. Updated 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Glidein
Target version:
Start date:
02/18/2021
Due date:
% Done:

0%

Estimated time:
Stakeholders:

Internship, CMS, HEPCloud

Duration:

Description

Glideins at the beginning download files from both the Factory and the Frontend (setup, VO scripts, ...).
Glidein in a vacuum allows starting a Glidein directly on a site without a submission from a Factory.
Sites like Theta and Barcelona do not have network access, from the worker nodes and would need all the files in advance.
Availability of a Web proxy could allow access but at times not even a Web proxy is available.

The proposal is for a service that can run on a Factory host but runs independently from the Factory, e.g. triggered by a post-reconfig hook.
This service would:
  • start locally a modified pilot
  • download all the files from the Factory and Frontend
  • create a tarball with all the files (everything that the Glidein needs)
Additionally, the Glidein should be modified to allow besides the download (wget/curl) also the reading from a local file,
and the Factory should be modified to heve a configuration know for a site to enable local scripts:
  • Get the tarball from the service
  • Have a different submit file including the tarball (as condor download)

And the Glidein in a vacuum mechanism/instruction should be updated to allow to run in this network-less modality

Considerations:
  • In the future some files (setup, VO scripts, ...) could be downloaded also from somewhere else (e.g. GitHub). Make so that this extension is easy
  • There may be use for different specialized Glideins (that do something different besides starting HTC startd, like this one that is packaging the files). Think about the design so that future pilot adaptations will be easier

Related issues

Related to GlideinWMS - Feature #25225: Integrate glidein in a vacuum in the factoryNew11/18/2020

History

#1 Updated by Marco Mambelli 2 months ago

  • Related to Feature #25225: Integrate glidein in a vacuum in the factory added

#2 Updated by Marco Mambelli 2 months ago

  • Category set to Glidein

Also available in: Atom PDF