Feature #25535

Create a Glidein bundle that includes all the files downloaded from Factory and Frontend to be able to start without network connectivity

Added by Marco Mambelli 2 months ago. Updated 2 months ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:

Internship, CMS, HEPCloud



Glideins at the beginning download files from both the Factory and the Frontend (setup, VO scripts, ...).
Glidein in a vacuum allows starting a Glidein directly on a site without a submission from a Factory.
Sites like Theta and Barcelona do not have network access, from the worker nodes and would need all the files in advance.
Availability of a Web proxy could allow access but at times not even a Web proxy is available.

The proposal is for a service that can run on a Factory host but runs independently from the Factory, e.g. triggered by a post-reconfig hook.
This service would:
  • start locally a modified pilot
  • download all the files from the Factory and Frontend
  • create a tarball with all the files (everything that the Glidein needs)
Additionally, the Glidein should be modified to allow besides the download (wget/curl) also the reading from a local file,
and the Factory should be modified to heve a configuration know for a site to enable local scripts:
  • Get the tarball from the service
  • Have a different submit file including the tarball (as condor download)

And the Glidein in a vacuum mechanism/instruction should be updated to allow to run in this network-less modality

  • In the future some files (setup, VO scripts, ...) could be downloaded also from somewhere else (e.g. GitHub). Make so that this extension is easy
  • There may be use for different specialized Glideins (that do something different besides starting HTC startd, like this one that is packaging the files). Think about the design so that future pilot adaptations will be easier

Related issues

Related to GlideinWMS - Feature #25225: Integrate glidein in a vacuum in the factoryNew11/18/2020


#1 Updated by Marco Mambelli 2 months ago

  • Related to Feature #25225: Integrate glidein in a vacuum in the factory added

#2 Updated by Marco Mambelli 2 months ago

  • Category set to Glidein

Also available in: Atom PDF