Feature #25535
Create a Glidein bundle that includes all the files downloaded from Factory and Frontend to be able to start without network connectivity
Start date:
02/18/2021
Due date:
% Done:
0%
Estimated time:
Stakeholders:
Internship, CMS, HEPCloud
Description
Glideins at the beginning download files from both the Factory and the Frontend (setup, VO scripts, ...).
Glidein in a vacuum allows starting a Glidein directly on a site without a submission from a Factory.
Sites like Theta and Barcelona do not have network access, from the worker nodes and would need all the files in advance.
Availability of a Web proxy could allow access but at times not even a Web proxy is available.
This service would:
- start locally a modified pilot
- download all the files from the Factory and Frontend
- create a tarball with all the files (everything that the Glidein needs)
and the Factory should be modified to heve a configuration know for a site to enable local scripts:
- Get the tarball from the service
- Have a different submit file including the tarball (as condor download)
And the Glidein in a vacuum mechanism/instruction should be updated to allow to run in this network-less modality
Considerations:- In the future some files (setup, VO scripts, ...) could be downloaded also from somewhere else (e.g. GitHub). Make so that this extension is easy
- There may be use for different specialized Glideins (that do something different besides starting HTC startd, like this one that is packaging the files). Think about the design so that future pilot adaptations will be easier
Related issues
History
#1 Updated by Marco Mambelli 2 months ago
- Related to Feature #25225: Integrate glidein in a vacuum in the factory added
#2 Updated by Marco Mambelli 2 months ago
- Category set to Glidein