Add a portable condor_chirp for jobs running under GlideinWMS
GlideinWMS does include condor_chirp
As requested by CMS [#6424], [#6430] to allow jobs to use CHIRP.
With the introduction of Singularity condor_chirp is not always available:
- the new environment may hide the Glidein condor to the job
- the job may run on a different platform (e.g. Glidein uses base OS, EL7, and job asks for and runs in EL6)
Some VOs ran into errors while trying to use the Glidein provided condor_chirp
A solution could be to download and make available a basic condor for the platform of the Singularity image, or make always available binaries for a set of platform or provide a multy-platform client (e.g. the python client https://github.com/htcondor/htchirp )
#1 Updated by Marco Mambelli over 1 year ago
As from an email from Jaime:
ssh-to-job and chirp should work with singularity jobs. For chirp, the singularity image will need to include a chirp client. There are several options for this: + The condor_chirp tool, which depends on the condor and classad libraries. + A pure python module, available here: https://github.com/htcondor/htchirp + Write your own client, as documented here: http://ccl.cse.nd.edu/software/chirp/
#5 Updated by Marco Mambelli 8 months ago
- Assignee set to Bruno Coimbra
The htchirp library is just an API that allows to use chirp functionalities in Python.
Users use condor_chirp, a client program.
This ticket should produce a drop-in replacement of condor_chirp, https://research.cs.wisc.edu/htcondor/manual/v8.7.9/Condorchirp.html
e.g. condor_chirp.py - accepting the same command-line options and behaving similarly
This should be distributed in GWMS together w/ the needed requirements (htchirp)
Possibly should be compatible w/ both python 2 (2.6, 2.7) and 3 (3.6).
Note that the ND chirp server does not support HTCondor chirp default authentication. Tests should be done w/in a condor (interactive) job.
The goal once this is done is to package it got the Glideins and to contribute the code to the HTCondor team.
GlideinWMS will not maintain this code.
I started a github repo, pychirp, so we can test using the new development model. Please fork the repo and go from there: https://github.com/glideinWMS/pychirp
#6 Updated by Bruno Coimbra 5 months ago
- % Done changed from 0 to 100
- Status changed from New to Feedback
A new Python-based condor_chirp was added as a drop-in replacement for the original condor_chirp.
These changes also include gwms-python, a wrapper script for Python to be used by GlideinWMS scripts written in this language.
The new condor_chirp uses htchirp client which we are shipping as a tarball while the newest version of the package is not published.
The changes can be found on the following branch: