Project

General

Profile

Feature #21711

Add a portable condor_chirp for jobs running under GlideinWMS

Added by Marco Mambelli 11 months ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
01/16/2019
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

GlideinWMS does include condor_chirp
As requested by CMS [#6424], [#6430] to allow jobs to use CHIRP.
With the introduction of Singularity condor_chirp is not always available:
- the new environment may hide the Glidein condor to the job
- the job may run on a different platform (e.g. Glidein uses base OS, EL7, and job asks for and runs in EL6)

Some VOs ran into errors while trying to use the Glidein provided condor_chirp

A solution could be to download and make available a basic condor for the platform of the Singularity image, or make always available binaries for a set of platform or provide a multy-platform client (e.g. the python client https://github.com/htcondor/htchirp )


Related issues

Related to GlideinWMS - Milestone #19515: Roadmap for Singularity supportNew03/27/2018

History

#1 Updated by Marco Mambelli 11 months ago

As from an email from Jaime:

ssh-to-job and chirp should work with singularity jobs. For chirp, the singularity image will need to include a chirp client. There are several options for this:
  + The condor_chirp tool, which depends on the condor and classad libraries.
  + A pure python module, available here: https://github.com/htcondor/htchirp
  + Write your own client, as documented here: http://ccl.cse.nd.edu/software/chirp/

#2 Updated by Marco Mambelli 10 months ago

#3 Updated by Marco Mambelli 4 months ago

  • Target version changed from v3_5_1 to v3_6_1

#4 Updated by Marco Mambelli 3 months ago

  • Target version changed from v3_6_1 to v3_6_2

#5 Updated by Marco Mambelli about 1 month ago

  • Assignee set to Bruno Coimbra

The htchirp library is just an API that allows to use chirp functionalities in Python.
Users use condor_chirp, a client program.
This ticket should produce a drop-in replacement of condor_chirp, https://research.cs.wisc.edu/htcondor/manual/v8.7.9/Condorchirp.html
e.g. condor_chirp.py - accepting the same command-line options and behaving similarly

This should be distributed in GWMS together w/ the needed requirements (htchirp)
Possibly should be compatible w/ both python 2 (2.6, 2.7) and 3 (3.6).

http://research.cs.wisc.edu/htcondor/manual/current/condor_chirp.html
https://github.com/htcondor/htchirp

Note that the ND chirp server does not support HTCondor chirp default authentication. Tests should be done w/in a condor (interactive) job.
The goal once this is done is to package it got the Glideins and to contribute the code to the HTCondor team.
GlideinWMS will not maintain this code.

I started a github repo, pychirp, so we can test using the new development model. Please fork the repo and go from there: https://github.com/glideinWMS/pychirp



Also available in: Atom PDF