Feature #22843

Develop a Testing framework for distributed resources and GlideinWMS

Added by Marco Mambelli almost 2 years ago. Updated 2 months ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


Build a testing application for distributed resources (entries) and GlideinWMS.
This application will interact with GlideinWMS components and send Glideins to explore and validate the resources.
It will further allow to customize the Glideins and replace standard components of the GlideinWMS framework with new ones than the user decides to test.

This project includes participation in the following activities:
  • Provide input in the design of the testing application
  • Getting familiar with distributed computing and GlideinWMS
  • Development and integration related to distributed computing software for Grids, Clouds and Supercomputers (this will include Python and Web development):
    • Adapting components to talk the GlideinWMS protocol
    • Adding a UI to drive the testing
  • Testing on clusters, High-Performance Computers and clouds
  • Integration and release
Some shortcomings of the current system that should be improved and some suggestions:
  • No flexible way to send pilots to specific resources
  • No way to substitute standard scripts
  • Manual client checking which resources are available, what's the status and letting the user decide how many pilots to send and where
  • The code written should be in Python 3 (specifically 3.6)
  • Consider migrating some of the code in GlideinWMS (e.g. protocol sending ClassAds) using python 3.6 and htcondor python bindings

Here a project doing something similar with shell scripts:

Related issues

Related to GlideinWMS - Milestone #22673: Summer interns 2019New06/03/201909/30/2019


#1 Updated by Marco Mambelli almost 2 years ago

#2 Updated by Javier Rodriguez almost 2 years ago

  • % Done changed from 0 to 70

#3 Updated by Javier Rodriguez almost 2 years ago

  • % Done changed from 70 to 100
  • Status changed from New to Resolved

#4 Updated by Javier Rodriguez almost 2 years ago

  • % Done changed from 100 to 80
  • Status changed from Resolved to Assigned

#5 Updated by Marco Mambelli over 1 year ago

Here a concrete task:
Start with
Add the possibility to modify the parameters used in the submission (params dictionary) by reading them from a file in INI format. The file name will be specified in a command option, e.g. --parameters-file FILE_NAME , where FILE_NAME is the path to the INI file


Step 2 (more difficult)
Use the YAML format instead of INI and run the manual_glidein_submit with python3.
This may require some changes in the GlideinWMS code imported.

#6 Updated by Marco Mambelli over 1 year ago

  • % Done changed from 80 to 0
  • Target version set to v3_5_x
  • Assignee changed from Javier Rodriguez to Marco Mambelli
  • Status changed from Assigned to New

Javier worked on a change to to allow extra attr options loaded from an ini file.
The ticket is being opened for future re-assignment

#7 Updated by Marco Mambelli over 1 year ago

  • Target version changed from v3_5_x to v3_6_x

#8 Updated by Marco Mambelli 2 months ago

  • Target version changed from v3_6_x to v3_7_x

Also available in: Atom PDF