Project

General

Profile

Feature #24465

Testing framework for distributed resources and GlideinWMS

Added by Marco Mambelli 5 months ago. Updated 3 months ago.

Status:
New
Priority:
Normal
Category:
-
Target version:
Start date:
07/29/2020
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Stakeholders:
Duration:

Description

Testing application for distributed resources and GlideinWMS. This application will interact with GlideinWMS components and send Glideins to explore and validate the resources. It will further allow to customize the Glideins and replace standard components of the GlideinWMS framework with new ones that require testing.
A description of the project can be found here:
https://drive.google.com/file/d/1lW1hFO3RdUcnPBbT1c9TFMJthk4mAZDC/view?usp=sharing

More about it:
  • ishould allow testing and validation of new Entries (sites where Glideins are running)
  • should allow testing of an individual worker node
  • should allow push (the Glidein is sent) and pull (the Glidein is requested) tets
  • should allow testing using existing Frontend groups and Factory entries configurations
  • should allow easy Glidein management (decide how many to send, remove currently running ones, ...)
  • should allow replacing some of the existing attributes or files
    • May allow trying easily multiple values or multiple files
  • should allow replacing also implicit files used by the Glidein (including glidein_startup.sh, condor_strtup.sh, ...)
Steps will include:
  • Propose a design
    • should it be integrated w/ GWMS or a separate app using it?
      • Some minimal modification of GWMS is needed to allow customizing custom scripts, the rest could be a separate project
    • which technologies should be used (languages, framework, DB)?
  • Development
    • Iterative phase, adding capabilities and components
    • Command line, maybe GUI
  • Integration testing
    • On Fermicloud first, on outside resources later
  • Release
Some interesting resources that can be integrated or provide examples:

Subtasks

Feature #24677: Add the possibility to override "attributes" (-params...) in manual_glidein_startupNewLerayah Neely-brown

History

#1 Updated by Marco Mambelli 3 months ago

  • Start date changed from 05/26/2020 to 07/29/2020
  • Due date set to 07/29/2020

due to changes in a related task: #24677

#2 Updated by Marco Mambelli 3 months ago

Adding here the reply form Marco Mascheroni to my email about the glidein in a vacuum:

Hi,

in our test factory (the ITB factory), we have a test entry that is dedicated to site testing:

https://github.com/opensciencegrid/osg-gfactory/blob/master/20-local-itb.xml#L4907

I wrote a cronjob that runs every day and uses the manual_glidein_startup tool to generate a wrapper script for the site admins. I do generate a wrapper script for each VO we have in the factory.

The result of the cronjob is shown here:

http://gfactory-itb-1.opensciencegrid.org/vacuum/

For example, for the CMS frontend (user fecmsglobalitb) we generate a wrapper script for each frontend group present. The one for the main group is:

http://gfactory-itb-1.opensciencegrid.org/vacuum/fecmsglobalitb/main_glidein_startup_wrapper

The idea is that an admin that wants to test his resources can just download this script and execute it on a worker node (running what we call a pilot in a vacuum)

Cheers,

Marco Mascheroni

[1]

[0926] root@gfactory-itb-1 ~# cat /root/mmascher/test_entry/generate_wrappers.sh
#!/bin/bash
BASEDIR="/var/www/html/vacuum/" 
ERR="$BASEDIR/err" 
OUT="$BASEDIR/out" 
rm $ERR $OUT
fename_clientname=$(condor_status -pool gfactory-itb-1.opensciencegrid.org -any -const 'MyType=="glideclient" && ReqName == "TEST_ENTRY@gfactory_instance@OSG-ITB"' -af 'strcat(regexps("(.*)@", AuthenticatedIdentity, "\1"), ":", ClientName)')

for item in $fename_clientname; do
  fename=${item%:*}
  clientname=${item#*:}
  groupname=${clientname#*.}
  fedir="$BASEDIR/${fename}" 
  mkdir -p "$fedir" 
  echo "Handling $fename (Client name is $clientname)" >> $OUT;
  /root/mmascher/test_entry/manual_glidein_startup --wms-collector="gfactory-itb-1.opensciencegrid.org" --client-name=$clientname --req-name="TEST_ENTRY@gfactory_instance@OSG-ITB" --cmd-out-file="$fedir/${groupname}_glidein_startup_wrapper" --glidein-startup="./glidein_startup.sh" >>$OUT 2>$ERR;
done



Also available in: Atom PDF