Project

General

Profile

Bug #23109

can jobsub server create haproxy heartbeat file related to schedd health

Added by Dennis Box about 1 month ago. Updated 27 days ago.

Status:
New
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
08/13/2019
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

this could streamline a lot of code and make scaling out servers easier

History

#1 Updated by Dennis Box 27 days ago

I am trying this out on a jobsub server on a fermicloud node. Comments and suggestions are welcome.

  • part one, the cron to run this once a minute non blocking
    * * * * * /usr/bin/flock -n /tmp/is_schedd_healthy.lockfile /var/lib/condor/modules/is_schedd_healthy 
    
  • part two, the actual script. If httpd or condor are healthy, create /var/www/html/heartbeat . If not, remove heartbeat
    #!/bin/bash
    www=/var/www/html  #for deployment
    #www=$(pwd)          #for testing
    HEALTH=${www}/schedd_health
    HEART=${www}/heartbeat
    if  wget --no-check-certificate  "https://$HOSTNAME/cigetcertopts.txt" -qO /dev/null &&
        rdcd=$(condor_status -constraint "name=?=\"$HOSTNAME\"" -schedd -af  recentdaemoncoredutycycle) &&
        echo $rdcd | grep -q [0-9] &&
        (( $(echo "0.0E-09 < $rdcd" |bc -l) )) && 
        (( $(echo "$rdcd < 9.0E-01" |bc -l) ))
    then
        echo "$rdcd" > $HEALTH
        touch $HEART
    else
        rm -f $HEALTH
        rm -f $HEART
        false
    fi
    
    


Also available in: Atom PDF