Project

General

Profile

Release usage

Keeping track of release usage, or it's not just NSA watching you

It is often useful for the software librarian to know what releases are in common daily usage. This allows one to know what releases are being used and by whom. The scripts below form a system where messages are formatted by the MINOS setup script, handed to the client to be sent to the server; the server then writes these to log files for review and analysis.

The exact message that is sent is up to whomever is calling the client script. In MINOS we send the following:

  • user shell (e.g. csh vs. bash)
  • user (best guess in case of condor glidein jobs based on ${X509_USER_PROXY})
  • release version
  • SRT optimization flag (i.e. debug vs. maxopt)
  • kernel version of the node
  • OS version of the node

The server automagically records the following bits:

  • time the message was received
  • node from whence message was sent

In the setup script there is code similar to:

  AUSER="$USER" 
  if [ -z "$AUSER" ] ; then AUSER=`whoami` ; fi
  if [ -z "$AUSER" ] ; then AUSER=`id -u`  ; fi
  if [ -z "$AUSER" ] ; then AUSER="nouser" ; fi
  if [ -n "${X509_USER_PROXY}" ] ; then
     PUSER=`basename ${X509_USER_PROXY}`
     AUSER="$AUSER/$PUSER" 
  fi

  kernel=`uname -r`
  sl=`cat /etc/redhat-release | tr -d "[:alpha:] ()" `

  MSG="${SHELL} ${AUSER} ${release} ${optflags} ${kernel} ${sl}" 
  /grid/fermiapp/minos/minossoft/setup/datagram/datagram_client.py "${MSG}"   # "" to make it single arg, preserve spaces

The minos logs can be found in the directory: /minos/app/minsoft/datagram/ as zzz_YYYYMMDD_hhmmss.log where the log file name encapsulates when the server was started (in a manner that sorts the files naturally by name (YYYY=year, MM=month, DD=day, hh=hour, mm=minute, ss=second).

Sender

The job of the sender script is to send a UDP packet to the client. The client script also has the ability to send a special message to the server to tell it to cleanly shutdown. The datagram_client.py script looks like:

#! /usr/bin/env python
# The basis for this UDP (User Datagram Protocol) portion of this code
# came from:  http://en.wikipedia.org/wiki/User_Datagram_Protocol  (2008-05-01)

import os, sys, getopt
import socket

SERVER_ADDRESS = 'minos27.fnal.gov'
SERVER_PORT = 12345

def Usage():
    print "usage: datagram_client \"message\"" 
    print "  -a --addr=   address [minos26.fnal.gov]" 
    print "  -p --port=   port #  [12345]" 
    print "  --shutdown   request server shutdown" 

try:
    optpairs, args = \
          getopt.getopt(sys.argv[1:],\
                        'a:p:',\
                        ["addr=","port=","shutdown"])
except getopt.GetoptError:
    Usage()
    sys.exit(1)

client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)

for io in range(len(optpairs)):
    kv = optpairs[io]
    opt1 = kv[0]
    arg1 = kv[1]
    for k in ('-a','--addr'):
        if (k == opt1):
            SERVER_ADDRESS = arg1
    for k in ('-p','--port'):
        if (k == opt1):
            SERVER_PORT = int(arg1)
    if (opt1 == '--shutdown'):
        args.append('SPECIAL_SHUTDOWN_MESSAGE')

for message in args:
    #print 'Sending packet "%s"' % message
    client.sendto(message,(SERVER_ADDRESS, SERVER_PORT))

client.close()

Receiver

On the receiver side there is a server that receives the UDP packets and prints them to standard out (the datagram_server.py script). This is wrapped in a script that starts the server and redirects output to a named log file (the run_datagram.sh script). The driver script is restarted nightly to keep manageable logs.

The entry in the crontab to start the server looks like:

# datagram server for recording setup invocations
# restart this every day so that log files are manageable and easily searched
#   script now sets $SRT_DIST
#   if run from other than minos27, need to change datagram_client.py
#   to send packets to the right node
#
0 0 * * * /grid/fermiapp/minos/minossoft/setup/datagram/run_datagram.sh

The actual run_datagram.sh code looks like:

#! /usr/bin/env bash
# This script runs the datagram server that receives UDP packets from 
# clients.  The data sent by the client is basically a string and the
# server simply writes it down into the log file.
#
# in FNAL crontab
#   # was:  sg e875 " <script> "; now just <script>
#   # sg = "set group" because minsoft account was "mysql" rather than "e875" 
#   0 0 * * * /grid/fermiapp/minos/minossoft/setup/datagram/run_datagram.sh
# this restarts the server daily to keep the logs easily searched and
# manageable sized.

# we need to know where this was installed so we can run it
export SRT_DIST=/grid/fermiapp/minos/minossoft
cd ${SRT_DIST}/setup/datagram

echo "in `pwd`/run_datagram.sh" 
echo "SRT_DIST=${SRT_DIST}" 
#
./datagram_client.py --shutdown "run_datagram.sh requests shutdown" 
#
logtag=`date +%Y%m%d_%H%M%S`
export OUTFILE=/minos/app/minsoft/datagram/zzz_${logtag}.log
echo restarted server ${logtag}
echo log output to $OUTFILE
touch                  $OUTFILE
chmod g+w              $OUTFILE
./datagram_server.py > $OUTFILE 2>&1
#
endtime=`date +%Y%m%d_%H%M%S`
echo completed at ${endtime}
# end-of-script

The actual datagram_server.py script is also small:

#! /usr/bin/env python
# The basis for this UDP (User Datagram Protocol) portion of this code
# came from:  http://en.wikipedia.org/wiki/User_Datagram_Protocol  (2008-05-01)

import socket, time, sys

PORT = 12345
BUFLEN = 512

server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
server.bind(('', PORT))

docontinue = True
while docontinue:
    (message, address) = server.recvfrom(BUFLEN)
    rtime = time.strftime("%Y-%m-%d %H:%M:%S",time.localtime())
    rmach = socket.gethostbyaddr(address[0])[0]
    if ( message == 'SPECIAL_SHUTDOWN_MESSAGE'):
        sys.stdout.write(rtime)
        sys.stdout.write(' Received SHUTDOWN message from %s\n'%rmach)
        docontinue = False
    else:
        sys.stdout.write(rtime)
        sys.stdout.write(' %s : \"%s\"\n'%(rmach, message))
    sys.stdout.flush()

server.close()