Project

General

Profile

AWS VM probe

Current Version:1.14.3-3

https://cdcvs.fnal.gov/redmine/attachments/download/31050/gratia-probe-common-1.14.3-3.el6.noarch.rpm
https://cdcvs.fnal.gov/redmine/attachments/download/31051/gratia-probe-awsvm-1.14.3-3.el6.noarch.rpm

Installing (SLF6)

1. Install boto3 library (AWS API)

yum install epel-release
yum --enablerepo=epel install python-pip
pip install boto3

2. Install gratia-probe-common and gratia-probe-awsvm RPMs (see above links for latest)

3. Configure /etc/gratia/awsvm/ProbeConfig

Key values to change:

    ProbeName="awsvm:<hostname>" 
    SiteName="<site name registered in OIM>" 
    EnableProbe="1" 

Collector:
Change CollectorHost, SSLHost, and SSLRegistrationHost if not using OSG production collectors.
For HEP Cloud production the settings are:

    CollectorHost="gratia-fermi.fnal.gov:80" 
    SSLHost="gratia-fermi.fnal.gov:443" 
    SSLRegistrationHost="gratia-fermi.fnal.gov:80" 

For pre-prod:

    CollectorHost="fermicloud054.fnal.gov:8880" 
    SSLHost="fermicloud054.fnal.gov:8443" 
    SSLRegistrationHost="fermicloud054.fnal.gov:8880" 

4. AWS credentials in /root/.aws/credentials (or other file as set in AWSCredentialsFile in ProbeConfig).

This should contain a profile for each account being monitored. The profile name is used as the VO name, unless the
instance is tagged with the VO name. Example credentials file format:

[nova]
aws_access_key_id = <id>
aws_secret_access_key = <key>
[rnd]
aws_access_key_id = <id>
aws_secret_access_key = <key>
[cms]
aws_access_key_id = <id>
aws_secret_access_key = <key>
[fermilab]
aws_access_key_id = <id>
aws_secret_access_key = <key>

5. Enable probe

service gratia-probes-cron start
chkconfig --level 345 gratia-probes-cron on 

Maintenance

On-demand pricing data

The on-demand instance prices are not available via the AWS API, and so are distributed with the probe in /usr/share/gratia/awsvm/aws_demand_data_<date>.json.
However, the prices do change from time-to-time, and can be obtained from https://aws.amazon.com/ec2/pricing/.
There is an unofficial, unsupported "API" that is used to populate that webpage. The probe can use this directly, by setting HardwareDetailsURL to https://a0.awsstatic.com/pricing/1/ec2/linux-od.min.js (not recommended).
A better option is to manually run the included script and save the output to a file; set HardwareDetailsFile in ProbeConfig to this file.

python /usr/lib/python2.6/site-packages/gratia/awsvm/demand_data.py > new_demand_data.json

WARNING: Since this not using an official API it could disappear or stop functioning at any time! If that happens, the file will have to be manually updated (or a new "API" used to generate it).