Cloud glideins do not shut down VMs all the time
Steve Timm pointed this from his observation in the new glideinwms launched VMs in Open Nebula but the problem is generic. Problem could occur when VMs are booted manually or if the pilot-launcher throws exception before dropping privileges to the user glidein_pilot.
He traced down the problem to additional line in /etc/sudoers file that is shipped with redhat
If the pilot launcher throws exception before dropping privileges, rest of the glidein process continues as root. Although this is not a big security issue, the above requiretty line prevents the pilot-launcher to shutdown as root. glidein rpms adds following line to the sudoers file giving glidein_pilot user access to shutdown the VM
Much cleaner way to handle this is to always make sure that we drop privileges in case of exceptions during the startup.