Bug #9234
Updating EOS mount sensor
100%
Description
Need to improve the sensor because the restart was leaving the eosd daemon dead.
History
#1 Updated by Gerard Bernabeu Altayo over 5 years ago
Thanks Natalia,
I could not find anything in the foswiki about this (https://cmsweb.fnal.gov/bin/view/Monitoring/WebSearch?tab=search&search=uscms_zabbix_plugins&scope=all&web=all&recurse=on), but as you said the history can be resonstructed somehow:
809 rm -rf uscms_zabbix_plugins-1.8.17 810 cd /usr/src/redhat/ 811 ls 812 cd SOURCES 813 ls 814 cp uscms_zabbix_plugins-1.8.18.tar.gz /root/ma 815 cd /root/ma 832 tar xvzf uscms_zabbix_plugins-1.8.18.tar.gz 833 ls -ltr 834 cd uscms_zabbix_plugins-1.8.18 835 ls 836 cd plugins/ 837 ls 838 cd src 839 ls 840 ls -ltr 841 scp root@cmsadmin1:/root/check_eos* /tmp 842 diff check_eos_mount /tmp/check_eos_mount 843 ls 844 cp /tmp/check_eos_mount . 845 ls 867 rm uscms_zabbix_plugins-1.8.18.tar.gz 868 history |grep tar 869 tar cvzf uscms_zabbix_plugins-1.8.18.tar.gz uscms_zabbix_plugins-1.8.18/ 870 ls -ltr 871 cp uscms_zabbix_plugins-1.8.18.tar.gz /usr/src/redhat/SOURCES/ 872 cd /usr/src/redhat/ 873 ls 874 cd SPECS 878 history |grep rpm 879 rpmbuild -ba --target=noarch uscms_zabbix_plugins.spec 880 cd /root/rpmbuild/ 881 ls 882 cd SOURCES 883 ls 884 pwd 885 cd - 886 cd /usr/src/ #A few (failed?) attempts to build the rpm 912 cp ../SOURCES/uscms_zabbix_plugins-1.8.18.tar.gz /root/rpmbuild/SOURCES 913 rpmbuild -ba --target=noarch uscms_zabbix_plugins.spec 914 cd /root/rpmbuild/ 939 cd /root/rpmbuild/ 940 ls 941 cd RP 942 cd RPMS/noarch/ 943 ls 944 scp uscms_zabbix_plugins-1.8.18-2.noarch.rpm root@cmsadmin1:/root 947 rpm -Uvh /tmp/uscms_zabbix_plugins-1.8.18-2.noarch.rpm
Will give it a try.... This seems to work:
[root@cmszabbix1 ma]# vim /usr/src/redhat/SPECS/uscms_zabbix_plugins.spec #Increase version number to 1.8.19 [root@cmszabbix1 ~]# ll /usr/src/redhat/SOURCES/uscms_zabbix_plugins-1.8.18.tar.gz -rw-r--r-- 1 root root 47035 May 28 13:19 /usr/src/redhat/SOURCES/uscms_zabbix_plugins-1.8.18.tar.gz [root@cmszabbix1 ~]# cd /usr/src/redhat/SOURCES/ [root@cmszabbix1 SOURCES]# tar -xvf /usr/src/redhat/SOURCES/uscms_zabbix_plugins-1.8.18.tar.gz [root@cmszabbix1 SOURCES]# mv uscms_zabbix_plugins-1.8.18 uscms_zabbix_plugins-1.8.19 [root@cmslpc33 ~]# scp /usr/local/bin/check_eos_mount cmszabbix1.fnal.gov:/usr/src/redhat/SOURCES/uscms_zabbix_plugins-1.8.19/plugins/src/check_eos_mount check_eos_mount 100% 5007 4.9KB/s 00:00 [root@cmslpc33 ~]# [root@cmszabbix1 SOURCES]# tar -czf uscms_zabbix_plugins-1.8.19.tar.gz uscms_zabbix_plugins-1.8.19 [root@cmszabbix1 SPECS]# cd /usr/src/redhat/SPECS [root@cmszabbix1 SPECS]# cp /usr/src/redhat/SOURCES/uscms_zabbix_plugins-1.8.19.tar.gz /root/rpmbuild/SOURCES/uscms_zabbix_plugins-1.8.19.tar.gz #Not sure why I have to do this... [root@cmszabbix1 SPECS]# rpmbuild -ba --target=noarch uscms_zabbix_plugins.spec [root@cmszabbix1 SPECS]# ll /root/rpmbuild/RPMS/noarch/uscms_zabbix_plugins-1.8.19-1.noarch.rpm -rw-r--r-- 1 root root 52270 Jun 17 15:16 /root/rpmbuild/RPMS/noarch/uscms_zabbix_plugins-1.8.19-1.noarch.rpm [root@cmszabbix1 SPECS]# rpm -vUh /root/rpmbuild/RPMS/noarch/uscms_zabbix_plugins-1.8.19-1.noarch.rpm Preparing... ########################################### [100%] 1:uscms_zabbix_plugins ########################################### [100%] [root@cmszabbix1 SPECS]# rpm -q uscms_zabbix_plugins uscms_zabbix_plugins-1.8.19-1.noarch [root@cmszabbix1 SPECS]# vim /usr/local/bin/check_eos_mount [root@cmszabbix1 SPECS]# grep -i version /usr/local/bin/check_eos_mount # Version: 3.4
So this works!
Will now put it in the repo:
ssh root@cms-install.fnal.gov scp root@cmszabbix1.fnal.gov:/root/rpmbuild/RPMS/noarch/uscms_zabbix_plugins-1.8.19-1.noarch.rpm /srv/repo/slf6-x86_64/ cd /srv/repo make uscmst1
Testing it now:
[root@cmslpc37 ~]# yum clean all --disablerepo=* --enablerepo=uscmst1 Loaded plugins: priorities, security Cleaning repos: uscmst1 Cleaning up Everything [root@cmslpc37 ~]# yum update --exclude=kernel* Loaded plugins: priorities, security Setting up Update Process uscmst1 | 2.5 kB 00:00 uscmst1/primary_db | 138 kB 00:00 7749 packages excluded due to repository priority protections Resolving Dependencies --> Running transaction check ---> Package uscms_zabbix_plugins.noarch 0:1.8.18-2 will be updated ---> Package uscms_zabbix_plugins.noarch 0:1.8.19-1 will be an update --> Finished Dependency Resolution
Adding all this in https://cdcvs.fnal.gov/redmine/issues/9234
For the SL5 machines I don't see this RPM, not sure how the check should be distributed....
Gerard
#2 Updated by Gerard Bernabeu Altayo over 5 years ago
- Due date set to 06/17/2015
- Status changed from New to Resolved
- % Done changed from 0 to 100
- Estimated time set to 1.00 h
Update done on the SL5 nodes manually:
[root@cmslpc33 ~]# scp cmslpc37:/usr/local/bin/check_eos_mount /usr/local/bin/check_eos_mount
check_eos_mount 100% 5007 4.9KB/s 00:00
[root@cmslpc33 ~]# for i in `seq 28 35`; do rsh cmslpc$i uname -a; done
Linux cmslpc28.fnal.gov 2.6.18-400.1.1.el5 #1 SMP Wed Dec 17 14:22:42 CST 2014 x86_64 x86_64 x86_64 GNU/Linux
Linux cmslpc29.fnal.gov 2.6.18-400.1.1.el5 #1 SMP Wed Dec 17 14:22:42 CST 2014 x86_64 x86_64 x86_64 GNU/Linux
Linux cmslpc30.fnal.gov 2.6.18-400.1.1.el5 #1 SMP Wed Dec 17 14:22:42 CST 2014 x86_64 x86_64 x86_64 GNU/Linux
Linux cmslpc31.fnal.gov 2.6.18-400.1.1.el5 #1 SMP Wed Dec 17 14:22:42 CST 2014 x86_64 x86_64 x86_64 GNU/Linux
Linux cmslpc32.fnal.gov 2.6.18-400.1.1.el5 #1 SMP Wed Dec 17 14:22:42 CST 2014 x86_64 x86_64 x86_64 GNU/Linux
Linux cmslpc33.fnal.gov 2.6.18-400.1.1.el5 #1 SMP Wed Dec 17 14:22:42 CST 2014 x86_64 x86_64 x86_64 GNU/Linux
Linux cmslpc34.fnal.gov 2.6.18-400.1.1.el5 #1 SMP Wed Dec 17 14:22:42 CST 2014 x86_64 x86_64 x86_64 GNU/Linux
Linux cmslpc35.fnal.gov 2.6.18-398.el5 #1 SMP Tue Sep 16 01:03:15 CDT 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@cmslpc33 ~]# for i in `seq 28 35`; do rsh -F cmslpc$i scp cmslpc37:/usr/local/bin/check_eos_mount /usr/local/bin/check_eos_mount; done
[root@cmslpc33 ~]#
The SL6 machines will pickup the new version from the new RPM :).
#3 Updated by Gerard Bernabeu Altayo over 5 years ago
I actually didn't make the script in a way that zabbix like the way we have it deployed, had to update and redeploy with:
-bash-4.1$ pssh -l root --verbose --outdir /tmp/pssh_output.gba --timeout 600 -h /usr/local/etc/farmlets/all 'yum clean all --disablerepo=* --enablerepo=uscmst1; yum update -y uscms_zabbix_plugins'
And then "rerun" zabbix like:
-bash-4.1$ pssh -l root --verbose --outdir /tmp/pssh_output.gba --timeout 600 -h /usr/local/etc/farmlets/workers 'service zabbix-agent restart'