Bug #10982
Upgrade FW on FSTs with qlcnic NIC
0%
Description
This is within PRB000000001100.
In order to check if the QLA is #1 one needs to run:
[root@cmschimeradiskbackup qlcnic]# echo -e "1\n1\n1\n\n99\n" | /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli > /tmp/out.gba [root@cmschimeradiskbackup qlcnic]# grep QLE32 /tmp/out.gba 1: CNA Model: QLE3242 SN: RFE1321H85939 Adapter Model : QLE3242 1: CNA Model: QLE3242 SN: RFE1321H85939 [root@cmschimeradiskbackup qlcnic]#
In order to update the command is:
mkdir /root/qlcnic; cd /root/qlcnic; scp cmsstor266:/root/qlcnic/CLI/QConvergeConsoleCLI-1.1.04-65.x86_64.rpm . && rpm -vUh QConvergeConsoleCLI-1.1.04-65.x86_64.rpm; scp cmsstor266:/root/qlcnic/p3p23204.bin /root/qlcnic/p3p23204.bin; lsmod | grep qlcnic && lspci | grep cLOM8214 && echo -e "3\n1\n1\n/root/qlcnic/p3p23204.bin\nyes\nyes\n\n99\n" | /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli
I will first check that all nodes have the NIC as #1 and if so then I will perform the FW upgrade on them.
History
#1 Updated by Gerard Bernabeu Altayo about 5 years ago
I'm getting the list of nodes from EOS itself:
[root@cmssrv238 ~]# eos node ls | grep nodesview | awk '{print $2}' | cut -d: -f1 | sort cmseos11.fnal.gov cmseos12.fnal.gov cmseos13.fnal.gov cmseos14.fnal.gov cmseos15.fnal.gov cmseos16.fnal.gov cmseos17.fnal.gov cmseos1.fnal.gov cmseos25.fnal.gov cmseos26.fnal.gov cmseos27.fnal.gov cmseos28.fnal.gov cmseos29.fnal.gov cmseos2.fnal.gov cmseos30.fnal.gov cmseos31.fnal.gov cmseos32.fnal.gov cmseos33.fnal.gov cmseos34.fnal.gov cmseos35.fnal.gov cmseos36.fnal.gov cmseos37.fnal.gov cmseos38.fnal.gov cmseos39.fnal.gov cmseos3.fnal.gov cmseos40.fnal.gov cmseos41.fnal.gov cmseos42.fnal.gov cmseos43.fnal.gov cmseos44.fnal.gov cmseos45.fnal.gov cmseos46.fnal.gov cmseos47.fnal.gov cmseos48.fnal.gov cmseos49.fnal.gov cmseos4.fnal.gov cmseos50.fnal.gov cmseos51.fnal.gov cmseos52.fnal.gov cmseos53.fnal.gov cmseos54.fnal.gov cmseos55.fnal.gov cmseos56.fnal.gov cmseos57.fnal.gov cmseos58.fnal.gov cmseos59.fnal.gov cmseos5.fnal.gov cmseos60.fnal.gov cmseos61.fnal.gov cmseos62.fnal.gov cmseos63.fnal.gov
First I will install the util on all nodes:
pssh -l root -p50 --hosts=eos.fst.list -o eosfsts-qlcnic.out -e eosfsts-qlcnic.err -t 0 'rpm -q QConvergeConsoleCLI && exit; mkdir /root/qlcnic; cd /root/qlcnic; scp cmsstor266:/root/qlcnic/CLI/QConvergeConsoleCLI-1.1.04-65.x86_64.rpm . && rpm -vUh QConvergeConsoleCLI-1.1.04-65.x86_64.rpm;'
Now I check if there is such an interface:
-bash-4.1$ pssh -l root -p50 --hosts=eos.fst.list -o eosfsts-qlcnic.out -e eosfsts-qlcnic.err -t 0 'echo -e "1\n1\n1\n\n99\n" | /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli' ... -bash-4.1$ cd eosfsts-qlcnic.out -bash-4.1$ grep QLE32 cmseos* | cut -d. -f1 | sort -u cmseos1 cmseos11 cmseos12 cmseos13 cmseos14 cmseos15 cmseos16 cmseos17
Now I perform the actual upgrade:
-bash-4.1$ ssh -l root -p50 --hosts=qlcnic.eos.list -o eosfsts-qlcnic.out -e eosfsts-qlcnic.err -t 0 'scp cmsstor266:/root/qlcnic/p3p23204.bin /root/qlcnic/p3p23204.bin; lsmod | grep qlcnic && lspci | grep cLOM8214 && echo -e "3\n1\n1\n1\n/root/qlcnic/p3p23204.bin\nyes\nyes\n\n99\n" | /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli '
I'll let this run, in the mean time I'll make sure no other node has this NIC and actually failed the pssh.
#2 Updated by Gerard Bernabeu Altayo about 5 years ago
- Status changed from New to Resolved
I manually checked and all the servers with the QLCNIC have been patched.
Most had the NIC as eth0, some as eth1 (cmseos16, 17) and some as eth3 (cmseos14).
Closing task.
-bash-4.1$ pssh -l root -p50 --hosts=qlcnic.eos.list -o eosfsts-qlcnic.out -e eosfsts-qlcnic.err -t 0 'ethtool -i eth0'
[1] 13:59:40 [SUCCESS] cmseos13
[2] 13:59:40 [SUCCESS] cmseos16
[3] 13:59:40 [SUCCESS] cmseos15
[4] 13:59:40 [SUCCESS] cmseos11
[5] 13:59:40 [SUCCESS] cmseos17
[6] 13:59:40 [SUCCESS] cmseos12
[7] 13:59:40 [SUCCESS] cmseos1
[8] 13:59:41 [FAILURE] cmseos14 Exited with error code 71
-bash-4.1$ grep firmware-v eosfsts-qlcnic.out/cmseos*
eosfsts-qlcnic.out/cmseos1:firmware-version: 4.20.1
eosfsts-qlcnic.out/cmseos11:firmware-version: 4.20.1
eosfsts-qlcnic.out/cmseos12:firmware-version: 4.20.1
eosfsts-qlcnic.out/cmseos13:firmware-version: 4.20.1
eosfsts-qlcnic.out/cmseos15:firmware-version: 4.20.1
eosfsts-qlcnic.out/cmseos16:firmware-version: 1.4.3
eosfsts-qlcnic.out/cmseos17:firmware-version: 1.4.3
[root@cmseos17 ~]# ethtool -i eth1 | grep firmware-v
firmware-version: 4.20.1
[root@cmseos17 ~]#
-bash-4.1$ root cmseos16 ethtool -i eth1 | grep firmware-v
firmware-version: 4.20.1
-bash-4.1$ root cmseos14 ethtool -i eth3 | grep firmware-v
firmware-version: 4.20.1