Project

General

Profile

Support #15799

Milestone #15057: Minos SLF5 retirement

Support #15792: Individual Minos SLF5 node shutdowns

minos52 retirement

Added by Arthur Kreymer over 3 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
High
Start date:
03/08/2017
Due date:
03/31/2017
% Done:

100%

Estimated time:
10.00 h
Duration: 24

Description

Retire minos52 by 2017 Apr 01

History

#1 Updated by Arthur Kreymer over 3 years ago

  • Status changed from Accepted to Assigned

#2 Updated by Arthur Kreymer over 3 years ago

  • Status changed from Assigned to Work in progress
  • % Done changed from 0 to 10

#3 Updated by Arthur Kreymer over 3 years ago

  • % Done changed from 10 to 20

Summary

Task Completed Comment
Ganglia 03/24 idle since 03/17 mgabriel root process
procsum 03/24 idle from 03/23 except mkiveni strip2stip cron
opt 03/20 nothing needed
home 03/20 nothing needed
scratch 03/21 non-root to /minos/data/users/mindata/localscratch

Ganglia averages final day

Idle    99.8 %
Load 1   44 m
Net in    3 k
Net out   6 k

#4 Updated by Arthur Kreymer over 3 years ago

  • % Done changed from 20 to 80

#5 Updated by Arthur Kreymer over 3 years ago

Date: Wed, 22 Mar 2017 16:03:12 +0000
From: Arthur Kreymer <kreymer@fnal.gov>
To: mkiveni@fnal.gov
Cc: minos_batch@fnal.gov
Subject: Strip2Strip cronjob on minos52

   We are about to shut down the old SLF5 node minos52 .

    We see mkiveni cron jobs running on minos52 .    

    The processes include
/minos/app/mkiveni/S2Sdev/Strip2Strip/scripts/CronJob.sh kNear
    and
/minos/app/mkiveni/S2Sdev/Strip2Strip/scripts/CronJob.sh kFar

    Please disable these jobs, with
crontab -l
crontab -r

These weekly calibration jobs are running on elm4, which has not been produced since December 2016.
There should be no harm letting them expire with minos52 tomorrow.

#6 Updated by Arthur Kreymer over 3 years ago

In procsum summaries, there are many entries like

  20170322_12:02:01_procs.gz
16601 grzelakk  15   0 71692 2652 1176 S  0.0  0.0   0:00.09 -tcsh
16669 grzelakk  16   0 42236 2644 1228 S  0.0  0.0   2:12.73 ftp fndca1.fnal.gov 24126
wall minos52 is shutting down tomorrow. Please use minos6* nodes instead.
Broadcast message from mindata (pts/1) (Thu Mar 23 08:23:38 2017):
minos52 is shutting down tomorrow. Please use minos6* nodes instead.
Date: Thu, 23 Mar 2017 13:42:36 +0000
From: Arthur Kreymer <kreymer@fnal.gov>
To: grzelakk@fnal.gov
Cc: minos-admin@fnal.gov
Subject: grzelakk ftp processes on minos52.fnal.gov

  I see recent grzelakk processes running ftp on minos52.fnal.gov, like

20170323_08:15:01_procs.gz
  814 grzelakk  15   0 71692 2648 1172 S  0.0  0.0   0:00.10 -tcsh
  888 grzelakk  16   0 42240 2652 1228 D  0.0  0.0   0:21.36 ftp fndca1.fnal.gov 24126

  minos52 will be shut down tomorrow.

  Feel free to move data handling work to minos60 through minos63 .

  Thanks !
Date: Thu, 23 Mar 2017 13:50:46 +0000
From: Katarzyna Grzelak <Katarzyna.Grzelak@fuw.edu.pl>
To: Arthur Kreymer <kreymer@fnal.gov>
Subject: Re: grzelakk ftp processes on minos52.fnal.gov

Thank you. I got used to minos52 and forgot that I have
to move to minos60-

Many thanks,
Katarzyna

#7 Updated by Arthur Kreymer over 3 years ago

More agressive procsum just before shutdown.
The latest procsum runs locally then copies the summary to the web.

crontab 
MAILTO='minos-data@fnal.gov'
00,10,20,30,40,50 * * * *  ${HOME}/procsum today

date
Fri Mar 24 09:07:29 CDT 2017

RITM0542958 03/24 minos52 shutdown

At your convenience, please shut down the minos52 system.
This SLF5 system is being retired.

See preparation details in https://cdcvs.fnal.gov/redmine/issues/15799

Please hold the hardware for a month before disposal,
in case we overlooked something.
2017-03-24 10:24:17 CDT - Christophe Bonnaud (Additional comments)
The server has been shut down.
Got a clean final procsum sample at 10:20
http://minos.fnal.gov/computing/dh/procsum/minos52/minos52-20170324

#8 Updated by Arthur Kreymer over 3 years ago

  • % Done changed from 80 to 90

#9 Updated by Arthur Kreymer over 3 years ago

  • Status changed from Work in progress to Resolved
  • % Done changed from 90 to 100

#10 Updated by Arthur Kreymer about 3 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF