Project

General

Profile

Maint-20140605

Last Modified {{last_modified}}

  • Bluearc APP area migration to new disks
    • offline 00:00 to 08:00
  • Minos server kernel updates
    • shut down at 23:45, reboot after Bluearc is up
  • Fermigrid draining Jun 4 10:00
    • reboot all workers with new kernels Jun 5 08:00 to 13:00
  • minos-db1 patches 09:00 to 10:00

FEF status link
http://fefweb.fnal.gov/mediawiki/index.php/IF/EAG/GPGrid/GPCF_Downtime_5_June_2014

SERVICE DOWN UP COMMENT
Bluearc 00:00 06:45
Fermigrid 08:00 10:00 drain started 18:00, up around noon, subsequent problem 13:00
minos-db1 09:00 09:45 ifdb02
SERVER DOWN UP COMMENT
minos25 09:00 09:10 up overnight
minos27 23:45 09:10
minos50 23:45 09:50 fsck
minos51 23:45 11:10 fsck
minos52 23:45 09:10
minos53 23:45 11:10 fsck
minos54 23:45 09:10 up 08:54 with old kernel
minos-sam04 09:00 09:10
minos-nearline 09:00 09:10 up overnight
minos-slf4 xxxxx xxxxx app areas stale, remounted Jun 11
minos-slf5 09:00 09:10 up overnight
minos-slf6 09:00 09:10 up overnight
minos-mysql2 23:45 09:10

Issues after planned maintenance -

  • Expected :
    • remounted /pnfs/minos
      • on minos27/50/52/53/54/gpvm03
    • restarted predator, roundup
  • Pending
  • Fixed

Announcement to minos-users,minos-admin,minosdb-support,minos-run,ecl

Minos Fermilab scheduled computing maintenance Thu 2014 Jun 05

The following Fermilab computing services will be down
during the scheduled maintenance day this Thursday 2014 Jun 05:

23:45-08:00 Bluearc APP areas offline - /minos/app /grid/fermiapp /nusoft/app
23:45-12:00 Minos and GP server kernel updates after Bluearc maintenance
Jun 4 10:00 Fermigid starts draining - no new starts
08:00-13:00 Fermigrid worker kernel updates
09:00-10:00 minos-db1 server patches

Jobs running on Fermigrid will be stopped or restarted.

As usual, see details and progress notes at
https://cdcvs.fnal.gov/redmine/projects/admin/wiki/Maint-20140605