Support #3590
Move /minos/app data files
Description
This is to track movement of most data files from /minos/app to /minos/data.
We presently have 7 TB in app.
This should be well under 2.
Reducing the content will help with the March 21 separation
globally of /*/app from /*/data volumes in Bluearc.
History
#1 Updated by Arthur Kreymer over 7 years ago
2013 02 11
#########
Need space on /minos/app 98% in use now, of 7 TB.
http://www-numi.fnal.gov/computing/dh/masum/NOW.txt
Top 13 users ( over 100 GB )
100 avgavril/ 432 jdejong/ 421 jjling/ 338 kopp2/ 02/13 597 loiacono/ 496 pawloski/ 135 rhatcher/ 509 rmehdi/ to 198 MB 03/14 102 rtoner/ 276 rustem/ 02/12 213 sanjay/ 333 scavan/ 03/14 101 tinti/
This is a net 4 TB, which would be a good start if moved.
#2 Updated by Arthur Kreymer over 7 years ago
Moved rustem Feb 11
du -sm /minos/app/rustem/*
79485 /minos/app/rustem/entire
98 /minos/app/rustem/farcat
991 /minos/app/rustem/FDfiles_2008
1 /minos/app/rustem/rsync-entire.log
196206 /minos/app/rustem/sigmip
Not accessed for 794 daysfind . -type f -atime -794
./entire/lock/nlock
Moveddate
cp -ax /minos/app/rustem /minos/data/users/rustem/app
date
diff -r /minos/app/rustem /minos/data/users/rustem/app
date
du -sm /minos/app/rustem /minos/data/users/rustem/app
Mon Feb 11 17:10:33 CST 2013
Mon Feb 11 18:59:38 CST 2013
Mon Feb 11 21:21:38 CST 2013
276778 /minos/app/rustem
276898 /minos/data/users/rustem/app
unalias rm
date
rm -r /minos/app/rustem
ln -s /minos/data/users/rustem/app /minos/app/rustem
date
Tue Feb 12 09:29:34 CST 2013
Tue Feb 12 09:33:30 CST 2013
Filesystem Size Used Avail Use% Mounted on
blue3.fnal.gov:/minos/app
7.5T 7.1T 435G 95% /minos/app
#3 Updated by Arthur Kreymer over 7 years ago
Moved kopp2 Feb 12
Mostly Jan 7 2010, plus a jpg from Mar 10, 2010
find . -type f | wc -l
19215
find . -type f -atime -790 # latest access 791 days ago
date -d '791 days ago'
Tue Dec 14 17:09:57 GMT 2010
There is no /minos/data/users/kopp,
so shift these directly to the top there.
minos-admin
date
cp -ax /minos/app/kopp2 /minos/data/users/kopp
date
diff -r /minos/app/kopp2 /minos/data/users/kopp
date
du -sm /minos/app/kopp2 /minos/data/users/kopp
Tue Feb 12 11:18:06 CST 2013
Tue Feb 12 13:57:16 CST 2013
Tue Feb 12 16:39:41 CST 2013
338909 /minos/app/kopp2
339165 /minos/data/users/kopp
unalias rm
date
rm -r /minos/app/kopp2
ln -s /minos/data/users/kopp /minos/app/kopp2
date
df -h /minos/app
Wed Feb 13 14:43:22 CST 2013
Wed Feb 13 15:00:37 CST 2013
Filesystem Size Used Avail Use% Mounted on
blue3.fnal.gov:/minos/app
7.5T 6.8T 761G 91% /minos/app
#4 Updated by Arthur Kreymer over 6 years ago
Usage jumped from 97 to 98% May 26
20:40 97 %
20:45 97.5 %
20:50 98 %
Getting warnings from batch sysetem
check_quota: too little space on /minos/app (based on "df")
if-nas-0.fnal.gov:/minos/app 7.5T 7.3T 230G 98%
USED=7999872828 QUOTA=8246337209 FREE=246464381(>7500000) PERCENT=2(<3)Found large set of mbeam.root files under
120073 /minos/app/users/loiacono/BeamDataPro
BDL=/minos/app/users/loiacono/BeamDataPro/mbeam
FILES=`find ${BDL} -type f -name \*mbeam.root`
Two of these were 'recovered', not yet in /minos/data/beam_data of /pnfs/minos/beam_data
minosraw@minos-slf6
for FILE in B100904_183016.mbeam.root B100910_001320.mbeam.root ; do
cp -a /minos/app/users/loiacono/BeamDataPro/mbeam/2010-09/${FILE} \
/pnfs/minos/beam_data/2010-09/${FILE}
donecp: writing `/pnfs/minos/beam_data/2010-09//B100904_183016.mbeam.root': Input/output error
cp: writing `/pnfs/minos/beam_data/2010-09//B100910_001320.mbeam.root': Input/output error
root@if-admin-minosBDL=/minos/app/users/loiacono/BeamDataPro/mbeam
FILES=`find ${BDL} -type f -name \*mbeam.root`
mkdir /minos/app/root/log/appx
find ${BDL} -type f -name \*mbeam.root -exec ls -l {} \; \
> /minos/app/root/log/appx/loiacono-mbeam
wc -l /minos/app/root/log/appx/loiacono-mbeam
1384 /minos/app/root/log/appx/loiacono-mbeam
unalias rm
for FILE in ${FILES} ; do
FD=`echo ${FILE} | cut f 8 -d /`
SD=`ls -l /minos/data/beam_data/${FD} | cut -f 5,5 -d ' '`
SL=`ls -l ${FILE} | cut -f 5,5 -d ' '`
[ "${SD}" != "${SL}" ] && ls -l ${FILE} /minos/data/beam_data/${FD}
[ "${SD}" == "${SL}" ] && rm -v ${FILE}
done 2>&1 | tee /minos/app/root/log/appx/loiacono-mbeam-rm
du -sm /minos/app/users/loiacono/BeamDataPro
20 /minos/app/users/loiacono/BeamDataPro
#5 Updated by Arthur Kreymer almost 6 years ago
Moving all 77 GB deb4
No /minos/data/users/deb4, so make and move
mkdir /minos/data/users/deb4
chown 11231 /minos/data/users/deb4
date
cp -ax /minos/app/deb4 /minos/data/users/deb4/app
date
diff -r /minos/app/deb4 /minos/data/users/deb4/app
date
Wed Nov 26 16:33:29 CST 2014
Wed Nov 26 17:01:52 CST 2014
diff: /minos/app/deb4/daikon04_mc/for_field/shw_compare/analysis/.#nu_passes.C: No such file or directory
diff: /minos/data/users/deb4/app/daikon04_mc/for_field/shw_compare/analysis/.#nu_passes.C: No such file or directory
diff: /minos/app/deb4/test.R1.26/include/MCReweight: No such file or directory
diff: /minos/data/users/deb4/app/test.R1.26/include/MCReweight: No such file or directory
diff: /minos/app/deb4/test.R1.26/include/NeugenInterface: No such file or directory
diff: /minos/data/users/deb4/app/test.R1.26/include/NeugenInterface: No such file or directory
Wed Nov 26 17:35:24 CST 2014
These are broken symlinks, OK
/minos/app/deb4/daikon04_mc/for_field/shw_compare/analysis/.#nu_passes.C -> deb4@minos25.fnal.gov.7208
/minos/app/deb4/test.R1.26/include/MCReweight -> ../MCReweight
/minos/app/deb4/test.R1.26/include/NeugenInterface -> ../NeugenInterfacedu -sm /minos/app/deb4 /minos/data/users/deb4/app
77710 /minos/app/deb4
77854 /minos/data/users/deb4/app
find /minos/app/deb4 -type f | wc -l
6116
find /minos/data/users/deb4/app -type f | wc -l
6116
df -m /minos/app
chmod -R 755 /minos/app/deb4
rm -r /minos/app/deb4
df -m /minos/app
Filesystem 1M-blocks Used Available Use% Mounted on
if-nas-0.fnal.gov:/minos/app 7864320 7630678 233643 98% /minos/app
if-nas-0.fnal.gov:/minos/app 7864320 7552982 311338 97% /minos/app
ln -s /minos/data/users/deb4/app /minos/app/deb4
date
Wed Nov 26 22:52:25 CST 2014