Project

General

Profile

Support #16830

duplicate spilltime data files

Added by Arthur Kreymer about 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Start date:
06/13/2017
Due date:
06/14/2017
% Done:

100%

Estimated time:
1.00 h
Duration: 2

Description

All of the 442 spilltime files persisting in
LOOT=/nusoft/app/web/htdoc/minos/controlroom/daqlogs/NEAR/timing/spilllogs
are duplicates, as reported each hour in logs such as
TL=/minos/app/home/minos/log/TimeGoblinLooter/loot.201706.log

To get the log size under control, and avoid thrashing,
I will move the duplicate files to a DUPLICATE subdirectory.

History

#1 Updated by Arthur Kreymer about 3 years ago

  • Status changed from Work in progress to Resolved
  • % Done changed from 20 to 90

The files are moved.

I will run loot manually to verify the log cleanup.

LOOT=/nusoft/app/web/htdoc/minos/controlroom/daqlogs/NEAR/timing/spilllogs

find ${LOOT} -type d
/nusoft/app/web/htdoc/minos/controlroom/daqlogs/NEAR/timing/spilllogs
/nusoft/app/web/htdoc/minos/controlroom/daqlogs/NEAR/timing/spilllogs/already_looted
/nusoft/app/web/htdoc/minos/controlroom/daqlogs/NEAR/timing/spilllogs/tmp_2016

date ; mkdir ${LOOT}/DUPLICATE
Tue Jun 13 15:20:08 CDT 2017

date
for FILE in `ls ${LOOT} | grep ^spilltime.*gz$ | sort` ; do 
    if [ -r "${LOOT}/already_looted/${FILE}" ] ; then 
        mv ${LOOT}/${FILE} ${LOOT}/DUPLICATE/${FILE}
    fi
done

Tue Jun 13 15:20:40 CDT 2017

ls ${LOOT}/DUPLICATE | wc -w
442

#2 Updated by Arthur Kreymer about 3 years ago

  • % Done changed from 90 to 100
Tested loot run

    mindata@minos-nearline

cd TimeGoblinLooter

./loot &
usleep 100000 ; ./loot

Running TimeGoblinLooter 20170606 on minos-nearline Tue Jun 13 15:29:07 CDT 2017
 OOPS - 31537 is still running at Tue Jun 13 15:29:08 CDT 2017
Running looter: /minos/app/home/minos/TimeGoblinLooter/bin/Linux2.6-GCC_3_4/looter
Warning in <TClassTable::Add>: class timespec already in TClassTable
Successfully opened connection to: mysql:odbc://minos-db1.fnal.gov/offline
This client, and MySQL server (MySQL 5.6.19-log) does support prepared statements.
DbiCascader Status:- 
Status   URL

Closed (auth)  mysql:odbc://minos-db1.fnal.gov/offline

DatabaseInterface shutdown not requested
ListenerThreadManager::scheduleShutdown(): Unknown exception caught.
Exiting TimeGoblinLooter 20170606 on minos-nearline Tue Jun 13 15:29:13 CDT 2017

Tested cleaned up rerun protection message

./loot.20170613 &
usleep 100000 ; ./loot.20170613

Deployed the cleaner message

date ; ln -sf loot.20170613 loot # was loot.20170606
Tue Jun 13 15:33:13 CDT 2017



Also available in: Atom PDF