Project

General

Profile

Bug #24402

Not all the logfiles are rotated in the factory

Added by Marco Mascheroni 7 months ago. Updated about 2 months ago.

Status:
New
Priority:
Normal
Category:
-
Target version:
Start date:
05/12/2020
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:

Factory Ops

Duration:

Description

This is the log configuation in the factory:

   <log_retention>
      <condor_logs max_days="14.0" max_mbytes="100.0" min_days="3.0"/>
      <job_logs max_days="7.0" max_mbytes="100.0" min_days="3.0"/>
      <summary_logs max_days="31.0" max_mbytes="100.0" min_days="3.0"/>
      <process_logs>
                <process_log extension="info" max_days="7.0" max_mbytes="100.0" min_days="3.0" msg_types="INFO"/>
                <process_log extension="err" max_days="7.0" max_mbytes="100.0" min_days="3.0" msg_types="DEBUG,ERR,WARN,EXCEPTION"/>
      </process_logs>
   </log_retention>

However, it looks like some of the entry logfiles never rotates:

/var/log/gwms-factory/server/entry_Gluex_US_NUMEP_grid1/:
total 721832
-rw-r--r--   1 gfactory gfactory 104861331  6 feb 17.57 Gluex_US_NUMEP_grid1.info.log.2020-02-06_17-57
-rw-r--r--   1 gfactory gfactory 104860362 21 feb 16.35 Gluex_US_NUMEP_grid1.info.log.2020-02-21_16-35
-rw-r--r--   1 gfactory gfactory 104858829 10 mar 23.24 Gluex_US_NUMEP_grid1.info.log.2020-03-10_23-24
-rw-r--r--   1 gfactory gfactory 104857800 14 mar 00.19 Gluex_US_NUMEP_grid1.err.log.2020-03-14_00-19
-rw-r--r--   1 gfactory gfactory 104858131 29 mar 04.48 Gluex_US_NUMEP_grid1.info.log.2020-03-29_04-48
-rw-r--r--   1 gfactory gfactory 104861166 22 apr 07.00 Gluex_US_NUMEP_grid1.info.log.2020-04-22_07-00

The old OSG production factory used to have a tmpwatch in a cronjob to clean them, but the new one run into disk issues because of these files never rotating. Also, check if files are rotated for disabled entries as well.

History

#1 Updated by Marco Mambelli 2 months ago

  • Target version changed from v3_6_4 to v3_6_5

#2 Updated by Marco Mambelli about 2 months ago

  • Target version changed from v3_6_5 to v3_6_6

Also available in: Atom PDF