Project

General

Profile

Feature #6983

Add option to compress log files

Added by Joe Boyd over 5 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Category:
-
Target version:
Start date:
09/11/2014
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

The <entry>.info.log files compress from 100MB down to about 5MB. We run on SAN based VM's where disk isn't free so it would be really nice if glideinwms would compress the log file when it rotates them. We'd like to keep a long period of log files for debugging. We have about a dozen entry points for various reasons so when everything gets multiplied out compression would be helpful.

History

#1 Updated by Joe Boyd over 5 years ago

This request started from ticket INC000000445637 where Gerard logged:


I'd be happy to see how GWMS relies on standard logrotate daemon and
provides a /etc/logrotate.d/gwms* file where Ops teams can setup the
desired rotation policy.

It would simplify both GWMS code and it's management. After looking a bit
this morning I believe Condor allows to do that if one disables condor
rotation and uses copytruncate option in logrotate. Perhaps we could do the
same with GWMS.

Gerard


I don't care how it's done but I think compression of log files is a reasonable request and we shouldn't roll our own.

#2 Updated by Parag Mhashilkar over 5 years ago

  • Target version set to v3_2_x

#3 Updated by Parag Mhashilkar over 5 years ago

  • Assignee set to Parag Mhashilkar
  • Target version changed from v3_2_x to v3_2_8

#4 Updated by Marco Mambelli over 5 years ago

  • Assignee changed from Parag Mhashilkar to Marco Mambelli

#5 Updated by Marco Mambelli about 5 years ago

Possible alternatives:
1. support an encoding for GlideinHandler. BaseRotatingHandler supports bz2 (encoding='bz2-codec' - only Python2, python3 has custom logrotators):

class GlideinHandler( BaseRotatingHandler):
...
   def __init__(self, filename, maxDays=1, minDays=0, maxMBytes=10, backupCount=5, encoding=None):
...
        BaseRotatingHandler.__init__(self, filename, mode, encoding)
...

def add_processlog_handler(logger_name, log_dir, msg_types, extension, maxDays, minDays, maxMBytes, backupCount=5, encoding=None):
...
    handler = GlideinHandler(logfile, maxDays, minDays, maxMBytes, backupCount, encoding)

2. add manual compression for the files importing gzip or zipfile, e.g.:

    def doRollover(self):
...
      if self.doCompress:
        if os.path.exists(dfn + ".zip"):
            os.remove(dfn + ".zip")
        file = zipfile.ZipFile(dfn + ".zip", "wb")
        file.write(dfn, os.path.basename(dfn), zipfile.ZIP_DEFLATED)
        file.close()
        os.remove(dfn)


This would still require the same parameter passing as 1.
Then in both 1 and 2 calls to logSupport.add_processlog_handler should access the configuration to decide wether to use compression or not (similar to the other logging parameters)

3. let logrotate handling log rotation. It is standard and will make RHEL/SL sysadmin happy. Not all systems will have it.
Check how the logging works to see if log files are opened and closed each time or some trigger command is needed.
Options to handle this: copytruncate, prescript, postscript, delaycompress

I need some feedback on which alternative would be better.
Thanks,
Marco

#6 Updated by Marco Mambelli about 5 years ago

Committed a first version (needs to be tested).

In the unit test compressed files are verified checking the file name (extension), not the actual content.

Marco

#7 Updated by Marco Mambelli about 5 years ago

for testing I creted a big file:

And I added the following lines.
On a RH5 factory: # to print a lot #MMDB
logSupport.log.info("################### Printing filling file:")
#with open('/opt/file-1m', 'r') as f:
f = open('/opt/file-1m', 'r')
s = f.read()
logSupport.log.info(s)
f.close()

Before the log update in the loop: # Aggregate Monitoring data periodically
logSupport.log.info("Aggregate monitoring data")
aggregate_stats(factory_downtimes.checkDowntime())

And in a RH6 frontend: # to print a lot #MMDB
logSupport.log.info("################### Printing filling file:")
with open('/opt/file-1m', 'r') as f:
s = f.read()
logSupport.log.info(s)

Before the log update in the loop:
logSupport.log.info("Aggregate monitoring data") # KEL - can we just call the monitor aggregator method directly? see above
aggregate_stats()

I added to the documentation a note that the max_byte value is truncated (whenever read) . Any value <1 will cause no rotation.
Marco

#8 Updated by Marco Mambelli about 5 years ago

  • Status changed from New to Feedback
  • Assignee changed from Marco Mambelli to Parag Mhashilkar

ready for review

#9 Updated by Marco Mambelli about 5 years ago

I forgot to add that to make sure that no strange chars were used I generated the test files used to files the log files with:
< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c 1M > /opt/file-1m

#10 Updated by Parag Mhashilkar about 5 years ago

  • Assignee changed from Parag Mhashilkar to Marco Mambelli

sent feedback separately.

#11 Updated by Marco Mambelli about 5 years ago

  • Status changed from Feedback to Resolved

merged

#12 Updated by Parag Mhashilkar about 5 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF