THe production stuff is growing memory massively again; so reviewing for possible memory leaks.
Read a big writeup on https://github.com/kennethreitz/requests/issues/1685 about leaks using the requests library etc. doing http calls out. Concluded that there are two places we have possible issues: 1)failing to close() http request results, and 2) the exception leaks we've seen before. So I went through various routines to make sure everywere we do an http get or post we close the result object, even in exception handlers.
Apparently just letting http connection objects get destroyed does not call all the close() code. :-(.
#2 Updated by Marc Mengel almost 4 years ago
- Status changed from Work in progress to Resolved
- % Done changed from 0 to 100
Development instance of jobsub_q_scraper is steady at 219m vsize overnight since this cleanup (yay!).
The uwsgi threads in development do not seem to be obviously leaking memory anymore either, although