Bug #4489: Reduce the memory consumption of the factory
Reduce glideinEntryGroup memory footprint
For a test 2.7.1 factory with a lot of entries (but a single frontend), the factoryEntryGroup takes 240 MB of memory.
On a production factory, it's more like 1.4 GB.
This is fine, except we fork and collect a lot of work, and the proportional set size per child might be growing over time as memory gets more and more fragmented.
Reducing the memory footprint significantly would help a lot.
#1 Updated by Igor Sfiligoi over 6 years ago
- Status changed from Assigned to Feedback
- Assignee changed from Igor Sfiligoi to Parag Mhashilkar
I think the easiest thing to do is to push most of the "find work" activity into a different process,
and then pickle back the results.
I have written the code in
Parag: Can you have a look? And if you have a test factory handy, try it out?
(I don't have one at this time)
#3 Updated by Igor Sfiligoi over 6 years ago
- Status changed from Feedback to Assigned
- Assignee changed from Parag Mhashilkar to Burt Holzman
I think my approach is not good enough;
testing at SDSC production factory show a saving in the order of 20%, not much more.
(for either/both VZISE and RSS as reported by top, from ~1.9G/1.6G -> 1.6G/1.4G)
Bottom line, not worth the additional complexity.
#4 Updated by Igor Sfiligoi over 6 years ago
- Assignee changed from Burt Holzman to Parag Mhashilkar
I am getting convinced that the memory footprint is due mostly to the post-entry monitoring.
structure persists between iterations, and my bet is that it is to be blamed for much of the memory use.
Assigning the ticket to Parag, since he is likely the most familiar with that part.