Project

General

Profile

Bug #10491

memory spike at the end of a job

Added by Andrei Gaponenko almost 4 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Metadata
Target version:
Start date:
10/12/2015
Due date:
% Done:

100%

Estimated time:
Spent time:
Occurs In:
Scope:
Internal
Experiment:
Mu2e
SSI Package:
art
Duration:

Description

Hello,

A Mu2e production job that tries to merge our "beam flash" dataset into a single file fails with "bad_alloc", even on a lightly loaded detsim machine with 60GB of RAM. I ran massif on a subset of the inputs. The output indicates a large memory spike at the end of job.

Attached are a test fcl file and the formatted massif output.
That was run with Mu2e Offline v5_4_7 (art v1_15_00).
Input data files can be found in
mu2egpvm05.fnal.gov:/mu2e/data/users/gandr/20151010a.flt4-massif/inputs/

Andrei

flttest-5files.fcl (722 Bytes) flttest-5files.fcl Andrei Gaponenko, 10/12/2015 01:57 PM
formatted.txt (1.13 MB) formatted.txt Andrei Gaponenko, 10/12/2015 02:09 PM

Associated revisions

Revision 0ae2d869 (diff)
Added by Christopher Green almost 4 years ago

Mitigation for issue #10491.

Revision f23e968f (diff)
Added by Christopher Green almost 4 years ago

Mitigation for issue #10491.

History

#1 Updated by Christopher Green almost 4 years ago

  • Category set to Metadata
  • Status changed from New to Resolved
  • Target version set to 1.17.00
  • % Done changed from 0 to 100
  • SSI Package art added
  • SSI Package deleted ()

New source option readParameterSets (default true) and new output option writeParameterSets (default true) are for use only when one is ABSOLUTELY SURE that historical parameter set information will never be needed. The only legitimate case for this currently known is production of mixing (overlay) files.

#2 Updated by Christopher Green almost 4 years ago

Implemented with commits 0ae2d86, bcb4c85 and 39e3722.

#3 Updated by Christopher Green almost 4 years ago

This mitigation has also been pushed to v1_16-branch, meaning that you should be able to clone this repository on any machine where art v1_16_02 is installed, build for installation into a private products area, and build a mu2e against it.

Let me know if you wish to do this and need any help or pointers.

#4 Updated by Kyle Knoepfel over 3 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF