Project

General

Profile

Bug #3295

Handle issues found out in v2_7_alpha1

Added by Parag Mhashilkar almost 8 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Parag Mhashilkar
Category:
-
Target version:
Start date:
01/15/2013
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

This will be master ticket to address the issues
Adding description as a comment so it is easy to update it.

History

#1 Updated by Parag Mhashilkar almost 8 years ago

  1. We encountered an unusually high load after leaving the 2_7 factory
    up for 24h. We couldn't even log into the factory initially, but once I
    got in I saw system CPU was approaching 80 and % i/0 wait was about
    70%. I noticed we still had high ulimits set from previous tests (50k),
    so when i lowered that back to 1024 and restarted the factory, the
    problem never came back.

    Unable to reproduce
  2. After shutting down the factory to address 1., even though it said
    "OK" we still saw 8 python processes running, of the form:
    /usr/bin/python
    /home/gfactory/glideinWMS/factory/glideFactoryEntryGroup.py 21096 60 5
    /usr/local/share/gfactory/glideinsubmit/glidein_v1_1 <followed by a
    bunch of : separated entry names here>

    These eventually went away on their own and I haven't noticed this on
    subsequent shutdowns
  3. Watching top during normal operation frequently reveals a few of these:
    python <defunct>
    Not sure if that is cause for concern or not

    This is expected
  4. factoryCompletedStats.html looks completely broken:
    http://glidein-itb.grid.iu.edu/glidefactory/monitor/glidein_v1_1/factoryCompletedStats.html
    we've been running glideins all week but those plots there are empty

    Fixed in v2_7_alpha2

#2 Updated by Parag Mhashilkar almost 8 years ago

  • Status changed from Assigned to Resolved

Released v2_7_alpha2 that addresses the issues pointed out in v2_7_alpha1

#3 Updated by Parag Mhashilkar almost 8 years ago

  • Subject changed from Handle issues found out in v2_7_alpha to Handle issues found out in v2_7_alpha1

#4 Updated by Parag Mhashilkar over 7 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF