Project

General

Profile

ncis_error_email.txt

E-mail containing traceback. - Michael Zalokar, 08/04/2015 02:00 PM

 
1
From tissue@gibbs.fnal.gov Wed Jun 10 14:20:16 2015
2
Date: Wed, 10 Jun 2015 14:20:14 -0500
3
From: tissue@gibbs.fnal.gov
4
To: tissue-dev@fnal.gov
5
Subject: IMPORTANT ERROR LOG ENTRY FROM TISSUE on gibbs, database tissue
6

    
7
2015-06-10 14:20:14 21158    ERROR RobotImpl> UNEXPECTED ROBOT FAILURE on job 187752 (event_id 16452, work_code B):
8
EXCEPTION: DatabaseError deadlock detected
9
DETAIL:  Process 21172 waits for ExclusiveLock on tuple (200,74) of relation 17237 of database 16422; blocked by process 20887.
10
Process 20887 waits for ShareLock on transaction 1273584961; blocked by process 21172.
11
HINT:  See server log for query details.
12

    
13
Traceback:
14
  File "/fnal/ups/prd/tissue_core/v3_2a/TissueCoreImpl/RobotImpl.py", line 166, in _doProcessOne
15
    result = the_method(authenticated_agent, this_job.pk, **kwargs)
16
  File "/fnal/ups/prd/tissue_core/v3_2a/TissueCoreImpl/EventImpl.py", line 301, in processScheduledBlock
17
    self._processScheduledJob(agent, event_schedule_id, EventImpl.RCODE_BBOT, self.eventFactory.handleBlockJob)
18
  File "/fnal/ups/prd/tissue_core/v3_2a/TissueCoreImpl/EventImpl.py", line 292, in _processScheduledJob
19
    handler(agent, this_job, *args, **kwargs)
20
  File "/fnal/ups/prd/tissue_core/v3_2a/TissueEvent/EventFactory.py", line 1105, in handleBlockJob
21
    self.handleBlock(authenticated_agent, this_job.event, this_job.event_work)
22
  File "/fnal/ups/prd/tissue_core/v3_2a/TissueEvent/EventFactory.py", line 1082, in handleBlock
23
    self.blockImpl.blockEvent(authenticated_agent, event.pk)
24
  File "/fnal/ups/prd/tissue_core/v3_2a/TissueCoreImpl/BlockImpl.py", line 162, in blockEvent
25
    self._set_event_error(this_event, this_agent, errmsg)
26
  File "/fnal/ups/prd/tissue_core/v3_2a/TissueCoreImpl/BlockImpl.py", line 292, in _set_event_error
27
    this_event.save()
28
  File "/usr/lib/python2.6/site-packages/django/db/models/base.py", line 546, in save
29
    force_update=force_update, update_fields=update_fields)
30
  File "/usr/lib/python2.6/site-packages/django/db/models/base.py", line 591, in save_base
31
    update_fields=update_fields)
32
  File "/usr/lib/python2.6/site-packages/django/db/models/base.py", line 626, in save_base
33
    rows = manager.using(using).filter(pk=pk_val)._update(values)
34
  File "/usr/lib/python2.6/site-packages/django/db/models/query.py", line 605, in _update
35
    return query.get_compiler(self.db).execute_sql(None)
36
  File "/usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 1014, in execute_sql
37
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
38
  File "/usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
39
    cursor.execute(sql, params)
40
  File "/usr/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 58, in execute
41
    six.reraise(utils.DatabaseError, utils.DatabaseError(*tuple(e.args)), sys.exc_info()[2])
42
  File "/usr/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 54, in execute
43
    return self.cursor.execute(query, args)