Project

General

Profile

Bug #8444

jobsub_submit_dag leaves entries in /tmp, eventually breaking job submission

Added by Philip Rodrigues almost 5 years ago. Updated almost 5 years ago.

Status:
Closed
Priority:
High
Assignee:
Category:
-
Target version:
Start date:
04/28/2015
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

jobsub_submit_dag creates temporary directories like /tmp/tmp6Pjmyq containing, eg:

$ ls l /tmp/tmp6Pjmyq/
total 8
-rw-r--r-
1 rodriges e938 67 Apr 28 11:46 foo.dag_input
rw-r--r- 1 rodriges e938 511 Apr 28 11:46 payload.tgz

When too many (32k, I think) of these build up, job submission breaks with a "too many links" error:

Traceback (most recent call last):
File "/grid/fermiapp/products/common/db/../prd/jobsub_client/v1_1_0_2/NULL/jobsub_submit_dag", line 222, in main
http_code = js_client.submit_dag()
File "/grid/fermiapp/products/common/prd/jobsub_client/v1_1_0_2/NULL/jobsubClient.py", line 258, in submit_dag
payloadFileName=self.makeDagPayload(uri2path(self.jobExeURI))
File "/grid/fermiapp/products/common/prd/jobsub_client/v1_1_0_2/NULL/jobsubClient.py", line 290, in makeDagPayload
dirpath=tempfile.mkdtemp()
File "/grid/fermiapp/minerva/software_releases/v10r8p1/lcg/external/Python/2.6.5p2/x86_64-slc6-gcc44-opt/lib/python2.6/tempfile.py", line 318, in mkdtemp
_os.mkdir(file, 0700)
OSError: [Errno 31] Too many links: '/tmp/tmpYWlXpG'

I suppose there's also the risk of filling up /tmp, which is always bad.

History

#1 Updated by Parag Mhashilkar almost 5 years ago

  • Assignee set to Dennis Box
  • Priority changed from Normal to High
  • Target version set to v1.1.2

I agree this is bad. Thanks for reporting it. We will fix it in next release.

#2 Updated by Dennis Box almost 5 years ago

  • Status changed from New to Feedback

see branch 8444

#3 Updated by Dennis Box almost 5 years ago

  • Assignee changed from Dennis Box to Parag Mhashilkar

#4 Updated by Parag Mhashilkar almost 5 years ago

  • Target version changed from v1.1.2 to v1.1.3

#5 Updated by Parag Mhashilkar almost 5 years ago

  • Assignee changed from Parag Mhashilkar to Dennis Box
  • Target version changed from v1.1.3 to v1.1.2

Looks ok to merge. Please add this to release as well

#6 Updated by Parag Mhashilkar almost 5 years ago

  • Status changed from Feedback to Resolved

#7 Updated by Parag Mhashilkar almost 5 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF