Project

General

Profile

Bug #21440

make the dagbegins request less memory

Added by Joe Boyd 6 months ago. Updated 3 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
11/28/2018
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

Not sure what a dagbegin is or why it's there since it literally does NOTHING!!!

Anyway, don't require 2GB for it at least. Request 100Kbytes or something so it runs right away and gets out of the way if you can't get rid of it compeletely.

-bash-4.1$ condor_q -nobatch -constraint 'owner == "kotwal"' -af cmd requestmemory
/usr/bin/condor_dagman 10
/fife/local/scratch/uploads/cdf/kotwal/2018-11-28_073337.616242_5719/run_pdfloop.sh_20181128_073338_3857981.dagbegin.sh 2000

They can sit for a long period of time for experiments with a low quota sometimes holding up jobs that are requesting much less memory which would also run in a busy batch system if this dagbegin wasn't stuck waiting for 2GB of memory.

History

#1 Updated by Dennis Box 6 months ago

  • Status changed from New to Feedback
  • Assignee changed from Dennis Box to Shreyas Bhat
  • Target version set to v1.2.9

See jobsub branch 21440
commit message:
configurable request_memory parameter for 'dag_begin dag_end sam_begin sam_end' files.
Default value is 100kb. Can be changed in jobsub.ini in [default] section, example
dag_start_memory = 33kb

#2 Updated by Shreyas Bhat 6 months ago

  • Status changed from Feedback to Accepted
  • Assignee changed from Shreyas Bhat to Dennis Box

Looks good to me.

#3 Updated by Dennis Box 6 months ago

  • Status changed from Accepted to Resolved

changes merged to branch_1.2.9_rc0 for testing

#4 Updated by Joe Boyd 5 months ago

This seems to have broken things. Looks like 4MB is a minimum.

HoldReason = "Error from : Error response from daemon: Minimum memory limit allowed is 4MB "

The default in the code should be set to probably 100MB or something. Does everyone submitting a dag have a dagstart or is this just a CDF thing? A requirement of 100MB is very small but should handle I would think most shell scripts unless maybe people do large sam commands in them. If other people have dag starts do they actually put whatever they want in the dag start or does jobsub always create a dag start?

I don't want to role this out and have a break a bunch of things because the limit is too small.

#5 Updated by Dennis Box 3 months ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF