JobSub Users Meeting: July 10 2014


  • Stdout/Stderr Size limit
  • Jobsub User Interface
  • AOB

Meeting Notes

Present: Parag Mhashilkar, Neha Sharma, Tanya Levshina, Marek Zielinski, Art Kreymer, Joe Boyd, Mike Kirby, Dennis Box, Gerard, Yun Tse, Adam


Architecture: 50K Jobs running at a time, 2G input sandbox per job. How long should the sandbox stick.
Based on the Fermigrid capacity: 50k Job per day.

2MB stdout & stderr per condor procid (job).

Users can redirect the stdout/err to data directory and make jobsub tools transfer the output using ifdh

We need means to fetch a process rather than entire cluster

Can we compress the individual files when accessed first.

Jobsub User Interface

Microboone: Yun Tse: So far working good. Working on 8G VM. But dont need 8G since the memory consumption is low now after the fix to memory leaks. Huge production spikes in early August.


  • Done with preliminary jobsub and will be using HA only.
  • Adam is in charge of data production jobs
  • SES Data production group (Marek)
  • Going directly to Jobsub HA
  • Ran MC production run.
  • Sign off on moving fifebatch1 to HA
  • Discussion about the Jobsub job id.
  • Operations: Jobsub brings back many other files which we should stop doing.