Project

General

Profile

Review Request #6144

Review Request #6135: Data Management Workflow Umbrella Task

File Access Audit

Added by Gabriel Perdue almost 6 years ago. Updated about 5 years ago.

Status:
Assigned
Priority:
Normal
Assignee:
Daniel Ruterbories
Start date:
05/02/2014
Due date:
09/02/2014
% Done:

90%

Estimated time:
Duration: 124

Description

Perform the following audits of file access methods:
a. Minerva should conduct an audit of their jobs to detect and enumerate
all open file descriptors. This can be done through the use of “lsof” or “strace” on a given worker node while it is running a Minerva job.
i. Remove all instances of direct file access and replace with appropriate access patterns
b. Minerva should conduct an audit of their jobs to detect and enumerate all direct calls of I/O locking to ensure they are not capable of causing stalls and deadlocks. In particular the experiment should purge all hard coded calls to the deprecated “CPN” utility. Proper I/O throttling and locking patterns should be followed and number of locks the jobs take should be consolidated to take advantage of bulk transfers.

History

#1 Updated by Gabriel Perdue almost 6 years ago

  • Parent task set to #6135

#2 Updated by Heidi Schellman almost 6 years ago

  • Assignee set to Philip Rodrigues

Phil, I'm assigning this to you, can you find a bright young algorithms person to do this? It will be highly educational for them.

#3 Updated by Daniel Ruterbories about 5 years ago

  • Status changed from New to Assigned
  • Assignee changed from Philip Rodrigues to Daniel Ruterbories
  • % Done changed from 0 to 90

Taking this from Phil. Have a method suggested by Marc and Art. Need to do it.

Obvious issues like cpn usage have been correct with new version of software.

Also available in: Atom PDF