Review Request #6144
Review Request #6135: Data Management Workflow Umbrella Task
File Access Audit
90%
Description
Perform the following audits of file access methods:
a. Minerva should conduct an audit of their jobs to detect and enumerate
all open file descriptors. This can be done through the use of “lsof” or “strace” on a given worker node while it is running a Minerva job.
i. Remove all instances of direct file access and replace with appropriate access patterns
b. Minerva should conduct an audit of their jobs to detect and enumerate all direct calls of I/O locking to ensure they are not capable of causing stalls and deadlocks. In particular the experiment should purge all hard coded calls to the deprecated “CPN” utility. Proper I/O throttling and locking patterns should be followed and number of locks the jobs take should be consolidated to take advantage of bulk transfers.
History
#1 Updated by Gabriel Perdue over 6 years ago
- Parent task set to #6135
#2 Updated by Heidi Schellman over 6 years ago
- Assignee set to Philip Rodrigues
Phil, I'm assigning this to you, can you find a bright young algorithms person to do this? It will be highly educational for them.
#3 Updated by Daniel Ruterbories almost 6 years ago
- Status changed from New to Assigned
- Assignee changed from Philip Rodrigues to Daniel Ruterbories
- % Done changed from 0 to 90
Taking this from Phil. Have a method suggested by Marc and Art. Need to do it.
Obvious issues like cpn usage have been correct with new version of software.