Project

General

Profile

Feature #3605

Statistics collection

Added by Marc Mengel over 7 years ago. Updated almost 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Start date:
03/15/2013
Due date:
% Done:

100%

Estimated time:
Spent time:
Duration:

Description

Can we get better statisitics collection (i.e. i/o stats on cp/gridftp sub processes? data rates?)

History

#1 Updated by Marc Mengel about 7 years ago

  • Description updated (diff)
  • Status changed from New to Assigned
  • Target version set to v1_2_5
  • % Done changed from 0 to 30

We have a rough implementation of this using getrusage(); but this only
works properly on SLF6. And it's still a bit of a lie, because it ignores
buffer-cached reads. But it's better that nothing.

what's there so far is in 0742ecba and 7c105bb0

#2 Updated by Marc Mengel almost 7 years ago

Updated to try to stat() all our arguments, and if we found files, to keep a sum of the st_size values for the source and destination files separately. If we don't get rusage blocks in and out, we take the st_size sums/512 and report that. That should give us some approximation of block count vs time that we can use
to collect througput numbers.

#3 Updated by Marc Mengel almost 7 years ago

Okay, now we stat our inputs and outputs after we copied the files, and total up any of them we can see, and use the max of inputs/outputs (because often one is remote and we get zero) to report bytes transferred, and we use higher resolution time to do our difference so we get a delta time for quick (i.e. local) copies. Downside is we include fork/excec time in the timing, but for larger transfers that should come out in the wash.

#4 Updated by Marc Mengel almost 7 years ago

  • % Done changed from 30 to 100

#5 Updated by Marc Mengel almost 7 years ago

  • Status changed from Assigned to Closed


Also available in: Atom PDF