Project

General

Profile

Feature #13707

Better split types

Added by Marc Mengel about 4 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
Start date:
09/23/2016
Due date:
% Done:

90%

Estimated time:
Scope:
Internal
Experiment:
-
Stakeholders:
Duration:

Description

We should make a better new-file splitter, which takes parameters;
we should also make mod_n accept mod(n) to make the parameter clear.

New-file splitter should allow new(window=1w,round=d,localtime=t) to say that it
should split in one-week sections rounded to the nearest day; and optionally
use localtime to convert the string (some experiments use localtime in the metadata?)
and it should actually split on file's end_time metadata for keep-up usage.

History

#1 Updated by Marc Mengel about 4 years ago

  • Target version set to v1_1_0

#2 Updated by Marc Mengel almost 4 years ago

  • Start date changed from 08/30/2016 to 09/23/2016
  • % Done changed from 0 to 20

#3 Updated by Anna Mazzacane almost 4 years ago

  • Status changed from New to Work in progress

#4 Updated by Marc Mengel over 3 years ago

  • % Done changed from 20 to 50

Roughed out new and ostensibly improved one.

#5 Updated by Marc Mengel over 3 years ago

get new does even time windows -- you can set it for 2 hour windows, and launch it hourly, and it will actually launch every other one, and then catch up if it missed a window

#6 Updated by Marc Mengel over 3 years ago

  • Status changed from Work in progress to Resolved
  • % Done changed from 50 to 90

This isn't as well tested as I would like, so we may have to update it in a bugfix update.

#7 Updated by Marc Mengel over 3 years ago

Smoke tested the new split types tonight; hopefully they'll be nice for folks.

Made a dataset with 15 files with different start/end times in SAM, so I could test the time-based "new" file splitter; also used it to test the mod(n) file splitter; both now seem to work as they ought.

#8 Updated by Marc Mengel over 3 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF