Feature #4319: Scale SFA - File Aggregation - Fermilab Redmine

Feature #4319

Scale SFA

Added by Alexander Moibenko over 6 years ago. Updated over 6 years ago.

Status:

New

Priority:

Normal

Assignee:

Alexander Moibenko

Start date:

07/11/2013

Due date:

08/01/2013

% Done:

100%

Estimated time:

30.00 h

Spent time:

30.00 h

Duration: 22

Description

To scale SFA for increasing number of users we need to be able to add more cache servers.
Cache servers provide cache as NAS and for some configuration packaging and unpackaging files locally on cache servers.
This is how SFA currently operates in production.
The current approach is described in SFA HLD document (https://cdcvs.fnal.gov/redmine/attachments/download/6908/small_files_hld_v21.pdf). In current implementation a new cache server can also be added with its cache mounted on all disk and migrator nodes, but then packaging and unpackaging must be done over NAS (and not locally on cache server), which has proven to be an issue for NFS currently used as NAS provider.
The new approach is to group disk movers and migrators around dedicated disk library (cluster) to access a dedicated cache provided by certain cache server. In such an approach only members of the cluster can access data.

Scaling SFA through clusters.docx (141 KB) Scaling SFA through clusters.docx		Alexander Moibenko, 07/15/2013 09:32 AM
Scaling SFA through clusters.pdf (613 KB) Scaling SFA through clusters.pdf		Alexander Moibenko, 07/17/2013 04:21 PM
Scaling SFA through clusters.pdf (614 KB) Scaling SFA through clusters.pdf		Alexander Moibenko, 07/26/2013 10:18 AM

History

#1 Updated by Alexander Moibenko over 6 years ago

Assignee set to Alexander Moibenko

#2 Updated by Alexander Moibenko over 6 years ago

Due date set to 08/01/2013
% Done changed from 0 to 50
Estimated time set to 30.00 h

#3 Updated by Alexander Moibenko over 6 years ago

Policy applies only for writes.
If we want reads come through new cache we need to assign a new library to corresponding disk volumes just as we do for tape volumes when we move then from one library to another.

Question. If we install a new system around a new disk library manager cluster, and move an existing policy to this new cluster, will they be able to access files they wrote prior to the move?

Gene