Feature #4319
Scale SFA
Description
To scale SFA for increasing number of users we need to be able to add more cache servers.
Cache servers provide cache as NAS and for some configuration packaging and unpackaging files locally on cache servers.
This is how SFA currently operates in production.
The current approach is described in SFA HLD document (https://cdcvs.fnal.gov/redmine/attachments/download/6908/small_files_hld_v21.pdf). In current implementation a new cache server can also be added with its cache mounted on all disk and migrator nodes, but then packaging and unpackaging must be done over NAS (and not locally on cache server), which has proven to be an issue for NFS currently used as NAS provider.
The new approach is to group disk movers and migrators around dedicated disk library (cluster) to access a dedicated cache provided by certain cache server. In such an approach only members of the cluster can access data.
History
#1 Updated by Alexander Moibenko over 7 years ago
- Assignee set to Alexander Moibenko
#2 Updated by Alexander Moibenko over 7 years ago
- Due date set to 08/01/2013
- % Done changed from 0 to 50
- Estimated time set to 30.00 h
#3 Updated by Alexander Moibenko over 7 years ago
Policy applies only for writes.
If we want reads come through new cache we need to assign a new library to corresponding disk volumes just as we do for tape volumes when we move then from one library to another.
Question. If we install a new system around a new disk library manager cluster, and move an existing policy to this new cluster, will they be able to access files they wrote prior to the move?
Gene
#4 Updated by Alexander Moibenko over 7 years ago
HLD added
#5 Updated by Alexander Moibenko over 7 years ago
Added description of multi-processed migrator and its communication with migration dispatcher in case when it ran out of allowed number of processes.
#6 Updated by Alexander Moibenko over 7 years ago
- % Done changed from 50 to 70
#7 Updated by Alexander Moibenko over 7 years ago
Project finished
#8 Updated by Alexander Moibenko over 7 years ago
- % Done changed from 70 to 100