Novel Data Placement Algorithm for Distributed Storage System Based on Fault-Tolerant Domain

被引：2

作者：

Shi L. ^{[1
]}

Wang Z. ^{[2
]}

Li X. ^{[1
]}

机构：

[1] School of Cyber Science and Engineering, Shanghai Jiao Tong University, Shanghai

[2] School of Electronic Information Engineering, Shanghai DianJi University, Shanghai

来源：

Journal of Shanghai Jiaotong University (Science) | 2021年 / 26卷 / 04期

关键词：

A; data placement; data reliability; distributed system; failure domain; fault-tolerant domain; storage system; TP; 391;

D O I：

10.1007/s12204-020-2253-5

中图分类号：

学科分类号：

摘要：

The 3-replica redundancy strategy is widely used to solve the problem of data reliability in large-scale distributed storage systems. However, its storage capacity utilization is only 33%. In this paper, a data placement algorithm based on fault-tolerant domain (FTD) is proposed. Owing to the fine-grained design of the FTD, the data reliability of systems using two replicas is comparable to that of current mainstream systems using three replicas, and the capacity utilization is increased to 50%. Moreover, the proposed FTD provides a new concept for the design of distributed storage systems. Distributed storage systems can take FTDs as the units for data placement, data migration, data repair and so on. In addition, fault detection can be performed independently and concurrently within the FTDs. © 2020, Shanghai Jiao Tong University and Springer-Verlag GmbH Germany, part of Springer Nature.

引用

页码：463 / 470

页数：7

共 16 条

[1]

Pinheiro E., Weber W.D., Barroso L.A., Failure trends in a large disk drive population [C], Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’ 07), pp. 17-29, (2007)

[2]

Chemawat S., Gobioff H., Leung S.T., The Google file system [J], ACM SIGOPS Operating Systems Review, 37, 5, pp. 29-43, (2003)

[3]

Shvachko K., Kuang H.R., Radia S., Et al., The hadoop distributed file system [C], 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1-10, (2010)

[4]

Weil S.A., Brandt S.A., Miller E.L., Et al., Ceph: A scalable, high-performance distributed file system [C], Proceedings of the 7th Symposium on Operating Systems Design and Implementation, pp. 307-320, (2006)

[5]

Adya A.J., Farsite: Federated, available, and reliable storage for an incompletely trusted environment [C]//Proceedings of the 5th symposium on Operating systems design and implementation (OSDI 2002), Boston, MA, USA: ACM, 36, pp. 1-14, (2002)

[6]

Yang S.L., Zhang G.Y., Review of data recovery in storage systems based on erasure codes [J], Journal of Frontiers of Computer Science and Technology, 11, 10, pp. 1531-1544, (2017)

[7]

Reed I.S., Solomon G., Polynomial codes over certain finite fields [J], Journal of the Society for Industrial and Applied Mathematics, 8, 2, pp. 300-304, (1960)

[8]

Weil S.A., Brandt S.A., Miller E.L., Et al., CRUSH: Controlled, scalable, decentralized placement of replicated data [C], Proceedings of the 2006 ACM/IEEE Conference on Supercomputing.Tampa, pp. 122-133, (2006)

[9]

Honicky R.J., Miller E.L., Replication under ucal-able uashing: A family of algorithms for scalable decentralized data distribution [C], Proceedings of 18th International Parallel and Distributed Processing Symposium (IPDPS 2004), pp. 1-10, (2004)

[10]

Swift’s overview and concepts: The rings [EB/OL]., (2019)

← 1 2 →