Modeling and Optimization of Latency in Erasure-coded Storage Systems

被引:1
作者
Aggarwal, Vaneet [1 ]
Lan, Tian [2 ]
机构
[1] Purdue Univ, W Lafayette, IN 47907 USA
[2] George Washington Univ, Washington, DC 20052 USA
来源
FOUNDATIONS AND TRENDS IN COMMUNICATIONS AND INFORMATION THEORY | 2021年 / 18卷 / 03期
关键词
DISTRIBUTED STORAGE; REGENERATING CODES; TAMING LATENCY; MULTI-TENANT; HEAVY TAILS; PARALLEL; PERFORMANCE; SYNCHRONIZATION; ASYMPTOTICS; ALGORITHM;
D O I
10.1561/0100000108
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
As consumers are increasingly engaged in social networking and E-commerce activities, businesses grow to rely on Big Data analytics for intelligence, and traditional IT infrastructures continue to migrate to the cloud and edge, these trends cause distributed data storage demand to rise at an unprecedented speed. Erasure coding has seen itself quickly emerged as a promising technique to reduce storage cost while providing similar reliability as replicated systems, widely adopted by companies like Facebook, Microsoft and Google. However, it also brings new challenges in characterizing and optimizing the access latency when data objects are erasure coded in distributed storage. The aim of this monograph is to provide a review of recent progress (both theoretical and practical) on systems that employ erasure codes for distributed storage. In this monograph, we will first identify the key challenges and taxonomy of the research problems and then give an overview of different models and approaches that have been developed to quantify latency of erasure-coded storage. This includes recent work leveraging MDS-Reservation, Fork-Join,Probabilistic, and Delayed-Relaunch scheduling policies, as well as their applications to characterizing access latency (e.g., mean, tail, and asymptotic latency) of erasure-coded distributed storage systems. We will also extend the discussions to video streaming from erasure-coded distributed storage systems. Next, we will bridge the gap between theory and practice, and discuss lessons learned from prototype implementations. In particular, we will discuss exemplary implementations of erasure-coded storage, illuminate key design degrees of freedom and tradeoffs, and summarize remaining challenges in real-world storage systems such as in content delivery and caching. Open problems for future research are discussed at the end of each chapter.
引用
收藏
页码:380 / 525
页数:146
相关论文
共 128 条
[1]  
A. S3, AM SIMPL STOR SERV
[2]  
Abdelkefi Atef, 2011, Proceedings 2011 9th Annual Communication Networks and Services Research Conference (CNSR 2011), P41, DOI 10.1109/CNSR.2011.15
[3]  
Aggarwal V., 2016, DISTRIBUTED COMPUTIN
[4]  
Aggarwal V, 2017, IEEE INFOCOM SER
[5]   Sprout: A Functional Caching Approach to Minimize Service Latency in Erasure-Coded Storage [J].
Aggarwal, Vaneet ;
Chen, Yih-Farn Robin ;
Lan, Tian ;
Xiang, Yu .
IEEE-ACM TRANSACTIONS ON NETWORKING, 2017, 25 (06) :3683-3694
[6]  
Aggarwal V, 2014, IEEE INFOCOM SER, P1833, DOI 10.1109/INFOCOM.2014.6848122
[7]   Using erasure codes efficiently for storage in a distributed system [J].
Aguilera, MK ;
Janakiraman, R ;
Xu, LH .
2005 INTERNATIONAL CONFERENCE ON DEPENDABLE SYSTEMS AND NETWORKS, PROCEEDINGS, 2005, :336-345
[8]   Straggler Mitigation at Scale [J].
Aktas, Mehmet Fatih ;
Soljanin, Emina .
IEEE-ACM TRANSACTIONS ON NETWORKING, 2019, 27 (06) :2266-2279
[9]  
Al-Abbasi A., 2019, IEEE TRANS CLOUD COM
[10]  
Al-Abbasi A. O., ACM T MODELING PERFO, V5, P1