A Heterogeneous Hybrid Storage Method Based on Ceph Erasure Code

被引:1
|
作者
Qin, Mingyuan [1 ]
Fu, Zheng [1 ]
机构
[1] Chongqing Univ Posts & Telecommun, Coll Comp Sci & Technol, Chongqing, Peoples R China
来源
2021 ASIA-PACIFIC CONFERENCE ON COMMUNICATIONS TECHNOLOGY AND COMPUTER SCIENCE (ACCTCS 2021) | 2021年
关键词
Ceph; TOPSIS; heterogeneous hybrid storage; multi-attribute decision-making; Erasure Code; LATENCY;
D O I
10.1109/ACCTCS52002.2021.00044
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In order to achieve the goal of reducing storage redundancy, Ceph provides support for erasure code technology, which can more effectively use storage space. However, when Ceph selects storage nodes, it only uses storage capacity as the only indicator, without considering the resource load status and the heterogeneous situation of nodes, which makes a large number of read and write tasks allocated to storage nodes with lower performance. The above causes the problem of low reading and writing efficiency in Ceph erasure code mode. Based on this problem, we propose a heterogeneous hybrid storage method based on Ceph erasure code. Under the premise of comprehensively considering node CPU utilization, memory utilization and other performance indicators, Using TOPSIS method to implement multi-attribute decision-making and assigning Ceph erasure code tasks to the storage node with best performance. Experiments show that the method can effectively improve the read and write performance of large objects in Ceph erasure code mode.
引用
收藏
页码:182 / 186
页数:5
相关论文
共 50 条
  • [31] Optimization Design and Realization of Ceph Storage System Based on Software Defined Network
    Wu, Duo
    Wang, Yong
    Feng, Hao
    Huan, Yiming
    2017 13TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND SECURITY (CIS), 2017, : 277 - 281
  • [32] UHUM: An Efficient Hybrid Update Mechanism in Distributed Storage Systems with Erasure Coding
    Luo, Qian
    Wang, Yun
    PROCEEDINGS OF THE 2019 IEEE 23RD INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED COOPERATIVE WORK IN DESIGN (CSCWD), 2019, : 158 - 163
  • [33] A Ceph-based storage strategy for big gridded remote sensing data
    Tang, Xinyu
    Yao, Xiaochuang
    Liu, Diyou
    Zhao, Long
    Li, Li
    Zhu, Dehai
    Li, Guoqing
    BIG EARTH DATA, 2022, 6 (03) : 323 - 339
  • [34] On Endurance of Erasure Codes in SSD-based Storage Systems
    Chamazcoti, Saeideh Alinezhad
    Miremadi, Seyed Ghassem
    Asadi, Hossein
    2013 17TH CSI INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE AND DIGITAL SYSTEMS (CADS 2013), 2013, : 67 - 72
  • [35] Construction and Performance Analysis of Unified Storage Cloud Platform Based on OpenStack with Ceph RBD
    Ding, Weichao
    Gu, Chunhua
    Luo, Fei
    Chang, Yaohui
    2018 IEEE 3RD INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND BIG DATA ANALYSIS (ICCCBDA), 2018, : 135 - 141
  • [36] An adaptive redundancy control method for erasure-code-based real-time data transmission over the Internet
    Yuk, SW
    Kang, MG
    Shin, BC
    Cho, DH
    IEEE TRANSACTIONS ON MULTIMEDIA, 2001, 3 (03) : 366 - 374
  • [37] Device and Placement Aware Framework to Optimize Single Failure Recoveries and Reads for Erasure Coded Storage System with Heterogeneous Storage Devices
    Fu, Yingxun
    Liu, Xun
    Shu, Jiwu
    Shen, Zhirong
    Zhang, Shiye
    Wu, Jun
    Duan, Jianyong
    Ma, Li
    2020 INTERNATIONAL SYMPOSIUM ON RELIABLE DISTRIBUTED SYSTEMS (SRDS 2020), 2020, : 225 - 235
  • [38] An Erasure Code-based Approach to Improve Data Recovery and Update Capability
    Zhou, Hang
    Yang, Yahui
    Li, Weiping
    PROCEEDINGS OF THE 2018 INTERNATIONAL CONFERENCE ON MECHANICAL, ELECTRONIC, CONTROL AND AUTOMATION ENGINEERING (MECAE 2018), 2018, 149 : 614 - 622
  • [39] An Empirical Study of Performance, Power Consumption, and Energy Cost of Erasure Code Computing for HPC Cloud Storage Systems
    Chen, Hsing-bung
    Grider, Gary
    Inman, Jeff
    Fields, Parks
    Kuehn, Jeff Alan
    PROCEEDINGS OF THE 2015 IEEE INTERNATIONAL CONFERENCE ON NETWORKING, ARCHITECTURE AND STORAGE (NAS), 2015, : 71 - 80
  • [40] Load-Balanced Recovery Schemes for Single-disk Failure in Storage Systems with Any Erasure Code
    Luo, Xianghong
    Shu, Jiwu
    2013 42ND ANNUAL INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING (ICPP), 2013, : 552 - 561