Massive Data Storage Algorithm Based on Node Performance Evaluation

被引：1
作者：
Chen, Qiang [1 ]
机构：
[1] Guangdong Univ Sci & Technol, Dept Comp Sci, Dongguan 523083, Guangdong, Peoples R China
来源：
2018 14TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND SECURITY (CIS) | 2018年
关键词：
Hadoop; Massive Data Storage; Node Performance Evaluation;
D O I：
10.1109/CIS2018.2018.00097
中图分类号：
TP31 [计算机软件];
学科分类号：
081202 ; 0835 ;
摘要：
In order to solve the problem of massive network data storage in heterogeneous environment, this paper introduces the node performance evaluation method and data storage node selection algorithm in HDFS data storage mechanism. First, this article describes how to evaluate the performance of various nodes in a Hadoop cluster. Then,based on the performance evaluation results of the nodes, select the appropriate nodes to store the data blocks when storing the data. This algorithm is used to achieve the goal of distributing the number ratio of data blocks in each node according to the performance value of the node.In this paper, the related technology of distributed data storage was studied in detail, based on which the heterogeneous environment was studied.The problem of distributed data storage is analyzed and then based on the service performance evaluation technology.Node performance evaluation of massive data storage algorithm..
引用
收藏
页码：408 / 411
页数：4
相关论文
共 10 条
  • [1] Banoori Farhad, P 2018 INT C ADV CON
  • [2] Huang Baiyi, 2011, P 2011 4 IEEE INT C, V6, P4
  • [3] Juan Yanhong, 2012, J HARBIN I TECHNOLOG, V19, P13
  • [4] Kaniezhil R., 2012, Journal of Electronic Science and Technology, V10, P342, DOI 10.3969/j.issn.1674-862X.2012.04.010
  • [5] Review of data storage and management technologies for massive remote sensing data
    Lu XueFeng
    Cheng ChengQi
    Gong JianYa
    Guan Li
    [J]. SCIENCE CHINA-TECHNOLOGICAL SCIENCES, 2011, 54 (12) : 3220 - 3232
  • [6] Qu LL, 2013, PROCEEDINGS OF 2013 IEEE 11TH INTERNATIONAL CONFERENCE ON ELECTRONIC MEASUREMENT & INSTRUMENTS (ICEMI), P84, DOI 10.1109/ICEMI.2013.6743046
  • [7] Xiang Guangli, 2018, Wuhan University Journal of Natural Sciences, V23, P163, DOI 10.1007/s11859-018-1306-1
  • [8] Xie Xiaolan, 2013, P 2013 INT C INF ENG, P8
  • [9] XU Pandeng, 2015, HIGH TECHNOLOGY LETT, P231
  • [10] Performance analysis of optical burst switching node with limited wavelength conversion capabilities
    Xu Yi
    Shi Kai-yuan
    Fan Ge
    [J]. JOURNAL OF CENTRAL SOUTH UNIVERSITY OF TECHNOLOGY, 2009, 16 (05): : 794 - 801
1