DESIGN AND IMPLEMENTATION OF PARALLEL STATIATICAL ALGORITHM BASED ON HADOOP'S MAPREDUCE MODEL

被引：0
|
作者：
Duan, Songqing [1 ]
Wu, Bin [1 ]
Wang, Bai [1 ]
Yang, Juan [1 ]
机构：
[1] Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China
来源：
2011 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS | 2011年
基金：
中国国家自然科学基金;
关键词：
Hadoop; MapReduce; Parallel Statistical Algorithm; Central Tendency; Dispersion; Distribution Tendency;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The rapid growth of data promotes the development of parallel computing. MapReduce, which is a simplified programming model of distributed parallel computing, is becoming more and more popular. In this paper, we design and implementation of parallel statistical algorithm based on Hadoop ' s MapReduce model. The algorithm, which is used to grasp the overall characteristics of massive data, involves the calculation of central tendency, dispersion and distribution tendency. By experiment, we come to the conclusion that the algorithm is suitable for dealing with large-scale data.
引用
收藏
页码：134 / 138
页数：5
相关论文
共 50 条
  • [1] DESIGN AND IMPLEMENTATION OF PARALLEL TERM CONTRIBUTION ALGORITHM BASED ON MAPREDUCE MODEL
    Chao, Peng
    Bin, Wu
    Chao, Deng
    PROCEEDINGS OF THE 2012 SEVENTH OPEN CIRRUS SUMMIT (OCS 2012), 2012, : 43 - 47
  • [2] Implementation of Parallel CASINO Algorithm Based on MapReduce
    Zhang, Li
    Shi, Yijie
    PROCEEDINGS OF THE 2016 INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND ENGINEERING APPLICATIONS, 2016, 63 : 104 - 109
  • [3] Parallel Reachability Testing Based on Hadoop MapReduce
    Qi, Xiaofang
    Li, Yueran
    SOFTWARE ANALYSIS, TESTING, AND EVOLUTION, SATE 2018, 2018, 11293 : 173 - 184
  • [4] Verification and Validation of Parallel Support Vector Machine Algorithm based on MapReduce Program Model on Hadoop Cluster
    Kiran, M.
    Kumar, Amresh
    Prathap, B. R.
    PROCEEDINGS OF THE 2013 INTERNATIONAL CONFERENCE ON ADVANCED COMPUTING & COMMUNICATION SYSTEMS (ICACCS), 2013,
  • [5] Parallel Implementation of Apriori Algorithm Based on MapReduce
    Li, Ning
    Zeng, Li
    He, Qing
    Shi, Zhongzhi
    INTERNATIONAL JOURNAL OF NETWORKED AND DISTRIBUTED COMPUTING, 2013, 1 (02) : 89 - 96
  • [6] Design and implementation of K-means parallel algorithm based on Hadoop
    Jia, Jiyang
    Xie, Hui
    Xu, Tao
    PROCEEDINGS OF 2021 2ND INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND INFORMATION SYSTEMS (ICAIIS '21), 2021,
  • [7] Parallel implementation of Apriori algorithm based on MapReduce
    Li N.
    Zeng L.
    He Q.
    Shi Z.
    International Journal of Networked and Distributed Computing, 2013, 1 (2) : 89 - 96
  • [8] A Parallel Genetic Algorithms Framework based on Hadoop MapReduce
    Ferrucci, Filomena
    Salza, Pasquale
    Kechadi, M-Tahar
    Sarro, Federica
    30TH ANNUAL ACM SYMPOSIUM ON APPLIED COMPUTING, VOLS I AND II, 2015, : 1664 - 1667
  • [9] Verification and Validation of MapReduce Program model for Parallel K-Means algorithm on Hadoop Cluster
    Kumar, Amresh
    Kiran, M.
    Prathap, B. R.
    2013 FOURTH INTERNATIONAL CONFERENCE ON COMPUTING, COMMUNICATIONS AND NETWORKING TECHNOLOGIES (ICCCNT), 2013,
  • [10] Implementation of Page Rank Algorithm in Hadoop MapReduce Framework
    Bhawivuga, Adhitya
    Kirana, Annisa Puspa
    2016 INTERNATIONAL SEMINAR ON INTELLIGENT TECHNOLOGY AND ITS APPLICATIONS (ISITIA): RECENT TRENDS IN INTELLIGENT COMPUTATIONAL TECHNOLOGIES FOR SUSTAINABLE ENERGY, 2016, : 231 - 235
12345