DESIGN AND IMPLEMENTATION OF PARALLEL STATIATICAL ALGORITHM BASED ON HADOOP'S MAPREDUCE MODEL

被引：0
作者：
Duan, Songqing [1 ]
Wu, Bin [1 ]
Wang, Bai [1 ]
Yang, Juan [1 ]
机构：
[1] Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China
来源：
2011 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS | 2011年
基金：
中国国家自然科学基金;
关键词：
Hadoop; MapReduce; Parallel Statistical Algorithm; Central Tendency; Dispersion; Distribution Tendency;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The rapid growth of data promotes the development of parallel computing. MapReduce, which is a simplified programming model of distributed parallel computing, is becoming more and more popular. In this paper, we design and implementation of parallel statistical algorithm based on Hadoop ' s MapReduce model. The algorithm, which is used to grasp the overall characteristics of massive data, involves the calculation of central tendency, dispersion and distribution tendency. By experiment, we come to the conclusion that the algorithm is suitable for dealing with large-scale data.
引用
收藏
页码：134 / 138
页数：5
相关论文
共 50 条
  • [41] Hadoop Based Parallel Binary Bat Algorithm for Network Intrusion Detection
    P. Natesan
    R. R. Rajalaxmi
    G. Gowrison
    P. Balasubramanie
    International Journal of Parallel Programming, 2017, 45 : 1194 - 1213
  • [42] Algorithm implementation and tested of crop growth model based on hadoop of cloud computing
    Jiang, H. (jianghy@njau.edu.cn), 1600, Chinese Society of Agricultural Engineering (29): : 179 - 186
  • [43] Research on Parallel Particle Filtering Target Tracking Algorithm Based on Hadoop
    Sun, Fu
    Song, JianXin
    PROCEEDINGS OF THE 2015 5TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCES AND AUTOMATION ENGINEERING, 2016, 42 : 922 - 928
  • [44] Hadoop Based Parallel Binary Bat Algorithm for Network Intrusion Detection
    Natesan, P.
    Rajalaxmi, R. R.
    Gowrison, G.
    Balasubramanie, P.
    INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING, 2017, 45 (05) : 1194 - 1213
  • [45] A MapReduce based Parallel Algorithm for CIM Data Verification
    Liu, Yang
    Shen, Xiaodong
    Xu, Lixiong
    Li, Maozhen
    2014 11TH INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS AND KNOWLEDGE DISCOVERY (FSKD), 2014, : 704 - 709
  • [46] PARALLEL COREGISTRATION ALGORITHM FOR SAR IMAGES BASED ON HADOOP
    Li, Jiawei
    Zeng, Guobing
    Xu, Huaping
    IGARSS 2023 - 2023 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM, 2023, : 7957 - 7960
  • [47] Parallel k-modes Algorithm based on MapReduce
    Guo Tao
    Ding Xiangwu
    Li Yefeng
    2015 THIRD INTERNATIONAL CONFERENCE ON DIGITAL INFORMATION, NETWORKING, AND WIRELESS COMMUNICATIONS (DINWC), 2015, : 176 - 179
  • [48] Network Intrusion Detection with a Hashing Based Apriori Algorithm Using Hadoop MapReduce
    Azeez, Nureni Ayofe
    Ayemobola, Tolulope Jide
    Misra, Sanjay
    Maskeliunas, Rytis
    Damasevicius, Robertas
    COMPUTERS, 2019, 8 (04)
  • [49] Parallel Diffrential Evolution Clustering Algorithm based on MapReduce
    Daoudi, Meroua
    Hamena, Soumiya
    Benmounah, Zakaria
    Batouche, Mohamed
    2014 6TH INTERNATIONAL CONFERENCE OF SOFT COMPUTING AND PATTERN RECOGNITION (SOCPAR), 2014, : 337 - 341
  • [50] A MapReduce based Parallel Algorithm for CIM Data Verification
    Liu, Yang
    Shen, Xiaodong
    Xu, Lixiong
    Li, Maozhen
    2014 11TH INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS AND KNOWLEDGE DISCOVERY (FSKD), 2014, : 745 - 750
12345