DESIGN AND IMPLEMENTATION OF PARALLEL STATIATICAL ALGORITHM BASED ON HADOOP'S MAPREDUCE MODEL

被引：0
作者：
Duan, Songqing [1 ]
Wu, Bin [1 ]
Wang, Bai [1 ]
Yang, Juan [1 ]
机构：
[1] Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China
来源：
2011 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS | 2011年
基金：
中国国家自然科学基金;
关键词：
Hadoop; MapReduce; Parallel Statistical Algorithm; Central Tendency; Dispersion; Distribution Tendency;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The rapid growth of data promotes the development of parallel computing. MapReduce, which is a simplified programming model of distributed parallel computing, is becoming more and more popular. In this paper, we design and implementation of parallel statistical algorithm based on Hadoop ' s MapReduce model. The algorithm, which is used to grasp the overall characteristics of massive data, involves the calculation of central tendency, dispersion and distribution tendency. By experiment, we come to the conclusion that the algorithm is suitable for dealing with large-scale data.
引用
收藏
页码：134 / 138
页数：5
相关论文
共 50 条
  • [31] Parallel Bat Algorithm-Based Clustering Using MapReduce
    Ashish, Tripathi
    Kapil, Sharma
    Manju, Bala
    NETWORKING COMMUNICATION AND DATA KNOWLEDGE ENGINEERING, VOL 2, 2018, 4 : 73 - 82
  • [32] Parallel K-Medoids Clustering Algorithm Based on Hadoop
    Jiang, Yaobin
    Zhang, Jiongmin
    2014 5TH IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING AND SERVICE SCIENCE (ICSESS), 2014, : 649 - 652
  • [33] Parallel knowledge Community Detection Algorithm Research based on MapReduce
    Xu, Min
    Yang, Panpan
    Ma, Jie
    ADVANCED RESEARCH ON INDUSTRY, INFORMATION SYSTEMS AND MATERIAL ENGINEERING, PTS 1-7, 2011, 204-210 : 1646 - 1650
  • [34] A parallel clustering algorithm for Logs Data Based on Hadoop Platform
    Huo, Jiuyuan
    Weng, Jian
    Qu, Hong
    2019 THE 3RD INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPILATION, COMPUTING AND COMMUNICATIONS (HP3C 2019), 2019, : 90 - 94
  • [35] A Survey of Whole Genome Alignment Tools and Frameworks based on Hadoop's MapReduce
    Purbarani, Sumarsih C.
    Sanabila, Hadaiq R.
    Bowolaksono, Anom
    Wiweko, Budi
    2016 INTERNATIONAL WORKSHOP ON BIG DATA AND INFORMATION SECURITY (IWBIS), 2016, : 65 - 69
  • [36] The High-Activity Parallel Implementation of Data Preprocessing Based on MapReduce
    He, Qing
    Tan, Qing
    Ma, Xudong
    Shi, Zhongzhi
    ROUGH SET AND KNOWLEDGE TECHNOLOGY (RSKT), 2010, 6401 : 646 - 654
  • [37] An improved chaotic image encryption algorithm using Hadoop-based MapReduce framework for massive remote sensed images in parallel IoT applications
    Mahmoud Ahmad Al-Khasawneh
    Irfan Uddin
    Syed Atif Ali Shah
    Ahmad M. Khasawneh
    Laith Abualigah
    Marwan Mahmoud
    Cluster Computing, 2022, 25 : 999 - 1013
  • [38] An improved chaotic image encryption algorithm using Hadoop-based MapReduce framework for massive remote sensed images in parallel IoT applications
    Al-Khasawneh, Mahmoud Ahmad
    Uddin, Irfan
    Shah, Syed Atif Ali
    Khasawneh, Ahmad M.
    Abualigah, Laith
    Mahmoud, Marwan
    CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2022, 25 (02): : 999 - 1013
  • [39] Parallel Particle Swarm Optimization Clustering Algorithm based on MapReduce Methodology
    Aljarah, Ibrahim
    Ludwig, Simone A.
    PROCEEDINGS OF THE 2012 FOURTH WORLD CONGRESS ON NATURE AND BIOLOGICALLY INSPIRED COMPUTING (NABIC), 2012, : 104 - 111
  • [40] Performance Evaluation of a MapReduce Hadoop-based Implementation for Processing Large Virtual Campus Log Files
    Xhafa, Fatos
    Garcia, Daniel
    Ramirez, Daniel
    Caballe, Santi
    2015 10TH INTERNATIONAL CONFERENCE ON P2P, PARALLEL, GRID, CLOUD AND INTERNET COMPUTING (3PGCIC), 2015, : 200 - 206
12345