An Overview of Hadoop MapReduce, Spark, and Scalable Graph Processing Architecture

被引：2
|
作者：
Talan, Pooja P. [1 ]
Sharma, Kartik U. [1 ]
Nawade, Pratiksha P. [1 ]
Talan, Karishma P. [2 ]
机构：
[1] Prof Ram Meghe Coll Engn & Management, Badnera Amravati, India
[2] KPIT Technol Ltd, Mumbai, Maharashtra, India
来源：
RECENT DEVELOPMENTS IN MACHINE LEARNING AND DATA ANALYTICS | 2019年 / 740卷
关键词：
Big Data; Hadoop MapReduce; Apache Spark; Graph processing;
D O I：
10.1007/978-981-13-1280-9_3
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
In today's technology era, Big Data has become a buzzword. Various frameworks are available in order to process this Big Data. Both Hadoop and Spark are open source framework to process Big Data. Hadoop provides batch processing while Spark supports both batch as well as stream processing, i.e., it is a hybrid processing framework. Both frameworks have their own advantages and drawback. The contribution of this paper is to provide a comparative analysis of Hadoop MapReduce and Apache Spark. In this paper, we also propose a scalable graph processing architecture that could be used to overcome traditional limitations of Hadoop framework.
引用
收藏
页码：35 / 42
页数：8
相关论文
共 50 条
  • [1] An overview and an Approach for Graph Data Processing using Hadoop MapReduce
    Talan, Pooja P.
    Sharma, Kartik U.
    PROCEEDINGS OF THE 2ND INTERNATIONAL CONFERENCE ON COMPUTING METHODOLOGIES AND COMMUNICATION (ICCMC 2018), 2018, : 59 - 63
  • [2] Scalable Big Graph Processing in MapReduce
    Qin, Lu
    Yu, Jeffrey Xu
    Chang, Lijun
    Cheng, Hong
    Zhang, Chengqi
    Lin, Xuemin
    SIGMOD'14: PROCEEDINGS OF THE 2014 ACM SIGMOD INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 2014, : 827 - 838
  • [3] A comparative between Hadoop MapReduce and Apache Spark on HDFS
    Saouabi, Mohamed
    Ezzati, Abdellah
    PROCEEDINGS OF THE 1ST INTERNATIONAL CONFERENCE ON INTERNET OF THINGS AND MACHINE LEARNING (IML'17), 2017,
  • [4] A Comparison of Big Remote Sensing Data Processing with Hadoop MapReduce and Spark
    Chebbi, I.
    Boulila, W.
    Mellouli, N.
    Lamolle, M.
    Farah, I. R.
    2018 4TH INTERNATIONAL CONFERENCE ON ADVANCED TECHNOLOGIES FOR SIGNAL AND IMAGE PROCESSING (ATSIP), 2018,
  • [5] Big Data Management Processing with Hadoop MapReduce and Spark Technology: A Comparison
    Verma, Ankush
    Mansuri, Ashik Hussain
    Jain, Neelesh
    2016 SYMPOSIUM ON COLOSSAL DATA ANALYSIS AND NETWORKING (CDAN), 2016,
  • [6] Architecture of Efficient Word Processing using Hadoop MapReduce for Big Data Applications
    Mandal, Bichitra
    Sahoo, Ramesh Kumar
    Sethi, Srinivas
    PROCEEDINGS 2015 INTERNATIONAL CONFERENCE ON MAN AND MACHINE INTERFACING (MAMI), 2015,
  • [7] Comparative Analysis of Apache Spark and Hadoop MapReduce Using Various Parameters and Execution Time
    Meena, Bhagavathula
    Sarwani, I. S. L.
    Archana, M.
    Supriya, P.
    INTELLIGENT COMPUTING AND COMMUNICATION, ICICC 2019, 2020, 1034 : 719 - 725
  • [8] High-throughput and scalable protein function identification with Hadoop and Map-only pattern of the MapReduce processing model
    Dariusz Mrozek
    Marek Suwała
    Bożena Małysiak-Mrozek
    Knowledge and Information Systems, 2019, 60 : 145 - 178
  • [9] High-throughput and scalable protein function identification with Hadoop and Map-only pattern of the MapReduce processing model
    Mrozek, Dariusz
    Suwala, Marek
    Malysiak-Mrozek, Bozena
    KNOWLEDGE AND INFORMATION SYSTEMS, 2019, 60 (01) : 145 - 178
  • [10] IRPDP_HT2: a scalable data pre-processing method in web usage mining using Hadoop MapReduce
    Srivastava, Atul Kumar
    Srivastava, Mitali
    SOFT COMPUTING, 2023, 27 (12) : 7907 - 7923
12345