A Comparative Study of Big Data Processing : Hadoop vs. Spark

被引：0

作者：

Sharma, Meghna ^{[1
]}

Kaur, Jagdeep ^{[1
]}

机构：

[1] NorthCap Univ, Comp Sci & Engn, Gurugram, Haryana, India

来源：

PROCEEDINGS OF THE 2019 6TH INTERNATIONAL CONFERENCE ON COMPUTING FOR SUSTAINABLE GLOBAL DEVELOPMENT (INDIACOM) | 2019年

关键词：

Big Data; Hadoop; Spark; In-Memory Processing; On-Disk Processing;

D O I：

暂无

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

Apache Spark and Hadoop's MapReduce are two very important tools used for Big Data processing. The processing started with Hadoop's MapReduce Framework but suffers from many disadvantages due to multiple disc processing operations. The drawbacks of the traditional big data processing have been overcome by in memory handling framework like Spark. In some aspects they go hand in hand as due to lack of file system in Spark, it needs to depend upon MapReduce. This paper has shown the extensive study on various tools related to Big Data processing and has done extensive comparison on MapReduce Vs Spark. The frameworks have been studied on real time datasets and finally compared in terms of processing time. Spark showing the remarkable improvement over MapReduce.

引用

页码：1073 / 1077

页数：5

共 16 条

[1] Big Data 2.0 Processing Systems: Taxonomy and Open Challenges [J].

Bajaber, Fuad ;

Elshawi, Radwa ;

Batarfi, Omar ;

Altalhi, Abdulrahman ;

Barnawi, Ahmed ;

Sakr, Sherif .

JOURNAL OF GRID COMPUTING, 2016, 14 (03) :379-405

[2] Integration of Cloud computing and Internet of Things: A survey [J].

Botta, Alessio ;

de Donato, Walter ;

Persico, Valerio ;

Pescape, Antonio .

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2016, 56 :684-700

[3]

Coronel C., 2016, DATABASE SYSTEMS DES

[4]

data.cityofne, 2017, YELL TAX TRIP DAT

[5] Cloud-Assisted Computation Offloading to Support Mobile Services [J].

Elgazzar, Khalid ;

Martin, Patrick ;

Hassanein, Hossam S. .

IEEE TRANSACTIONS ON CLOUD COMPUTING, 2016, 4 (03) :279-292

[6]

Hoffman Steve., 2015, Apache Flume: Distributed Log Collection for Hadoop - Second Edition

[7] Big data and analytics in healthcare: Introduction to the special section [J].

Kankanhalli, Atreyi ;

Hahn, Jungpil ;

Tan, Sharon ;

Gao, Gordon .

INFORMATION SYSTEMS FRONTIERS, 2016, 18 (02) :233-235

[8]

Lydia E.L., 2015, Int. J. Sci., Eng. Comput. Technol., V5, P390

[9]

Mazumder S., 2016, Big Data Concepts, Theories, and Applications, P29, DOI DOI 10.1007/978-3-319-27763-9_2

[10]

Prasad B.R., 2016, Int. J. Database Theory Appl, V9, P45

← 1 2 →