Apache Hadoop-MapReduce on YARN framework latency

被引:0
作者
El Yazidi, Abdelaziz [1 ]
Azizi, Mohamed Saad [1 ]
Benlachmi, Yassine [1 ]
Hasnaoui, Moulay Lahcen [1 ]
机构
[1] ENSAAd Moulay Ismatl Univ, LMMI Lab, Meknes 50000, Morocco
来源
12TH INTERNATIONAL CONFERENCE ON AMBIENT SYSTEMS, NETWORKS AND TECHNOLOGIES (ANT) / THE 4TH INTERNATIONAL CONFERENCE ON EMERGING DATA AND INDUSTRY 4.0 (EDI40) / AFFILIATED WORKSHOPS | 2021年 / 184卷
关键词
Hadoop; big data; YARN; MapReduce; Resource Manager; IoT;
D O I
10.1016/j.procs.2021.03.100
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Big Data is currently a fertile field for researchers and scientific companies around the world, due to the emergence of new technologies, Internet of Things (IoT) and means of communication such as social networking sites, which has led to a notable increase in the amount of data produced each day. In this paper, we discuss the challenges of Big Data and we are going to present the famous Apache Hadoop framework and we are going to test the latency at the level of execution of a MapReduce job based on YARN (Yet Another Resource Negotiator) by applying a Java wordcount application on different text files which resumes the sales of a store, which has prepared beforehand, finally going to make a synthesis of the results. (C) 2021 The Authors. Published by Elsevier B.V.
引用
收藏
页码:803 / 808
页数:6
相关论文
共 10 条
  • [1] [Anonymous], 2008, P 8 USENIX C OP SYST
  • [2] Appuswamy R, 2013, ACM S CLOUD COMP 2 O ACM S CLOUD COMP 2 O
  • [3] Beyond the hype: Big data concepts, methods, and analytics
    Gandomi, Amir
    Haider, Murtaza
    [J]. INTERNATIONAL JOURNAL OF INFORMATION MANAGEMENT, 2015, 35 (02) : 137 - 144
  • [4] Hashem H., 2014, 9 INT C INT SYST THE 9 INT C INT SYST THE
  • [5] Oguntimilehin A., 2014, Journal of Emerging Trends in Computing and Information Sciences, P433
  • [6] A comprehensive view of Hadoop research-A systematic literature review
    Polato, Ivanilton
    Re, Reginaldo
    Goldman, Alfredo
    Kon, Fabio
    [J]. JOURNAL OF NETWORK AND COMPUTER APPLICATIONS, 2014, 46 : 1 - 25
  • [7] The Hadoop Distributed Filesystem: Balancing Portability and Performance
    Shafer, Jeffrey
    Rixner, Scott
    Cox, Alan L.
    [J]. 2010 IEEE INTERNATIONAL SYMPOSIUM ON PERFORMANCE ANALYSIS OF SYSTEMS AND SOFTWARE (ISPASS 2010), 2010, : 122 - 133
  • [8] White T., 2012, HADOOP DE NITIVE GUI
  • [9] Yao Y., HASTE HADOOP YARN HASTE HADOOP YARN
  • [10] You H.-H., 2011, Proceedings of the 2011 ACM Symposium on Applied Computing, P127