Performance analysis of MapReduce program in heterogeneous cloud computing

被引：8
|
作者：
Lin, Wenhui [1 ,2 ]
Liu, Jun [1 ]
机构：
[1] Beijing Key Laboratory of Network System Architecture and Convergence, School of Information and Communication Engineering, Beijing University of Posts and Telecommunications
[2] Technology Research Institute, Aisino Corporation
来源：
Journal of Networks | 2013年 / 8卷 / 08期
关键词：
Benchmark; Cloud computing; Hadoop; Performance analysis;
D O I：
10.4304/jnw.8.8.1734-1741
中图分类号：
学科分类号：
摘要：
The research of Hadoop is an important part of cloud computing industry, and Hadoop performance research is a key research direction. The Hadoop performance analysis as a basic work can provide important reference for other performance optimization researches. In this paper, based on previous researches of server performance analysis, we propose a node performance measurement method on Hadoop. We describe in detail how to measure the performance value of each node in heterogeneous Hadoop cluster and evaluate measurement results by running MapReduce programs. Meanwhile, the method has also been implemented and evaluated in real-world Hadoop cluster. Experiment results show that the method can accurately measure the performance value of each node. Based on this research, users can have a comprehensive and objective understanding of their own Hadoop cluster and then make optimization and improvement on Hadoop. © 2013 ACADEMY PUBLISHER.
引用
收藏
页码：1734 / 1741
页数：7
相关论文
共 50 条
  • [21] Analysis of Memory Management Policies for Heterogeneous Cloud Computing
    Son, Dong Oh
    Choi, Hong Jun
    Park, Jae Hyung
    Hong, Cheol
    2013 INTERNATIONAL CONFERENCE ON INFORMATION SCIENCE AND APPLICATIONS (ICISA 2013), 2013,
  • [22] DISTRIBUTED LOG ANALYSIS ON THE CLOUD USING MapReduce
    Aydin, Galip
    Hallac, Ibrahim R.
    TEHNICKI VJESNIK-TECHNICAL GAZETTE, 2016, 23 (04): : 1011 - 1016
  • [23] Scanning Files for Signatures with a MapReduce System on Cloud Computing
    Huang, Tzu-Chi
    Chu, Kuo-Chih
    Shieh, Ce-Kuen
    Chiu, Chui-Ming
    Huang, Sheng-Wei
    2014 INTERNATIONAL SYMPOSIUM ON COMPUTER, CONSUMER AND CONTROL (IS3C 2014), 2014, : 248 - 251
  • [24] Smart Intermediate Data Transfer for MapReduce on Cloud Computing
    Huang, Tzu-Chi
    Chu, Kuo-Chih
    Rao, Yu-Ruei
    2013 INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND BIG DATA (CLOUDCOM-ASIA), 2013, : 9 - 14
  • [25] An optimized MapReduce workflow scheduling algorithm for heterogeneous computing
    Zhuo Tang
    Min Liu
    Almoalmi Ammar
    Kenli Li
    Keqin Li
    The Journal of Supercomputing, 2016, 72 : 2059 - 2079
  • [26] Cuckoo: Opportunistic MapReduce on Ephemeral and Heterogeneous Cloud Resources
    Dartois, Jean-Emile
    Ribeiro, Heverson B.
    Boukhobza, Jalil
    Barais, Olivier
    2019 IEEE 12TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (IEEE CLOUD 2019), 2019, : 396 - 403
  • [27] An optimized MapReduce workflow scheduling algorithm for heterogeneous computing
    Tang, Zhuo
    Liu, Min
    Ammar, Almoalmi
    Li, Kenli
    Li, Keqin
    JOURNAL OF SUPERCOMPUTING, 2016, 72 (06) : 2059 - 2079
  • [28] Morpho: A decoupled MapReduce framework for elastic cloud computing
    Lu, Lu
    Shi, Xuanhua
    Jin, Hai
    Wang, Qiuyue
    Yuan, Daxing
    Wu, Song
    FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF GRID COMPUTING AND ESCIENCE, 2014, 36 : 80 - 90
  • [29] A New Mechanism to Ensure Integrity for MapReduce in Cloud Computing
    Bendahmane, Ahmed
    Essaaidi, Mohammad
    el Moussaoui, Ahmed
    Younes, Ali
    2012 INTERNATIONAL CONFERENCE ON MULTIMEDIA COMPUTING AND SYSTEMS (ICMCS), 2012, : 785 - 790
  • [30] Metaheuristics for scheduling of heterogeneous tasks in cloud computing environments: Analysis, performance evaluation, and future directions
    Singh, Harvinder
    Tyagi, Sanjay
    Kumar, Pardeep
    Gill, Sukhpal Singh
    Buyya, Rajkumar
    SIMULATION MODELLING PRACTICE AND THEORY, 2021, 111
12345