Performance Evaluation and Estimation Model Using Regression Method for Hadoop WordCount

被引：8

作者：

Issa, Joseph A. ^{[1
]}

机构：

[1] Notre Dame Univ Louaize, Dept Elect & Comp Engn, Zouk Mosbeh 25011305, Lebanon

来源：

IEEE ACCESS | 2015年 / 3卷

关键词：

Performance analysis; cloud computing; Hadoop WordCount; MAPREDUCE;

D O I：

10.1109/ACCESS.2015.2509598

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Given the rapid growth in cloud computing, it is important to analyze the performance of different Hadoop MapReduce applications and to understand the performance bottleneck in a cloud cluster that contributes to higher or lower performance. It is also important to analyze the underlying hardware in cloud cluster servers to enable the optimization of software and hardware to achieve the maximum performance possible. Hadoop is based on MapReduce, which is one of the most popular programming models for big data analysis in a parallel computing environment. In this paper, we present a detailed performance analysis, characterization, and evaluation of Hadoop MapReduce WordCount application. We also propose an estimation model based on Amdahl's law regression method to estimate performance and total processing time versus different input sizes for a given processor architecture. The estimation regression model is verified to estimate performance and run time with an error margin of <5%.

引用

页码：2784 / 2793

页数：10

共 21 条

[1]

[Anonymous], P 9 ICPDCS INT C PAR

[2]

[Anonymous], 2014, INT C INF COMM EMB S

[3]

Bent J, 2004, USENIX ASSOCIATION PROCEEDINGS OF THE FIRST SYMPOSIUM ON NETWORKED SYSTEMS DESIGN AND IMPLEMENTATION (NSDI'04), P365

[4] Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities [J].

Buyya, Rajkumar ;

Yeo, Chee Shin ;

Venugopal, Srikumar .

HPCC 2008: 10TH IEEE INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS, PROCEEDINGS, 2008, :5-13

[5] An energy case for hybrid datacenters [J].

Chun B.-G. ;

Iannaccone G. ;

Katz R. ;

Lee G. ;

Niccolini L. .

Operating Systems Review (ACM), 2010, 44 (01) :76-80

[6]

Fox A, 1997, P 16 ACM S OP SYST P, V31, P78, DOI DOI 10.1145/269005266662

[7]

Ghemawat S, 2003, ACM SIGOPS Operating Systems Review, P29, DOI [10.1145/1165389.945450, 10.1145/945445.945450]

[8]

Hoste K, 2007, PERF E R SI, V35, P375

[9]

Ibrahim S, 2009, LECT NOTES COMPUT SC, V5931, P519, DOI 10.1007/978-3-642-10665-1_47

[10]

Issa J., 2010, P SPECTS C JUL, P127

← 1 2 3 →