An Approach to Enhance the Performance of Hadoop MapReduce Framework for Big Data

被引:2
作者
Chandra, Subhash [1 ]
Motwani, Deepak [1 ]
机构
[1] ITM Univ, Dept Comp Sci, Gwalior, India
来源
2016 INTERNATIONAL CONFERENCE ON MICRO-ELECTRONICS AND TELECOMMUNICATION ENGINEERING (ICMETE) | 2016年
关键词
K-mean clustering; MapReduce; Hadoop; HDFS;
D O I
10.1109/ICMETE.2016.64
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Data analysis is becoming one of the highest research topic among researchers. Information is the baseline of every small and big organization. Everyone wants relevant information for their business to grow faster and bigger. Every organization wants to know what their customers like and dislike. This desirable information requires analysis of very large information stored in various places in different format. Hadoop MapReduce framework becoming a popular platform for processing so large amount of data in very efficient manner. It is used by organizations to process their customers information data sets. Hadoop process datasets in distributed parallel processes by using its HDFS and MapReduce model. Hadoop optimization is requiring more attention from researchers and programmers. Many approaches is already developed to make Hadoop framework optimized. These approaches includes performances tuning and efficient clustering formation. In this research work we have developed Optimal Approach to Improve the Performance of Hadoop framework. K-Means and KMedoids are well known clustering approaches for clustering inside Hadoop. In proposed approach a modified K-Medoids clustering algorithm has been developed which gives better result for processing inside Hadoop. The research work is tested inside multi node Hadoop environment.
引用
收藏
页码:178 / 182
页数:5
相关论文
共 17 条
[1]  
[Anonymous], WORKSH SMART PLAN BI
[2]  
Bide Pramod, 2015, IEEE INT C EL COMP C
[3]   MapReduce: A Flexible Data Processing Tool [J].
Dean, Jeffrey ;
Ghemawat, Sanjay .
COMMUNICATIONS OF THE ACM, 2010, 53 (01) :72-77
[4]  
Gandhi Gopi, INT J RES ENG TECHNO, V03
[5]  
Garg Dweepna, 2015, IEEE INT C EL COMP 1
[6]  
Gohil P., 2015, P 2015 IEEE INT C EL, DOI [10.1109/ICECCT.2015.7226044, DOI 10.1109/ICECCT.2015.7226044]
[7]   SHadoop: Improving MapReduce performance by optimizing job execution mechanism in Hadoop clusters [J].
Gu, Rong ;
Yang, Xiaoliang ;
Yan, Jinshuang ;
Sun, Yuanhao ;
Wang, Bing ;
Yuan, Chunfeng ;
Huang, Yihua .
JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING, 2014, 74 (03) :2166-2179
[8]   Performance Evaluation and Estimation Model Using Regression Method for Hadoop WordCount [J].
Issa, Joseph A. .
IEEE ACCESS, 2015, 3 :2784-2793
[9]  
Jiang Yaobin, 2014, 5 IEEE INT C SOFTW E
[10]   MapReduce: Limitations, Optimizations and Open Issues [J].
Kalavri, Vasiliki ;
Vlassov, Vladimir .
2013 12TH IEEE INTERNATIONAL CONFERENCE ON TRUST, SECURITY AND PRIVACY IN COMPUTING AND COMMUNICATIONS (TRUSTCOM 2013), 2013, :1031-1038