DYNAMIC SCHEDULING ALGORITHM FOR REDUCING START TIME IN HADOOP

被引:5
作者
Gunasekaran, S. [1 ]
SaiRamesh, L. [2 ]
Sabena, S. [3 ]
Selvakumar, K. [4 ]
Ganapathy, S. [5 ]
Kannan, A. [2 ]
机构
[1] Anna Univ Chennai, Chennai, Tamil Nadu, India
[2] Anna Univ, Chennai, Tamil Nadu, India
[3] Anna Univ, Reg Ctr, Tirunelveli, India
[4] VIT Univ Vellore, Vellore, Tamil Nadu, India
[5] VIT Univ, Chennai, Tamil Nadu, India
来源
PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON INFORMATICS AND ANALYTICS (ICIA' 16) | 2016年
关键词
Big data; Mapreduce; Self Adaptive Scheduling; Dynamic Scheduling; Task scheduling; MAPREDUCE;
D O I
10.1145/2980258.2982115
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Map Reduce is a model associated with a programming and implementation method and is used for formulating on large datasets. The main challenge is scaling of start blocks and present implementations might end in a block of scale back tasks. In this work, In this work, a new start up model is proposed using temporal constraints and hence, the map task gives a massive output then the performance of Map Reduce reduces drastically. Through this analysis the map reduce planning mechanism is modified to reduce the waste resources in the system slot. This tends to an end within the scale back tasks waiting around the proposed model scale back the planning policy for reducing the waiting of scales back tasks and begin times within the Hadoop platform. It also decides the beginning time and purpose of every scale back task dynamically based on the context of each job, together with the task completion time and therefore the size of map as output. Thereafter, scale back completion time and system average latent period job completion time have been estimated. The experimental results illustrate that the scale back completion time has been decreased sharply due to the rise of the temporal rules and map reduce techniques.
引用
收藏
页数:4
相关论文
共 10 条
[1]   Mapreduce: Simplified data processing on large clusters [J].
Dean, Jeffrey ;
Ghemawat, Sanjay .
COMMUNICATIONS OF THE ACM, 2008, 51 (01) :107-113
[2]   SHadoop: Improving MapReduce performance by optimizing job execution mechanism in Hadoop clusters [J].
Gu, Rong ;
Yang, Xiaoliang ;
Yan, Jinshuang ;
Sun, Yuanhao ;
Wang, Bing ;
Yuan, Chunfeng ;
Huang, Yihua .
JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING, 2014, 74 (03) :2166-2179
[3]  
Gu T, 2014, 2014 5TH IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING AND SERVICE SCIENCE (ICSESS), P190, DOI 10.1109/ICSESS.2014.6933542
[4]  
Hammoud M., 2012, 2012 IEEE 5th International Conference on Cloud Computing (CLOUD), P49, DOI 10.1109/CLOUD.2012.92
[5]   MAP-JOIN-REDUCE: Toward Scalable and Efficient Data Analysis on Large Clusters [J].
Jiang, Dawei ;
Tung, Anthony K. H. ;
Chen, Gang .
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2011, 23 (09) :1299-1311
[6]   Workload Characteristic Oriented Scheduler for MapReduce [J].
Lu, Peng ;
Lee, Young Choon ;
Wang, Chen ;
Zhou, Bing Bing ;
Chen, Junliang ;
Zomaya, Albert Y. .
PROCEEDINGS OF THE 2012 IEEE 18TH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED SYSTEMS (ICPADS 2012), 2012, :156-163
[7]   Deadline-Based MapReduce Workload Management [J].
Polo, Jorda ;
Becerra, Yolanda ;
Carrera, David ;
Steinder, Malgorzata ;
Whalley, Ian ;
Torres, Jordi ;
Ayguade, Eduard .
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, 2013, 10 (02) :231-244
[8]   Hierarchical attribute reduction algorithms for big data using MapReduce [J].
Qian, Jin ;
Lv, Ping ;
Yue, Xiaodong ;
Liu, Caihui ;
Jing, Zhengjun .
KNOWLEDGE-BASED SYSTEMS, 2015, 73 :18-31
[9]  
Verma A., 2012, 2012 IEEE 20th International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), P11, DOI 10.1109/MASCOTS.2012.12
[10]  
Xiang Gao, 2012, 2012 Seventh ChinaGrid Annual Conference (ChinaGrid 2012), P17, DOI 10.1109/ChinaGrid.2012.27