Efficient & Accurate Scheduling Algorithm For Cloudera Hadoop

被引:1
作者
Yadav, Swati [1 ]
Vishwakarma, Santosh [1 ]
Verma, Ashok [1 ]
机构
[1] Gyan Ganga Inst Sci & Technol, Dept Comp Sci, Jabalpur, India
来源
2015 INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND COMMUNICATION NETWORKS (CICN) | 2015年
关键词
MapReduce; huge Data; Hadoop; Cloud Computing; Improved Scheduling;
D O I
10.1109/CICN.2015.322
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
the term immense data was coined to capture which suggests of this rising trend. To boot to its sheer volume, immense data to boot exhibits completely different distinctive characteristics as compared with ancient data. For instance, immense data is typically unstructured and wish extra amount analysis. This development incorporates new system architectures for data acquisition, transmission, storage, and large-scale process mechanisms. Recent technological advancements have semiconductor diode to a deluge of information from distinctive domains (e.g., health care and sciatic sensors, user generated data, net and money corporations, and supply chain systems). the buildup of information over the past twenty years has enlarged to large volumes. Apache Hadoop have introduced a economical and possible tool for distributed computing of such immense data for filtering and extracting massive volumes of knowledge. MapReduce can be a good used parallel computing framework for giant scale process. The two major performance metrics in MapReduce area unit job execution time and cluster production. MapReduce uses inventory accounting job programming by default and completely different programming algorithms area unit being introduced in proprietary domain. This work introduces a metric primarily based programming algorithmic rule to reinforce the potency and utilization of the server resources.
引用
收藏
页码:839 / 844
页数:6
相关论文
共 50 条
[11]   Towards Thermal-Efficient Hadoop Clusters through Scheduling [J].
Zhou, Yi ;
Alghamdi, Mohammed ;
Dudeja, Gautam ;
Taneja, Shubbhi ;
Qin, Xiao .
2017 IEEE INTERNATIONAL CONFERENCE ON INTERNET OF THINGS (ITHINGS) AND IEEE GREEN COMPUTING AND COMMUNICATIONS (GREENCOM) AND IEEE CYBER, PHYSICAL AND SOCIAL COMPUTING (CPSCOM) AND IEEE SMART DATA (SMARTDATA), 2017, :530-537
[12]   MUS: a novel deadline-constrained scheduling algorithm for Hadoop [J].
Teng, Fei ;
Yang, Hao ;
Li, Tianrui ;
Magoules, Frederic ;
Fan, Xiaoliang .
INTERNATIONAL JOURNAL OF COMPUTATIONAL SCIENCE AND ENGINEERING, 2015, 11 (04) :360-367
[13]   An Adaptive Scheduling Algorithm for Heterogeneous Hadoop Systems [J].
Han, Jiazhen ;
Yuan, Zhengheng ;
Han, Yiheng ;
Peng, Cheng ;
Liu, Jing ;
Li, Guangli .
2017 16TH IEEE/ACIS INTERNATIONAL CONFERENCE ON COMPUTER AND INFORMATION SCIENCE (ICIS 2017), 2017, :845-850
[14]   Improving Fair Scheduling Performance on Hadoop [J].
Cheng, Ya-Wen ;
Lo, Shou-Chih .
2017 INTERNATIONAL CONFERENCE ON PLATFORM TECHNOLOGY AND SERVICE (PLATCON), 2017, :1-6
[15]   MOMC: Multi-Objective and Multi-Constrained Scheduling Algorithm of Many Tasks in Hadoop [J].
Voicu, Cristiana ;
Pop, Florin ;
Dobre, Ciprian ;
Xhafa, Fatos .
2014 NINTH INTERNATIONAL CONFERENCE ON P2P, PARALLEL, GRID, CLOUD AND INTERNET COMPUTING (3PGCIC), 2014, :89-96
[16]   A Scheduling Algorithm for Hadoop MapReduce Workflows with Budget Constraints in the Heterogeneous Cloud [J].
Wylie, Andrew ;
Shi, Wei ;
Corriveau, Jean-Pierre ;
Wang, Yang .
2016 IEEE 30TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM WORKSHOPS (IPDPSW), 2016, :1433-1442
[17]   An Efficient Implementation of the Algorithm by Lukas et al. on Hadoop [J].
Cattaneo, Giuseppe ;
Petrillo, Umberto Ferraro ;
Nappi, Michele ;
Narducci, Fabio ;
Roscigno, Gianluca .
GREEN, PERVASIVE, AND CLOUD COMPUTING (GPC 2017), 2017, 10232 :475-489
[18]   An Efficient Improved Join Algorithm Using Map Reduce in Hadoop [J].
Patel, Warish D. ;
Vaghela, Dineshkumar B. .
2014 INTERNATIONAL CONFERENCE ON SIGNAL PROPAGATION AND COMPUTER TECHNOLOGY (ICSPCT 2014), 2014, :263-272
[19]   HybSMRP: a hybrid scheduling algorithm in Hadoop MapReduce framework [J].
Gandomi, Abolfazl ;
Reshadi, Midia ;
Movaghar, Ali ;
Khademzadeh, Ahmad .
JOURNAL OF BIG DATA, 2019, 6 (01)
[20]   HybSMRP: a hybrid scheduling algorithm in Hadoop MapReduce framework [J].
Abolfazl Gandomi ;
Midia Reshadi ;
Ali Movaghar ;
Ahmad Khademzadeh .
Journal of Big Data, 6