Research on job scheduling algorithm in Hadoop

被引：0

作者：

Xia, Yang ^{[1
]}

Wang, Lei ^{[1
]}

Zhao, Qiang ^{[1
]}

Zhang, Gongxuan ^{[2
]}

机构：

[1] School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221116, China

[2] School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing 210094, China

来源：

Journal of Computational Information Systems | 2011年 / 7卷 / 16期

关键词：

Scheduling - Scheduling algorithms - Clustering algorithms - Health;

D O I：

暂无

中图分类号：

学科分类号：

摘要：

On the basis of researching Fair Scheduling Strategy deeply in Hadoop cluster, the Node Health Degree is defined by constructing the relationship function between node load and job fail rate, and a job scheduling algorithm based on Node Health Degree is proposed in this paper. Nodes are grouped, according to Node Health Degree, into three categories in order to assign corresponding job in accordance with load and guarantee resource load balance. By comparing with FIFO and Fair scheduling algorithm, the simulation results show that this algorithm can ensure to reduce job fail rate and improve cluster throughput. © 2011 by Binary Information Press.

引用

页码：5769 / 5775