A genetic algorithm-based job scheduling model for big data analytics

被引：10

作者：

Lu, Qinghua ^{[1
]}

Li, Shanshan ^{[1
]}

Zhang, Weishan ^{[1
]}

Zhang, Lei ^{[1
]}

机构：

[1] China Univ Petr, Coll Comp & Commun Engn, Qingdao, Peoples R China

来源：

EURASIP JOURNAL ON WIRELESS COMMUNICATIONS AND NETWORKING | 2016年

基金：

中国国家自然科学基金;

关键词：

Big data; Hadoop; MapReduce; Job scheduling; Genetic algorithm; OPTIMIZATION; FRAMEWORK; MAPREDUCE;

D O I：

10.1186/s13638-016-0651-z

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

Big data analytics (BDA) applications are a new category of software applications that process large amounts of data using scalable parallel processing infrastructure to obtain hidden value. Hadoop is the most mature open-source big data analytics framework, which implements the MapReduce programming model to process big data with MapReduce jobs. Big data analytics jobs are often continuous and not mutually separated. The existing work mainly focuses on executing jobs in sequence, which are often inefficient and consume high energy. In this paper, we propose a genetic algorithm-based job scheduling model for big data analytics applications to improve the efficiency of big data analytics. To implement the job scheduling model, we leverage an estimation module to predict the performance of clusters when executing analytics jobs. We have evaluated the proposed job scheduling model in terms of feasibility and accuracy.

引用

页数：9

共 50 条

[21] Genetic algorithm-based optimization of routing and scheduling for logistics
Hu, XD
Wei, QF
CONCURRENT ENGINEERING: THE WORLDWIDE ENGINEERING GRID, PROCEEDINGS, 2004, : 959 - 962
[22] Genetic Algorithm based Data-aware Group Scheduling for Big Data Clouds
Kune, Raghavendra
Konugurthi, Pramod Kumar
Agarwal, Arun
Chillarige, Raghavendra Rao
Buyya, Rajkumar
2014 IEEE/ACM INTERNATIONAL SYMPOSIUM ON BIG DATA COMPUTING (BDC), 2014, : 96 - 104
[23] Genetic algorithm-based multi-objective model for scheduling of linear construction projects
Senouci, Ahmed
Al-Derham, Hassan R.
ADVANCES IN ENGINEERING SOFTWARE, 2008, 39 (12) : 1023 - 1028
[24] A genetic algorithm-based task scheduling for cloud resource crowd-funding model
Zhang, Nan
Yang, Xiaolong
Zhang, Min
Sun, Yan
Long, Keping
INTERNATIONAL JOURNAL OF COMMUNICATION SYSTEMS, 2018, 31 (01)
[25] Flexible job shop scheduling model with parallel processes based on genetic algorithm
Bao, Bo
Zhang, Lin
Zhang, Bo
PROCEEDINGS OF THE 2017 2ND INTERNATIONAL CONFERENCE ON MATERIALS SCIENCE, MACHINERY AND ENERGY ENGINEERING (MSMEE 2017), 2017, 123 : 953 - 958
[26] Hybrid genetic algorithm-based workflow scheduling in cloud environment
1600, CESER Publications, Post Box No. 113, Roorkee, 247667, India (48):
[27] Genetic algorithm-based method for printer scheduling in ubiquitous computing
Wen, Yong-He
Yoon, Tae-Bok
Jung, Hye-Wuk
Jung, Young-Mo
Park, Doo-Kyeong
Lee, Jee-Hyong
PROCEEDINGS OF THE IASTED INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND APPLICATIONS, 2007, : 463 - +
[28] Genetic algorithm-based e-manufacturing scheduling system
Zhang, Ying-Feng
Jiang, Ping-Yu
Zhou, Guang-Hui
Jisuanji Jicheng Zhizao Xitong/Computer Integrated Manufacturing Systems, CIMS, 2004, 10 (08): : 955 - 961
[29] Genetic Algorithm-Based Batch Filling Scheduling in the Steel Industry
Kovacic, Miha
Sarler, Bozidar
MATERIALS AND MANUFACTURING PROCESSES, 2011, 26 (03) : 464 - 474
[30] A genetic algorithm-based method for scheduling repetitive construction projects
Long, Luong Duc
Ohsato, Ario
AUTOMATION IN CONSTRUCTION, 2009, 18 (04) : 499 - 511

← 1 2 3 4 5 →