Deadline scheduling algorithm for sustainable computing in Hadoop environment

被引：13

作者：

Varga, Mihai ^{[1
]}

Petrescu-Nita, Alina ^{[2
]}

Pop, Florin ^{[1
,3
]}

机构：

[1] Univ Politehn Bucuresti, Fac Automat Control & Comp, Comp Sci Dept, Bucharest, Romania

[2] Univ Politehn Bucuresti, Dept Math Methods & Models, Bucharest, Romania

[3] Natl Inst Res & Dev Informat ICI, Bucharest, Romania

来源：

COMPUTERS & SECURITY | 2018年 / 76卷

基金：

欧盟地平线“2020”;

关键词：

Scheduling; Distributed systems; Deadline scheduling; Hadoop; Energy efficiency; Sustainable computing; SERVICE;

D O I：

10.1016/j.cose.2017.12.014

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Cloud computing is popular choice for processing and analyzing large amounts of data. Organizations can easily manage and deploy powerful clusters that run different software environments and enable distributed processing. Scheduling is an important part of distributed computing that allows users to leverage the available resources for a faster computation time. In this paper we propose a generic scheduling algorithm that takes deadline constraints into consideration. We develop a cost model that estimates the remaining work load which allows the scheduler to properly prioritize jobs according to their upcoming deadlines. The cost model works with generic abstract resources requests such as virtual cores, memory and containers and determines the remaining running time based on the completed tasks. We validate the cost model and measure the performance of the scheduler by running several experiments on a cluster on Amazon EC2 and our algorithm performs as expected under different scenarios. (C) 2018 Elsevier Ltd. All rights reserved.

引用

页码：354 / 366

页数：13

共 31 条

[1]

[Anonymous], P 4 ANN S CLOUD COMP, DOI [10.1145/2523616.2523633, DOI 10.1145/2523616.2523633]

[2] Energy-aware resource allocation heuristics for efficient management of data centers for Cloud computing [J].

Beloglazov, Anton ;

Abawajy, Jemal ;

Buyya, Rajkumar .

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2012, 28 (05) :755-768

[3] Analyzing Hadoop power consumption and impact on application QoS [J].

Conejero, Javier ;

Rana, Omer ;

Burnap, Peter ;

Morgan, Jeffrey ;

Caminero, Blanca ;

Carrion, Carmen .

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2016, 55 :213-223

[4] The Mahalanobis distance [J].

De Maesschalck, R ;

Jouan-Rimbaud, D ;

Massart, DL .

CHEMOMETRICS AND INTELLIGENT LABORATORY SYSTEMS, 2000, 50 (01) :1-18

[5] A cost model for managing information security hazards [J].

Ekenberg, L ;

Oberoi, S ;

Orci, I .

COMPUTERS & SECURITY, 1995, 14 (08) :707-717

[6]

Gomes CarlaP., 2009, The Bridge, V39, P5

[7]

Hassan MA, 2015, STUD INFORM CONTROL, V24, P387

[8]

Iordache GV, 2006, LECT NOTES COMPUT SC, V4276, P1234

[9] Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing [J].

Iosup, Alexandru ;

Ostermann, Simon ;

Yigitbasi, M. Nezih ;

Prodan, Radu ;

Fahringer, Thomas ;

Epema, Dick H. J. .

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2011, 22 (06) :931-945

[10] Makespan minimization for MapReduce systems with different servers [J].

Jiang, Yiwei ;

Zhu, Yuqing ;

Wu, Weili ;

Li, Deying .

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2017, 67 :13-21

← 1 2 3 4 →