Improving Load Balance for Data-Intensive Computing on Cloud Platforms

被引：9

作者：

Dai, Wei ^{[1
]}

Ibrahim, Ibrahim ^{[1
]}

Bassiouni, Mostafa ^{[2
]}

机构：

[1] Univ Cent Florida, Dept Elect & Comp Engn, Orlando, FL 32816 USA

[2] Univ Cent Florida, Dept Comp Sci, Orlando, FL 32816 USA

来源：

2016 IEEE INTERNATIONAL CONFERENCE ON SMART CLOUD (SMARTCLOUD) | 2016年

关键词：

Cloud Computing; Data-Intensive Computing; Load Balance; Replica Placement; Hadoop Distributed File System; Hadoop; MapReduce;

D O I：

10.1109/SmartCloud.2016.44

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

Nowadays, big data problems are ubiquitous, which in turn creates huge demand for data-intensive computing. The advent of Cloud Computing has made data-intensive computing much more accessible and affordable than ever before. One of the crucial issues that can significantly affect the performance of data-intensive applications is the load balance among cluster nodes. In this paper, we address the load balance problem in the context of Hadoop Distributed File System (HDFS), a widely used file system for data-intensive computing on Cloud platforms, and present an innovative replica placement policy for HDFS, which can perfectly balance the computing load among all cluster nodes in both homogeneous and heterogeneous cluster environments.

引用

页码：140 / 145

页数：6

共 10 条

[1] A New Replica Placement Policy for Hadoop Distributed File System [J].

Dai, Wei ;

Ibrahim, Ibrahim ;

Bassiouni, Mostafa .

2016 IEEE 2ND INTERNATIONAL CONFERENCE ON BIG DATA SECURITY ON CLOUD (BIGDATASECURITY), IEEE INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE AND SMART COMPUTING (HPSC), AND IEEE INTERNATIONAL CONFERENCE ON INTELLIGENT DATA AND SECURITY (IDS), 2016, :262-267

[2] An improved task assignment scheme for Hadoop running in the clouds [J].

Dai, Wei ;

Bassiouni, Mostafa .

JOURNAL OF CLOUD COMPUTING-ADVANCES SYSTEMS AND APPLICATIONS, 2013, 2 (02) :1-16

[3]

Dean J, 2004, USENIX ASSOCIATION PROCEEDINGS OF THE SIXTH SYMPOSIUM ON OPERATING SYSTEMS DESIGN AND IMPLEMENTATION (OSDE '04), P137

[4] CoHadoop: Flexible Data Placement and Its Exploitation in Hadoop [J].

Eltabakh, Mohamed Y. ;

Tian, Yuanyuan ;

Ozcan, Fatma ;

Gemulla, Rainer ;

Krettek, Aljoscha ;

McPherson, John .

PROCEEDINGS OF THE VLDB ENDOWMENT, 2011, 4 (09) :575-585

[5] Cost-Aware Multimedia Data Allocation for Heterogeneous Memory Using Genetic Algorithm in Cloud Computing [J].

Gai, Keke ;

Qiu, Longfei ;

Zhao, Hui ;

Qiu, Meikang .

IEEE TRANSACTIONS ON CLOUD COMPUTING, 2020, 8 (04) :1212-1222

[6] Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing [J].

Gai, Keke ;

Qiu, Meikang ;

Zhao, Hui ;

Tao, Lixin ;

Zong, Ziliang .

JOURNAL OF NETWORK AND COMPUTER APPLICATIONS, 2016, 59 :46-54

[7]

Li Y., 2015, IEEE TRANSACTIONS ON

[8] Phase-Change Memory Optimization for Green Cloud with Genetic Algorithm [J].

Qiu, Meikang ;

Ming, Zhong ;

Li, Jiayin ;

Gai, Keke ;

Zong, Ziliang .

IEEE TRANSACTIONS ON COMPUTERS, 2015, 64 (12) :3528-3540

[9]

Shabeera TP, 2013, 2013 IEEE RECENT ADVANCES IN INTELLIGENT COMPUTATIONAL SYSTEMS (RAICS), P64, DOI 10.1109/RAICS.2013.6745448

[10] Aurora: Adaptive Block Replication in Distributed File Systems [J].

Zhang, Qi ;

Zhang, Sai Qian ;

Leon-Garcia, Alberto ;

Boutaba, Raouf .

2015 IEEE 35TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS, 2015, :442-451

← 1 →