EVALUATING THE SCALABILITY OF BIG DATA FRAMEWORKS

被引：3

作者：

Sanchez, David ^{[2
]}

Solarte, Oswaldo ^{[2
]}

Bucheli, Victor ^{[2
]}

Ordonez, Hugo ^{[1
]}

机构：

[1] Univ San Buenaventura, Cali, Colombia

[2] Univ Valle, Cali, Colombia

来源：

SCALABLE COMPUTING-PRACTICE AND EXPERIENCE | 2018年 / 19卷 / 03期

关键词：

Scalability; Isoefficiency; Big Data; Hadoop; Spark; MapReduce;

D O I：

10.12694/scpe.v19i3.1402

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

The aim of this paper is to present a method based on the isoefficiency model for assessing the scalability in big data environments. The programs word count and sort were implemented and compared in Hadoop and Spark. The results confirm that isoefficiency presented a linear growth as the size of the data sets was increased. They were checked experimentally to ensure that the evaluated frameworks are scalable and a sublinear function was obtained. This paper discusses how scalability in big data is governed by a constant of scalability (beta).

引用

页码：301 / 307

页数：7

共 30 条

[1]

Bondi A. B., 2000, Proceedings Second International Workshop on Software and Performance. WOSP2000, P195, DOI 10.1145/350391.350432

[2] Big Data: A Survey [J].

Chen, Min ;

Mao, Shiwen ;

Liu, Yunhao .

MOBILE NETWORKS & APPLICATIONS, 2014, 19 (02) :171-209

[3]

COSTA C., 2017, IAENG INT J COMPUTER

[4]

Dean J, 2004, USENIX ASSOCIATION PROCEEDINGS OF THE SIXTH SYMPOSIUM ON OPERATING SYSTEMS DESIGN AND IMPLEMENTATION (OSDE '04), P137

[5] Isoefficiency Maps for Divisible Computations [J].

Drozdowski, Maciej ;

Wielebski, Lukasz .

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2010, 21 (06) :872-880

[6]

GANDOMI A., 2015, INT J INFORM MANAGEM

[7]

Grama A. Y., 1993, IEEE Parallel & Distributed Technology: Systems & Applications, V1, P12, DOI 10.1109/88.242438

[8]

Grama Ananth, 2003, INTRO PARALLEL COMPU

[9] Memory or Time: Performance Evaluation for Iterative Operation on Hadoop and Spark [J].

Gu, Lei ;

Li, Huan .

2013 IEEE 15TH INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS & 2013 IEEE INTERNATIONAL CONFERENCE ON EMBEDDED AND UBIQUITOUS COMPUTING (HPCC_EUC), 2013, :721-727

[10]

KUMAR V., 1988, PARALLEL DEPTH 1 SEA

← 1 2 3 →