Analysis and Performance Evaluation of Deep Learning on Big Data

被引：6

作者：

Matteussi, Kassiano J. ^{[1
]}

Zanchetta, Breno F. ^{[1
]}

Bertoncello, Germano ^{[1
]}

Dos Santos, Jobe D. D. ^{[1
]}

Dos Anjos, Julio C. S. ^{[1
]}

Geyer, Claudio F. R. ^{[1
]}

机构：

[1] Univ Fed Rio Grande do Sul, Inst Informat, BR-91509900 Porto Alegre, RS, Brazil

来源：

2019 IEEE SYMPOSIUM ON COMPUTERS AND COMMUNICATIONS (ISCC) | 2019年

关键词：

Deep Learning; Distributed Deep Learning; Big Data; BigDL; Apache Spark; Parallel Processing;

D O I：

10.1109/iscc47284.2019.8969762

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Deep Learning (DL) and Big Data (BD) have converged to a hybrid computing paradigm that merges the dynamic processing in DL models with the computational power of the distributed processing of the BD frameworks. In this context, this work aims to conduct an analysis and performance evaluation of DL applications in BD. The experiments evaluate how the application training completion time can be related to the model's precision loss and the impacts of distributed computing in DL models. The experiments were performed in Microsoft Azure using BigDL framework, which allows using both Spark and TensorFlow on top of a Yarn cluster. The outcomes revealed a speedup of up to 8x and accuracy higher than 95%.

引用

页码：682 / 687

页数：6

共 28 条

[1]

Ahn H, 2018, I C INF COMM TECH CO, P1261, DOI 10.1109/ICTC.2018.8539474

[2] Enabling Strategies for Big Data Analytics in Hybrid Infrastructures [J].

Anjos, Julio C. S. ;

Matteussi, Kassiano J. ;

De Souza, Paulo R. R. ;

Geyer, Claudio F. R. ;

Veith, Alexandre S. ;

Fedak, Gilles ;

Victoria Barbosa, Jorge Luis .

PROCEEDINGS 2018 INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING & SIMULATION (HPCS), 2018, :869-876

[3]

Bahdanau D, 2016, Arxiv, DOI arXiv:1409.0473

[4]

Dai J., 2017, BUILDING LARGE SCALE

[5]

De Falco I, 2018, IEEE SYMP COMP COMMU, P1187, DOI 10.1109/ISCC.2018.8538572

[6] Aten: A Dispatcher for Big Data Applications in Heterogeneous Systems [J].

de Souza Junior, Paulo R. R. ;

Matteussi, Kassiano J. ;

dos Anjos, Julio C. S. ;

dos Santos, Jobe D. D. ;

Resin Geyer, Claudio Fernando ;

Veith, Alexandre da Silva .

PROCEEDINGS 2018 INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING & SIMULATION (HPCS), 2018, :585-592

[7]

Feng A., CAFFEONSPARK OPEN SO

[8]

Feng A., OPEN SOURCING TENSOR

[9] Vision-based Obstacle Avoidance Using Deep Learning [J].

Gaya, Joel O. ;

Goncalves, Lucas T. ;

Duarte, Amanda C. ;

Zanchetta, Breno ;

Drews-, Paulo, Jr. ;

Botelho, Silvia S. C. .

PROCEEDINGS OF 13TH LATIN AMERICAN ROBOTICS SYMPOSIUM AND 4TH BRAZILIAN SYMPOSIUM ON ROBOTICS - LARS/SBR 2016, 2016, :7-12

[10]

Goodfellow I, 2016, ADAPT COMPUT MACH LE, P1

← 1 2 3 →