Model Selection in Committees of Evolved Convolutional Neural Networks Using Genetic Algorithms

被引:6
作者
Baldominos, Alejandro [1 ]
Saez, Yago [1 ]
Isasi, Pedro [1 ]
机构
[1] Univ Carlos III Madrid, Dept Comp Sci, Ave Univ 30, Leganes 28911, Spain
来源
INTELLIGENT DATA ENGINEERING AND AUTOMATED LEARNING - IDEAL 2018, PT I | 2018年 / 11314卷
关键词
Genetic algorithms; Neuroevolution; Convolutional neural network; Committees; Ensembles;
D O I
10.1007/978-3-030-03493-1_39
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Neuroevolution is a technique that has been successfully applied for over three decades in order to optimize certain aspects of neural networks by applying evolutionary algorithms. However, only in the recent years, the increase of computational resources has enabled to apply such techniques to deep and convolutional neural networks, where the number of hyperparameters is significantly large. In recent years, deep and convolutional neural networks are outperforming classical machine learning for many different tasks, including computer vision, natural language processing, signal processing, activity recognition, etc. In this context, neuroevolution can be useful since there are no analytic approaches for determining optimal network architectures or hyperparameters, therefore attaining better performance. Moreover, in some cases, committees (also called ensembles) are used, which combine two or more models in order to improve results even more. Neuroevolution can be of particular importance in this case, since evolutionary algorithms evolve a whole population of individuals, making it easy to build ensembles out of models chosen among those in the evolved population. In this paper, we explore the application of genetic algorithms for carrying out model selection in a context of neuroevolution. Thus, the best models will be selected from a population of evolved individuals in order to maximize an overall objective function. This approach is tested using the well-known MNIST database as benchmark, and it obtains results which are highly competitive when compared with the state of the art.
引用
收藏
页码:364 / 373
页数:10
相关论文
共 29 条
[1]   A Large-Scale CNN Ensemble for Medication Safety Analysis [J].
Akhtyamova, Liliya ;
Ignatov, Andrey ;
Cardiff, John .
NATURAL LANGUAGE PROCESSING AND INFORMATION SYSTEMS, NLDB 2017, 2017, 10260 :247-253
[2]  
[Anonymous], 2015, Batch-normalized maxout network in network
[3]  
[Anonymous], 2017, DEVOL AUTOMATED DEEP
[4]  
[Anonymous], 2012, NEURAL NETWORKS MA E
[5]  
Baker C, 2017, TLS-TIMES LIT SUPPL, P5
[6]   Evolutionary Design of Convolutional Neural Networks for Human Activity Recognition in Sensor-Rich Environments [J].
Baldominos, Alejandro ;
Saez, Yago ;
Isasi, Pedro .
SENSORS, 2018, 18 (04)
[7]   Evolutionary convolutional neural networks: An application to handwriting recognition [J].
Baldominos, Alejandro ;
Saez, Yago ;
Isasi, Pedro .
NEUROCOMPUTING, 2018, 283 :38-52
[8]   Random forests [J].
Breiman, L .
MACHINE LEARNING, 2001, 45 (01) :5-32
[9]   Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing [J].
Desell, Travis .
PROCEEDINGS OF THE 2017 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE COMPANION (GECCO'17 COMPANION), 2017, :127-128
[10]   Convolution by Evolution Differentiable Pattern Producing Networks [J].
Fernando, Chrisantha ;
Banarse, Dylan ;
Reynolds, Malcolm ;
Besse, Frederic ;
Pfau, David ;
Jaderberg, Max ;
Lanctot, Marc ;
Wierstra, Daan .
GECCO'16: PROCEEDINGS OF THE 2016 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE, 2016, :109-116