A Hybrid Evolutionary Algorithm, Utilizing Novelty Search and Local Optimization, Used to Design Convolutional Neural Networks for Handwritten Digit Recognition

Convolutional neural networks (CNNs) are deep learning models that have been successfully applied to various computer vision tasks. The design of CNN topologies often requires extensive domain knowledge and a high degree of trial and error. In recent years, numerous Evolutionary Algorithms (EAs) have been proposed to automate the design of CNNs. The search space of these EAs is very large and often deceptive, which entails great computational cost. In this work, we investigate the design of CNNs using Cartesian Genetic Programming (CGP), an EA variant. We then augment the basic CGP with methods for identifying potential/actual local optima within the solution space (via Novelty Search), followed by further local optimization of each of the optima (via Simulated Annealing). This hybrid EA methodology is evaluated using the MNIST data-set for handwritten digit recognition. We demonstrate that the use of the proposed method results in considerable reduction of computational effort, when compared to the basic CGP approach, while still returning competitive results. Also, the CNNs designed by our method achieve competitive recognition results compared to other neuroevolutionary methods.