Simplified swarm optimization for hyperparameters of convolutional neural networks

Convolutional neural networks (CNNs) are widely used in image recognition. Numerous CNN models, such as LeNet, AlexNet, VGG, ResNet, and GoogLeNet, have been developed by increasing the number of layers to improve performance. However, performance deteriorates beyond a certain number of layers. Hence, hyper -parameter optimization is a more efficient way to improve CNNs. To validate this concept, in the present study, an algorithm based on simplified swarm optimization was developed for optimizing the hyperparameters of the simplest CNN model: LeNet. The results of experiments involving the MNIST, Fashion-MNIST, and CIFAR-10 datasets indicated that the accuracy of the proposed algorithm was higher than those of LeNet and PSO-LeNet and that the proposed algorithm can be applied to more complex models such as AlexNet.