APENAS: An Asynchronous Parallel Evolution Based Multi-objective Neural Architecture Search

Machine learning is widely used in pattern classification, image processing and speech recognition. Neural architecture search (NAS) could reduce the dependence of human experts on machine learning effectively. Due to the high complexity of NAS, the tradeoff between time consumption and classification accuracy is vital. This paper presents APENAS, an asynchronous parallel evolution based multi-objective neural architecture search, using the classification accuracy and the number of parameters as objectives, encoding the network architectures as individuals. To make full use of computing resource, we propose a multi-generation undifferentiated fusion scheme to achieve asynchronous parallel evolution on multiple GPUs or CPUs, which speeds up the process of NAS. Accordingly, we propose an election pool and a buffer pool for two-layer filtration of individuals. The individuals are sorted in the election pool by non-dominated sorting and filtered in the buffer pool by the roulette algorithm to improve the elitism of the Pareto front. APENAS is evaluated on the CIFAR-10 and CIFAR-100 datasets [25]. The experimental results demonstrate that APENAS achieves 90.05% accuracy on CIFAR-10 with only 0.07 million parameters, which is comparable to state of the art. Especially, APENAS has high parallel scalability, achieving 92.5% parallel efficiency on 64 nodes.