Evolutionary Neural Architecture Search with Predictor of Ranking-Based Score

The exceptional performance of neural networks has been extensively validated across various practical applications, with architecture serving as the primary determinant of their efficacy. Presently, the state-of-the-art architectures necessitate manual design, heavily relying on expert experience and iterative trial-and-error methodologies for performance validation. In recent years, the emergence of Evolutionary Neural Architecture Search (ENAS) has alleviated the burden associated with manual design. However, despite the ability of ENAS methods to autonomously identify superior architectures, their widespread application remains impeded by the substantial time and computational resources required. Surrogate models can mitigate this excessive resource consumption to some extent. However, existing surrogate model-assisted evolutionary neural architecture searches fail to fully integrate the search and surrogate processes. Moreover, it is difficult for the current surrogate methods to accurately predict network architectural rankings with similar accuracies. Furthermore, existing surrogate models typically necessitate substantial amounts of architectural information as training data to attain satisfactory surrogate accuracy. These limitations hinder the effective assistance of surrogate models in ENAS. thereby constraining its advancement. In this paper, we propose a Rank Score Predictor-assisted Evolutionary Neural Architecture Search method (RSP-ENAS). By introducing a novel loss function specifically designed for rank score prediction, the Multi-Layer Perceptron (MLP) employed as a score predictor can optimally align the ranking of individual performance scores within the population with their actual performance order. During the search process utilizing this method, the predicted scores are directly applicable for elite selection. We introduce a two-stage search strategy in the search phase, initially focusing on accumulating historical information for the surrogate dataset from evaluating a small population and subsequently emphasizing the use of the surrogate model to predict fitness values for a larger population in the later stages. The experiments conducted in this study were performed on the EvoXBench platform, yielding superior results across all benchmark datasets. Additionally, we validated our approach on the ImageNet dataset. Compared to alternative methodologies, our approach successfully identifies the optimal architecture within the NASBench-101 space. On the three datasets within the NASBench-201 space, accuracy improvements of 0. 35%. 1. 12%, and 0. 55% were achieved relative to other optimal methods. In experiments utilizing real datasets on ImageNet, our method demonstrated a 2. 2% enhancement in classification accuracy. Moreover, with the same quantity of data, the ranking results generated by the proposed rank score prediction model exhibited a 1. 55% improvement in Kendall's Tau coefficient when compared to other optimal approaches. We further validated the effectiveness of One-hot encoding and the proposed rank loss within the surrogate model, demonstrating the efficacy of these two components for the overall algorithm. This research underscores the potential of advanced surrogate models to enhance the efficiency and accuracy of neural architecture search processes. By reducing computational costs and improving the precision of architecture rankings, our RSP-ENAS method could significantly advance the practical application and accessibility of neural network design, potentially catalyzing more rapid advancements in the fields of machine learning and artificial intelligence. Future work may explore the surrogate models for less training data, which could yield even more substantial enhancements in neural architecture search methodologies. © 2024 Science Press. All rights reserved.