Bandit-NAS: Bandit sampling and training method for Neural Architecture Search

Existing Neural Architecture Search algorithms achieve a low error rate in vision tasks, such as image classification, by training child networks with equal resources during the search. However, it is unnecessary to allocate equal resources or fully converge scores to assess which child architectures should be adopted, resulting in computational redundancy. In this study, we present Bandit-NAS, an approach that automatically computes data slicing and training time for each child network. Firstly, we formulate the search for the optimal training time for a given resource as an M -armed bandit problem. Secondly, we extend the original NAS methods by proposing an end -to -end bandit algorithm, combined with reinforcement learning -based NAS algorithms, to determine an update strategy. Bandit-NAS enables simultaneous training of M child networks within a specified resource constraint (one epoch training time), with the allocation of training data based on the current accuracy of the child networks, thereby minimizing their error rate. Experimental results on 3 different datasets, MNIST , CIFAR-10 and CIFAR-100 demonstrate the superiority of Bandit-NAS over baseline NAS algorithms, such as ENAS and DQNAS, achieving lower error rates and faster search time.