Exploring Neural Architecture Search Space via Deep Deterministic Sampling

Recent developments in Neural Architecture Search (NAS) resort to training the supernet of a predefined search space with weight sharing to speed up architecture evaluation. These include random search schemes, as well as various schemes based on optimization or reinforcement learning, in particular policy gradient, that aim to optimize a parametric architecture distribution and the shared model weights simultaneously. In this paper, we focus on efficiently exploring the important region of a neural architecture search space with reinforcement learning. We propose Deep Deterministic Architecture Sampling (DDAS) based on deep deterministic policy gradient and the actor-critic framework, to selectively sample important architectures in the supernet for training. Through balancing exploitation and exploration, DDAS is designed to combat the disadvantages of prior random supernet warm-up schemes and optimization schemes. Gradient-based NAS approaches require the execution of multiple short experiments in order to combat the random stochastic nature of gradient descent, while still only producing a single architecture. Contrary to this approach, DDAS employs a reinforcement learning-based agent and focuses on discovering a Pareto frontier containing many architectures over the course of a single experiment requiring 1 GPU day. Experimental results for CIFAR-10 and CIFAR-100 on the DARTS search space show that DDAS can depict in a single search, the accuracy-FLOPs (or model size) Pareto frontier, which outperforms random sampling and search. With a test accuracy of 97.27%, the best architecture found on CIFAR-10 outperforms the original second-order DARTS while using 600M fewer parameters. Additionally, DDAS finds an architecture capable of achieving 82.00% test accuracy on CIFAR-100 while using only 3.14M parameters and outperforming GDAS.