Microarchitecture Aware Neural Architecture Search for TinyML Devices

Designing accurate and efficient models for tiny machine learning (TinyML) devices is challenging because of its strict power consumption limitations and accuracy requirements. This paper proposed a Microarchitecture Aware Neural Architecture Search (M-NAS) method to consider the detailed microarchitecture into NAS and improve the energy efficiency of TinyML devices. We first build a microarchitecture aware candidate operators pool with the limitation of microarchitecture to balance the inference efficiency and accuracy. Moreover, we generate a microarchitecture aware search space by integrating the microarchitecture-level power and latency into NAS with the help of back-end layout simulation based on the microarchitecture of NPU. These strategies assist our M-NAS to find the optimal model for the microarchitecture of the target NPU. Extensive experiments demonstrate that with the help of the M-NAS, the target NPU yields a considerable 67.1% Top-1 accuracy on ImageNet and achieves a record ultra-high energy efficiency of 3809.52 Frames/J.