Array-Aware Neural Architecture Search

Convolutional Neural Networks (CNNs) have exceeded human accuracy in many Computer Vision tasks, such as Image Classification, Object Detection, Image Segmentation, etc. This advancement is due to the efficient manual design of CNNs in initially, followed by automated design through Neural Architecture Search (NAS). In parallel to neural network design, advances in Accelerator hardware design, such as Google's Tensor Processing Unit (TPU), Eyeriss, etc., also occurred for efficient processing of CNN forward propagation. The heart of these accelerators is an array processor (Systolic Array) of a fixed dimension, that limits the amount of CNN computation that can be carried out in a single clock cycle. While NAS is able to produce efficient neural architectures, the networks need to be co-designed with respect to the underlying array dimensions to obtain the best performance. In this paper, we introduce "Array Aware Neural Architecture Search" to automatically design efficient CNNs for a fixed array-based neural network accelerator. Previous Hardware Aware NAS methods consider a fixed search space for different hardware platforms and search within its predefined space. We explore the search space based on the underlying hardware array dimensions to design a more efficient CNN architectures for optimal performance. We observe that our proposed NAS methods on the CIFAR-10 dataset produce similar accuracy as the baseline network while saving a substantial number of cycles on the Array.