Multi-Objective Neural Architecture Search for Efficient and Fast Semantic Segmentation on Edge

Deploying efficient and fast semantic segmentation networks on edge computing platforms in real-world environments is desired and challenging. To address this challenge, we propose RealtimeSeg, one of the first semantic segmentation models to be searched by neural architecture search(NAS), capable of running at real-time speed on edge devices. In our neural architecture search, we incorporate the inference time and FLOPs (floating-point operations) of the target edge devices and the semantic segmentation accuracy as objectives. In this way, we construct a multi-objective neural architecture search. Specifically, the multi-objective NAS's loss function is decomposed into three sub-objective loss functions, which are weighted and summed. We employed knowledge distillation to further enhance the accuracy, latency, and FLOPs of the discovered network architecture during the search process. As a result, we successfully obtained our RealtimeSeg model. Lastly, we utilized NVIDIA TensorRT to accelerate RealtimeSeg and deployed the accelerated RealtimeSeg on the target platform for real-time semantic segmentation. Using a single NVIDIA Titan XP GPU, RealtimeSeg can be obtained within 1.5 days. The experimental results demonstrate that RealtimeSeg achieved an accuracy of 71.7 mIoU(%) while maintaining a frame rate of 25.25 FPS on the NVIDIA Jetson NX, using the input resolution of 1024 x 2048. And the RealtimeSeg has a lower FLOPs value of 1.52 G, which is 17-18x less than SOTA methods. In realistic scenarios, RealtimeSeg has been successfully deployed on edge computing platforms, achieving efficient and fast semantic segmentation results.