Real-Time Semantic Segmentation of LiDAR Point Clouds on Edge Devices for Unmanned Systems

Real-time semantic segmentation of LiDAR measurements is crucial for high-level perception in unmanned systems, such as unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs). The limited computation and memory capacity of onboard devices, however, restricts most existing methods to offline analyses. To solve this problem, this article proposes an attention-based 3-D semantic segmentation approach, 3D-ARSS, which is able to classify measurements from a Velodyne HDL-64E sensor at the speed of 5 FPS on AGX Xavier. In our approach, we present two plug-and-play attention modules, a spatial attention module and a channel attention module. The former is to learn local-global context by reweighting features from different regions; the latter is to model the importance of different-scale features for semantic information fusion. To effectively process large-scale point clouds, a sparse-tensor (ST)-based implementation is introduced. Two kinds of sparse convolution operations are used to reduce unnecessary computation and memory costs on free spaces. Experimental results on the SemanticKITTI and NuScenes datasets demonstrate that our method outperforms state-of-the-art real-time methods by +3.1% and +2.2% mean intersection over union (mIoU), respectively. The floating-point operations of our method are reduced to 2/5 of SPVNAS and 1/4 of RPVNet. Our source code is available at https://github.com/wuzhaoo/3D-ARSS.