Real-Time Semantic Segmentation of Point Clouds Based on an Attention Mechanism and a Sparse Tensor

被引:5
作者
Wang, Fei [1 ]
Yang, Yujie [1 ]
Wu, Zhao [1 ]
Zhou, Jingchun [1 ]
Zhang, Weishi [1 ]
机构
[1] Dalian Maritime Univ, Coll Informat Sci & Technol, Dalian 116026, Peoples R China
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 05期
关键词
3D point cloud; attention mechanism; semantic segmentation; sparse tensor; NETWORK;
D O I
10.3390/app13053256
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
A 3D point cloud is one of the main data sources for robot environmental cognition and understanding. Due to the limited computation and memory capacities of the robotic platform, existing semantic segmentation models of 3D point clouds cannot meet the requirements of real-time applications. To solve this problem, a lightweight, fully convolutional network based on an attention mechanism and a sparse tensor is proposed to better balance the accuracy and real-time performance of point cloud semantic segmentation. On the basis of the 3D-Unet structure, a global feature-learning module and a multi-scale feature fusion module are designed. The former improves the ability of features to describe important areas by learning the importance of spatial neighborhoods. The latter realizes the fusion of multi-scale semantic information and suppresses useless information through the task correlation learning of multi-scale features. Additionally, to efficiently process the large-scale point clouds acquired in real time, a sparse tensor-based implementation method is introduced. It is able to reduce unnecessary computation according to the sparsity of the 3D point cloud. As demonstrated by the results of experiments conducted with the SemanticKITTI and NuScenes datasets, our model improves the mIoU metric by 6.4% and 5%, respectively, over existing models that can be applied in real time. Our model is a lightweight model that can meet the requirements of real-time applications.
引用
收藏
页数:15
相关论文
共 38 条
[21]   SPLATNet: Sparse Lattice Networks for Point Cloud Processing [J].
Su, Hang ;
Jampani, Varun ;
Sun, Deqing ;
Maji, Subhransu ;
Kalogerakis, Evangelos ;
Yang, Ming-Hsuan ;
Kautz, Jan .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :2530-2539
[22]  
Sun YB, 2020, IEEE WINT CONF APPL, P61, DOI [10.1109/wacv45572.2020.9093430, 10.1109/WACV45572.2020.9093430]
[23]  
Tang H., 2020, EUR C COMP VIS, P685, DOI [10.1007/978-3-030-58604-1_41, DOI 10.1007/978-3-030-58604-1_41]
[24]   KPConv: Flexible and Deformable Convolution for Point Clouds [J].
Thomas, Hugues ;
Qi, Charles R. ;
Deschaud, Jean-Emmanuel ;
Marcotegui, Beatriz ;
Goulette, Francois ;
Guibas, Leonidas J. .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :6420-6429
[25]   Cross self-attention network for 3D point cloud [J].
Wang, Gaihua ;
Zhai, Qianyu ;
Liu, Hong .
KNOWLEDGE-BASED SYSTEMS, 2022, 247
[26]   Online Spatial Crowdsensing With Expertise-Aware Truth Inference and Task Allocation [J].
Wang, Xiong ;
Jia, Riheng ;
Fu, Luoyi ;
Jin, Haiming ;
Tian, Xiaohua ;
Gan, Xiaoying ;
Wang, Xinbing .
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2022, 40 (01) :412-427
[27]  
Wen X., 2020, P 28 ACM INT C MULTI
[28]   Unraveling the Detectability of Stochastic Block Model With Overlapping Communities [J].
Wu, Huaying ;
Fu, Luoyi ;
Long, Huan ;
Meng, Guie ;
Gan, Xiaoying ;
Wu, Yuanhao ;
Zhang, Haisong ;
Wang, Xinbing .
IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, 2021, 8 (02) :1443-1455
[29]  
Xu J., 2021, P 2021 IEEE INT C CO
[30]  
Yan X., 2020, P AAAI C ARTIFICIAL