Real-Time Semantic Segmentation of Point Clouds Based on an Attention Mechanism and a Sparse Tensor

被引:5
作者
Wang, Fei [1 ]
Yang, Yujie [1 ]
Wu, Zhao [1 ]
Zhou, Jingchun [1 ]
Zhang, Weishi [1 ]
机构
[1] Dalian Maritime Univ, Coll Informat Sci & Technol, Dalian 116026, Peoples R China
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 05期
关键词
3D point cloud; attention mechanism; semantic segmentation; sparse tensor; NETWORK;
D O I
10.3390/app13053256
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
A 3D point cloud is one of the main data sources for robot environmental cognition and understanding. Due to the limited computation and memory capacities of the robotic platform, existing semantic segmentation models of 3D point clouds cannot meet the requirements of real-time applications. To solve this problem, a lightweight, fully convolutional network based on an attention mechanism and a sparse tensor is proposed to better balance the accuracy and real-time performance of point cloud semantic segmentation. On the basis of the 3D-Unet structure, a global feature-learning module and a multi-scale feature fusion module are designed. The former improves the ability of features to describe important areas by learning the importance of spatial neighborhoods. The latter realizes the fusion of multi-scale semantic information and suppresses useless information through the task correlation learning of multi-scale features. Additionally, to efficiently process the large-scale point clouds acquired in real time, a sparse tensor-based implementation method is introduced. It is able to reduce unnecessary computation according to the sparsity of the 3D point cloud. As demonstrated by the results of experiments conducted with the SemanticKITTI and NuScenes datasets, our model improves the mIoU metric by 6.4% and 5%, respectively, over existing models that can be applied in real time. Our model is a lightweight model that can meet the requirements of real-time applications.
引用
收藏
页数:15
相关论文
共 38 条
[11]   Joint Scheduling and Incentive Mechanism for Spatio-Temporal Vehicular Crowd Sensing [J].
Fan, Guiyun ;
Jin, Haiming ;
Liu, Qihong ;
Qin, Wei ;
Gan, Xiaoying ;
Long, Huan ;
Fu, Luoyi ;
Wang, Xinbing .
IEEE TRANSACTIONS ON MOBILE COMPUTING, 2021, 20 (04) :1449-1464
[12]  
Fang Y., 2020, ARXIV
[13]   Point attention network for semantic segmentation of 3D point clouds [J].
Feng, Mingtao ;
Zhang, Liang ;
Lin, Xuefei ;
Gilani, Syed Zulqarnain ;
Mian, Ajmal .
PATTERN RECOGNITION, 2020, 107 (107)
[14]   3D Semantic Segmentation with Submanifold Sparse Convolutional Networks [J].
Graham, Benjamin ;
Engelcke, Martin ;
van der Maaten, Laurens .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :9224-9232
[15]   HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds [J].
Gu, Xiuye ;
Wang, Yijie ;
Wu, Chongruo ;
Lee, Yong Jae ;
Wang, Panqu .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :3249-3258
[16]   A Multiscale Multi-Feature Deep Learning Model for Airborne Point-Cloud Semantic Segmentation [J].
He, Peipei ;
Ma, Zheng ;
Fei, Meiqi ;
Liu, Wenkai ;
Guo, Guihai ;
Wang, Mingwei .
APPLIED SCIENCES-BASEL, 2022, 12 (22)
[17]  
Hu J, 2018, PROC CVPR IEEE, P7132, DOI [10.1109/CVPR.2018.00745, 10.1109/TPAMI.2019.2913372]
[18]   Object-Level Semantic Map Construction for Dynamic Scenes [J].
Kang, Xujie ;
Li, Jing ;
Fan, Xiangtao ;
Jian, Hongdeng ;
Xu, Chen .
APPLIED SCIENCES-BASEL, 2021, 11 (02) :1-20
[19]   PointVGG: Graph convolutional network with progressive aggregating features on point clouds [J].
Li, Rongkang ;
Zhang, Yumeng ;
Niu, Dongmei ;
Yang, Guangchao ;
Zafar, Numan ;
Zhang, Caiming ;
Zhao, Xiuyang .
NEUROCOMPUTING, 2021, 429 :187-198
[20]  
Rosu R.A., 2019, arXiv