Multiattention Mechanism 3D Object Detection Algorithm Based on RGB and LiDAR Fusion for Intelligent Driving

被引:5
|
作者
Zhang, Xiucai [1 ]
He, Lei [1 ]
Chen, Junyi [1 ]
Wang, Baoyun [1 ]
Wang, Yuhai [1 ]
Zhou, Yuanle [1 ]
机构
[1] Jilin Univ, State Key Lab Automot Simulat & Control, Changchun 130022, Peoples R China
关键词
multimodal fusion; attention mechanism; 3D target detection; deep learning; REPRESENTATION;
D O I
10.3390/s23218732
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
This paper proposes a multimodal fusion 3D target detection algorithm based on the attention mechanism to improve the performance of 3D target detection. The algorithm utilizes point cloud data and information from the camera. For image feature extraction, the ResNet50 + FPN architecture extracts features at four levels. Point cloud feature extraction employs the voxel method and FCN to extract point and voxel features. The fusion of image and point cloud features is achieved through regional point fusion and voxel fusion methods. After information fusion, the Coordinate and SimAM attention mechanisms extract fusion features at a deep level. The algorithm's performance is evaluated using the DAIR-V2X dataset. The results show that compared to the Part-A2 algorithm; the proposed algorithm improves the mAP value by 7.9% in the BEV view and 7.8% in the 3D view at IOU = 0.5 (cars) and IOU = 0.25 (pedestrians and cyclists). At IOU = 0.7 (cars) and IOU = 0.5 (pedestrians and cyclists), the mAP value of the SECOND algorithm is improved by 5.4% in the BEV view and 4.3% in the 3D view, compared to other comparison algorithms.
引用
收藏
页数:17
相关论文
共 50 条
  • [11] A Frustum-based probabilistic framework for 3D object detection by fusion of LiDAR and camera data
    Gong, Zheng
    Lin, Haojia
    Zhang, Dedong
    Luo, Zhipeng
    Zelek, John
    Chen, Yiping
    Nurunnabi, Abdul
    Wang, Cheng
    Li, Jonathan
    ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2020, 159 : 90 - 100
  • [12] Recent advances in 3D object detection based on RGB-D: A survey
    Wang, Yangfan
    Wang, Chen
    Long, Peng
    Gu, Yuzong
    Li, Wenfa
    DISPLAYS, 2021, 70
  • [13] LiDAR-camera fusion: Dual transformer enhancement for 3D object detection
    Chen, Mu
    Liu, Pengfei
    Zhao, Huaici
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2023, 120
  • [14] A comprehensive survey of LIDAR-based 3D object detection methods with deep learning for autonomous driving
    Zamanakos, Georgios
    Tsochatzidis, Lazaros
    Amanatiadis, Angelos
    Pratikakis, Ioannis
    COMPUTERS & GRAPHICS-UK, 2021, 99 : 153 - 181
  • [15] LXL: LiDAR Excluded Lean 3D Object Detection With 4D Imaging Radar and Camera Fusion
    Xiong, Weiyi
    Liu, Jianan
    Huang, Tao
    Han, Qing-Long
    Xia, Yuxuan
    Zhu, Bing
    IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, 2024, 9 (01): : 79 - 92
  • [16] Multi-Modal and Multi-Scale Fusion 3D Object Detection of 4D Radar and LiDAR for Autonomous Driving
    Wang, Li
    Zhang, Xinyu
    Li, Jun
    Xv, Baowei
    Fu, Rong
    Chen, Haifeng
    Yang, Lei
    Jin, Dafeng
    Zhao, Lijun
    IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2023, 72 (05) : 5628 - 5641
  • [17] Multi-Modal Fusion Based on Depth Adaptive Mechanism for 3D Object Detection
    Liu, Zhanwen
    Cheng, Juanru
    Fan, Jin
    Lin, Shan
    Wang, Yang
    Zhao, Xiangmo
    IEEE TRANSACTIONS ON MULTIMEDIA, 2025, 27 : 707 - 717
  • [18] LiDAR-only 3D object detection based on spatial context
    Wang, Qiang
    Li, Ziyu
    Zhu, Dejun
    Yang, Wankou
    JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 2023, 93
  • [19] 3D object detection based on fusion of point cloud and image by mutual attention
    Chen J.-Y.
    Bai T.-Y.
    Zhao L.
    Guangxue Jingmi Gongcheng/Optics and Precision Engineering, 2021, 29 (09): : 2247 - 2254
  • [20] Research on 3D Object Detection Based on Laser Point Cloud and Image Fusion
    Liu Y.
    Yu F.
    Zhang X.
    Chen Z.
    Qin D.
    Jixie Gongcheng Xuebao/Journal of Mechanical Engineering, 2022, 58 (24): : 289 - 299