Multiattention Mechanism 3D Object Detection Algorithm Based on RGB and LiDAR Fusion for Intelligent Driving

被引:5
|
作者
Zhang, Xiucai [1 ]
He, Lei [1 ]
Chen, Junyi [1 ]
Wang, Baoyun [1 ]
Wang, Yuhai [1 ]
Zhou, Yuanle [1 ]
机构
[1] Jilin Univ, State Key Lab Automot Simulat & Control, Changchun 130022, Peoples R China
关键词
multimodal fusion; attention mechanism; 3D target detection; deep learning; REPRESENTATION;
D O I
10.3390/s23218732
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
This paper proposes a multimodal fusion 3D target detection algorithm based on the attention mechanism to improve the performance of 3D target detection. The algorithm utilizes point cloud data and information from the camera. For image feature extraction, the ResNet50 + FPN architecture extracts features at four levels. Point cloud feature extraction employs the voxel method and FCN to extract point and voxel features. The fusion of image and point cloud features is achieved through regional point fusion and voxel fusion methods. After information fusion, the Coordinate and SimAM attention mechanisms extract fusion features at a deep level. The algorithm's performance is evaluated using the DAIR-V2X dataset. The results show that compared to the Part-A2 algorithm; the proposed algorithm improves the mAP value by 7.9% in the BEV view and 7.8% in the 3D view at IOU = 0.5 (cars) and IOU = 0.25 (pedestrians and cyclists). At IOU = 0.7 (cars) and IOU = 0.5 (pedestrians and cyclists), the mAP value of the SECOND algorithm is improved by 5.4% in the BEV view and 4.3% in the 3D view, compared to other comparison algorithms.
引用
收藏
页数:17
相关论文
共 50 条
  • [1] 3D object detection based on image and LIDAR fusion for autonomous driving
    Chen G.
    Yi H.
    Mao Z.
    International Journal of Vehicle Information and Communication Systems, 2023, 8 (03) : 237 - 251
  • [2] 3D Object Detection with Fusion Point Attention Mechanism in LiDAR Point Cloud
    Liu Weili
    Zhu Deli
    Luo Huahao
    Li Yi
    ACTA PHOTONICA SINICA, 2023, 52 (09)
  • [3] RangeLVDet: Boosting 3D Object Detection in LIDAR With Range Image and RGB Image
    Zhang, Zehan
    Liang, Zhidong
    Zhang, Ming
    Zhao, Xian
    Li, Hao
    Yang, Ming
    Tan, Wenming
    Pu, Shiliang
    IEEE SENSORS JOURNAL, 2022, 22 (02) : 1391 - 1403
  • [4] TEMPORAL AXIAL ATTENTION FOR LIDAR-BASED 3D OBJECT DETECTION IN AUTONOMOUS DRIVING
    Carranza-Garcia, Manuel
    Riquelme, Jose C.
    Zakhor, Avideh
    2022 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP, 2022, : 201 - 205
  • [5] A Survey on Deep-Learning-Based LiDAR 3D Object Detection for Autonomous Driving
    Alaba, Simegnew Yihunie
    Ball, John E.
    SENSORS, 2022, 22 (24)
  • [6] BRTPillar: boosting real-time 3D object detection based point cloud and RGB image fusion in autonomous driving
    Zhang, Zhitian
    Zhao, Hongdong
    Zhao, Yazhou
    Chen, Dan
    Zhang, Ke
    Li, Yanqi
    INTERNATIONAL JOURNAL OF INTELLIGENT COMPUTING AND CYBERNETICS, 2025, 18 (01) : 217 - 235
  • [7] Transformer-Based Optimized Multimodal Fusion for 3D Object Detection in Autonomous Driving
    Alaba, Simegnew Yihunie
    Ball, John E.
    IEEE ACCESS, 2024, 12 : 50165 - 50176
  • [8] CrossFusion net: Deep 3D object detection based on RGB images and point clouds in autonomous driving
    Hong, Dza-Shiang
    Chen, Hung-Hao
    Hsiao, Pei-Yung
    Fu, Li-Chen
    Siao, Siang-Min
    IMAGE AND VISION COMPUTING, 2020, 100
  • [9] Point cloud 3D object detection algorithm based on local information fusion
    Zhang, Linjie
    Chai, Zhilei
    Wang, Ning
    Zhejiang Daxue Xuebao (Gongxue Ban)/Journal of Zhejiang University (Engineering Science), 2024, 58 (11): : 2219 - 2229
  • [10] Improved AVOD Model Based on a Bidirectional Fusion Mechanism for 3D Object Detection
    Tao, Zhonghan
    Wan, Yi
    Hou, Jiarui
    Wu, Weili
    Zhang, Shilei
    PROCEEDINGS OF THE 2024 3RD INTERNATIONAL SYMPOSIUM ON INTELLIGENT UNMANNED SYSTEMS AND ARTIFICIAL INTELLIGENCE, SIUSAI 2024, 2024, : 309 - 312