Object Detection and Tracking Based on Image and Point Clouds Instance Matching for Intelligent Vehicles

被引：0
作者：
Li, Shangjie [1 ]
Yin, Guodong [1 ]
Geng, Keke [1 ]
Liu, Shuaipeng [1 ]
机构：
[1] School of Mechanical Engineering, Southeast University, Nanjing
来源：
Jixie Gongcheng Xuebao/Journal of Mechanical Engineering | 2024年 / 60卷 / 22期
关键词：
instance segmentation; intelligent vehicle; object detection; object tracking; perspective projection; sensor fusion;
D O I：
10.3901/JME.2024.22.302
中图分类号：
学科分类号：
摘要：
In the environment perception task of intelligent vehicles, in order to combine the rich semantic information in the camera image with the accurate spatial information in the lidar point clouds, a fusion detection method based on image and point clouds instance matching is proposed. To achieve the fusion detection, the instance masks of the targets in the image are predicted by the instance segmentation network, the point clouds are projected to the image plane through perspective projection transformation, the point clouds belonging to the target are extracted according to the instance mask of each target, and then the clustering algorithm is used to remove the noise, and the convex hull approximating algorithm is used to fit the 3D bounding box of the target. Based on the fusion detection method, a gate is designed to realize multi-target data association and management, and the Kalman filter is used to track the target and estimate the motion state of the target. The experimental results show that the method can effectively fuse the information from image data and point clouds data, accurately and quickly fit the position, size, and direction of the target and estimate the speed of the target, and show robustness in different experimental scenarios. © 2024 Chinese Mechanical Engineering Society. All rights reserved.
引用
收藏
页码：302 / 310
页数：8
相关论文
共 14 条
  • [1] PENG P, GENG K, YIN G, Et al., Adaptive multi-modal fusion instance segmentation for CAEVs in complex conditions: Dataset, framework and verifications, Chinese Journal of Mechanical Engineering, 34, 1, pp. 1-11, (2021)
  • [2] LIU L, OUYANG W, WANG X, Et al., Deep learning for generic object detection:A survey[J], International Journal of Computer Vision, 128, 2, pp. 261-318, (2020)
  • [3] REN S, HE K, GIRSHICK R, Et al., Faster R-CNN: Towards real-time object detection with region proposal networks[J], IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 6, pp. 1137-1149, (2017)
  • [4] REDMON J, DIVVALA S, GIRSHICK R, Et al., You only look once: Unified, real-time object detection[C], Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779-788, (2016)
  • [5] LIU W, ANGUELOV D, ERHAN D, Et al., SSD:Single shot multibox detector[C], European Conference on Computer Vision, pp. 21-37, (2016)
  • [6] HE K, GKIOXARI G, DOLLAR P, Et al., Mask R-CNN[C], Proceedings of the IEEE International Conference on Computer Vision, pp. 2961-2969, (2017)
  • [7] LI Y, QI H, DAI J, Et al., Fully convolutional instance-aware semantic segmentation[C], Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2359-2367, (2017)
  • [8] BOLYA D, ZHOU C, XIAO F, Et al., YOLACT:Real-time instance segmentation, Proceedings of the IEEE International Conference on Computer Vision, pp. 9157-9166, (2019)
  • [9] ARNOLD E, AL-JARRAH O Y, DIANATI M, Et al., A survey on 3d object detection methods for autonomous driving applications[J], IEEE Transactions on Intelligent Transportation Systems, 20, 10, pp. 3782-3795, (2019)
  • [10] FENG D, HAASE-SCHUTZ C, ROSENBAUM L, Et al., Deep multi-modal object detection and semantic segmentation for autonomous driving:Datasets, methods, and challenges[J], IEEE Transactions on Intelligent Transportation Systems, 22, 3, pp. 1341-1360, (2020)
12