Deep Neural Network-Based Dynamical Object Recognition and Robust Multiobject Tracking Technique for Onboard Unmanned Aerial Vehicle's Computer Vision-Based Systems

被引:2
作者
Saetchnikov I.V. [1 ]
Skakun V.V. [1 ]
Tcherniavskaia E.A. [2 ]
机构
[1] Belarusian State University, Radio Physics Department, Minsk
[2] Belarusian State University, Physics Department, Minsk
来源
IEEE Journal on Miniaturization for Air and Space Systems | 2023年 / 4卷 / 03期
关键词
Image analysis; machIne vislon; neural networks; object detection; object recognition; recurrent neural networks; road vehlcles; tracklng;
D O I
10.1109/JMASS.2023.3274929
中图分类号
学科分类号
摘要
Computer vision-based systems seem highly perspective for semantic analysis of the dynamical objects. However, considering dynamical object recognition and tracking from the unmanned aerial vehicle (UAV) the task to design a robust model for data association is highly challenging due to additional issues, e.g., image degradation, nonfixed object camera distance and shooting focus, and real-time issues. Thus, we propose an accurate deep neural network-based dynamical object recognition and robust multiobject tracking technique based on bidirectional LSTM with the optimized motion and appearance gates as a multiobject tracking backbone, supported by an advanced single-shot detector network improved with residual prediction model and implemented a DenseNet network as well as a YOLOv4eff network as feature extraction. The technique has been trained on VisDrone 2022 and UAVDT datasets with the side-shoot dynamical objects at a height of up to 50 m. The performance analysis on the test stage performed on seven metrics demonstrate that the proposed technique surpasses, by accuracy and robustness ability, other state-of-the-art techniques based on two cumulative MOTA and MOTP, as well as MT and IDsw. In particular, we have dramatically decreased the number of IDsw which implies a better capability to handle several occlusions, which is a desirable property in real-time multiple object tracking. We have pointed out the sensitivity of the tracking performance of our technique on the number of utilizing different sequence lengths and have defined an optimum. Finally, the applicability and reliability of the proposed technique for onboard UAV computer-based systems have been discussed. © 2019 IEEE.
引用
收藏
页码:250 / 256
页数:6
相关论文
共 32 条
  • [1] He K., Zhang X., Ren S., Sun J., Deep residual learning for image recognition, Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 770-778, (2016)
  • [2] Schumann A., Sommer L., Klatte J., Schuchert T., Beyerer J., Deep cross-domain flying object classification for robust UAV detection, Proc. 14th IEEE Int. Conf. Adv. Video Signal Based Surveillance (AVSS), pp. 1-6, (2017)
  • [3] Long J., Shelhamer E., Darrell T., Fully convolutional networks for semantic segmentation, Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 3431-3440, (2015)
  • [4] Cho H., Seo Y.-W., Kumar B.V.K.V., Rajkumar R.R., A multisensor fusion system for moving object detection and tracking in urban driving environments, Proc. IEEE Int. Conf. Robot. Autom. (ICRA), pp. 1836-1843, (2014)
  • [5] Li Y., Fu C., Ding F., Huang Z., Lu G., AutoTrack: Towards high-performance visual tracking for UAV with automatic spatio-temporal regularization, Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2020, pp. 11920-11929
  • [6] Jacobson A., Zeng F., Smith D., Boswell N., Peynot T., Milford M., Semi-supervised SLAM: Leveraging low-cost sensors on underground autonomous vehicles for position tracking, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), pp. 3970-3977, (2018)
  • [7] Ren X., Wang D., Laskey M., Goldberg K., Learning traffic behaviors by extracting vehicle trajectories from online video streams, Proc. IEEE 14th Int. Conf. Autom. Sci. Eng. (CASE), pp. 1276-1283, (2018)
  • [8] Huang Z., Fu C., Li Y., Lin F., Lu P., Learning aberrance repressed correlation filters for real-time UAV tracking, Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), pp. 2891-2900, (2019)
  • [9] Wu Y., Sui Y., Wang G., Vision-based real-time aerial object localization and tracking for UAV sensing system, IEEE Access, 5, pp. 23969-23978, (2017)
  • [10] Zhu J., Et al., Urban traffic density estimation based on ultrahigh-resolution UAV video and deep neural network, IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., 11, 12, pp. 4968-4981, (2018)