A video inpainting method for unmanned vehicle based on fusion of time series optical flow information and spatial information

被引:0
|
作者
Zhao, Rui [1 ]
Li, Hengyu [1 ]
Liu, Jingyi [1 ]
Pu, Huayan [1 ]
Xie, Shaorong [1 ]
Luo, Jun [1 ,2 ]
机构
[1] Shanghai Univ, Sch Mechatron Engn & Automat, 99 Shangda Rd, Shanghai 200444, Peoples R China
[2] Chongqing Univ, Coll Mech & Vehicle Engn, Chongqing, Peoples R China
来源
INTERNATIONAL JOURNAL OF ADVANCED ROBOTIC SYSTEMS | 2021年 / 18卷 / 05期
基金
中国国家自然科学基金;
关键词
Optical flow; deep learning; generative adversarial networks; video inpainting; SYSTEM;
D O I
10.1177/17298814211053103
中图分类号
TP24 [机器人技术];
学科分类号
080202 ; 1405 ;
摘要
In this article, the problem of video inpainting combines multiview spatial information and interframe information between video sequences. A vision system is an important way for autonomous vehicles to obtain information about the external environment. Loss or distortion of visual images caused by camera damage or pollution seriously makes an impact on the vision system ability to correctly perceive and understand the external environment. In this article, we solve the problem of image restoration by combining the optical flow information between frames in the video with the spatial information from multiple perspectives. To solve the problems of noise in the single-frame images of video frames, we propose a complete two-stage video repair method. We combine the spatial information of images from different perspectives and the optical flow information of the video sequence to assist and constrain the repair of damaged images in the video. This method combines the interframe information of the front and rear image frames with the multiview image information in the video and performs video repair based on optical flow and a conditional generation adversarial network. This method regards video inpainting as a pixel propagation problem, uses the interframe information in the video for video inpainting, and introduces multiview information to assist the repair based on a conditional generative adversarial network. This method was trained and tested in Zurich using a data set recorded by a pair of cameras mounted on a mobile platform.
引用
收藏
页数:11
相关论文
共 50 条
  • [21] Locating Method for Vehicle at Intersection Based on TPVM and Optical Flow
    Chen, Tao
    Fan, Linkun
    Li, Xuchuan
    Guo, Congshuai
    Qiao, Miaomiao
    CICTP 2020: TRANSPORTATION EVOLUTION IMPACTING FUTURE MOBILITY, 2020, : 764 - 776
  • [22] Real-time detecting and tracking of traffic shockwaves based on weighted consensus information fusion in distributed video network
    Yang, Deliang
    Chen, Yangzhou
    Xin, Le
    Zhang, Yuan
    IET INTELLIGENT TRANSPORT SYSTEMS, 2014, 8 (04) : 377 - 387
  • [23] Side Information and Noise Learning for Distributed Video Coding Using Optical Flow and Clustering
    Huynh Van Luong
    Raket, Lars Lau
    Huang, Xin
    Forchhammer, Soren
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2012, 21 (12) : 4782 - 4796
  • [24] Detection and tracking of floating objects based on spatial-temporal information fusion
    Renfei, Chen
    Jian, Wu
    Yong, Peng
    Zhongwen, Li
    Hua, Shang
    EXPERT SYSTEMS WITH APPLICATIONS, 2023, 225
  • [25] Saliency Calculation Based on the Fusion of Enhanced Contour Features and Spatial Semantic Information
    Ji C.
    Wang D.
    Huang X.
    Cao W.
    Jiang B.
    1813, Institute of Computing Technology (32): : 1813 - 1821
  • [26] Research on Object Detection Algorithm Based on Coordinate Information Fusion and Spatial Perception
    Liuha, Youyong
    Luo, Liang
    Wang, Xusheng
    Zheng, Yuansheng
    2024 5TH INTERNATIONAL CONFERENCE ON COMPUTER ENGINEERING AND APPLICATION, ICCEA 2024, 2024, : 1289 - 1293
  • [27] An LSTM based method for stage performance degradation early warning with consideration of time-series information
    Li, Xingshuo
    Liu, Jinfu
    Bai, Mingliang
    Li, Jiajia
    Li, Xianling
    Yan, Peigang
    Yu, Daren
    ENERGY, 2021, 226
  • [28] Multiscale information fusion-based deep learning framework for campus vehicle detection
    Xu, Zengyong
    Rao, Meili
    INTERNATIONAL JOURNAL OF IMAGE AND DATA FUSION, 2021, 12 (01) : 83 - 97
  • [29] Hybrid Side Information Generation Algorithm Based on Probability Fusion for Distributed Video Coding
    Wang, Wei
    Chen, Jianhua
    2020 5TH INTERNATIONAL CONFERENCE ON COMPUTER AND COMMUNICATION SYSTEMS (ICCCS 2020), 2020, : 291 - 295
  • [30] Physics-Based and Human-derived Information Fusion Video Activity Analysis
    Blasch, Erik P.
    Aved, Alex J.
    2018 21ST INTERNATIONAL CONFERENCE ON INFORMATION FUSION (FUSION), 2018, : 997 - 1004