Local-Global Dynamic Filtering Network for Video Super-Resolution

被引:1
作者
Zhang, Chaopeng [1 ]
Wang, Xingtao [1 ]
Xiong, Ruiqin [2 ]
Fan, Xiaopeng [1 ,3 ]
Zhao, Debin [1 ,3 ]
机构
[1] Harbin Inst Technol, Sch Comp Sci & Technol, Harbin 150001, Peoples R China
[2] Peking Univ, Inst Digital Media, Sch Elect Engn & Comp Sci, Beijing 100871, Peoples R China
[3] Peng Cheng Lab, Shenzhen 519055, Peoples R China
基金
国家重点研发计划; 中国国家自然科学基金;
关键词
Video super-resolution; alignment; divide-and-donquer; self-calibrated dynamic filtering; long-range dependencies; SUPER RESOLUTION;
D O I
10.1109/TCI.2023.3321980
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Video super-resolution (VSR) has been greatly advanced by the use of deep learning techniques, but the challenge of handling motion variability has remained a bottleneck. Many previous methods have treated motions equally, leading to suboptimal alignment. In this article, we propose a Local-Global Dynamic Filtering Network (LGDFNet) to address this issue. LGDFNet uses a divide-and-conquer strategy to handle motion-varying features, where the overall feature is split into local features and assigned specialized sub-networks to align and fuse them from local to global. To align the features and adaptively aggregate several kernels for calibration, we propose the Self-Calibrated Dynamic Filtering (SCDF) module. Additionally, we introduce the Cross-Attention Feature Fusing (CAFF) module to capture long-range dependencies and fuse each feature. Our extensive experiments on different benchmark datasets demonstrate the effectiveness of LGDFNet, both subjectively and objectively.
引用
收藏
页码:963 / 976
页数:14
相关论文
共 69 条
  • [1] MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement
    Bao, Wenbo
    Lai, Wei-Sheng
    Zhang, Xiaoyun
    Gao, Zhiyong
    Yang, Ming-Hsuan
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2021, 43 (03) : 933 - 948
  • [2] Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
    Caballero, Jose
    Ledig, Christian
    Aitken, Andrew
    Acosta, Alejandro
    Totz, Johannes
    Wang, Zehan
    Shi, Wenzhe
    [J]. 30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, : 2848 - 2857
  • [3] Cao JZ, 2023, Arxiv, DOI arXiv:2106.06847
  • [4] BasicVSR plus plus : Improving Video Super-Resolution with Enhanced Propagation and Alignment
    Chan, Kelvin C. K.
    Zhou, Shangchen
    Xu, Xiangyu
    Loy, Chen Change
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 5962 - 5971
  • [5] BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
    Chan, Kelvin C. K.
    Wang, Xintao
    Yu, Ke
    Dong, Chao
    Loy, Chen Change
    [J]. 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 4945 - 4954
  • [6] Chan KCK, 2021, AAAI CONF ARTIF INTE, V35, P973
  • [7] Dynamic Convolution: Attention over Convolution Kernels
    Chen, Yinpeng
    Dai, Xiyang
    Liu, Mengchen
    Chen, Dongdong
    Yuan, Lu
    Liu, Zicheng
    [J]. 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, : 11027 - 11036
  • [8] Learning a Deep Convolutional Network for Image Super-Resolution
    Dong, Chao
    Loy, Chen Change
    He, Kaiming
    Tang, Xiaoou
    [J]. COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 : 184 - 199
  • [9] Drulea M, 2011, IEEE INT C INTELL TR, P318, DOI 10.1109/ITSC.2011.6082986
  • [10] Efficient Video Super-Resolution through Recurrent Latent Space Propagation
    Fuoli, Dario
    Gu, Shuhang
    Timofte, Radu
    [J]. 2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW), 2019, : 3476 - 3485