Efficient Image Super-Resolution With Feature Interaction Weighted Hybrid Network

被引:0
作者
Li, Wenjie [1 ]
Li, Juncheng [2 ]
Gao, Guangwei [3 ,4 ,5 ]
Deng, Weihong [1 ]
Yang, Jian [6 ]
Qi, Guo-Jun [7 ,8 ,9 ]
Lin, Chia-Wen [10 ,11 ]
机构
[1] Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Pattern Recognit & Intelligent Syst Lab, Beijing 100080, Peoples R China
[2] Shanghai Univ, Sch Commun & Informat Engn, Shanghai 200444, Peoples R China
[3] Nanjing Univ Posts & Telecommun, Inst Adv Technol, IVIPLab, Nanjing 210046, Peoples R China
[4] Minist Educ, Key Lab Artificial Intelligence, Shanghai 200240, Peoples R China
[5] Soochow Univ, Prov Key Lab Comp Informat Proc Technol, Suzhou 215006, Peoples R China
[6] Nanjing Univ Sci & Technol, Sch Comp Sci & Technol, Nanjing 210094, Peoples R China
[7] Westlake Univ, Res Ctr Ind Future, Hangzhou 310024, Peoples R China
[8] Westlake Univ, Sch Engn, Hangzhou 310024, Peoples R China
[9] OPPO Res, Seattle, WA 98101 USA
[10] Natl Tsing Hua Univ, Dept Elect Engn, Hsinchu 300044, Taiwan
[11] Natl Tsing Hua Univ, Inst Commun Engn, Hsinchu 300044, Taiwan
关键词
Transformers; Convolutional neural networks; Computational modeling; Feature extraction; Training; Superresolution; Adaptation models; Computer architecture; Telecommunications; Image reconstruction; Single-image super-resolution; wide-residual distillation interaction; hybrid network; transformer; ACCURATE;
D O I
10.1109/TMM.2024.3521753
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Lightweight image super-resolution aims to reconstruct high-resolution images from low-resolution images using low computational costs. However, existing methods result in the loss of middle-layer features due to activation functions. To minimize the impact of intermediate feature loss on reconstruction quality, we propose a Feature Interaction Weighted Hybrid Network (FIWHN), which comprises a series of Wide-residual Distillation Interaction Block (WDIB) as the backbone. Every third WDIB forms a Feature Shuffle Weighted Group (FSWG) by applying mutual information shuffle and fusion. Moreover, to mitigate the negative effects of intermediate feature loss, we introduce Wide Residual Weighting units within WDIB. These units effectively fuse features of varying levels of detail through a Wide-residual Distillation Connection (WRDC) and a Self-Calibrating Fusion (SCF). To compensate for global feature deficiencies, we incorporate a Transformer and explore a novel architecture to combine CNN and Transformer. We show that our FIWHN achieves a favorable balance between performance and efficiency through extensive experiments on low-level and high-level tasks.
引用
收藏
页码:2256 / 2267
页数:12
相关论文
共 45 条
[1]  
Ahn N., 2018, PROC EUR C COMPUT VI
[2]   SCTANet: A Spatial Attention-Guided CNN-Transformer Aggregation Network for Deep Face Image Super-Resolution [J].
Bao, Qiqi ;
Liu, Yunmeng ;
Gang, Bowen ;
Yang, Wenming ;
Liao, Qingmin .
IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 :8554-8565
[3]   Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding [J].
Bevilacqua, Marco ;
Roumy, Aline ;
Guillemot, Christine ;
Morel, Marie-Line Alberi .
PROCEEDINGS OF THE BRITISH MACHINE VISION CONFERENCE 2012, 2012,
[4]   Segmentation and Recognition Using Structure from Motion Point Clouds [J].
Brostow, Gabriel J. ;
Shotton, Jamie ;
Fauqueur, Julien ;
Cipolla, Roberto .
COMPUTER VISION - ECCV 2008, PT I, PROCEEDINGS, 2008, 5302 :44-+
[5]   Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model [J].
Cai, Jianrui ;
Zeng, Hui ;
Yong, Hongwei ;
Cao, Zisheng ;
Zhang, Lei .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :3086-3095
[6]   N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution [J].
Choi, Haram ;
Lee, Jeongmin ;
Yang, Jihoon .
2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR, 2023, :2071-2081
[7]  
Chu X., 2021, PROC IEEE INT C PATT, P5964
[8]   Image Super-Resolution Using Deep Convolutional Networks [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2016, 38 (02) :295-307
[9]   Fast and Memory-Efficient Network Towards Efficient Image Super-Resolution [J].
Du, Zongcai ;
Liu, Ding ;
Liu, Jie ;
Tang, Jie ;
Wu, Gangshan ;
Fu, Lean .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2022, 2022, :852-861
[10]   Context-Patch Representation Learning With Adaptive Neighbor Embedding for Robust Face Image Super-Resolution [J].
Gao, Guangwei ;
Yu, Yi ;
Lu, Huimin ;
Yang, Jian ;
Yue, Dong .
IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 :1879-1889