BAM: a balanced attention mechanism to optimize single image super-resolution

被引:3
作者
Wang, Fanyi [1 ]
Hu, Haotian [1 ]
Shen, Cheng [2 ]
Feng, Tianpeng [3 ]
Guo, Yandong [3 ]
机构
[1] Zhejiang Univ, Hangzhou 310027, Peoples R China
[2] CALTECH, Pasadena, CA 91125 USA
[3] OPPO Res Inst, Shenzhen, Peoples R China
关键词
Single image super-resolution; Texture aliasing; Inference acceleration; Lightweight attention mechanism;
D O I
10.1007/s11554-022-01235-x
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Recovering texture information from the aliasing regions has always been a major challenge for single image super-resolution (SISR) task. These regions are often submerged in noise so that we have to restore texture details while suppressing noise. To address this issue, we propose an efficient Balanced Attention Mechanism (BAM), which consists of Avgpool Channel Attention Module (ACAM) and Maxpool Spatial Attention Module (MSAM) in parallel. ACAM is designed to suppress extreme noise in the large-scale feature maps, while MSAM preserves high-frequency texture details. Thanks to the parallel structure, these two modules not only conduct self-optimization, but also mutual optimization to obtain the balance of noise reduction and high-frequency texture restoration during the back propagation process, and the parallel structure makes the inference faster. To verify the effectiveness and robustness of BAM, we applied it to 10 state-of-the-art SISR networks. The results demonstrate that BAM can efficiently improve the networks' performance, and for those originally with attention mechanism, the substitution with BAM further reduces the amount of parameters and increases the inference speed. Information multi-distillation network (IMDN), a representative lightweight SISR network with attention, when the input image size is 200 x 200, the FPS of proposed IMDN-BAM precedes IMDN {8.1%, 8.7%, 8.8%} under the three SR magnifications of x 2, x 3, x 4, respectively. Densely residual Laplacian network (DRLN), a representative heavyweight SISR network with attention, when the scale is 60 x 60, the proposed DRLN-BAM is {11.0%, 8.8%, 10.1%} faster than DRLN under x 2, x 3, x 4. Moreover, we present a dataset with rich texture aliasing regions in real scenes, named realSR7. Experiments prove that BAM achieves better super-resolution results on the aliasing area.
引用
收藏
页码:941 / 955
页数:15
相关论文
共 50 条
  • [31] Pixel attention convolutional network for image super-resolution
    Xin Wang
    Shufen Zhang
    Yuanyuan Lin
    Yanxia Lyu
    Jiale Zhang
    Neural Computing and Applications, 2023, 35 : 8589 - 8599
  • [32] Feature Fusion Attention Network for Image Super-resolution
    Zhou D.-W.
    Ma L.-Y.
    Tian J.-Y.
    Sun X.-X.
    Zidonghua Xuebao/Acta Automatica Sinica, 2022, 48 (09): : 2233 - 2241
  • [33] Single-Image Super-Resolution: A Survey
    Yao, Tingting
    Luo, Yu
    Chen, Yantong
    Yang, Dongqiao
    Zhao, Lei
    COMMUNICATIONS, SIGNAL PROCESSING, AND SYSTEMS, CSPS 2018, VOL II: SIGNAL PROCESSING, 2020, 516 : 119 - 125
  • [34] Learn to Zoom in Single Image Super-Resolution
    Zhang, Zili
    Favaro, Paolo
    Tian, Yan
    Li, Jianxiang
    IEEE SIGNAL PROCESSING LETTERS, 2022, 29 : 1237 - 1241
  • [35] Single-Image Super-Resolution: A Benchmark
    Yang, Chih-Yuan
    Ma, Chao
    Yang, Ming-Hsuan
    COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 : 372 - 386
  • [36] Gradient boosting for single image super-resolution
    Xiong, Dongping
    Gui, Qiuling
    Hou, Wenguang
    Ding, Mingyue
    INFORMATION SCIENCES, 2018, 454 : 328 - 343
  • [37] Single Image Super-Resolution Using ConvNeXt
    You, Chenghui
    Hong, Chaoqun
    Liu, Lijuan
    Lin, Xuehan
    2022 IEEE INTERNATIONAL CONFERENCE ON VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2022,
  • [38] Revealing the Dark Side of Non-Local Attention in Single Image Super-Resolution
    Su, Jian-Nan
    Fan, Guodong
    Gan, Min
    Chen, Guang-Yong
    Guo, Wenzhong
    Chen, C. L. Philip
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2024, 46 (12) : 11476 - 11490
  • [39] A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution
    Park, Karam
    Soh, Jae Woong
    Cho, Nam Ik
    IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 907 - 918
  • [40] Single Image Super-Resolution: Depthwise Separable Convolution Super-Resolution Generative Adversarial Network
    Jiang, Zetao
    Huang, Yongsong
    Hu, Lirui
    APPLIED SCIENCES-BASEL, 2020, 10 (01):