Reinforced Swin-Convs Transformer for Simultaneous Underwater Sensing Scene Image Enhancement and Super-resolution

被引:96
作者
Ren, Tingdi [1 ]
Xu, Haiyong [1 ]
Jiang, Gangyi [2 ]
Yu, Mei [2 ]
Zhang, Xuan [1 ]
Wang, Biao [1 ]
Luo, Ting [2 ]
机构
[1] Ningbo Univ, Sch Math & Stat, Ningbo 315211, Peoples R China
[2] Ningbo Univ, Fac Informat Sci & Engn, Ningbo 315211, Peoples R China
来源
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING | 2022年 / 60卷
基金
浙江省自然科学基金;
关键词
Transformers; Atmospheric modeling; Generative adversarial networks; Image resolution; Image enhancement; Convolutional neural networks; Superresolution; Super-resolution (SR); Swin-Convs Transformer; U-Net; underwater image enhancement (UIE);
D O I
10.1109/TGRS.2022.3205061
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Underwater image enhancement (UIE) technology aims to tackle the challenge of restoring the degraded underwater images due to light absorption and scattering. Meanwhile, the ever-increasing requirement for higher resolution images from a lower resolution in the underwater domain cannot be overlooked. To address these problems, a novel U-Net-based reinforced Swin-Convs Transformer for simultaneous enhancement and superresolution (URSCT-SESR) method is proposed. Specifically, with the deficiency of U-Net based on pure convolutions, the Swin Transformer is embedded into U-Net for improving the ability to capture the global dependence. Then, given the inadequacy of the Swin Transformer capturing the local attention, the reintroduction of convolutions may capture more local attention. Thus, an ingenious manner is presented for the fusion of convolutions and the core attention mechanism to build a reinforced Swin-Convs Transformer block (RSCTB) for capturing more local attention, which is reinforced in the channel and the spatial attention of the Swin Transformer. Finally, experimental results on available datasets demonstrate that the proposed URSCT-SESR achieves the state-of-the-art performance compared with other methods in terms of both subjective and objective evaluations. The code is publicly available at https://github.com/TingdiRen/URSCT-SESR.
引用
收藏
页数:16
相关论文
共 69 条
[1]   What Is the Space of Attenuation Coefficients in Underwater Computer Vision? [J].
Akkaynak, Derya ;
Treibitz, Tali ;
Shlesinger, Tom ;
Tamir, Raz ;
Loya, Yossi ;
Iluz, David .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :568-577
[2]   Localization of Acoustically Tagged Marine Animals in Under-Ranked Conditions [J].
Alexandri, Talmon ;
Shamir, Ziv Zemah ;
Bigal, Eyal ;
Scheinin, Aviad ;
Tchernov, Dan ;
Diamant, Roee .
IEEE TRANSACTIONS ON MOBILE COMPUTING, 2021, 20 (03) :1126-1137
[3]   Applications of Generative Adversarial Networks (GANs): An Updated Review [J].
Alqahtani, Hamed ;
Kavakli-Thorne, Manolya ;
Kumar, Gulshan .
ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING, 2021, 28 (02) :525-552
[4]   Color Balance and Fusion for Underwater Image Enhancement [J].
Ancuti, Codruta O. ;
Ancuti, Cosmin ;
De Vleeschouwer, Christophe ;
Bekaert, Philippe .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2018, 27 (01) :379-393
[5]  
Ancuti C, 2012, PROC CVPR IEEE, P81, DOI 10.1109/CVPR.2012.6247661
[6]   Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset [J].
Berman, Dana ;
Levy, Deborah ;
Avidan, Shai ;
Treibitz, Tali .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2021, 43 (08) :2822-2837
[7]   A NEW COLOR CORRECTION METHOD FOR UNDERWATER IMAGING [J].
Bianco, G. ;
Muzzupappa, M. ;
Bruno, F. ;
Garcia, R. ;
Neumann, L. .
UNDERWATER 3D RECORDING AND MODELING, 2015, 45 (W5) :25-32
[8]  
Cao H., 2021, arXiv
[9]   Real-world single image super-resolution: A brief review [J].
Chen, Honggang ;
He, Xiaohai ;
Qing, Linbo ;
Wu, Yuanyuan ;
Ren, Chao ;
Sheriff, Ray E. ;
Zhu, Ce .
INFORMATION FUSION, 2022, 79 :124-145
[10]  
Chen R., 2022, IEEE T GEOSCI ELECT, V60, P1