Reinforced Swin-Convs Transformer for Simultaneous Underwater Sensing Scene Image Enhancement and Super-resolution

被引:96
作者
Ren, Tingdi [1 ]
Xu, Haiyong [1 ]
Jiang, Gangyi [2 ]
Yu, Mei [2 ]
Zhang, Xuan [1 ]
Wang, Biao [1 ]
Luo, Ting [2 ]
机构
[1] Ningbo Univ, Sch Math & Stat, Ningbo 315211, Peoples R China
[2] Ningbo Univ, Fac Informat Sci & Engn, Ningbo 315211, Peoples R China
来源
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING | 2022年 / 60卷
基金
浙江省自然科学基金;
关键词
Transformers; Atmospheric modeling; Generative adversarial networks; Image resolution; Image enhancement; Convolutional neural networks; Superresolution; Super-resolution (SR); Swin-Convs Transformer; U-Net; underwater image enhancement (UIE);
D O I
10.1109/TGRS.2022.3205061
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Underwater image enhancement (UIE) technology aims to tackle the challenge of restoring the degraded underwater images due to light absorption and scattering. Meanwhile, the ever-increasing requirement for higher resolution images from a lower resolution in the underwater domain cannot be overlooked. To address these problems, a novel U-Net-based reinforced Swin-Convs Transformer for simultaneous enhancement and superresolution (URSCT-SESR) method is proposed. Specifically, with the deficiency of U-Net based on pure convolutions, the Swin Transformer is embedded into U-Net for improving the ability to capture the global dependence. Then, given the inadequacy of the Swin Transformer capturing the local attention, the reintroduction of convolutions may capture more local attention. Thus, an ingenious manner is presented for the fusion of convolutions and the core attention mechanism to build a reinforced Swin-Convs Transformer block (RSCTB) for capturing more local attention, which is reinforced in the channel and the spatial attention of the Swin Transformer. Finally, experimental results on available datasets demonstrate that the proposed URSCT-SESR achieves the state-of-the-art performance compared with other methods in terms of both subjective and objective evaluations. The code is publicly available at https://github.com/TingdiRen/URSCT-SESR.
引用
收藏
页数:16
相关论文
共 69 条
[31]   Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network [J].
Ledig, Christian ;
Theis, Lucas ;
Huszar, Ferenc ;
Caballero, Jose ;
Cunningham, Andrew ;
Acosta, Alejandro ;
Aitken, Andrew ;
Tejani, Alykhan ;
Totz, Johannes ;
Wang, Zehan ;
Shi, Wenzhe .
30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :105-114
[32]   Underwater Image Enhancement via Medium Transmission-Guided Multi-Color Space Embedding [J].
Li, Chongyi ;
Anwar, Saeed ;
Hou, Junhui ;
Cong, Runmin ;
Guo, Chunle ;
Ren, Wenqi .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 :4985-5000
[33]   An Underwater Image Enhancement Benchmark Dataset and Beyond [J].
Li, Chongyi ;
Guo, Chunle ;
Ren, Wenqi ;
Cong, Runmin ;
Hou, Junhui ;
Kwong, Sam ;
Tao, Dacheng .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2020, 29 :4376-4389
[34]   Underwater scene prior inspired deep underwater image and video enhancement [J].
Li, Chongyi ;
Anwar, Saeed ;
Porikli, Fatih .
PATTERN RECOGNITION, 2020, 98
[35]  
Li CY, 2016, INT CONF ACOUST SPEE, P1731, DOI 10.1109/ICASSP.2016.7471973
[36]   WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images [J].
Li, Jie ;
Skinner, Katherine A. ;
Eustice, Ryan M. ;
Johnson-Roberson, Matthew .
IEEE ROBOTICS AND AUTOMATION LETTERS, 2018, 3 (01) :387-394
[37]   FG-SRGAN: A Feature-Guided Super-Resolution Generative Adversarial Network for Unpaired Image Super-Resolution [J].
Lian, Shuailong ;
Zhou, Hejian ;
Sun, Yi .
ADVANCES IN NEURAL NETWORKS - ISNN 2019, PT I, 2019, 11554 :151-161
[38]   SwinIR: Image Restoration Using Swin Transformer [J].
Liang, Jingyun ;
Cao, Jiezhang ;
Sun, Guolei ;
Zhang, Kai ;
Van Gool, Luc ;
Timofte, Radu .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), 2021, :1833-1844
[39]   Enhanced Deep Residual Networks for Single Image Super-Resolution [J].
Lim, Bee ;
Son, Sanghyun ;
Kim, Heewon ;
Nah, Seungjun ;
Lee, Kyoung Mu .
2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2017, :1132-1140
[40]  
Lin H., 2021, arXiv, DOI DOI 10.48550/ARXIV.2106.05786