Reinforced Swin-Convs Transformer for Simultaneous Underwater Sensing Scene Image Enhancement and Super-resolution

被引:96
作者
Ren, Tingdi [1 ]
Xu, Haiyong [1 ]
Jiang, Gangyi [2 ]
Yu, Mei [2 ]
Zhang, Xuan [1 ]
Wang, Biao [1 ]
Luo, Ting [2 ]
机构
[1] Ningbo Univ, Sch Math & Stat, Ningbo 315211, Peoples R China
[2] Ningbo Univ, Fac Informat Sci & Engn, Ningbo 315211, Peoples R China
来源
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING | 2022年 / 60卷
基金
浙江省自然科学基金;
关键词
Transformers; Atmospheric modeling; Generative adversarial networks; Image resolution; Image enhancement; Convolutional neural networks; Superresolution; Super-resolution (SR); Swin-Convs Transformer; U-Net; underwater image enhancement (UIE);
D O I
10.1109/TGRS.2022.3205061
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Underwater image enhancement (UIE) technology aims to tackle the challenge of restoring the degraded underwater images due to light absorption and scattering. Meanwhile, the ever-increasing requirement for higher resolution images from a lower resolution in the underwater domain cannot be overlooked. To address these problems, a novel U-Net-based reinforced Swin-Convs Transformer for simultaneous enhancement and superresolution (URSCT-SESR) method is proposed. Specifically, with the deficiency of U-Net based on pure convolutions, the Swin Transformer is embedded into U-Net for improving the ability to capture the global dependence. Then, given the inadequacy of the Swin Transformer capturing the local attention, the reintroduction of convolutions may capture more local attention. Thus, an ingenious manner is presented for the fusion of convolutions and the core attention mechanism to build a reinforced Swin-Convs Transformer block (RSCTB) for capturing more local attention, which is reinforced in the channel and the spatial attention of the Swin Transformer. Finally, experimental results on available datasets demonstrate that the proposed URSCT-SESR achieves the state-of-the-art performance compared with other methods in terms of both subjective and objective evaluations. The code is publicly available at https://github.com/TingdiRen/URSCT-SESR.
引用
收藏
页数:16
相关论文
共 69 条
[41]   A Probabilistic Model Based on Bipartite Convolutional Neural Network for Unsupervised Change Detection [J].
Liu, Jia ;
Zhang, Wenhua ;
Liu, Fang ;
Xiao, Liang .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2022, 60
[42]   Swin Transformer: Hierarchical Vision Transformer using Shifted Windows [J].
Liu, Ze ;
Lin, Yutong ;
Cao, Yue ;
Hu, Han ;
Wei, Yixuan ;
Zhang, Zheng ;
Lin, Stephen ;
Guo, Baining .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :9992-10002
[43]  
Loshchilov I, 2019, Arxiv, DOI [arXiv:1711.05101, 10.48550/arXiv.1711.05101]
[44]   An efficient learning-based method for underwater image enhancement [J].
Lyu, Zhangkai ;
Peng, Andrew ;
Wang, Qingwei ;
Ding, Dandan .
DISPLAYS, 2022, 74
[45]  
Mathieu M, 2016, Arxiv, DOI [arXiv:1511.05440, 10.48550/arXiv.1511.05440, DOI 10.48550/ARXIV.1511.05440]
[46]   Human-Visual-System-Inspired Underwater Image Quality Measures [J].
Panetta, Karen ;
Gao, Chen ;
Agaian, Sos .
IEEE JOURNAL OF OCEANIC ENGINEERING, 2016, 41 (03) :541-551
[47]  
Peng LT, 2022, Arxiv, DOI arXiv:2111.11843
[48]   Generalization of the Dark Channel Prior for Single Image Restoration [J].
Peng, Yan-Tsung ;
Cao, Keming ;
Cosman, Pamela C. .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2018, 27 (06) :2856-2868
[49]  
Raffel C, 2023, Arxiv, DOI [arXiv:1910.10683, 10.48550/arXiv.1910.10683, DOI 10.48550/ARXIV.1910.10683]
[50]   U-Net: Convolutional Networks for Biomedical Image Segmentation [J].
Ronneberger, Olaf ;
Fischer, Philipp ;
Brox, Thomas .
MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION, PT III, 2015, 9351 :234-241