MCWESRGAN: Improving Enhanced Super-Resolution Generative Adversarial Network for Satellite Images

被引:7
作者
Karwowska, Kinga [1 ]
Wierzbicki, Damian [1 ]
机构
[1] Mil Univ Technol, Fac Civil Engn & Geodesy, Dept Imagery Intelligence, PL-00908 Warsaw, Poland
关键词
Spatial resolution; Training; Generators; Superresolution; Generative adversarial networks; Satellite images; Computational modeling; Convolutional neural networks; deep learning; enhanced super-resolution generative adversarial network (ESRGAN); neural networks; power spectral density (PSD); single-image super-resolution (SISR); super resolution (SR); RESOLUTION;
D O I
10.1109/JSTARS.2023.3322642
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
With the dynamic technological development, we are witnessing a major progress in solutions that allow for the observation of Earth's surface. Small satellites have a significant drawback. Due to their limitations, the installed optic systems are not perfect. As a result, the quality of the obtained images is lower, including lower resolution, although the satellites move on the Low Earth Orbit. In the case of images lacking a high-resolution counterpart, the spatial resolution of the imagery can be improved using single-image super-resolution algorithms. In this article, we present an SISR solution based on a new network called MCWESRGAN, which is a modification of the popular ESRGAN network. We propose a novel strategy that introduces a multi-column discriminator model. The generator model is trained using Wasserstein loss. The introduced modifications enable a tenfold reduction in the training time of the network. The proposed algorithm is verified using images obtained from space, aerial imagery, and the Dataset for Object deTection in Aerial Images (DOTA) database. A set of evaluation methods for super-resolution (SR) images is proposed to verify the results. These evaluation methods indicate areas that are poorly estimated by the algorithm. Furthermore, as part of the conducted experiments, an absolute assessment method for interpretational potential based on the power spectral density of the image (PSD) is proposed, allowing for determining the magnitude of interpretational improvement after applying resolution enhancement algorithms. The conducted research demonstrates that we achieve better qualitative and quantitative results than classical ESRGAN methods and other state-of-the-art (SOTA) approaches.
引用
收藏
页码:9886 / 9906
页数:21
相关论文
共 74 条
[41]  
Lu Z., 2021, arXiv, DOI 10.48550/arXiv.2108.11084
[42]   NTIRE 2021 Learning the Super-Resolution Space Challenge [J].
Lugmayr, Andreas ;
Danelljan, Martin ;
Timofte, Radu ;
Busch, Christoph ;
Chen, Yang ;
Cheng, Jian ;
Chudasama, Vishal ;
Gang, Ruipeng ;
Gao, Shangqi ;
Gao, Kun ;
Gong, Laiyun ;
Han, Qingrui ;
Huang, Chao ;
Jin, Zhi ;
Jo, Younghyun ;
Kim, Seon Joo ;
Kim, Younggeun ;
Lee, Seungjun ;
Lei, Yuchen ;
Li, Chu-Tak ;
Li, Chenghua ;
Li, Ke ;
Liu, Zhi-Song ;
Liu, Youming ;
Nan, Nan ;
Park, Seung-Ho ;
Patel, Heena ;
Peng, Shichong ;
Prajapati, Kalpesh ;
Qi, Haoran ;
Raja, Kiran ;
Ramachandra, Raghavendra ;
Siu, Wan-Chi ;
Son, Donghee ;
Song, Ruixia ;
Upla, Kishor ;
Wang, Li-Wen ;
Wang, Yatian ;
Wang, Junwei ;
Wu, Qianyu ;
Xu, Xinhua ;
Yang, Sejong ;
Yuan, Zhen ;
Zhang, Liting ;
Zhang, Huanrong ;
Zhang, Junkai ;
Zhang, Yifan ;
Zhang, Zhenzhou ;
Zhou, Hangqi ;
Zhu, Aichun .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2021, 2021, :596-612
[43]  
Martin D, 2001, EIGHTH IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION, VOL II, PROCEEDINGS, P416, DOI 10.1109/ICCV.2001.937655
[44]   Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery [J].
Mookambiga, A. ;
Gomathi, V. .
MULTIDIMENSIONAL SYSTEMS AND SIGNAL PROCESSING, 2016, 27 (04) :863-889
[45]   Automatic SAR Change Detection Based on Visual Saliency and Multi-Hierarchical Fuzzy Clustering [J].
Peng, Yao ;
Bin Cui ;
Yin, Hujun ;
Zhang, Yonghong ;
Du, Peijun .
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2022, 15 :7755-7769
[46]   Occluded Object Detection in High-Resolution Remote Sensing Images Using Partial Configuration Object Model [J].
Qiu, Shaohua ;
Wen, Gongjian ;
Fan, Yaxiang .
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2017, 10 (05) :1909-1925
[47]   Multi-spectral multi-image super-resolution of Sentinel-2 with radiometric consistency losses and its effect on building delineation [J].
Razzak, Muhammed T. ;
Mateo-Garcia, Gonzalo ;
Lecuyer, Gurvan ;
Gomez-Chova, Luis ;
Gal, Yarin ;
Kalaitzis, Freddie .
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2023, 195 :1-13
[48]   Real-World Super-Resolution using Generative Adversarial Networks [J].
Ren, Haoyu ;
Kheradmand, Amin ;
El-Khamy, Mostafa ;
Wang, Shuangquan ;
Bai, Dongwoon ;
Lee, Jungwon .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW 2020), 2020, :1760-1768
[49]  
Shaham TR, 2019, Arxiv, DOI [arXiv:1905.01164, 10.48550/arXiv.1905.01164, DOI 10.48550/ARXIV.1905.01164]
[50]   Super-Resolution of Sentinel-2 Imagery Using Generative Adversarial Networks [J].
Salgueiro Romero, Luis ;
Marcello, Javier ;
Vilaplana, Veronica .
REMOTE SENSING, 2020, 12 (15)