Unsupervised Cycle-Consistent Generative Adversarial Networks for Pan Sharpening

被引:44
作者
Zhou, Huanyu [1 ]
Liu, Qingjie [1 ]
Weng, Dawei [1 ]
Wang, Yunhong [1 ,2 ]
机构
[1] Beihang Univ, Hangzhou Innovat Inst, State Key Lab Virtual Real Technol & Syst, Beijing 100191, Peoples R China
[2] Capital Med Univ, Sch Biomed Engn, Beijing 100054, Peoples R China
来源
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING | 2022年 / 60卷
关键词
Feature extraction; Deep learning; Training; Task analysis; Generators; Generative adversarial networks; Spatial resolution; Cycle consistency; generative adversarial network (GAN); image fusion; pan sharpening; unsupervised learning; FUSION TECHNIQUE; IMAGE FUSION; QUALITY;
D O I
10.1109/TGRS.2022.3166528
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Deep learning-based pan sharpening has received significant research interest in recent years. Most of the existing methods fall into the supervised learning framework in which they downsample the multispectral (MS) and panchromatic (PAN) images and regard the original MS images as ground truths to form training samples based on Wald's protocol. Although impressive performance could be achieved, they have difficulties when generalizing to the original full-scale images due to the scale gap, which makes them lack of practicability. In this article, we propose an unsupervised generative adversarial framework that learns from the full-scale images without the ground truths to alleviate this problem. We first extract the modality-specific features from the PAN and MS images with a two-stream generator, perform fusion in the feature domain, and then reconstruct the pan-sharpened images. Furthermore, we introduce a novel hybrid loss based on the cycle-consistency and adversarial scheme to improve the performance. Comparison experiments with the state-of-the-art methods are conducted on GaoFen-2 (GF-2) and WorldView-3 satellites. Results demonstrate that the proposed method can greatly improve the pan-sharpening performance on the full-scale images, which clearly shows its practical value. Codes are available at https://github.com/zhysora/UCGAN.
引用
收藏
页数:14
相关论文
共 50 条
  • [21] PSGAN: A GENERATIVE ADVERSARIAL NETWORK FOR REMOTE SENSING IMAGE PAN-SHARPENING
    Liu, Xiangyu
    Wang, Yunhong
    Liu, Qingjie
    2018 25TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2018, : 873 - 877
  • [22] Deep cycle autoencoder for unsupervised domain adaptation with generative adversarial networks
    Zhou, Qiang
    Zhou, Wen'an
    Yang, Bin
    Huan, Jun
    IET COMPUTER VISION, 2019, 13 (07) : 659 - 665
  • [23] Underwater Image Color Correction and Enhancement Based on Improved Cycle-consistent Generative Adversarial Networks
    Li Q.-Z.
    Bai W.-X.
    Niu J.
    Zidonghua Xuebao/Acta Automatica Sinica, 2023, 49 (04): : 820 - 829
  • [24] Unsupervised Standard Plane Synthesis in Population Cine MRI via Cycle-Consistent Adversarial Networks
    Zhang, Le
    Pereanez, Marco
    Bowles, Christopher
    Piechnik, Stefan K.
    Neubauer, Stefan
    Petersen, Steffen E.
    Frangi, Alejandro F.
    MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION - MICCAI 2019, PT II, 2019, 11765 : 660 - 668
  • [25] GRAY-SCALE IMAGE COLORIZATION USING CYCLE-CONSISTENT GENERATIVE ADVERSARIAL NETWORKS WITH RESIDUAL STRUCTURE ENHANCER
    Johari, Mohammad Mahdi
    Behroozi, Hamid
    2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2020, : 2223 - 2227
  • [26] Seismic impedance inversion based on cycle-consistent generative adversarial network
    Wang, Yu-Qing
    Wang, Qi
    Lu, Wen-Kai
    Ge, Qiang
    Yan, Xin-Fei
    PETROLEUM SCIENCE, 2022, 19 (01) : 147 - 161
  • [27] Image-to-image Translation Based on Improved Cycle-consistent Generative Adversarial Network
    Zhang Jinglei
    Hou Yawei
    JOURNAL OF ELECTRONICS & INFORMATION TECHNOLOGY, 2020, 42 (05) : 1216 - 1222
  • [28] Machinery Health Monitoring Based on Unsupervised Feature Learning via Generative Adversarial Networks
    Dai, Jun
    Wang, Jun
    Huang, Weiguo
    Shi, Juanjuan
    Zhu, Zhongkui
    IEEE-ASME TRANSACTIONS ON MECHATRONICS, 2020, 25 (05) : 2252 - 2263
  • [29] Unsupervised image-to-image translation via long-short cycle-consistent adversarial networks
    Gang Wang
    Haibo Shi
    Yufei Chen
    Bin Wu
    Applied Intelligence, 2023, 53 : 17243 - 17259
  • [30] Unsupervised Deformable Image Registration Using Cycle-Consistent CNN
    Kim, Boah
    Kim, Jieun
    Lee, June-Goo
    Kim, Dong Hwan
    Park, Seong Ho
    Ye, Jong Chul
    MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION - MICCAI 2019, PT VI, 2019, 11769 : 166 - 174