Conditional Diffusion for SAR to Optical Image Translation

被引:17
作者
Bai, Xinyu [1 ]
Pu, Xinyang [1 ]
Xu, Feng [1 ]
机构
[1] Fudan Univ, Sch Informat Sci & Technol, Key Lab Informat Sci Electromagnet Waves, Minist Educ, Shanghai 200232, Peoples R China
关键词
Radar polarimetry; Optical imaging; Optical sensors; Task analysis; Optical noise; Computational modeling; Training; Diffusion model; image translation; synthetic aperture radar (SAR);
D O I
10.1109/LGRS.2023.3337143
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Synthetic aperture radar (SAR) offers all-weather and all-day high-resolution imaging, yet its unique imaging mechanism often necessitates expert interpretation, limiting its broader applicability. Addressing this challenge, this letter proposes a generative model that bridges SAR and optical imaging, facilitating the conversion of SAR images into more human-recognizable optical aerial images. This assists in the interpretation of SAR data, making it easier to recognize. Specifically, our model backbone is based on the recent diffusion models, which have powerful generative capabilities. We have innovatively tailored the diffusion model framework, incorporating SAR images as conditional constraints in the sampling process. This adaptation enables the effective translation from SAR to optical images. We conduct experiments on the satellite GF3 and SEN12 datasets and use structural similarity (SSIM) and Frechet inception distance (FID) for quantitative evaluation. The results show that our model not only surpasses previous methods in quantitative evaluation but also significantly improves the visual quality of the generated images. This advancement underscores the model's potential to enhance SAR image interpretation.
引用
收藏
页码:1 / 5
页数:5
相关论文
共 26 条
[1]  
[Anonymous], 2021, ADV NEURAL INFORM PR, DOI DOI 10.1080/20477724.2021.1951556
[2]   Reusing Discriminators for Encoding: Towards Unsupervised Image-to-Image Translation [J].
Chen, Runfa ;
Huang, Wenbing ;
Huang, Binghui ;
Sun, Fuchun ;
Fang, Bin .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, :8165-8174
[3]   Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Model [J].
Cheng, Shin-I ;
Chen, Yu-Jie ;
Chiu, Wei-Chen ;
Tseng, Hung-Yu ;
Lee, Hsin-Ying .
2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2023, :4043-4051
[4]  
Choi Jooyoung, 2021, ARXIV210802938
[5]   Reciprocal translation between SAR and optical remote sensing images with cascaded-residual adversarial networks [J].
Fu, Shilei ;
Xu, Feng ;
Jin, Ya-Qiu .
SCIENCE CHINA-INFORMATION SCIENCES, 2021, 64 (02)
[6]   Generative Adversarial Networks [J].
Goodfellow, Ian ;
Pouget-Abadie, Jean ;
Mirza, Mehdi ;
Xu, Bing ;
Warde-Farley, David ;
Ozair, Sherjil ;
Courville, Aaron ;
Bengio, Yoshua .
COMMUNICATIONS OF THE ACM, 2020, 63 (11) :139-144
[7]  
Heusel M, 2017, ADV NEUR IN, V30
[8]  
Ho Jonathan., 2020, P 34 INT C NEURAL IN, P6840
[9]  
Hou XY, 2019, INT GEOSCI REMOTE SE, P9486, DOI [10.1109/igarss.2019.8900619, 10.1109/IGARSS.2019.8900619]
[10]   Adaptive Fuzzy Output Feedback Fault-Tolerant Compensation for Uncertain Nonlinear Systems With Infinite Number of Time-Varying Actuator Failures and Full-State Constraints [J].
Jing, Yan-Hui ;
Yang, Guang-Hong .
IEEE TRANSACTIONS ON CYBERNETICS, 2021, 51 (02) :568-578