Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical Matching

被引:2
作者
Qian, Lixin [1 ,2 ]
Liu, Xiaochun [1 ]
Huang, Meiyu [2 ]
Xiang, Xueshuang [2 ]
机构
[1] Wuhan Univ, Sch Math & Stat, Wuhan 430072, Peoples R China
[2] China Acad Space Technol, Qian Xuesen Lab Space Technol, Beijing 100086, Peoples R China
关键词
SAR-optical fusion; image matching; self-supervised learning; representation learning;
D O I
10.3390/rs14122749
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Due to the vast geometric and radiometric differences between SAR and optical images, SAR-optical image matching remains an intractable challenge. Despite the fact that the deep learning-based matching model has achieved great success, SAR feature embedding ability is not fully explored yet because of the lack of well-designed pre-training techniques. In this paper, we propose to employ the self-supervised learning method in the SAR-optical matching framework, in order to serve as a pre-training strategy for improving the representation learning ability of SAR images as well as optical images. We first use a state-of-the-art self-supervised learning method, Momentum Contrast (MoCo), to pre-train an optical feature encoder and an SAR feature encoder separately. Then, the pre-trained encoders are transferred to an advanced common representation learning model, Bridge Neural Network (BNN), to project the SAR and optical images into a more distinguishable common feature representation subspace, which leads to a high multi-modal image matching result. Experimental results on three SAR-optical matching benchmark datasets show that our proposed MoCo pre-training method achieves a high matching accuracy up to 0.873 even for the complex QXS-SAROPT SAR-optical matching dataset. BNN pre-trained with MoCo outperforms BNN with the most commonly used ImageNet pre-training, and achieves at most 4.4% gains in matching accuracy.
引用
收藏
页数:10
相关论文
共 30 条
[1]  
Bao W., 2021, ARXIV, DOI [10.1109/JSTARS.2021.3109002, DOI 10.1109/JSTARS.2021.3109002]
[2]  
Burger W., 2010, PRINCIPLES DIGITAL I
[3]  
Caron M, 2020, ADV NEUR IN, V33
[4]  
Chen T, 2020, PR MACH LEARN RES, V119
[5]   SAR-SIFT: A SIFT-Like Algorithm for SAR Images [J].
Dellinger, Flora ;
Delon, Julie ;
Gousseau, Yann ;
Michel, Julien ;
Tupin, Florence .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2015, 53 (01) :453-466
[6]   Rethinking ImageNet Pre-training [J].
He, Kaiming ;
Girshick, Ross ;
Dollar, Piotr .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :4917-4926
[7]   Deep Residual Learning for Image Recognition [J].
He, Kaiming ;
Zhang, Xiangyu ;
Ren, Shaoqing ;
Sun, Jian .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778
[8]  
Henaff OJ, 2020, PR MACH LEARN RES, V119
[9]  
Huang M., 2021, ARXIV
[10]   Identifying Corresponding Patches in SAR and Optical Images With a Pseudo-Siamese CNN [J].
Hughes, Lloyd H. ;
Schmitt, Michael ;
Mou, Lichao ;
Wang, Yuanyuan ;
Zhu, Xiao Xiang .
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2018, 15 (05) :784-788