An Optical Image-Aided Approach for Zero-Shot SAR Image Scene Classification

Scene classification is one of the most significant tasks in synthetic aperture radar (SAR) image interpretation. However, most existing SAR image scene classification methods cannot effectively identify the scene categories without training samples, which seriously affects the classification performance of these unseen categories. It is an effective way to solve this problem by extracting information from easily accessible other-source aided information to assist SAR scene classification of unseen categories. To this end, a framework of optical image-aided zero-shot SAR image scene classification is established, including feature extraction, joint feature compatibility and calibration classification module. Specifically, the feature extraction module is employed to sufficiently extract features from optical and SAR images. The joint feature compatibility module can maximize the compatibility between extracted features. Based on the compatibility score, the calibration classification module combines superposition calibration and one-versus-all classifier, and finally achieves good performance in classification for zero-shot SAR scene. Experimental results based on multi-modal remote sensing scene classification (MRSSC) dataset have shown the superiority of the proposed method on zero-shot SAR image scene classification.