Scale-Aware Detailed Matching for Few-Shot Aerial Image Semantic Segmentation

被引:65
作者
Yao, Xiwen [1 ]
Cao, Qinglong [1 ]
Feng, Xiaoxu [1 ]
Cheng, Gong [1 ]
Han, Junwei [1 ]
机构
[1] Northwestern Polytech Univ, Sch Automat, Xian 710072, Peoples R China
来源
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING | 2022年 / 60卷
基金
中国博士后科学基金; 美国国家科学基金会;
关键词
Detailed matching match; few-shot semantic segmentation; scale-aware focal loss; RESOLUTION; CLASSIFICATION; NETWORK; FUSION; FOREST; LIDAR;
D O I
10.1109/TGRS.2021.3119852
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Few-shot semantic segmentation, aiming to segment query images with a few annotated support samples, has drawn increasing attention. Most existing few-shot methods leverage the single prototype obtained from global average pooling to represent all support information and further use the extracted prototype to segment the query images in a matching manner. Although promising results for natural images have been reported, these methods cannot be directly applied on aerial images. The main reason comes from that the extracted single support prototype can only provide a coarse guidance for matching between query and support images and could not handle the large variance of objects' appearances and scales. To deal with these challenges on aerial images, we propose a scale-aware few-shot semantic segmentation network to perform detailed matching with multiple prototypes. More specifically, the detailed matching module is first constructed to compute the pixel-level similarity between the query features and the extracted multiple support prototypes for providing more accurate parsing guidance. Subsequently, to address the problem of scale imbalance, the scale-aware focal loss is designed to dynamically down-weight the loss assigned to large well-parsed objects and focus training on tiny hard-parsed objects. To facilitate the reproducible research on the task of few-shot semantic segmentation in aerial images, we further provide a few-shot segmentation benchmark iSAID-5i constructed from the large-scale iSAID dataset [1]. Comprehensive experiments and comparisons with the state-of-the-art few-shot segmentation methods on the iSAID-5i dataset clearly demonstrate the superiority of our proposed method. The code and dataset are available at https://github.com/caoql98/SDM.
引用
收藏
页数:11
相关论文
共 42 条
[1]  
[Anonymous], 2014, COMPUT RES REPOSITOR
[2]  
[Anonymous], 2018, P C AR J GEOSC
[3]   Joint Learning from Earth Observation and OpenStreetMap Data to Get Faster Better Semantic Maps [J].
Audebert, Nicolas ;
Le Saux, Bertrand ;
Lefevre, Sebastien .
2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2017, :1552-1560
[4]  
Chaudhuri B, 2018, IEEE T GEOSCI REMOTE, V56, P1144, DOI [10.1109/TGRS.2017.2760909, 10.1109/tgrs.2017.2760909]
[5]   DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs [J].
Chen, Liang-Chieh ;
Papandreou, George ;
Kokkinos, Iasonas ;
Murphy, Kevin ;
Yuille, Alan L. .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2018, 40 (04) :834-848
[6]   When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs [J].
Cheng, Gong ;
Yang, Ceyuan ;
Yao, Xiwen ;
Guo, Lei ;
Han, Junwei .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2018, 56 (05) :2811-2821
[7]   Tree species classification in the Southern Alps based on the fusion of very high geometrical resolution multispectral/hyperspectral images and LiDAR data [J].
Dalponte, Michele ;
Bruzzone, Lorenzo ;
Gianelle, Damiano .
REMOTE SENSING OF ENVIRONMENT, 2012, 123 :258-270
[8]   Semantic Segmentation of Large-Size VHR Remote Sensing Images Using a Two-Stage Multiscale Training Architecture [J].
Ding, Lei ;
Zhang, Jing ;
Bruzzone, Lorenzo .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2020, 58 (08) :5367-5376
[9]   LANet: Local Attention Embedding to Improve the Semantic Segmentation of Remote Sensing Images [J].
Ding, Lei ;
Tang, Hao ;
Bruzzone, Lorenzo .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2021, 59 (01) :426-435
[10]   Deep Residual Learning for Image Recognition [J].
He, Kaiming ;
Zhang, Xiangyu ;
Ren, Shaoqing ;
Sun, Jian .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778