Asymmetric Cross-Attention Hierarchical Network Based on CNN and Transformer for Bitemporal Remote Sensing Images Change Detection

被引:95
作者
Zhang, Xiaofeng [1 ]
Cheng, Shuli [1 ,2 ]
Wang, Liejun [1 ]
Li, Haojin [1 ]
机构
[1] Xinjiang Univ, Coll Informat Sci & Engn, Urumqi 830046, Peoples R China
[2] Xinjiang Univ, Coll Math & Syst Sci, Urumqi 830046, Peoples R China
来源
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING | 2023年 / 61卷
基金
美国国家科学基金会;
关键词
Transformers; Feature extraction; Convolutional neural networks; Task analysis; Remote sensing; Semantics; Fuses; Asymmetric cross-attention; change detection (CD); convolutional neural network (CNN); deep learning (DL); transformer; BUILDING CHANGE DETECTION;
D O I
10.1109/TGRS.2023.3245674
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
As an important task in the field of remote sensing (RS) image processing, RS image change detection (CD) has made significant advances through the use of convolutional neural networks (CNNs). The transformer has recently been introduced into the field of CD due to its excellent global perception capabilities. Some works have attempted to combine CNN and transformer to jointly harvest local-global features; however, these works have not paid much attention to the interaction between the features extracted by both. Also, the use of the transformer has resulted in significant resource consumption. In this article, we propose the Asymmetric Cross-attention Hierarchical Network (ACAHNet) by combining CNN and transformer in a series-parallel manner. The proposed Asymmetric Multiheaded Cross Attention (AMCA) module reduces the quadratic computational complexity of the transformer to linear, and the module enhances the interaction between features extracted from the CNN and the transformer. Different from the early and late fusion strategies employed in previous work, the effectiveness of the mid-term fusion strategy employed by ACAHNet shows a new choice of timing for feature fusion in the CD task. Our experiments on the proposed method on three public datasets show that our network has a better performance in terms of effectiveness and computational resource consumption compared to other comparative methods.
引用
收藏
页数:15
相关论文
共 57 条
[1]   Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing and Applications for Agriculture and Forestry [J].
Adao, Telmo ;
Hruska, Jonas ;
Padua, Luis ;
Bessa, Jose ;
Peres, Emanuel ;
Morais, Raul ;
Sousa, Joaquim Joao .
REMOTE SENSING, 2017, 9 (11)
[2]   Extending post-classification change detection using semantic similarity metrics to overcome class heterogeneity: A study of 1992 and 2001 US National Land Cover Database changes [J].
Ahlqvist, OlA .
REMOTE SENSING OF ENVIRONMENT, 2008, 112 (03) :1226-1241
[3]   Street-view change detection with deconvolutional networks [J].
Alcantarilla, Pablo F. ;
Stent, Simon ;
Ros, German ;
Arroyo, Roberto ;
Gherardi, Riccardo .
AUTONOMOUS ROBOTS, 2018, 42 (07) :1301-1322
[4]   An Image Change Detection Algorithm Based on Multi-Feature Self-Attention Fusion Mechanism UNet Network [J].
Alimjan, Gulnaz ;
Jiaermuhamaiti, Yiliyaer ;
Jumahong, Huxidan ;
Zhu, Shuangling ;
Nurmamat, Pazilat .
INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE, 2021, 35 (14)
[5]   Edge-Guided Recurrent Convolutional Neural Network for Multitemporal Remote Sensing Image Building Change Detection [J].
Bai, Beifang ;
Fu, Wei ;
Lu, Ting ;
Li, Shutao .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2022, 60
[6]  
Bandara WGC, 2022, Arxiv, DOI arXiv:2201.01293
[7]   Multi-Decadal Mangrove Forest Change Detection and Prediction in Honduras, Central America, with Landsat Imagery and a Markov Chain Model [J].
Chen, Chi-Farn ;
Nguyen-Thanh Son ;
Chang, Ni-Bin ;
Chen, Cheng-Ru ;
Chang, Li-Yu ;
Valdez, Miguel ;
Centeno, Gustavo ;
Thompson, Carlos Alberto ;
Aceituno, Jorge Luis .
REMOTE SENSING, 2013, 5 (12) :6408-6426
[8]   Remote Sensing Image Change Detection With Transformers [J].
Chen, Hao ;
Qi, Zipeng ;
Shi, Zhenwei .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2022, 60
[9]   A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection [J].
Chen, Hao ;
Shi, Zhenwei .
REMOTE SENSING, 2020, 12 (10)
[10]   Change Detection in Multisource VHR Images via Deep Siamese Convolutional Multiple-Layers Recurrent Neural Network [J].
Chen, Hongruixuan ;
Wu, Chen ;
Du, Bo ;
Zhang, Liangpei ;
Wang, Le .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2020, 58 (04) :2848-2864