An attention-based multiscale transformer network for remote sensing image change detection

被引:86
作者
Liu, Wei [1 ,2 ]
Lin, Yiyuan [1 ]
Liu, Weijia [1 ]
Yu, Yongtao [3 ]
Li, Jonathan [4 ]
机构
[1] East China Jiaotong Univ, Sch Software, Nanchang 330013, Peoples R China
[2] Thinvent Digital Technol Co Ltd, Nanchang 330096, Peoples R China
[3] Huaiyin Inst Technol, Fac Comp & Software Engn, Huaian 223003, Peoples R China
[4] Univ Waterloo, Dept Geog & Environm Management, Waterloo, ON N2L 3G1, Canada
基金
中国国家自然科学基金;
关键词
Change detection; Attention mechanism; Transformer; Multiscale; CNN;
D O I
10.1016/j.isprsjprs.2023.07.001
中图分类号
P9 [自然地理学];
学科分类号
0705 ; 070501 ;
摘要
The bi-temporal change detection (CD) is still challenging for high-resolution optical remote sensing data analysis due to various factors such as complex textures, seasonal variations, climate changes, and new requirements. We propose an attention-based multiscale transformer network (AMTNet) that utilizes a CNN transformer structure to address this issue. Our Siamese network based on the CNN-transformer architecture uses ConvNets as the backbone to extract multiscale features from the raw input image pair. We then employ attention and transformer modules to model contextual information in bi-temporal images effectively. Additionally, we use feature exchange to bridge the domain gap between different temporal image domains by partially exchanging features between the two Siamese branches of our AMTNet. Experimental results on four commonly used CD datasets - CLCD, HRSCD, WHU-CD, and LEVIR-CD - demonstrate the effectiveness and efficiency of our proposed AMTNet approach. The code for this work will be available on GitHub.1
引用
收藏
页码:599 / 609
页数:11
相关论文
共 48 条
[1]   A TRANSFORMER-BASED SIAMESE NETWORK FOR CHANGE DETECTION [J].
Bandara, Wele Gedara Chaminda ;
Patel, Vishal M. .
2022 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS 2022), 2022, :207-210
[2]   PPCNET: A Combined Patch-Level and Pixel-Level End-to-End Deep Network for High-Resolution Remote Sensing Image Change Detection [J].
Bao, Tengfei ;
Fu, Chenqin ;
Fang, Tao ;
Huo, Hong .
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2020, 17 (10) :1797-1801
[3]   Vision Transformers for Remote Sensing Image Classification [J].
Bazi, Yakoub ;
Bashmal, Laila ;
Rahhal, Mohamad M. Al ;
Dayil, Reham Al ;
Ajlan, Naif Al .
REMOTE SENSING, 2021, 13 (03) :1-20
[4]   End-to-End Object Detection with Transformers [J].
Carion, Nicolas ;
Massa, Francisco ;
Synnaeve, Gabriel ;
Usunier, Nicolas ;
Kirillov, Alexander ;
Zagoruyko, Sergey .
COMPUTER VISION - ECCV 2020, PT I, 2020, 12346 :213-229
[5]   Remote Sensing Image Change Detection With Transformers [J].
Chen, Hao ;
Qi, Zipeng ;
Shi, Zhenwei .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2022, 60
[6]   A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection [J].
Chen, Hao ;
Shi, Zhenwei .
REMOTE SENSING, 2020, 12 (10)
[7]   Deep Siamese Multi-scale Convolutional Network for Change Detection in Multi-temporal VHR Images [J].
Chen, Hongruixuan ;
Wu, Chen ;
Du, Bo ;
Zhang, Liangpei .
2019 10TH INTERNATIONAL WORKSHOP ON THE ANALYSIS OF MULTITEMPORAL REMOTE SENSING IMAGES (MULTITEMP), 2019,
[8]   Change Detection in Multisource VHR Images via Deep Siamese Convolutional Multiple-Layers Recurrent Neural Network [J].
Chen, Hongruixuan ;
Wu, Chen ;
Du, Bo ;
Zhang, Liangpei ;
Wang, Le .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2020, 58 (04) :2848-2864
[9]   DASNet: Dual Attentive Fully Convolutional Siamese Networks for Change Detection in High-Resolution Satellite Images [J].
Chen, Jie ;
Yuan, Ziyang ;
Peng, Jian ;
Chen, Li ;
Huang, Haozhe ;
Zhu, Jiawei ;
Liu, Yu ;
Li, Haifeng .
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2021, 14 :1194-1206
[10]   FCCDN: Feature constraint network for VHR image change detection [J].
Chen, Pan ;
Zhang, Bing ;
Hong, Danfeng ;
Chen, Zhengchao ;
Yang, Xuan ;
Li, Baipeng .
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2022, 187 :101-119