Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multi-label aerial image classification

被引:137
作者
Hua, Yuansheng
Mou, Lichao
Zhu, Xiao Xiang [1 ]
机构
[1] German Aerosp Ctr DLR, Remote Sensing Technol Inst IMF, D-82234 Oberpfaffenhofen, Wessling, Germany
基金
欧洲研究理事会;
关键词
Multi-label classification; High-resolution aerial image; Convolutional Neural Network (CNN) I Class; Attention Learning; Bidirectional Long Short-Term Memory (BiLSTM); Class dependency; HIGH-RESOLUTION; SCENE CLASSIFICATION; NEURAL-NETWORKS; UAV; RGB;
D O I
10.1016/j.isprsjprs.2019.01.015
中图分类号
P9 [自然地理学];
学科分类号
0705 ; 070501 ;
摘要
Aerial image classification is of great significance in the remote sensing community, and many researches have been conducted over the past few years. Among these studies, most of them focus on categorizing an image into one semantic label, while in the real world, an aerial image is often associated with multiple labels, e.g., multiple object-level labels in our case. Besides, a comprehensive picture of present objects in a given high-resolution aerial image can provide a more in-depth understanding of the studied region. For these reasons, aerial image multi-label classification has been attracting increasing attention. However, one common limitation shared by existing methods in the community is that the co-occurrence relationship of various classes, so-called class dependency, is underexplored and leads to an inconsiderate decision. In this paper, we propose a novel end-to end network, namely class-wise attention-based convolutional and bidirectional LSTM network (CA-Cony-BiLSTM), for this task. The proposed network consists of three indispensable components: (1) a feature extraction module, (2) a class attention learning layer, and (3) a bidirectional LSTM-based sub-network. Particularly, the feature extraction module is designed for extracting fine-grained semantic feature maps, while the class attention learning layer aims at capturing discriminative class-specific features. As the most important part, the bidirectional LSTM-based sub-network models the underlying class dependency in both directions and produce structured multiple object labels. Experimental results on UCM multi-label dataset and DFC15 multi label dataset validate the effectiveness of our model quantitatively and qualitatively.
引用
收藏
页码:188 / 199
页数:12
相关论文
共 59 条
[11]  
Graves A, 2013, ARXIV
[12]   Deep Residual Learning for Image Recognition [J].
He, Kaiming ;
Zhang, Xiangyu ;
Ren, Shaoqing ;
Sun, Jian .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778
[13]  
Hu F., 2018, IEEE INT GEOSC REM S
[14]   Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery [J].
Hu, Fan ;
Xia, Gui-Song ;
Hu, Jingwen ;
Zhang, Liangpei .
REMOTE SENSING, 2015, 7 (11) :14680-14707
[15]  
Hua YS, 2018, INT GEOSCI REMOTE SE, P4728, DOI 10.1109/IGARSS.2018.8519576
[16]   Angular difference feature extraction for urban scene classification using ZY-3 multi-angle high-resolution satellite imagery [J].
Huang, Xin ;
Chen, Huijun ;
Gong, Jianya .
ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, 2018, 135 :127-141
[17]  
Kang J., 2018, BUILDING INSTANCE CL
[18]   Land Classification Using Remotely Sensed Data: Going Multilabel [J].
Karalas, Konstantinos ;
Tsagkatakis, Grigorios ;
Zervakis, Michael ;
Tsakalides, Panagiotis .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2016, 54 (06) :3548-3563
[19]   Spatial and Structured SVM for Multilabel Image Classification [J].
Koda, Satoru ;
Zeggada, Abdallah ;
Melgani, Farid ;
Nishii, Ryuei .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2018, 56 (10) :5948-5960
[20]   HSF-Net: Multiscale Deep Feature Embedding for Ship Detection in Optical Remote Sensing Imagery [J].
Li, Qingpeng ;
Mou, Lichao ;
Liu, Qingjie ;
Wang, Yunhong ;
Zhu, Xiao Xiang .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2018, 56 (12) :7147-7161