Efficient Discrimination and Localization of Multimodal Remote Sensing Images Using CNN-Based Prediction of Localization Uncertainty

被引:12
|
作者
Uss, Mykhail [1 ]
Vozel, Benoit [2 ]
Lukin, Vladimir [1 ]
Chehdi, Kacem [2 ]
机构
[1] Natl Aerosp Univ, Dept Informat Commun Technol, UA-61070 Kharkiv, Ukraine
[2] Univ Rennes 1, IETR UMR CNRS 6164, F-22305 Lannion, France
关键词
multimodal images; remote sensing; similarity measure; localization accuracy; localization uncertainty; regression uncertainty; deep learning; convolutional neural networks; two-channel CNN; REGISTRATION; SIMILARITY;
D O I
10.3390/rs12040703
中图分类号
X [环境科学、安全科学];
学科分类号
08 ; 0830 ;
摘要
Detecting similarities between image patches and measuring their mutual displacement are important parts in the registration of multimodal remote sensing (RS) images. Deep learning approaches advance the discriminative power of learned similarity measures (SM). However, their ability to find the best spatial alignment of the compared patches is often ignored. We propose to unify the patch discrimination and localization problems by assuming that the more accurately two patches can be aligned, the more similar they are. The uncertainty or confidence in the localization of a patch pair serves as a similarity measure of these patches. We train a two-channel patch matching convolutional neural network (CNN), called DLSM, to solve a regression problem with uncertainty. This CNN inputs two multimodal patches, and outputs a prediction of the translation vector between the input patches as well as the uncertainty of this prediction in the form of an error covariance matrix of the translation vector. The proposed patch matching CNN predicts a normal two-dimensional distribution of the translation vector rather than a simple value of it. The determinant of the covariance matrix is used as a measure of uncertainty in the matching of patches and also as a measure of similarity between patches. For training, we used the Siamese architecture with three towers. During training, the input of two towers is the same pair of multimodal patches but shifted by a random translation; the last tower is fed by a pair of dissimilar patches. Experiments performed on a large base of real RS images show that the proposed DLSM has both a higher discriminative power and a more precise localization compared to existing hand-crafted SMs and SMs trained with conventional losses. Unlike existing SMs, DLSM correctly predicts translation error distribution ellipse for different modalities, noise level, isotropic, and anisotropic structures.
引用
收藏
页数:22
相关论文
共 50 条
  • [41] CNN-based Methods for Survival Prediction using CT images for Lung Cancer Patients
    Amaro, Maria
    Oliveira, Helder P.
    Pereira, Tania
    2024 IEEE 37TH INTERNATIONAL SYMPOSIUM ON COMPUTER-BASED MEDICAL SYSTEMS, CBMS 2024, 2024, : 290 - 296
  • [42] LitCall: Learning Implicit Topology for CNN-based Aortic Landmark Localization
    Bian, Zhangxing
    Zhong, Jiayang
    Lu, Yanglong
    Hatt, Chuck R.
    Burris, Nicholas S.
    MEDICAL IMAGING 2022: IMAGE PROCESSING, 2022, 12032
  • [43] DOPNet: Dense Object Prediction Network for Multiclass Object Counting and Localization in Remote Sensing Images
    Cui, Mingpeng
    Ding, Guanchen
    Yang, Daiqin
    Chen, Zhenzhong
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 15
  • [44] Marker-Based Localization System Using an Active PTZ Camera and CNN-Based Ellipse Detection
    Oh, Xueyan
    Lim, Ryan
    Foong, Shaohui
    Tan, U-Xuan
    IEEE-ASME TRANSACTIONS ON MECHATRONICS, 2023, 28 (04) : 1984 - 1992
  • [45] Lightweight and Stable Multi-Feature Databases for Efficient Geometric Localization of Remote Sensing Images
    Zhao, Zilu
    Wang, Feng
    You, Hongjian
    REMOTE SENSING, 2024, 16 (07)
  • [46] Fuel Type Mapping Using a CNN-Based Remote Sensing Approach: A Case Study in Sardinia
    Carbone, Andrea
    Spiller, Dario
    Laneve, Giovanni
    FIRE-SWITZERLAND, 2023, 6 (10):
  • [47] Simultaneous Road Surface and Centerline Extraction From Large-Scale Remote Sensing Images Using CNN-Based Segmentation and Tracing
    Wei, Yao
    Zhang, Kai
    Ji, Shunping
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2020, 58 (12): : 8919 - 8931
  • [48] Detection and localization of splicing on remote sensing images using image-to-image transformation
    Alsughayer, Rawan
    Hussain, Muhammad
    Saeed, Fahman
    AboalSamh, Hatim
    APPLIED INTELLIGENCE, 2023, 53 (11) : 13275 - 13292
  • [49] Detection and localization of splicing on remote sensing images using image-to-image transformation
    Rawan Alsughayer
    Muhammad Hussain
    Fahman Saeed
    Hatim AboalSamh
    Applied Intelligence, 2023, 53 : 13275 - 13292
  • [50] Object Localization in Weakly Labeled Remote Sensing Images Based on Deep Convolutional Features
    Long, Yang
    Zhai, Xiaofang
    Wan, Qiao
    Tan, Xiaowei
    REMOTE SENSING, 2022, 14 (13)