GRiD: Guided Refinement for Detector-Free Multimodal Image Matching

被引:0
作者
Liu, Yuyan [1 ]
He, Wei [1 ]
Zhang, Hongyan [1 ,2 ]
机构
[1] Wuhan Univ, State Key Lab Informat Engn Surveying Mapping & Re, Wuhan 430072, Peoples R China
[2] China Univ Geosci, Sch Comp, Wuhan 430074, Peoples R China
基金
中国国家自然科学基金;
关键词
Feature extraction; Image matching; Transformers; Optical imaging; Detectors; Semantics; Image edge detection; Adaptive optics; Robustness; Remote sensing; detector-free; guided refinement; multimodal images; REGISTRATION; TRANSFORMER; MODEL;
D O I
10.1109/TIP.2024.3472491
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Multimodal image matching is essential in image stitching, image fusion, change detection, and land cover mapping. However, the severe nonlinear radiometric distortion (NRD) and geometric distortions in multimodal images severely limit the accuracy of multimodal image matching, posing significant challenges to existing methods. Additionally, detector-based methods are prone to feature point offset issues in regions with substantial modal differences, which also hinder the subsequent fine registration and fusion of images. To address these challenges, we propose a guided refinement for detector-free multimodal image matching (GRiD) method, which weakens feature point offset issues by establishing pixel-level correspondences and utilizes reference points to guide and correct matches affected by NRD and geometric distortions. Specifically, we first introduce a detector-free framework to alleviate the feature point offset problem by directly finding corresponding pixels between images. Subsequently, to tackle NRD and geometric distortion in multimodal images, we design a guided correction module that establishes robust reference points (RPs) to guide the search for corresponding pixels in regions with significant modality differences. Moreover, to enhance RPs reliability, we incorporate a phase congruency module during the RPs confirmation stage to concentrate RPs around image edge structures. Finally, we perform finer localization on highly correlated corresponding pixels to obtain the optimized matches. We conduct extensive experiments on four multimodal image datasets to validate the effectiveness of the proposed approach. Experimental results demonstrate that our method can achieve sufficient and robust matches across various modality images and effectively suppress the feature point offset problem.
引用
收藏
页码:5892 / 5906
页数:15
相关论文
共 50 条
  • [41] Accuracy of Fiducial Marker Based Multimodal Image Registration in Image Guided Surgery
    Zhang Hongwei
    Wang Guangzhi
    Ding Hui
    6TH WORLD CONGRESS OF BIOMECHANICS (WCB 2010), PTS 1-3, 2010, 31 : 1538 - +
  • [42] Multimodal Image Registration for Preoperative Planning and Image-Guided Neurosurgical Procedures
    Risholm, Petter
    Golby, Alexandra J.
    Wells, William, III
    NEUROSURGERY CLINICS OF NORTH AMERICA, 2011, 22 (02) : 197 - +
  • [43] Multimodal image matching based on Multimodality Robust Line Segment Descriptor
    Zhao, Chunyang
    Zhao, Huaici
    Lv, Jinfeng
    Sun, Shijie
    Li, Bo
    NEUROCOMPUTING, 2016, 177 : 290 - 303
  • [44] Fast and Robust Structure-based Multimodal Geospatial Image Matching
    Ye, Yuanxin
    Bruzzone, Lorenzo
    Shan, Jie
    Shen, Li
    2017 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS), 2017, : 5141 - 5144
  • [45] Two-Scale Multimodal Medical Image Fusion Based on Guided Filtering and Sparse Representation
    Pei, Chunyang
    Fan, Kuangang
    Wang, Wenshuai
    IEEE ACCESS, 2020, 8 : 140216 - 140233
  • [46] Elevation Information-Guided Multimodal Fusion Robust Framework for Remote Sensing Image Segmentation
    Fan, Junyu
    Li, Jinjiang
    Hua, Zhen
    Zhang, Fan
    Zhang, Caiming
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2024, 21 : 1 - 5
  • [47] Feature-guided Gaussian mixture model for image matching
    Ma, Jiayi
    Jiang, Xingyu
    Jiang, Junjun
    Gao, Yuan
    PATTERN RECOGNITION, 2019, 92 : 231 - 245
  • [48] Feature Guided Biased Gaussian Mixture Model for image matching
    Sun, Kun
    Li, Peiran
    Tao, Wenbing
    Tang, Yuanyan
    INFORMATION SCIENCES, 2015, 295 : 323 - 336
  • [49] Feedback Unilateral Grid-Based Clustering Feature Matching for Remote Sensing Image Registration
    Zheng, Zhaohui
    Zheng, Hong
    Ma, Yong
    Fan, Fan
    Ju, Jianping
    Xu, Bichao
    Lin, Mingyu
    Cheng, Shuilin
    REMOTE SENSING, 2019, 11 (12)
  • [50] Performance evaluation of state-of-the-art multimodal remote sensing image matching methods in the presence of noise
    Jovhari, Negar
    Sedaghat, Amin
    Mohammadi, Nazila
    Farhadi, Nima
    Mahtaj, Alireza Bahrami
    APPLIED GEOMATICS, 2024, 16 (01) : 215 - 233