Speed Matters, a robust infrared and visible image matching method at real-time speed

被引:0
作者
Chang, Rong [1 ]
Yang, Chuanxu [1 ]
Zhang, Hang [2 ]
Xie, Housheng [3 ]
Zhou, Chengjiang [3 ]
Pan, Anning [4 ,5 ]
Yang, Yang [3 ]
机构
[1] Yunnan Power Grid Corp, Yuxi Power Supply Bur, Yuxi 653100, Yunnan, Peoples R China
[2] Yunnan Power Grid Corp, Informat Ctr, Kunming 650500, Yunnan, Peoples R China
[3] Yunnan Normal Univ, Lab Pattern Recognit & Artificial Intelligence, Kunming 650500, Yunnan, Peoples R China
[4] Baoshan Univ, Sch Big Data, Baoshan 678000, Peoples R China
[5] Yunnan Normal Univ, Sch Phys & Elect Informat, Kunming 650500, Peoples R China
关键词
Multimodal; Real time; Feature matching; Deep learning; REGISTRATION;
D O I
10.1007/s11554-023-01395-4
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Image matching is a crucial step in executing many complex visual tasks, which involve identifying the same or similar visual patterns across various images. Matching images between infrared and visible becomes quite challenging due to the significant non-linear intensity differences. In this paper, we propose using a lightweight network for feature matching of infrared and visible images, combining global and local feature information, and reducing computational costs, enabling real-time inference on most desktop-level GPUs. To fully leverage the powerful matching capabilities of existing state-of-the-art models, we introduce knowledge distillation to obtain more robust features. Moreover, to address the issue of insufficient datasets for network training in existing methods, we propose using image style transfer techniques to synthesize paired datasets of infrared and visible. Experimental results show that our method achieves results comparable to the most advanced methods in infrared and visible image matching. Furthermore, our method has a significant advantage in inference speed, which is beneficial for tasks that require real-time completion.
引用
收藏
页数:10
相关论文
共 34 条
  • [1] Speeded-Up Robust Features (SURF)
    Bay, Herbert
    Ess, Andreas
    Tuytelaars, Tinne
    Van Gool, Luc
    [J]. COMPUTER VISION AND IMAGE UNDERSTANDING, 2008, 110 (03) : 346 - 359
  • [2] Joint Detection and Matching of Feature Points in Multimodal Images
    Ben Baruch, Elad
    Keller, Yosi
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (10) : 6585 - 6593
  • [3] PCNet: A structure similarity enhancement method for multispectral and multimodal image registration
    Cao, Si-Yuan
    Yu, Beinan
    Luo, Lun
    Zhang, Runmin
    Chen, Shu-Jie
    Li, Chunguang
    Shen, Hui-Liang
    [J]. INFORMATION FUSION, 2023, 94 : 200 - 214
  • [4] Cui S, 2019, INT GEOSCI REMOTE SE, P919, DOI [10.1109/igarss.2019.8900521, 10.1109/IGARSS.2019.8900521]
  • [5] ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning
    Deng, Yuxin
    Ma, Jiayi
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2023, 32 : 591 - 602
  • [6] SuperPoint: Self-Supervised Interest Point Detection and Description
    DeTone, Daniel
    Malisiewicz, Tomasz
    Rabinovich, Andrew
    [J]. PROCEEDINGS 2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2018, : 337 - 349
  • [7] D2-Net: A Trainable CNN for Joint Description and Detection of Local Features
    Dusmanu, Mihai
    Rocco, Ignacio
    Pajdla, Tomas
    Pollefeys, Marc
    Sivic, Josef
    Torii, Akihiko
    Sattler, Torsten
    [J]. 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 8084 - 8093
  • [8] LLVIP: A Visible-infrared Paired Dataset for Low-light Vision
    Jia, Xinyu
    Zhu, Chuang
    Li, Minzhen
    Tang, Wenqi
    Zhou, Wenli
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), 2021, : 3489 - 3497
  • [9] COTR: Correspondence Transformer for Matching Across Images
    Jiang, Wei
    Trulls, Eduard
    Hosang, Jan
    Tagliasacchi, Andrea
    Yi, Kwang Moo
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 6187 - 6197
  • [10] Katharopoulos A, 2020, PR MACH LEARN RES, V119