Visible and infrared image matching method based on generative adversarial model

被引：1
作者：
Chen T. [1 ]
Guo J.-F. [1 ]
Han X.-Z. [2 ]
Xie X.-L. [1 ]
Xi J.-X. [1 ]
机构：
[1] Department of Missile Engineering, Rocket Force University of Engineering, Xi'an
[2] The 96901 Unit of the Chinese People's Liberation Army, Beijing
来源：
Zhejiang Daxue Xuebao (Gongxue Ban)/Journal of Zhejiang University (Engineering Science) | 2022年 / 56卷 / 01期
关键词：
Aerial image processing; Deep learning; Generative adversarial network (GAN); Multi-sensor images matching; Style transfer;
D O I：
10.3785/j.issn.1008-973X.2022.01.007
中图分类号：
学科分类号：
摘要：
A visible-infrared image matching method based on generative adversarial model was proposed based on the style transfer of generative adversarial network and traditional local feature extraction capability in order to analyze the problems of large modal difference, difficult matching and poor robustness of existing multi-sensor images matching methods. The loss function calculation path was added and a new loss function was constructed according to the idea of style transfer in GAN network in order to improve the transfer effect of the model on the multi-sensor images. The feature information of the transformed homologous images was extracted by using SIFT algorithm. Then the position and scale of the points to be matched were determined. The feature matching and similarity measurement between the two images were indirectly completed according to the matching strategy. Experiments were conducted on the realistic aerial dataset. Results show that the proposed method can effectively deal with multi-modal data and reduce the difficulty of multi-sensor image matching. The method can provide a new solution for multi-sensor images matching. Copyright ©2022 Journal of Zhejiang University (Engineering Science). All rights reserved.
引用
收藏
页码：63 / 74
页数：11
相关论文
共 26 条
  • [1] BROWN L G., A survey of image registration techniques, ACM Computer Surveys (CSUR), 24, 4, pp. 325-376, (1992)
  • [2] BARBARA Z, FLUSSER J., Image registration methods: a survey, Image and Vision Computing, 21, 11, pp. 977-1000, (2003)
  • [3] XIONG Z, ZHANG Y., A critical review of image registration methods, International Journal of Image and Data Fusion, 1, 2, pp. 137-158, (2010)
  • [4] SHI W, SU F, WANG R, Et al., A visual circle based image registration algorithm for optical and SAR imagery, IEEE International Geoscience and Remote Sensing Symposium, pp. 2109-2112, (2012)
  • [5] SURI S, REINARTZ P., Mutual-information-based registration of terraSAR-X and ikonos imagery in urban areas, IEEE Transactions on Geoscience and Remote Sensing, 48, 2, pp. 939-949, (2010)
  • [6] HASAN M, PICKERING M R, JIA X., Robust automatic registration of multimodal satellite images using CCRE with partial volume interpolation, IEEE Transactions on Geoscience and Remote Sensing, 50, 10, pp. 4050-4061, (2012)
  • [7] GAO Xue-yan, PAN An-ning, YANG Yang, Urban green space remote sensing image registration using image mixed features, Journal of Zhejiang University: Engineering Science, 53, 6, pp. 1205-1217, (2019)
  • [8] ZHANG Li, LIU Ji-Lin, Human tracking method based on maximally stable extremal regions with multi cameras, Journal of Zhejiang University: Engineering Science, 44, 6, pp. 1091-1097, (2010)
  • [9] REN S Q, HE K M, GIRSHICK R, Et al., Faster R-CNN: towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 6, pp. 1137-1149, (2017)
  • [10] LIN T Y, DOLLAR P, GIRSHICK R, Et al., Feature pyramid networks for object detection, IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117-2125, (2017)
123