DUFuse: Deep U-Net for visual and infrared images fusion

被引：1
作者：
Pan Y. [1 ]
Pi D. [2 ]
Khan I.A. [2 ]
Meng H. [3 ]
机构：
[1] School of Information Science and Engineering, University of Jinan, Jinan
[2] College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing
[3] School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing
来源：
Journal of Ambient Intelligence and Humanized Computing | 2023年 / 14卷 / 09期
关键词：
Deep learning; Fusion strategy; Image fusion; Infrared image; Visual image;
D O I：
10.1007/s12652-022-04323-9
中图分类号：
学科分类号：
摘要：
The vision sensor is capable of capturing image detail features suitable for human observation, while the infrared sensor is capable of capturing the thermal characteristics of the target object. Therefore, the vision and infrared image fusion aim to retain both the rich detail features in the visual image and the thermal characteristics of the infrared image. This study proposes a novel deep U-Net network model to solve the fusion task. First, an improved deep model is proposed for better feature extraction by borrowing the process of decomposition, fusion and reconstruction in the multiscale decomposition process. Second, structural similarity is introduced into the loss function, which enables the network to enhance the quality of the detailed features of the generated images. Third, we propose a new hierarchical fusion strategy as well as average fusion and weighted fusion rules. Extensive experiments demonstrate that the proposed algorithm is superior to state-of-the-art algorithms. © 2022, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.
引用
收藏
页码：12549 / 12561
页数：12
相关论文
共 37 条
  • [1] Benediktsson A., Ghamisi P., Mura Mauro D., A survey on spectral-spatial classification techniques based on attribute profiles, IEEE Trans Geosci Remote Sens, 3, 53, pp. 2335-2353, (2015)
  • [2] Chavez P., Sides S., Anderson J., Comparison of three different methods to merge multiresolution and multispectral data-Landsat TM and SPOT panchromatic, Photogramm Eng Remote Sens, 3, 57, pp. 295-303, (1991)
  • [3] Cheng M., Wang C., Li J., Sparse representation based pansharpening using trained dictionary, IEEE Geosci Remote Sens Lett, 1, 11, pp. 293-297, (2014)
  • [4] Cunha A., Zhou J., Do M., The nonsubsampled contourlet transform: theory, design, and applications, IEEE Trans Image Process, 10, 15, pp. 3089-3101, (2006)
  • [5] Ghassemian H., A review of remote sensing image fusion methods, Inf Fusion, 1, 32, pp. 75-89, (2016)
  • [6] Fast R-CNN, In: 2015 IEEE International Conference on Computer Vision (ICCV), (2015)
  • [7] Guo M., Zhang H., Li J., Zhang L., Shen H., An Online coupled dictionary learning approach for remote sensing image fusion, IEEE J Sel Top Appl Earth Observ Remote Sens, 4, 7, pp. 1284-1294, (2014)
  • [8] Huang G., Liu Z., Maaten L., Weinberger K., Densely connected convolutional networks, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261-2269, (2017)
  • [9] Krizhevsky A., Sutskever I., Hinton G., Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems, pp. 1097-1105, (2012)
  • [10] Lecun Y., Boser B., Denker J., Henderson D., Howard R., Hubbard W., Jackel L., Backpropagation applied to handwritten zip code recognition, Neural Comput, 4, 1, pp. 541-551, (1989)
1234