An infrared and visible image fusion algorithm based on ResNet-152

被引:40
作者
Zhang, Liming [1 ,2 ]
Li, Heng [1 ,2 ]
Zhu, Rui [1 ,2 ]
Du, Ping [1 ,2 ]
机构
[1] Lanzhou Jiaotong Univ, Fac Geomat, Lanzhou 730070, Peoples R China
[2] Natl Local Joint Engn Res Ctr Technol & Applicat, Lanzhou 730070, Peoples R China
关键词
Image processing; Image fusion; ResNet-152; Infrared image; Visible image;
D O I
10.1007/s11042-021-11549-w
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The fusion of infrared and visible images can obtain a combined image with hidden objective and rich visible details. To improve the details of the fusion image from the infrared and visible images by reducing artifacts and noise, an infrared and visible image fusion algorithm based on ResNet-152 is proposed. First, the source images are decomposed into the low-frequency part and the high-frequency part. The low-frequency part is processed by the average weighting strategy. Second, the multi-layer features are extracted from high-frequency part by using the ResNet-152 network. Regularization L1, convolution operation, bilinear interpolation upsampling and maximum selection strategy on the feature layers to obtain the maximum weight layer. Multiplying the maximum weight layer and the high-frequency as new high-frequency. Finally, the fusion image is reconstructed by the low-frequency and the high-frequency. Experiments show that the proposed method can obtain more details from the image texture by retaining the significant features of the images. In addition, this method can effectively reduce artifacts and noise. The consistency in the objective evaluation and visual observation performs superior to the comparative algorithms.
引用
收藏
页码:9277 / 9287
页数:11
相关论文
共 27 条
[1]   Advances of Four Machine Learning Methods for Spatial Data Handling: a Review [J].
Du, Peijun ;
Bai, Xuyu ;
Tan, Kun ;
Xue, Zhaohui ;
Samat, Alim ;
Xia, Junshi ;
Li, Erzhu ;
Su, Hongjun ;
Liu, Wei .
JOURNAL OF GEOVISUALIZATION AND SPATIAL ANALYSIS, 2020, 4 (01)
[2]  
Haghighat M, 2014, I C APPL INF COMM TE, P424
[3]  
He K., 2016, ACM ECCV'16, P630, DOI [DOI 10.1007/978-3-319-46493-0_38, 10.1007/978-3-319-46493-0_38, 10.1007/978-3-319-46493-038]
[4]  
Huang Y, 2017, CHIN SOC OPT ENG C H
[5]   Joint patch clustering-based dictionary learning for multimodal image fusion [J].
Kim, Minjae ;
Han, David K. ;
Ko, Hanseok .
INFORMATION FUSION, 2016, 27 :198-214
[6]   Infrared and visible image fusion with ResNet and zero-phase component analysis [J].
Li, Hui ;
Wu, Xiao-jun ;
Durrani, Tariq S. .
INFRARED PHYSICS & TECHNOLOGY, 2019, 102
[7]   DenseFuse: A Fusion Approach to Infrared and Visible Images [J].
Li, Hui ;
Wu, Xiao-Jun .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (05) :2614-2623
[8]   Image Fusion with Guided Filtering [J].
Li, Shutao ;
Kang, Xudong ;
Hu, Jianwen .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2013, 22 (07) :2864-2875
[9]   Infrared and visible image fusion method based on saliency detection in sparse domain [J].
Liu, C. H. ;
Qi, Y. ;
Ding, W. R. .
INFRARED PHYSICS & TECHNOLOGY, 2017, 83 :94-102
[10]   A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter [J].
Liu, Shaopeng ;
Tian, Guohui ;
Xu, Yuan .
NEUROCOMPUTING, 2019, 338 :191-206