Gradient-based multi-focus image fusion method using convolution neural network

被引:8
作者
Zhou, Yang [1 ,2 ]
Yang, Xiaomin [1 ,2 ]
Zhang, Rongzhu [1 ]
Liu, Kai [3 ]
Anisetti, Marco [4 ]
Jeon, Gwanggil [5 ,6 ]
机构
[1] Sichuan Univ, Coll Elect & Informat Engn, Chengdu 610064, Sichuan, Peoples R China
[2] Sichuan Univ, Sch Aeronaut & Astronaut, Chengdu 610064, Sichuan, Peoples R China
[3] Sichuan Univ, Coll Elect Engn, Chengdu 610064, Sichuan, Peoples R China
[4] Univ Milan, Dipartimento Informat DI, Via Celoria 18, I-20133 Milan, MI, Italy
[5] Incheon Natl Univ, Dept Embedded Syst Engn, Incheon 22012, South Korea
[6] Xidian Univ, Sch Elect Engn, Xian 710071, Peoples R China
基金
中国国家自然科学基金;
关键词
Image fusion; Multi-focus fusion; Convolution neural network; Gradient-based; PERFORMANCE; TRANSFORM;
D O I
10.1016/j.compeleceng.2021.107174
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Due to limitation of optical lenses, obtaining all-in-focus images is difficult. However, lots of multi-focus image fusion methods cause undesirable artifacts around the focused and defocused boundaries in fusion images. Usually, these boundaries are at the edges of objects in images while the gradient information can reflect edge information intuitively. Based on the above ideas, a Gradient-based method using convolution neural network (CNN) is proposed to produce all-in-focus image. Specifically, we transmit the original images and corresponding four kinds of gradient images into five CNN models to generate the five initial focus score maps, respectively. Then, the final segmented focus map is obtained via merging the initial focus score maps. Finally, we combine the final segmented focus map and source images to obtain the fused image. The experimental results demonstrate that the proposed method has a better performance on both quality and quantitative evaluations than other state-of-the-art methods. Due to limitation of optical lenses, obtaining all-in-focus images is difficult. However, lots of multi-focus image fusion methods cause undesirable artifacts around the focused and defocused boundaries in fusion images. Usually, these boundaries are at the edges of objects in images while the gradient information can reflect edge information intuitively. Based on the above ideas, a Gradient-based method using convolution neural network (CNN) is proposed to produce all-in-focus image. Specifically, we transmit the original images and corresponding four kinds of gradient images into five CNN models to generate the five initial focus score maps, respectively. Then, the final segmented focus map is obtained via merging the initial focus score maps. Finally, we combine the final segmented focus map and source images to obtain the fused image. The experimental results demonstrate that the proposed method has a better performance on both quality and quantitative evaluations than other state-of-the-art methods.
引用
收藏
页数:15
相关论文
共 30 条
[1]   Ensemble of CNN for multi-focus image fusion [J].
Amin-Naji, Mostafa ;
Aghagolzadeh, Ali ;
Ezoji, Mehdi .
INFORMATION FUSION, 2019, 51 :201-214
[2]  
Ben Hamza A, 2005, INTEGR COMPUT-AID E, V12, P135
[3]   THE LAPLACIAN PYRAMID AS A COMPACT IMAGE CODE [J].
BURT, PJ ;
ADELSON, EH .
IEEE TRANSACTIONS ON COMMUNICATIONS, 1983, 31 (04) :532-540
[4]   Infrared and visible image fusion based on target-enhanced multiscale transform decomposition [J].
Chen, Jun ;
Li, Xuejiao ;
Luo, Linbo ;
Mei, Xiaoguang ;
Ma, Jiayi .
INFORMATION SCIENCES, 2020, 508 :64-78
[5]   FuseGAN: Learning to Fuse Multi-Focus Image via Conditional Generative Adversarial Network [J].
Guo, Xiaopeng ;
Nie, Rencan ;
Cao, Jinde ;
Zhou, Dongming ;
Mei, Liye ;
He, Kangjian .
IEEE TRANSACTIONS ON MULTIMEDIA, 2019, 21 (08) :1982-1996
[6]  
Haghighat MA, A non-reference image fusion metric based on mutual information of image features
[7]  
He K., 2016, P 29 IEEE C COMPUTER, P770
[8]  
Huang GL, 2017, IEEE ICC
[9]   Evaluation of focus measures in multi-focus image fusion [J].
Huang, Wei ;
Jing, Zhongliang .
PATTERN RECOGNITION LETTERS, 2007, 28 (04) :493-500
[10]   Pixel- and region-based image fusion with complex wavelets [J].
Lewis, John J. ;
O'Callaghan, Robert J. ;
Nikolov, Stavri G. ;
Bull, David R. ;
Canagarajah, Nishan .
INFORMATION FUSION, 2007, 8 (02) :119-130