Gradient-based multi-focus image fusion method using convolution neural network

被引:8
作者
Zhou, Yang [1 ,2 ]
Yang, Xiaomin [1 ,2 ]
Zhang, Rongzhu [1 ]
Liu, Kai [3 ]
Anisetti, Marco [4 ]
Jeon, Gwanggil [5 ,6 ]
机构
[1] Sichuan Univ, Coll Elect & Informat Engn, Chengdu 610064, Sichuan, Peoples R China
[2] Sichuan Univ, Sch Aeronaut & Astronaut, Chengdu 610064, Sichuan, Peoples R China
[3] Sichuan Univ, Coll Elect Engn, Chengdu 610064, Sichuan, Peoples R China
[4] Univ Milan, Dipartimento Informat DI, Via Celoria 18, I-20133 Milan, MI, Italy
[5] Incheon Natl Univ, Dept Embedded Syst Engn, Incheon 22012, South Korea
[6] Xidian Univ, Sch Elect Engn, Xian 710071, Peoples R China
基金
中国国家自然科学基金;
关键词
Image fusion; Multi-focus fusion; Convolution neural network; Gradient-based; PERFORMANCE; TRANSFORM;
D O I
10.1016/j.compeleceng.2021.107174
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Due to limitation of optical lenses, obtaining all-in-focus images is difficult. However, lots of multi-focus image fusion methods cause undesirable artifacts around the focused and defocused boundaries in fusion images. Usually, these boundaries are at the edges of objects in images while the gradient information can reflect edge information intuitively. Based on the above ideas, a Gradient-based method using convolution neural network (CNN) is proposed to produce all-in-focus image. Specifically, we transmit the original images and corresponding four kinds of gradient images into five CNN models to generate the five initial focus score maps, respectively. Then, the final segmented focus map is obtained via merging the initial focus score maps. Finally, we combine the final segmented focus map and source images to obtain the fused image. The experimental results demonstrate that the proposed method has a better performance on both quality and quantitative evaluations than other state-of-the-art methods. Due to limitation of optical lenses, obtaining all-in-focus images is difficult. However, lots of multi-focus image fusion methods cause undesirable artifacts around the focused and defocused boundaries in fusion images. Usually, these boundaries are at the edges of objects in images while the gradient information can reflect edge information intuitively. Based on the above ideas, a Gradient-based method using convolution neural network (CNN) is proposed to produce all-in-focus image. Specifically, we transmit the original images and corresponding four kinds of gradient images into five CNN models to generate the five initial focus score maps, respectively. Then, the final segmented focus map is obtained via merging the initial focus score maps. Finally, we combine the final segmented focus map and source images to obtain the fused image. The experimental results demonstrate that the proposed method has a better performance on both quality and quantitative evaluations than other state-of-the-art methods.
引用
收藏
页数:15
相关论文
共 30 条
[11]   Multifocus image fusion using region segmentation and spatial frequency [J].
Li, Shutao ;
Yang, Bin .
IMAGE AND VISION COMPUTING, 2008, 26 (07) :971-979
[12]   Image Fusion with Guided Filtering [J].
Li, Shutao ;
Kang, Xudong ;
Hu, Jianwen .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2013, 22 (07) :2864-2875
[13]   Multi-focus image fusion with a deep convolutional neural network [J].
Liu, Yu ;
Chen, Xun ;
Peng, Hu ;
Wang, Zengfu .
INFORMATION FUSION, 2017, 36 :191-207
[14]   A general framework for image fusion based on multi-scale transform and sparse representation [J].
Liu, Yu ;
Liu, Shuping ;
Wang, Zengfu .
INFORMATION FUSION, 2015, 24 :147-164
[15]   Multi-focus image fusion with dense SIFT [J].
Liu, Yu ;
Liu, Shuping ;
Wang, Zengfu .
INFORMATION FUSION, 2015, 23 :139-155
[16]   Pixel-based and region-based image fusion schemes using ICA bases [J].
Mitianoudis, Nikolaos ;
Stathaki, Tania .
INFORMATION FUSION, 2007, 8 (02) :131-142
[17]   Remote sensing image fusion using the curvelet transform [J].
Nencini, Filippo ;
Garzelli, Andrea ;
Baronti, Stefano ;
Alparone, Luciano .
INFORMATION FUSION, 2007, 8 (02) :143-156
[18]   A Survey on Transfer Learning [J].
Pan, Sinno Jialin ;
Yang, Qiang .
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2010, 22 (10) :1345-1359
[19]  
Piella G., 2003, Information Fusion, V4, P259, DOI 10.1016/S1566-2535(03)00046-0
[20]  
Qiang W, 2004, IEEE INSTRUMENTATION