Region- and Pixel-Level Multi-Focus Image Fusion through Convolutional Neural Networks

被引:11
作者
Zhao, Wenyi [1 ]
Yang, Huihua [1 ,2 ]
Wang, Jie [1 ]
Pan, Xipeng [2 ]
Cao, Zhiwei [1 ]
机构
[1] Beijing Univ Posts & Telecommun, Sch Automat, 10 Xitucheng Rd, Beijing, Haidian Distric, Peoples R China
[2] Guilin Univ Elect Technol, Sch Comp Sci & Informat Secur, Guilin, Peoples R China
关键词
Multi-focus image fusion; Convolutional neural network; Region; and pixel-based method; Image interpolation; SEGMENTATION;
D O I
10.1007/s11036-020-01719-9
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Capturing all-in-focus images with 3D scenes is typically a challenging task due to depth of field limitations, and various multi-focus image fusion methods have been employed to generate all-in-focus images. However, existing methods have difficulty simultaneously achieving real-time and superior fusion performance. In this paper, we propose a region- and pixel-based method that can recognize the focus and defocus regions or pixels by the neighborhood information in the source images. The proposed method can obtain satisfactory fusion results and achieve improved real-time performance. First, a convolutional neural network (CNN)-based classifier generates a coarse region-based trimap quickly, which contains focus, defocus and boundary regions. Then, precise fine-tuning is implemented at the boundary regions to address the boundary pixels that are difficult to discriminate by existing methods. Based on a public database, a high-quality dataset is constructed that provides abundant precise pixel-level labels, so that the proposed method can accurately classify the regions and pixels without artifacts. Furthermore, an image interpolation method called NEAREST_Gaussian is proposed to improve the recognition ability at the boundary. Experimental results show that the proposed method outperforms other state-of-the-art methods in visual perception and object metrics. Additionally, the proposed method has 80% improved to the conventional CNN-based methods.
引用
收藏
页码:40 / 56
页数:17
相关论文
共 41 条
[1]   Multi-focus image fusion using multi-scale image decomposition and saliency detection [J].
Bavirisetti, Durga Prasad ;
Dhuli, Ravindra .
AIN SHAMS ENGINEERING JOURNAL, 2018, 9 (04) :1103-1117
[2]   THE LAPLACIAN PYRAMID AS A COMPACT IMAGE CODE [J].
BURT, PJ ;
ADELSON, EH .
IEEE TRANSACTIONS ON COMMUNICATIONS, 1983, 31 (04) :532-540
[3]  
Chen Y. K., 2019, Detnas: Neural architecture search on object detection, P6638
[4]   Multi-focus image fusion using a morphology-based focus measure in a quad-tree structure [J].
De, Ishita ;
Chanda, Bhabatosh .
INFORMATION FUSION, 2013, 14 (02) :136-146
[5]  
Dollar P., 2015, CoRR
[6]   Image Segmentation-Based Multi-Focus Image Fusion Through Multi-Scale Convolutional Neural Network [J].
Du, Chaoben ;
Gao, Shesheng .
IEEE ACCESS, 2017, 5 :15750-15761
[7]   Multifocus image fusion using superpixel segmentation and superpixel-based mean filtering [J].
Duan, Junwei ;
Chen, Long ;
Chen, C. L. Philip .
APPLIED OPTICS, 2016, 55 (36) :10352-10362
[8]   Evaluation of focus measures in multi-focus image fusion [J].
Huang, Wei ;
Jing, Zhongliang .
PATTERN RECOGNITION LETTERS, 2007, 28 (04) :493-500
[9]  
Johnson D. H., 2006, Scholarpedia, V1, P2088, DOI [DOI 10.4249/SCHOLARPEDIA.2046, DOI 10.4249/SCHOLARPEDIA.2088]
[10]   MADNet: A Fast and Lightweight Network for Single-Image Super Resolution [J].
Lan, Rushi ;
Sun, Long ;
Liu, Zhenbing ;
Lu, Huimin ;
Pang, Cheng ;
Luo, Xiaonan .
IEEE TRANSACTIONS ON CYBERNETICS, 2021, 51 (03) :1443-1453