Fluid Micelle Network for Image Super-Resolution Reconstruction

被引:44
作者
Zhang, Mingjin [1 ]
Wu, Qianqian [1 ]
Zhang, Jing [2 ]
Gao, Xinbo [3 ,4 ]
Guo, Jie [1 ]
Tao, Dacheng [5 ]
机构
[1] Xidian Univ, State Key Lab Integrated Serv Networks, Xian 710071, Peoples R China
[2] Univ Sydney, Fac Engn, Sch Comp Sci, Darlington, NSW 2008, Australia
[3] Xidian Univ, Sch Elect Engn, State Key Lab Integrated Serv Networks, Xian 710071, Shaanxi, Peoples R China
[4] Chongqing Univ Posts & Telecommun, Chongqing Key Lab Image Cognit, Chongqing 400065, Peoples R China
[5] JD Com, JD Explore Acad, Beijing 100176, Peoples R China
基金
中国国家自然科学基金;
关键词
Feature extraction; Image reconstruction; Mathematical models; Image edge detection; Residual neural networks; Microscopy; Difference equations; Finite-difference equation; fluid dynamics (FD); fluid micelle network (FM-Net); image super-resolution (SR); Navier-Stokes (N-S) FD equation;
D O I
10.1109/TCYB.2022.3163294
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Most existing convolutional neural-network-based super-resolution (SR) methods focus on designing effective neural blocks but rarely describe the image SR mechanism from the perspective of image evolution in the SR process. In this study, we explore a new research routine by abstracting the movement of pixels in the reconstruction process as the flow of fluid in the field of fluid dynamics (FD), where explicit motion laws of particles have been discovered. Specifically, a novel fluid micelle network is devised for image SR based on the theory of FD that follows the residual learning scheme but learns the residual structure by solving the finite difference equation in FD. The pixel motion equation in the SR process is derived from the Navier-Stokes (N-S) FD equation, establishing a guided branch that is aware of edge information. Thus, the second-order residual drives the network for feature extraction, and the guided branch corrects the direction of the pixel stream to supplement the details. Experiments on popular benchmarks and a real-world microscope chip image dataset demonstrate that the proposed method outperforms other modern methods in terms of both objective metrics and visual quality. The proposed method can also reconstruct clear geometric structures, offering the potential for real-world applications.
引用
收藏
页码:578 / 591
页数:14
相关论文
共 46 条
[1]   Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network [J].
Ahn, Namhyuk ;
Kang, Byungkon ;
Sohn, Kyung-Ah .
COMPUTER VISION - ECCV 2018, PT X, 2018, 11214 :256-272
[2]   Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding [J].
Bevilacqua, Marco ;
Roumy, Aline ;
Guillemot, Christine ;
Morel, Marie-Line Alberi .
PROCEEDINGS OF THE BRITISH MACHINE VISION CONFERENCE 2012, 2012,
[3]   Modeling and Optimizing of the Multi-Layer Nearest Neighbor Network for Face Image Super-Resolution [J].
Chen, Liang ;
Pan, Jinshan ;
Hu, Ruimin ;
Han, Zhen ;
Liang, Chao ;
Wu, Yi .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2020, 30 (12) :4513-4525
[4]   Robust Face Super-Resolution via Position Relation Model Based on Global Face Context [J].
Chen, Liang ;
Pan, Jinshan ;
Jiang, Junjun ;
Zhang, Jiawei ;
Wu, Yi .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2020, 29 (9002-9016) :9002-9016
[5]   Robust Face Image Super-Resolution via Joint Learning of Subdivided Contextual Model [J].
Chen, Liang ;
Pan, Jinshan ;
Li, Qing .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (12) :5897-5909
[6]  
Chen RTQ, 2018, 32 C NEURAL INFORM P, V31
[7]   A Deep Convolutional Neural Network with Selection Units for Super-Resolution [J].
Choi, Jae-Seok ;
Kim, Munchurl .
2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2017, :1150-1156
[8]   Regularizing Hyperspectral and Multispectral Image Fusion by CNN Denoiser [J].
Dian, Renwei ;
Li, Shutao ;
Kang, Xudong .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2021, 32 (03) :1124-1135
[9]   Deep Hyperspectral Image Sharpening [J].
Dian, Renwei ;
Li, Shutao ;
Guo, Anjing ;
Fang, Leyuan .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2018, 29 (11) :5345-5355
[10]   Learning a Deep Convolutional Network for Image Super-Resolution [J].
Dong, Chao ;
Loy, Chen Change ;
He, Kaiming ;
Tang, Xiaoou .
COMPUTER VISION - ECCV 2014, PT IV, 2014, 8692 :184-199