Fluid Micelle Network for Image Super-Resolution Reconstruction

被引:44
作者
Zhang, Mingjin [1 ]
Wu, Qianqian [1 ]
Zhang, Jing [2 ]
Gao, Xinbo [3 ,4 ]
Guo, Jie [1 ]
Tao, Dacheng [5 ]
机构
[1] Xidian Univ, State Key Lab Integrated Serv Networks, Xian 710071, Peoples R China
[2] Univ Sydney, Fac Engn, Sch Comp Sci, Darlington, NSW 2008, Australia
[3] Xidian Univ, Sch Elect Engn, State Key Lab Integrated Serv Networks, Xian 710071, Shaanxi, Peoples R China
[4] Chongqing Univ Posts & Telecommun, Chongqing Key Lab Image Cognit, Chongqing 400065, Peoples R China
[5] JD Com, JD Explore Acad, Beijing 100176, Peoples R China
基金
中国国家自然科学基金;
关键词
Feature extraction; Image reconstruction; Mathematical models; Image edge detection; Residual neural networks; Microscopy; Difference equations; Finite-difference equation; fluid dynamics (FD); fluid micelle network (FM-Net); image super-resolution (SR); Navier-Stokes (N-S) FD equation;
D O I
10.1109/TCYB.2022.3163294
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Most existing convolutional neural-network-based super-resolution (SR) methods focus on designing effective neural blocks but rarely describe the image SR mechanism from the perspective of image evolution in the SR process. In this study, we explore a new research routine by abstracting the movement of pixels in the reconstruction process as the flow of fluid in the field of fluid dynamics (FD), where explicit motion laws of particles have been discovered. Specifically, a novel fluid micelle network is devised for image SR based on the theory of FD that follows the residual learning scheme but learns the residual structure by solving the finite difference equation in FD. The pixel motion equation in the SR process is derived from the Navier-Stokes (N-S) FD equation, establishing a guided branch that is aware of edge information. Thus, the second-order residual drives the network for feature extraction, and the guided branch corrects the direction of the pixel stream to supplement the details. Experiments on popular benchmarks and a real-world microscope chip image dataset demonstrate that the proposed method outperforms other modern methods in terms of both objective metrics and visual quality. The proposed method can also reconstruct clear geometric structures, offering the potential for real-world applications.
引用
收藏
页码:578 / 591
页数:14
相关论文
共 46 条
[11]   Soft-Edge Assisted Network for Single Image Super-Resolution [J].
Fang, Faming ;
Li, Juncheng ;
Zeng, Tieyong .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2020, 29 (29) :4656-4668
[12]   Deep Back-Projection Networks For Super-Resolution [J].
Haris, Muhammad ;
Shakhnarovich, Greg ;
Ukita, Norimichi .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :1664-1673
[13]  
HE KM, 2016, PROC CVPR IEEE, P770, DOI [DOI 10.1109/CVPR.2016.90, 10.1109/CVPR.2016.90]
[14]   ODE-inspired Network Design for Single Image Super-Resolution [J].
He, Xiangyu ;
Mo, Zitao ;
Wang, Peisong ;
Liu, Yang ;
Yang, Mingyuan ;
Cheng, Jian .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :1732-1741
[15]  
HUANG JB, 2015, PROC CVPR IEEE, P5197, DOI DOI 10.1109/CVPR.2015.7299156
[16]  
Kim J, 2016, PROC CVPR IEEE, P1637, DOI [10.1109/CVPR.2016.181, 10.1109/CVPR.2016.182]
[17]   Multi-scale Residual Network for Image Super-Resolution [J].
Li, Juncheng ;
Fang, Faming ;
Mei, Kangfu ;
Zhang, Guixu .
COMPUTER VISION - ECCV 2018, PT VIII, 2018, 11212 :527-542
[18]   Frequency Separation Network for Image Super-Resolution [J].
Li, Shanshan ;
Cai, Qiang ;
Li, Haisheng ;
Cao, Jian ;
Wang, Lei ;
Li, Zhuangzi .
IEEE ACCESS, 2020, 8 :33768-33777
[19]  
Li Wenbo, 2020, Advances in Neural Information Processing Systems, V33
[20]   Enhanced Deep Residual Networks for Single Image Super-Resolution [J].
Lim, Bee ;
Son, Sanghyun ;
Kim, Heewon ;
Nah, Seungjun ;
Lee, Kyoung Mu .
2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2017, :1132-1140