Generative Models for Inverse Imaging Problems: From mathematical foundations to physics-driven applications

被引:12
作者
Zhao, Zhizhen [1 ,2 ]
Ye, Jong Chul [3 ]
Bresler, Yoram [2 ,4 ]
机构
[1] Univ Illinois Urbana Champaign UIUC, William L Everitt Fac Fellow, Dept Elect & Comp Engn, Urbana, IL 61801 USA
[2] Univ Illinois Urbana Champaign UIUC, Coordinated Sci Lab, Urbana, IL 61801 USA
[3] Korea Adv Inst Sci & Technol KAIST, Kim Jaechul Grad Sch AI, Daejon, South Korea
[4] Univ Illinois, Dept Elect & Comp Engn, Urbana, IL 61801 USA
关键词
Inverse problems; Computational modeling; Imaging; Generative adversarial networks; Mathematical models; Physics; CRYO-EM; RECONSTRUCTION; CYCLEGAN;
D O I
10.1109/MSP.2022.3215282
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Physics-informed generative modeling for inverse problems in computational imaging is a fast-growing field encompassing a variety of methods and applications. Here, we review a few generative modeling techniques, such as variational autoencoders (VAEs) and generative adversarial networks (GANs), as well as more recent developments in score-based generative models. Through different imaging applications, we review how the generative modeling techniques are effectively combined with the physics of the imaging problem, e.g., the measurement forward model and physical properties of the target objects, to solve the inverse problems.
引用
收藏
页码:148 / 163
页数:16
相关论文
共 58 条
[11]   Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction [J].
Chung, Hyungjin ;
Sim, Byeongsu ;
Ye, Jong Chul .
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, :12403-12412
[12]   Score-based diffusion models for accelerated MRI [J].
Chung, Hyungjin ;
Ye, Jong Chul .
MEDICAL IMAGE ANALYSIS, 2022, 80
[13]   Trajectories of the ribosome as a Brownian nanomachine [J].
Dashti, Ali ;
Schwander, Peter ;
Langlois, Robert ;
Fung, Russell ;
Li, Wen ;
Hosseinizadeh, Ahmad ;
Liao, Hstau Y. ;
Pallesen, Jesper ;
Sharma, Gyanesh ;
Stupina, Vera A. ;
Simon, Anne E. ;
Dinman, Jonathan D. ;
Frank, Joachim ;
Ourmazd, Abbas .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2014, 111 (49) :17492-17497
[14]   Modular Assembly of the Bacterial Large Ribosomal Subunit [J].
Davis, Joseph H. ;
Tan, Yong Zi ;
Carragher, Bridget ;
Potter, Clinton S. ;
Lyumkis, Dmitry ;
Williamson, James R. .
CELL, 2016, 167 (06) :1610-+
[15]   Max-Sliced Wasserstein Distance and its use for GANs [J].
Deshpande, Ishan ;
Hu, Yuan-Ting ;
Sun, Ruoyu ;
Pyrros, Ayis ;
Siddiqui, Nasir ;
Koyejo, Sanmi ;
Zhao, Zhizhen ;
Forsyth, David ;
Schwing, Alexander .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, :10640-10648
[16]   Generative Modeling using the Sliced Wasserstein Distance [J].
Deshpande, Ishan ;
Zhang, Ziyu ;
Schwing, Alexander .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :3483-3491
[17]  
Du YL, 2019, ADV NEUR IN, V32
[18]  
Goodfellow IJ, 2014, ADV NEUR IN, V27, P2672
[19]  
Grathwohl W., 2019, PROC INT C LEARN REP
[20]   CryoGAN: A New Reconstruction Paradigm for Single-Particle Cryo-EM Via Deep Adversarial Learning [J].
Gupta, Harshit ;
McCann, Michael T. ;
Donati, Laurene ;
Unser, Michael .
IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING, 2021, 7 (07) :759-774