Guided Deep Generative Model-Based Spatial Regularization for Multiband Imaging Inverse Problems

被引:0
作者
Zhao M. [1 ,2 ]
Dobigeon N. [3 ,4 ]
Chen J. [1 ,2 ]
机构
[1] School of Marine Science and Technology, Northwestern Polytechnical University, Xi'an
[2] Research and Development Institute, Northwestern Polytechnical University, Shenzhen
[3] IRIT/INPENSEEIHT,CNRS, University of Toulouse, Toulouse
[4] Institut Universitaire de France (IUF), Paris
关键词
deep generative regularization; deep image prior; deep learning; guided image; inverse problems; Multiband imaging;
D O I
10.1109/TIP.2023.3321460
中图分类号
学科分类号
摘要
When adopting a model-based formulation, solving inverse problems encountered in multiband imaging requires to define spatial and spectral regularizations. In most of the works of the literature, spectral information is extracted from the observations directly to derive data-driven spectral priors. Conversely, the choice of the spatial regularization often boils down to the use of conventional penalizations (e.g., total variation) promoting expected features of the reconstructed image (e.g., piece-wise constant). In this work, we propose a generic framework able to capitalize on an auxiliary acquisition of high spatial resolution to derive tailored data-driven spatial regularizations. This approach leverages on the ability of deep learning to extract high level features. More precisely, the regularization is conceived as a deep generative network able to encode spatial semantic features contained in this auxiliary image of high spatial resolution. To illustrate the versatility of this approach, it is instantiated to conduct two particular tasks, namely multiband image fusion and multiband image inpainting. Experimental results obtained on these two tasks demonstrate the benefit of this class of informed regularizations when compared to more conventional ones. © 2023 IEEE.
引用
收藏
页码:5692 / 5704
页数:12
相关论文
共 50 条
[41]   Deep learning approaches to inverse problems in imaging: Past, present and future [J].
Lopez-Tapia, Santiago ;
Molina, Rafael ;
Katsaggelos, Aggelos K. .
DIGITAL SIGNAL PROCESSING, 2021, 119
[42]   Inverse problems in imaging science: from classical regularization methods to state of the art Bayesian methods [J].
Mohammad-Djafari, Ali .
2014 FIRST INTERNATIONAL IMAGE PROCESSING, APPLICATIONS AND SYSTEMS CONFERENCE (IPAS), 2014,
[43]   Probabilistic Representation and Inverse Design of Metamaterials Based on a Deep Generative Model with Semi-Supervised Learning Strategy [J].
Ma, Wei ;
Cheng, Feng ;
Xu, Yihao ;
Wen, Qinlong ;
Liu, Yongmin .
ADVANCED MATERIALS, 2019, 31 (35)
[44]   Model-based network architecture for image reconstruction in lensless imaging [J].
Zeng, Tianjiao ;
Lam, Edmund Y. .
HOLOGRAPHY, DIFFRACTIVE OPTICS, AND APPLICATIONS X, 2020, 11551
[45]   A Wavelet-Based Compressive Deep Learning Scheme for Inverse Scattering Problems [J].
Zong, Zheng ;
Wang, Yusong ;
Wei, Zhun .
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2022, 60
[46]   DCCM: DUAL DATA CONSISTENCY GUIDED CONSISTENCY MODEL FOR INVERSE PROBLEMS [J].
Tian, Jiahao ;
Zheng, Ziyang ;
Peng, Xinyu ;
Li, Yong ;
Dai, Wenrui ;
Xiong, Hongkai .
2024 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP, 2024, :1507-1513
[47]   Generative Models for Inverse Imaging Problems: From mathematical foundations to physics-driven applications [J].
Zhao, Zhizhen ;
Ye, Jong Chul ;
Bresler, Yoram .
IEEE SIGNAL PROCESSING MAGAZINE, 2023, 40 (01) :148-163
[48]   Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization [J].
Shlezinger, Nir ;
Eldar, Yonina C. ;
Boyd, Stephen P. .
IEEE ACCESS, 2022, 10 :115384-115398
[49]   Inverse Reflectance Model Based on Deep Learning [J].
Xi, Wang ;
Jian Zhenxiong ;
Ren Mingjun .
ACTA OPTICA SINICA, 2023, 43 (21)
[50]   SPATIAL AND ANGULAR RECONSTRUCTION OF LIGHT FIELD BASED ON DEEP GENERATIVE NETWORKS [J].
Meng, Nan ;
Zeng, Tianjiao ;
Lam, Edmund Y. .
2019 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2019, :4659-4663