Multispectral non-line-of-sight imaging via deep fusion photography

被引：0
作者：
Hao Liu [1 ]
Zhen Xu [2 ]
Yifan Wei [2 ]
Kai Han [1 ]
Xin Peng [2 ]
机构：
[1] College of Advanced Interdisciplinary Studies, National University of Defense Technology, Changsha
[2] School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing
来源：
Science China Information Sciences | 2025年 / 68卷 / 4期
基金：
中国国家自然科学基金;
关键词：
Kolmogorov-Arnold network; learning-based imaging method; multispectral deep fusion; non-line-of-sight imaging; VIS-SWIR-LWIR multispectral imaging;
D O I：
10.1007/s11432-024-4256-3
中图分类号：
学科分类号：
摘要：
Passive non-line-of-sight (NLOS) imaging is a promising technique that extends visual perception to hidden objects around the corner, offering advantages such as low-cost, portability, and real-time. However, the low quality of current passive NLOS images remains a significant barrier to field application of NLOS targets imaging at long standoffs. This study introduces a multispectral NLOS imaging approach utilizing a deep fusion framework to reconstruct images from visible, short-wavelength infrared, and long-wavelength infrared raw data captured by portable devices. The nonlinear representation capabilities and learnable activation function of the Kolmogorov-Arnold network (KAN) are particularly suited to the inverse light field transmission model in NLOS imaging, enhancing the interpretability of the deep neural network. Experimental results demonstrate that this deep fusion photography method provides satisfied performance to image the occluded individuals despite the polynomial attenuation of effective signals with increasing distance between hidden objects and the relay wall. Notably, the passive NLOS experiments reveal successful imaging of hidden people at distance >5 m from the relay wall. Remarkably, even at distances three times greater than those in previous studies, quantitative metrics validate the superior performance of the proposed method in the task of passive NLOS imaging. © Science China Press 2025.
引用
收藏
相关论文
共 59 条
  • [1] Faccio D., Velten A., Wetzstein G., Non-line-of-sight imaging, Nat Rev Phys, 2, pp. 318-327, (2020)
  • [2] O'Toole M., Lindell D.B., Wetzstein G., Confocal non-line-of-sight imaging based on the light-cone transform, Nature, 555, pp. 338-341, (2018)
  • [3] Batarseh M., Sukhov S., Shen Z., Et al., Passive sensing around the corner using spatial coherence, Nat Commun, 9, (2018)
  • [4] Xin S., Nousias S., Kutulakos K.N., Et al., A theory of Fermat paths for non-line-of-sight shape reconstruction, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6800-6809, (2019)
  • [5] Saunders C., Murray-Bruce J., Goyal V.K., Computational periscopy with an ordinary digital camera, Nature, 565, pp. 472-475, (2019)
  • [6] Rapp J., Saunders C., Tachella J., Et al., Seeing around corners with edge-resolved transient imaging, Nat Commun, 11, pp. 1-10, (2020)
  • [7] Metzler C.A., Heide F., Rangarajan P., Et al., Deep-inverse correlography: towards real-time high-resolution non-line-of-sight imaging, Optica, 7, pp. 63-71, (2020)
  • [8] Feng X., Gao L., Ultrafast light field tomography for snapshot transient and non-line-of-sight imaging, Nat Commun, 12, (2021)
  • [9] Wu C., Liu J., Huang X., Et al., Non-line-of-sight imaging over 1.43 km, Proc Natl Acad Sci USA, 118, (2021)
  • [10] Shen S., Wang Z., Liu P., Et al., Non-line-of-sight imaging via neural transient fields, IEEE Trans Pattern Anal Mach Intell, 43, pp. 2257-2268, (2021)
123456