Reinforcement Learning Based Tuning-free Plug-and-Play Image Reconstruction Method for Single Photon Imaging

Quantum image sensor (QIS) has ultra-high single-photon sensitivity and spatial resolution, making it a promising alternative to CMOS image sensor (CIS) as the next-generation image sensor. However, image reconstruction of QIS differs from traditional image reconstruction methods, it aims to recover the original scene from binary measurements. The existing methods include model-based QIS image reconstruction and deep learning-based QIS image reconstruction. Model-based methods are largely based on optimization and are highly sensitive to the selection of hyperparameters. While deep learning-based methods require designing and training separate models for QIS image reconstruction tasks with slight variations in detail, which is inflexible and limits its usefulness to a large extent. In order to tackle the problems in QIS image reconstruction, a tuning-free plug-and-play alternating direction method of multiplier (TFPnP-ADMM) QIS image reconstruction method is proposed in this paper, which can adaptively select appropriate parameters dynamically for different input images with various oversampling factors, so as to achieve better image reconstruction performance. Specifically, in this paper, the parameters that need to be manually tuned in the QIS image reconstruction process under the plug-and-play (PnP) framework are modeled as a sequential decision problem, and a mixed model-free and model-based reinforcement learning algorithm is introduced to learn an optimal strategy, which could determine optimal hyperparameters at each iteration for different input images. The experimental results on synthetic dataset and real dataset demonstrate that, compared with existing state-of-the-art methods, the proposed method improves the peak signal-to-noise ratio by approximately 0.44~ 0.60 dB under oversampling rates of 4, 6, and 8. Furthermore, the visual results demonstrate the superiority of the proposed method in retaining more texture details. Real extremely low light QIS image data is available at https://github.com/ying-fu/ Real-SPAD-Dataset. © 2024 Chinese Institute of Electronics. All rights reserved.