CASPI: collaborative photon processing for active single-photon imaging

Image sensors capable of capturing individual photons have made tremendous progress in recent years. However, this technology faces a major limitation. Because they capture scene information at the individual photon level, the raw data is sparse and noisy. Here we propose CASPI: Collaborative Photon Processing for Active Single-Photon Imaging, a technology-agnostic, application-agnostic, and training-free photon processing pipeline for emerging high-resolution single-photon cameras. By collaboratively exploiting both local and non-local correlations in the spatio-temporal photon data cubes, CASPI estimates scene properties reliably even under very challenging lighting conditions. We demonstrate the versatility of CASPI with two applications: LiDAR imaging over a wide range of photon flux levels, from a sub-photon to high ambient regimes, and live-cell autofluorescence FLIM in low photon count regimes. We envision CASPI as a basic building block of general-purpose photon processing units that will be implemented on-chip in future single-photon cameras. The sparse, noisy, and distorted raw photon data captured by single-photon cameras make it difficult to estimate scene properties under challenging illumination conditions. Here, the authors present Collaborative photon processing for Active Single-Photon Imaging (CASPI), a technology-agnostic, application-agnostic, and training-free photon processing pipeline for high-resolution single-photon cameras.