LiDAR-assisted image restoration for extreme low-light conditions

Research into enhancing images captured under low-light conditions has been a focus for several years. However, most existing image restoration techniques primarily address RGB-only images, overlooking the potential benefits of integrating additional modalities. With advancements in handheld technology, capturing images alongside depth data has become easily accessible through devices like smartphones. This presents a significant opportunity to explore the role of depth data in improving image quality in low-light environments. In this study, we introduce a novel dataset named LED (Low-light Image Enhanced with Depth Map), consisting of 1365 entries. Each entry includes a low-light image, its corresponding normal-light counterpart, and an associated depth map. To the best of our knowledge, LED is the first low-light image enhancement dataset that incorporates depth information. Building on this dataset, we propose model named DERNet with three innovative technologies to integrate depth data into the low-light image enhancement process. First, we utilize depth maps to extract precise edge details, improving object boundary clarity in low-light images. Second, we introduce a multi-scale multimodal fusion module, which combines features from both the image and depth map to enhance image quality. Third, we develop a multi-task refinement network that leverages an estimated illumination map to further refine the enhancement process. Experimental results demonstrate that our method achieves over a 0.52 improvement in PSNR compared to unimodal approaches and exceeds multimodal methods by nearly 0.3 with much less total parameters, highlighting the effectiveness of our approach. Moreover, our comprehensive ablation studies show that our method can be flexibly combined with previous approaches to enhance their accuracy, and our model demonstrates greater efficiency compared to previous methods.