Vision based nighttime pavement cracks pixel level detection by integrating infrared visible fusion and deep learning

Accurate detection of pavement cracks is an important task in road maintenance and safety management. However, accurate segmentation of road cracks at night is still challenging due to light conditions. In this study, an automatic segmentation method for nighttime road cracks based on infrared-visible fusion and deep learning is proposed. First, a fusion method of infrared and visible light images is proposed to improve the visibility of road cracks under low light conditions. Afterwards, a deep learning network integrating a dynamic sparse attention mechanism is proposed to segment the cracks in the enhanced road images. In this study, a dataset of infrared and visible light images of nighttime road cracks is acquired to test the effectiveness and sophistication of the proposed method. The results show that the proposed method can achieve accurate segmentation of nighttime pavement cracks (77.89 % mIoU,85.68 , 85.68 % mPA,97.74 , 97.74 % Accuracy,87.53 , 87.53 % Precision,81.55 , 81.55 % Recall,84.44 , 84.44 % F1-score) and better than the existing segmentation models (Unet, Pspnet, DeepLabv3+). +). Integration of the proposed method into an unmanned inspection robot helps to achieve 24/7 pixel-level pavement crack detection.