Detection of limestone spalling in 3D survey images using deep learning

Planning the restoration operations of cultural heritage buildings requires an accurate and up-to-date knowledge of the different areas of deterioration. This step is critical to perform the convenient reparation on time. Traditionally, the building analysis is performed by experts based on visual assessments. Which is highly time and resource consuming and complicated by each expert subjectivity. The rapid advancement of computer vision and deep learning inspire the development of automatic damage detection approaches. In this paper, we propose a novel architecture for automatic detection of stone deterioration from existing color image acquired for 3D modeling. The dataset consists of 1012 color images of spalling deterioration, a recurrent deterioration on limestone masonry of castles in the Loire valley. The proposed architecture combining YOLOv5 and transformer layers achieved a F1-score of 85% and an average precision of 81%, outperforming state-of-the-art approaches for automatic damage detection in color images using deep learning.