Convolutional neural network-based multimodal image information fusion for moisture damage assessment of cultural heritage buildings

Water is an essential factor causing the deterioration of historical and cultural heritage, and identifying and quantifying moisture damage is essential for conserving cultural heritage buildings. This study combines infrared thermography (IRT) and microwave technology to investigate moisture-induced diseases and the variation of the spatial distribution of water content in the Yungang Grottoes of China, a World Heritage Site. Information on building humidity distribution was extracted and combined from IRT and microwave information. Furthermore, multi-feature image fusion combines different types of monitoring information through the Convolutional Neural Network (CNN), which is used to diagnose moisture distribution and content accurately. The features obtained from different non-destructive methods were then jointly analyzed using statistical analysis methods such as absolute contrast, noise ratio (SNR), and spatial stability to quantify the artifact moisture defect data, thus improving the accuracy and range of features for detecting water-damaged pathologies. The results show that the combination of IRT and microwave information technology not only qualitatively but also quantitatively analyses the moisture in the building, significantly using microwave moisture meters to study water distribution at different depths and surface characterization by IRT. The use of non-destructive monitoring tools helps to guide the diagnosis of moisture damage, restoration of heritage buildings, and the planning of intervention strategies.