A damage assessment methodology for structural systems using transfer learning from the audio domain

Neural network-based strategies require balanced training datasets to avoid creating unreliable classification and prediction models. While these strategies are commonly used to model the dynamics of structural and mechanical systems, the imbalanced composition of monitoring data is a fundamental challenge for damage assessment in structural systems. The monitoring data often contain abundant observations from structures in their normal operating conditions (un-damaged state) and small and partial information from systems in the damaged state. Therefore, the model, trained by adopting deep learning approaches, tends to show an ill-conditioned nature, limited to specific structures in a narrow range of damage conditions. The current study presents a damage assessment strategy that overcomes the limitations of unbalanced datasets. To improve the model's ability to distinguish between different health conditions, informative features are utilized to facilitate the differentiation of multiple classes according to the frequency content of vibration signals. The model acquires this ability by learning from a rich dataset of human voices (source domain), where low-level features that denote the vibration traits of human waveforms are extracted. Subsequently, this knowledge is transferred to the features of a target domain that has limited data for damage detection. The proposed methodology relies on creating an informative feature extractor training a Time-Delay Neural Network (TDNN) using a collection of human voice recordings. Cepstral and pitch features derived from the speech data are used as input features for the TDNN. This network is used to derive low-level features at intermediate layers of the network, called "x -vectors". These features store non-case-dependent information about the frequency content of the signals and depict the ability to distinguish between different classes according to a change in the frequency content of the investigated system. This is not a unique attribute of the original audio source domain, and it can be employed to help differentiate categories for any vibrating system where a modification in the frequency content is representative of a transition between classes, including the structural and mechanical systems. Because of the generalization trait of the x-vector, they can be employed to construct a Probabilistic Linear Discriminant Analysis model able to classify various damage classes considering vibration measurements obtained from a different system, i.e., a structural system (target domain). Initially, the simulated acceleration response from the 12-degree of freedom structure are analyzed to affirm the effectiveness of the framework. Then, the method is further validated by using the field data of the Z24 bridge, to evaluate its reliability in real-world applications.