Gross outlier removal and fault data recovery for SHM data of dynamic responses by an annihilating filter-based Hankel-structured robust PCA method

In daily monitoring of structures instrumented with long-term structural health monitoring (SHM) systems, the acquired data is often corrupted with gross outliers due to hardware imperfection and/or electromagnetic interference. These unexpected spikes in data are not unusual and their existence may greatly influence the results of structural health evaluation and lead to false alarms. Hence, there is a high demand for executing data cleaning and data recovery, especially in harsh monitoring environment. In this paper, we propose a robust gross outlier removal method, termed Hankel-structured robust principal component analysis (HRPCA), to remove gross outliers in the monitoring data of structural dynamic responses. Different from the deep-learning-based approaches that possess only outlier identification or anomaly classification ability, HRPCA is a rapid and integrated methodology for data cleaning, which enables outlier detection, outlier identification, and recovery of fault data. It capitalizes on the fundamental duality between the sparsity of the signal and the rank of the structured matrix. Using annihilating filter-based fundamental duality, structural responses could be modeled as lying in a low-dimensional subspace with additional Hankel structure; thus, the gross outliers could be represented as a sparse component. Then the outlier removal issue turns into a matrix factorization problem, which could be successfully solved by robust principal component analysis (RPCA). To validate the denoising capability of HRPCA, a laboratory experiment is first conducted on a five-story building model where the reference clean signal is aware. Then real-world monitoring data with varying degrees of outliers (e.g., single outlier, multiple outliers, and periodic outliers) collected from a cable-stayed bridge and a high-rise structure is used to further illustrate the efficiency of the proposed approach.