A novel and robust data anomaly detection framework using LAL-AdaBoost for structural health monitoring

The development of structural health monitoring (SHM) on civil infrastructures has resulted in enormous amount of acquired data along with the pressure of data processing and data mining. Abnormalities in data can lead to serious analytical error in later assessment. Such anomalous data patterns generally account for a relatively small portion of the overall dataset, which can be easily misclassified as normal data by regular classifiers. In this paper, a novel and robust data anomaly detection framework was proposed. The core novelty in this framework is the utilization of learning active learning (LAL) and AdaBoost algorithm aiming to reduce the costly manual work of labeling and improve the classification of anomaly patterns. Furthermore, the problem of biased classification brought by imbalanced datasets has also been solved by the LAL. Wavelet packet transform was also utilized to extract features from the acceleration data. The methodologies were firstly introduced precisely in this paper followed by two study cases to verify the feasibility of the proposed framework for data anomaly detection. The first case was a dataset with the anomalies synthetically added to the acceleration time history data measured in dynamic tests of a grid structure, including five kinds of data abnormalities. Both the balanced and imbalanced datasets were studied and analyzed, where a comparative study was carried out between the LAL-AdaBoost and uncertainty sampling-based AdaBoost with the same training and testing sets. The results showed that LAL-AdaBoost outperformed in both scenarios with higher accuracies and faster convergence speed. Then, a further study was carried out using acceleration data collected from a long-span bridge. By querying only limited amount of the training set, the proposed framework could accurately detect and classify 97.95% anomaly patterns of the testing set, showing great potential for further and broader application in the field of SHM data processing.