A residual autoencoder-based transformer for fault detection of multivariate processes

The complexity of high-dimensional and noisy process signals reduces the effectiveness of conventional fault detection methods in industrial processes. Based on the hypothesis that data collected from normal and faulty processes has different characteristics, unsupervised deep neural networks, e.g., autoencoders, have been widely applied in process fault detection and achieved good performance. Many variants have been proposed to improve feature learning by combining different network structures. In this paper, a new transformer model, residual autoencoder-based transformer, is proposed for process fault detection. Firstly, autoencoder and transformer are integrated for better unsupervised feature learning of process signals. Secondly, linear embedding and attention mechanisms with bias are proposed to generate effective features from process signals. Finally, residual connections are constructed between the encoder and decoder of RATransformer to address overfitting in training. Four industrial cases are used to test the performance of RATransformer for process fault detection. The results show that the fault detection rate of RATransformer is at least 1 % higher than other comparison methods. Moreover, the testing results show that the model structure improves the fault detection performance of RATransformer. The complex models like RATransformer can be used in the industrial process when sufficient normal process data is available. An end-to-end training method can be further developed to improve the applicability of RATransformer in process fault detection in the future.