Efficient temporal flow Transformer accompanied with multi-head probsparse self-attention mechanism for remaining useful life prognostics

被引:51
作者
Chang, Yuanhong [1 ]
Li, Fudong [1 ]
Chen, Jinglong [1 ]
Liu, Yulang [1 ]
Li, Zipeng [2 ]
机构
[1] Xi An Jiao Tong Univ, State Key Lab Mfg & Syst Engn, Xian 710049, Peoples R China
[2] Northwestern Polytech Univ, Sch Marine Sci & Technol, Xian 710072, Peoples R China
基金
中国国家自然科学基金;
关键词
Remaining useful life prognostics; Transformer model; Probsparse self -attention mechanism; Rolling bearings; RECURRENT NEURAL-NETWORK; PREDICTION;
D O I
10.1016/j.ress.2022.108701
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Predictive maintenance, such as remaining useful life (RUL) prognostics, requires precise long time-series forecasting, which demands a higher predictive capability of data-driven models. Nevertheless, the typical convolution and recurrent frameworks are still inadequate in the feature extraction and temporal complexity analysis, which makes them difficult to efficiently capture the precise long-term dependency coupling. Recent research has demonstrated the potential of Transformer-based framework to improve the prediction capability by the massive success in sequence processing. Inspired by the above, this paper proposes an efficient end-to-end Temporal Flow Transformer (TFT) for RUL prognostics of rolling bearings. Its main framework is composed of multi-layer encoders, which can directly extract effective degradation features from the time-frequency repre-sentations of raw signals, with two distinctive characteristics: (1) Specially designed multi-head probsparse self -attention mechanism can effectively highlight the dominant attention, which makes the TFT have considerable performance in reducing the computational complexity of extremely long time-series; (2) The TFT trained by knowledge-induced distillation strategy can significantly improve its domain adaptability, making it possible to achieve accurate RUL prediction under cross-operating conditions. Extensive experiments on two life-cycle bearing datasets indicate that the TFT greatly outperforms the existing state-of-the-art methods and provides a new solution for RUL prognostics.
引用
收藏
页数:17
相关论文
共 31 条
[1]   A novel temporal convolutional network with residual self-attention mechanism for remaining useful life prediction of rolling bearings [J].
Cao, Yudong ;
Ding, Yifei ;
Jia, Minping ;
Tian, Rushuai .
RELIABILITY ENGINEERING & SYSTEM SAFETY, 2021, 215
[2]   Similarity Metric-Based Metalearning Network Combining Prior Metatraining Strategy for Intelligent Fault Detection Under Small Samples Prerequisite [J].
Chang, Yuanhong ;
Chen, Jinglong ;
He, Shuilong ;
Pan, Tongyang .
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2022, 71
[3]   Intelligent Fault Quantitative Identification for Industrial Internet of Things (IIoT) via a Novel Deep Dual Reinforcement Learning Model Accompanied With Insufficient Samples [J].
Chang, Yuanhong ;
Chen, Jinglong ;
Wu, Wenyang ;
Pan, Tongyang ;
Zhou, Zitong ;
He, Shuilong .
IEEE INTERNET OF THINGS JOURNAL, 2022, 9 (20) :19811-19822
[4]   Temporal convolution-based sorting feature repeat-explore network combining with multi-band information for remaining useful life estimation of equipment [J].
Chang, Yuanhong ;
Chen, Jinglong ;
Liu, Yulang ;
Xu, Enyong ;
He, Shuilong .
KNOWLEDGE-BASED SYSTEMS, 2022, 249
[5]   CFs-focused intelligent diagnosis scheme via alternative kernels networks with soft squeeze-and-excitation attention for fast-precise fault detection under slow & sharp speed variations [J].
Chang, Yuanhong ;
Chen, Jinglong ;
Chen, Qiang ;
Liu, Shen ;
Zhou, Zitong .
KNOWLEDGE-BASED SYSTEMS, 2022, 239
[6]   Heterogeneous bi-directional recurrent neural network combining fusion health indicator for predictive analytics of rotating machinery [J].
Chang, Yuanhong ;
Chen, Jinglong ;
Lv, Haixin ;
Liu, Shen .
ISA TRANSACTIONS, 2022, 122 :409-423
[7]   Intelligent fault diagnosis scheme via multi-module supervised-learning network with essential features capture-regulation strategy [J].
Chang, Yuanhong ;
Chen, Qiang ;
Chen, Jinglong ;
He, Shuilong ;
Li, Fudong ;
Zhou, Zitong .
ISA TRANSACTIONS, 2022, 129 :459-475
[8]   Intelligent fault diagnosis of Wind Turbines via a Deep Learning Network Using Parallel Convolution Layers with Multi-Scale Kernels [J].
Chang, Yuanhong ;
Chen, Jinglong ;
Qu, Cheng ;
Pan, Tongyang .
RENEWABLE ENERGY, 2020, 153 :205-213
[9]   Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process [J].
Chen Jinglong ;
Jing Hongjie ;
Chang Yuanhong ;
Liu Qian .
RELIABILITY ENGINEERING & SYSTEM SAFETY, 2019, 185 :372-382
[10]   Deep Attention Relation Network: A Zero-Shot Learning Method for Bearing Fault Diagnosis Under Unknown Domains [J].
Chen, Zuoyi ;
Wu, Jun ;
Deng, Chao ;
Wang, Xiaoqi ;
Wang, Yuanhang .
IEEE TRANSACTIONS ON RELIABILITY, 2023, 72 (01) :79-89