Online Knowledge Distillation for Machine Health Prognosis Considering Edge Deployment

Complex neural networks with deep structures are beneficial for solving problems, such as fault classification and health prediction of industrial equipment, due to their powerful feature extraction capabilities. Unfortunately, corresponding complex models designed based on deep learning algorithms require huge computational and memory resources, making them difficult to achieve effective edge deployment. In order to solve this difficulty with practical industrial significance, this article proposes an online knowledge distillation framework for machine health prognosis. Within this framework, the learned knowledge of complex networks can be distilled to simple networks that can be deployed on edge devices in sites. Specifically, the response-based knowledge distillation module, feature-based knowledge distillation module, and relation-based knowledge distillation module are, respectively, designed to achieve effective information transmission from different levels. Furthermore, the inherent differences between simple and complex networks have been fully considered for their impact on the efficiency of knowledge distillation, and an adaptive mutual learning strategy has been contrapuntally proposed to address this limitation. Multiple online knowledge distillation experiments were conducted on two different sets of run-to-failure data sets of mechanical key components with different pairs of complex and simple networks to verify the effectiveness of the proposed framework. The experimental results show that the simple student-networks can effectively improve prediction performance after receiving knowledge distillation from the complex teacher-networks, providing a new solution for machine health prognosis under the premise of edge deployment.