Heterogeneous Federated Learning Framework for IIoT Based on Selective Knowledge Distillation

The lack of complete labels and data heterogeneity are obstacles to the application of artificial intelligence-based methods in industrial scenarios, such as machinery fault diagnosis. To address these challenges, this article proposes a federated learning (FL) framework for the industrial Internet of Things based on bidirectional knowledge distillation (KD) and hard sample selection. In the framework, the cloud server provides a pretrained deep learning (DL) model based on the cross-domain public dataset to facilitate the cold start in real-world applications. Then during the training process, each participating factory trains its heterogeneous local DL model according to local data volume and computing resources. Bidirectional KD with feature maps and hard sample selection is then carried out on a shared dataset between the server and factories to share knowledge efficiently. Moreover, all the DL models used in the application of the proposed framework are designed based on expertise and attention mechanism to diagnose multiple types of machinery and faults. Case studies using the vibration data collected from multiple factories show that the proposed framework improves the fault diagnosis accuracy compared to other FL methods while significantly reducing communication overhead.