FS-SCF network: Neural network interpretability based on counterfactual generation and feature selection for fault diagnosis

Interpretability of neural networks aims at the development of models that can give information to the end-user about its inner workings and/or predictions, while keeping the high levels of performance of neural networks. In the context of fault diagnosis, interpretability is necessary for bias detection, model debugging and building trust. This is particularly relevant in the field due to the frequent applications in safety-critical systems, in which an undetected failure can result in highly detrimental outcomes. In this paper, a novel interpretable neural network model is proposed, referred to as Feature Selection and Sparse Counterfactual Generation (FS-SCF) network. It is a multi-task neural network divided into two branches: one for fault diagnosis and feature selection, and one for counterfactual generation. Thus, the model can be interpreted in terms of what are the most relevant features, and the obtained individual predictions are interpreted through counterfactuals. Also, the generated counterfactuals are used to evaluate necessity and sufficiency of each feature. The obtained rankings are then compared to the ranking obtained through the feature selection technique. The proposed approach is tested in two case studies with real data from the industry. Results show that the proposed neural network is able to achieve high levels of performance for fault diagnosis, generate interpretable counterfactuals, and determine the importance of each feature according to a criterion similar to both necessity and sufficiency.