Towards a More Reliable Interpretation of Machine Learning Outputs for Safety-Critical Systems Using Feature Importance Fusion

被引:16
作者
Rengasamy, Divish [1 ]
Rothwell, Benjamin C. [1 ]
Figueredo, Grazziela P. [2 ]
机构
[1] Univ Nottingham, Gas Turbine & Transmiss Res Ctr, Nottingham NG7 2TU, England
[2] Univ Nottingham, Sch Comp Sci, Adv Data Anal Ctr, Nottingham NG8 1BB, England
来源
APPLIED SCIENCES-BASEL | 2021年 / 11卷 / 24期
关键词
accountability; data fusion; deep learning; ensemble feature importance; explainable artificial intelligence; interpretability; machine learning; responsible artificial intelligence; NEURAL-NETWORKS; BLACK-BOX; PREDICTION; ALGORITHM; SELECTION; NOISE;
D O I
10.3390/app112411854
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
When machine learning supports decision-making in safety-critical systems, it is important to verify and understand the reasons why a particular output is produced. Although feature importance calculation approaches assist in interpretation, there is a lack of consensus regarding how features' importance is quantified, which makes the explanations offered for the outcomes mostly unreliable. A possible solution to address the lack of agreement is to combine the results from multiple feature importance quantifiers to reduce the variance in estimates and to improve the quality of explanations. Our hypothesis is that this leads to more robust and trustworthy explanations of the contribution of each feature to machine learning predictions. To test this hypothesis, we propose an extensible model-agnostic framework divided in four main parts: (i) traditional data pre-processing and preparation for predictive machine learning models, (ii) predictive machine learning, (iii) feature importance quantification, and (iv) feature importance decision fusion using an ensemble strategy. Our approach is tested on synthetic data, where the ground truth is known. We compare different fusion approaches and their results for both training and test sets. We also investigate how different characteristics within the datasets affect the quality of the feature importance ensembles studied. The results show that, overall, our feature importance ensemble framework produces 15% less feature importance errors compared with existing methods. Additionally, the results reveal that different levels of noise in the datasets do not affect the feature importance ensembles' ability to accurately quantify feature importance, whereas the feature importance quantification error increases with the number of features and number of orthogonal informative features. We also discuss the implications of our findings on the quality of explanations provided to safety-critical systems.
引用
收藏
页数:19
相关论文
共 51 条
  • [1] Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
    Adadi, Amina
    Berrada, Mohammed
    [J]. IEEE ACCESS, 2018, 6 : 52138 - 52160
  • [2] Permutation importance: a corrected feature importance measure
    Altmann, Andre
    Tolosi, Laura
    Sander, Oliver
    Lengauer, Thomas
    [J]. BIOINFORMATICS, 2010, 26 (10) : 1340 - 1347
  • [3] Alvarez-Melis D., 2018, On the robustness of interpretability methods
  • [4] Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
    Barredo Arrieta, Alejandro
    Diaz-Rodriguez, Natalia
    Del Ser, Javier
    Bennetot, Adrien
    Tabik, Siham
    Barbado, Alberto
    Garcia, Salvador
    Gil-Lopez, Sergio
    Molina, Daniel
    Benjamins, Richard
    Chatila, Raja
    Herrera, Francisco
    [J]. INFORMATION FUSION, 2020, 58 : 82 - 115
  • [5] Bergstra J, 2012, J MACH LEARN RES, V13, P281
  • [6] TRAINING WITH NOISE IS EQUIVALENT TO TIKHONOV REGULARIZATION
    BISHOP, CM
    [J]. NEURAL COMPUTATION, 1995, 7 (01) : 108 - 116
  • [7] Random forests
    Breiman, L
    [J]. MACHINE LEARNING, 2001, 45 (01) : 5 - 32
  • [8] Brundage M., 2020, Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
  • [9] A machine learning-based algorithm for processing massive data collected from the mechanical components of movable bridges
    Catbas, F. Necati
    Malekzadeh, Masoud
    [J]. AUTOMATION IN CONSTRUCTION, 2016, 72 : 269 - 278
  • [10] Chakraborty Supriyo, Interpretability of Deep Learning Models: A Survey of Results, 2017 IEEE SMARTWORLD