Exploring Evaluation Methods for Interpretable Machine Learning: A Survey

被引:8
|
作者
Alangari, Nourah [1 ]
Menai, Mohamed El Bachir [1 ]
Mathkour, Hassan [1 ]
Almosallam, Ibrahim [2 ]
机构
[1] King Saud Univ, Coll Comp & Informat Sci, Dept Comp Sci, Riyadh 11543, Saudi Arabia
[2] Saudi Informat Technol Co SITE, Riyadh 12382, Saudi Arabia
关键词
interpretability; explainable AI; evaluating interpretability; BLACK-BOX; RULES; CLASSIFICATION; ACCURACY; ISSUES;
D O I
10.3390/info14080469
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In recent times, the progress of machine learning has facilitated the development of decision support systems that exhibit predictive accuracy, surpassing human capabilities in certain scenarios. However, this improvement has come at the cost of increased model complexity, rendering them black-box models that obscure their internal logic from users. These black boxes are primarily designed to optimize predictive accuracy, limiting their applicability in critical domains such as medicine, law, and finance, where both accuracy and interpretability are crucial factors for model acceptance. Despite the growing body of research on interpretability, there remains a significant dearth of evaluation methods for the proposed approaches. This survey aims to shed light on various evaluation methods employed in interpreting models. Two primary procedures are prevalent in the literature: qualitative and quantitative evaluations. Qualitative evaluations rely on human assessments, while quantitative evaluations utilize computational metrics. Human evaluation commonly manifests as either researcher intuition or well-designed experiments. However, this approach is susceptible to human biases and fatigue and cannot adequately compare two models. Consequently, there has been a recent decline in the use of human evaluation, with computational metrics gaining prominence as a more rigorous method for comparing and assessing different approaches. These metrics are designed to serve specific goals, such as fidelity, comprehensibility, or stability. The existing metrics often face challenges when scaling or being applied to different types of model outputs and alternative approaches. Another important factor that needs to be addressed is that while evaluating interpretability methods, their results may not always be entirely accurate. For instance, relying on the drop in probability to assess fidelity can be problematic, particularly when facing the challenge of out-of-distribution data. Furthermore, a fundamental challenge in the interpretability domain is the lack of consensus regarding its definition and requirements. This issue is compounded in the evaluation process and becomes particularly apparent when assessing comprehensibility.
引用
收藏
页数:29
相关论文
共 50 条
  • [31] Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey
    Liu, Hongyu
    Lang, Bo
    APPLIED SCIENCES-BASEL, 2019, 9 (20):
  • [32] Interpretable Machine Learning with Gradual Argumentation Frameworks
    Spieler, Jonathan
    Potyka, Nico
    Staab, Steffen
    COMPUTATIONAL MODELS OF ARGUMENT, COMMA 2022, 2022, 353 : 373 - 374
  • [33] Interpretable Machine Learning with Boosting by Boolean Algorithm
    Neuhaus, Nathan
    Kovalerchuk, Boris
    2019 JOINT 8TH INTERNATIONAL CONFERENCE ON INFORMATICS, ELECTRONICS & VISION (ICIEV) AND 2019 3RD INTERNATIONAL CONFERENCE ON IMAGING, VISION & PATTERN RECOGNITION (ICIVPR) WITH INTERNATIONAL CONFERENCE ON ACTIVITY AND BEHAVIOR COMPUTING (ABC), 2019, : 307 - 311
  • [34] Exploring Interpretable AI Methods for ECG Data Classification
    Ojha, Jaya
    Haugerud, Harek
    Yazidi, Anis
    Lind, Pedro G.
    PROCEEDINGS OF THE 5TH ACM WORKSHOP ON INTELLIGENT CROSS-DATA ANALYSIS AND RETRIEVAL, ICDAR 2024, 2024, : 11 - 18
  • [35] Data mining and machine learning methods for sustainable smart cities traffic classification: A survey
    Shafiq, Survey Muhammad
    Tian, Zhihong
    Bashir, Ali Kashif
    Jolfaei, Alireza
    Yu, Xiangzhan
    SUSTAINABLE CITIES AND SOCIETY, 2020, 60
  • [36] A comparative analysis on the reliability of interpretable machine learning
    Yildirim, Mustafa
    Yildirim Okay, Feyza
    Ozdemir, Suat
    PAMUKKALE UNIVERSITY JOURNAL OF ENGINEERING SCIENCES-PAMUKKALE UNIVERSITESI MUHENDISLIK BILIMLERI DERGISI, 2024, 30 (04): : 494 - 508
  • [37] Advancing Computational Toxicology by Interpretable Machine Learning
    Jia, Xuelian
    Wang, Tong
    Zhu, Hao
    ENVIRONMENTAL SCIENCE & TECHNOLOGY, 2023, 57 (46) : 17690 - 17706
  • [38] Review of interpretable machine learning for process industries
    Carter, A.
    Imtiaz, S.
    Naterer, G. F.
    PROCESS SAFETY AND ENVIRONMENTAL PROTECTION, 2023, 170 : 647 - 659
  • [39] Evaluating interpretable machine learning predictions for cryptocurrencies
    El Majzoub, Ahmad
    Rabhi, Fethi A.
    Hussain, Walayat
    INTELLIGENT SYSTEMS IN ACCOUNTING FINANCE & MANAGEMENT, 2023, 30 (03) : 137 - 149
  • [40] Exploring the State of Machine Learning and Deep Learning in Medicine: A Survey of the Italian Research Community
    Bottrighi, Alessio
    Pennisi, Marzio
    INFORMATION, 2023, 14 (09)