Human-centered XAI: Developing design patterns for explanations of clinical decision support systems

被引:101
作者
Schoonderwoerd, Tjeerd A. J. [1 ]
Jorritsma, Wiard [1 ]
Neerincx, Mark A. [1 ,2 ]
van den Bosch, Karel [1 ]
机构
[1] TNO, Kampweg 55, Soesterberg, Netherlands
[2] Delft Univ Technol, Mekelweg 5, Delft, Netherlands
关键词
Explainable AI; Explainability; Causability; Human-centered design; Clinical decision making; Decision-support system; User study; Design patterns; EXPLAINABLE ARTIFICIAL-INTELLIGENCE; DIAGNOSIS; TRUST;
D O I
10.1016/j.ijhcs.2021.102684
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Much of the research on eXplainable Artificial Intelligence (XAI) has centered on providing transparency of machine learning models. More recently, the focus on human-centered approaches to XAI has increased. Yet, there is a lack of practical methods and examples on the integration of human factors into the development processes of AI-generated explanations that humans prove to uptake for better performance. This paper presents a case study of an application of a human-centered design approach for AI-generated explanations. The approach consists of three components: Domain analysis to define the concept & context of explanations, Requirements elicitation & assessment to derive the use cases & explanation requirements, and the consequential Multi-modal interaction design & evaluation to create a library of design patterns for explanations. In a case study, we adopt the DoReMi-approach to design explanations for a Clinical Decision Support System (CDSS) for child health. In the requirements elicitation & assessment, a user study with experienced paediatricians uncovered what explanations the CDSS should provide. In the interaction design & evaluation, a second user study tested the consequential interaction design patterns. This case study provided a first set of user requirements and design patterns for an explainable decision support system in medical diagnosis, showing how to involve expert end users in the development process and how to develop, more or less, generic solutions for general design problems in XAI.
引用
收藏
页数:25
相关论文
共 69 条
[1]   Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) [J].
Adadi, Amina ;
Berrada, Mohammed .
IEEE ACCESS, 2018, 6 :52138-52160
[2]  
Alexander C., 1977, A Pattern Language: Towns, Buildings, Construction
[3]   Guidelines for Human-AI Interaction [J].
Amershi, Saleema ;
Weld, Dan ;
Vorvoreanu, Mihaela ;
Fourney, Adam ;
Nushi, Besmira ;
Collisson, Penny ;
Suh, Jina ;
Iqbal, Shamsi ;
Bennett, Paul N. ;
Inkpen, Kori ;
Teevan, Jaime ;
Kikin-Gil, Ruth ;
Horvitz, Eric .
CHI 2019: PROCEEDINGS OF THE 2019 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, 2019,
[4]  
Anjomshoae S, 2019, AAMAS '19: PROCEEDINGS OF THE 18TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS, P1078
[5]   Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI [J].
Barredo Arrieta, Alejandro ;
Diaz-Rodriguez, Natalia ;
Del Ser, Javier ;
Bennetot, Adrien ;
Tabik, Siham ;
Barbado, Alberto ;
Garcia, Salvador ;
Gil-Lopez, Sergio ;
Molina, Daniel ;
Benjamins, Richard ;
Chatila, Raja ;
Herrera, Francisco .
INFORMATION FUSION, 2020, 58 :82-115
[6]  
Berner ES, 2007, Clinical decision support systems: theory and practice, P3, DOI [DOI 10.1007/978-0-387-38319-4_1, 10.1007/978-0-387-38319-4, DOI 10.1007/978-0-387-38319-4]
[7]   Explaining AI: Fairly? Well? [J].
Burnett, Margaret .
PROCEEDINGS OF THE 25TH INTERNATIONAL CONFERENCE ON INTELLIGENT USER INTERFACES, IUI 2020, 2020,
[8]   The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems [J].
Bussone, Adrian ;
Stumpf, Simone ;
O'Sullivan, Dympna .
2015 IEEE INTERNATIONAL CONFERENCE ON HEALTHCARE INFORMATICS (ICHI 2015), 2015, :160-169
[9]   The Effects of Example-Based Explanations in a Machine Learning Interface [J].
Cai, Carrie J. ;
Jongejan, Jonas ;
Holbrook, Jess .
PROCEEDINGS OF IUI 2019, 2019, :258-262
[10]  
Caro-Martinez M., 2018, ICCBR 2018, P52