User-Centric Explainability in Healthcare: A Knowledge-Level Perspective of Informed Machine Learning

Explaining increasingly complex machine learning will remain crucial to cope with risks, regulations, responsibilities, and human support in healthcare. However, extant explainable systems mostly provide explanations that mismatch clinical users' conceptions and fail their expectations to leverage validated and clinically relevant information. A key to more user-centric and satisfying explanations can be seen in combining data-driven and knowledge-based systems, i.e., to utilize prior knowledge jointly with the patterns learned from data. We conduct a structured review of knowledge-informed machine learning in healthcare. In this article, we build on a framework to characterize user knowledge and prior knowledge embodied in explanations. Specifically, we explicate the types and contexts of knowledge to examine the fit between knowledge-informed approaches and users. Our results highlight that knowledge-informed machine learning is a promising paradigm to enrich former data-driven systems, yielding explanations that can increase formal understanding, convey useful medical knowledge, and are more intuitive. Although complying with medical conception, it still needs to be investigated whether knowledge-informed explanations increase medical user acceptance and trust in clinical machine learning-based information systems. © 2020 IEEE.