The Role of Explanations in Human-Machine Learning

In this paper, we study explanations in a setting where human capabilities are in parity with Machine Learning (ML) capabilities. If an ML system is to be trusted in this situation, limitations in the trained ML model's abilities have to be exposed to the end-user. A majority of current approaches focus on the task of creating explanations for a proposed decision, but less attention is given to the equally important task of exposing limitations in the ML model's capabilities, limitations that in turn affect the validity of created explanations. Using a small-scale design experiment we compare human explanations with explanations created by an ML system. This paper explores and presents how the structure and terminology of scientific explanations can expose limitations in the ML models knowledge and be used as an approach for research and design in the area of explainable artificial intelligence.