To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making

被引：220
作者：
Buçinca Z. [1 ]
Malaya M.B. [2 ]
Gajos K.Z. [1 ]
机构：
[1] Harvard University, 33 Oxford St., Cambridge, 02138, MA
[2] Lodz University of Technology, ul. Stefana Zeromskiego 116, Lodz
来源：
Proceedings of the ACM on Human-Computer Interaction | 2021年 / 5卷 / CSCW1期
关键词：
artificial intelligence; cognition; explanations; trust;
D O I：
10.1145/3449287
中图分类号：
学科分类号：
摘要：
People supported by AI-powered decision support tools frequently overrely on the AI: they accept an AI's suggestion even when that suggestion is wrong. Adding explanations to the AI decisions does not appear to reduce the overreliance and some studies suggest that it might even increase it. Informed by the dual-process theory of cognition, we posit that people rarely engage analytically with each individual AI recommendation and explanation, and instead develop general heuristics about whether and when to follow the AI suggestions. Building on prior research on medical decision-making, we designed three cognitive forcing interventions to compel people to engage more thoughtfully with the AI-generated explanations. We conducted an experiment (N=199), in which we compared our three cognitive forcing designs to two simple explainable AI approaches and to a no-AI baseline. The results demonstrate that cognitive forcing significantly reduced overreliance compared to the simple explainable AI approaches. However, there was a trade-off: people assigned the least favorable subjective ratings to the designs that reduced the overreliance the most. To audit our work for intervention-generated inequalities, we investigated whether our interventions benefited equally people with different levels of Need for Cognition (i.e., motivation to engage in effortful mental activities). Our results show that, on average, cognitive forcing interventions benefited participants higher in Need for Cognition more. Our research suggests that human cognitive motivation moderates the effectiveness of explainable AI solutions. © 2021 ACM.
引用
收藏
相关论文
共 67 条
  • [21] Fitzsimons G.J., Lehmann D.R., Reactance to recommendations: When unsolicited advice yields contrary responses, Marketing Science, 23, 1, pp. 82-94, (2004)
  • [22] Gajos K.Z., Chauncey K., The Influence of Personality Traits and Cognitive Load on the Use of Adaptive User Interfaces, Proceedings of the 22Nd International Conference on Intelligent User Interfaces (Limassol, Cyprus) (IUI '17), pp. 301-306, (2017)
  • [23] Garg N., Sethupathy A., Tuwani R., Nk R., Dokania S., Iyer A., Gupta A., Agrawal S., Singh N., Shukla S., Et al., FlavorDB: a database of flavor molecules, Nucleic acids research, 46, D1, pp. D1210-D1216, (2018)
  • [24] Ghai B., Vera Liao Q., Zhang Y., Bellamy R., Mueller K., Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers, Proc. ACM Hum.-Comput. Interact., 4, CSCW3, (2021)
  • [25] Graber M.L., Kissam S., Payne V.L., Meyer A.N.D., Sorensen A., Lenfestey N., Tant E., Henriksen K., LaBresh K., Singh H., Cognitive interventions to reduce diagnostic error: a narrative review, BMJ quality & safety, 21, 7, pp. 535-557, (2012)
  • [26] Green B., Chen Y., The principles and limits of algorithm-in-the-loop decision making, Proceedings of the ACM on Human-Computer Interaction, 3, CSCW, pp. 1-24, (2019)
  • [27] Haugtvedt C.P., Petty R.E., Cacioppo J.T., Need for cognition and advertising: Understanding the role of personality variables in consumer behavior, Journal of Consumer Psychology, 1, 3, pp. 239-260, (1992)
  • [28] Holm S., A simple sequentially rejective multiple test procedure, Scandinavian Journal of Statistics, 6, 2, pp. 65-70, (1979)
  • [29] Hullman J., Adar E., Shah P., Benefitting infovis with visual difficulties, IEEE Transactions on Visualization and Computer Graphics, 17, 12, pp. 2213-2222, (2011)
  • [30] Jacobs M., Pradier M.F., McCoy T.H., Perlis R., Doshi-Velez F., Gajos K.Z., How machine learning recommendations influence clinician treatment selections: the example of antidepressant selection, Translational Psychiatry, (2021)
1234567