Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system

被引:88
作者
Henry, Katharine E. [1 ]
Kornfield, Rachel [2 ,3 ]
Sridharan, Anirudh [4 ]
Linton, Robert C. [4 ]
Groh, Catherine [5 ]
Wang, Tony [1 ]
Wu, Albert [6 ]
Mutlu, Bilge [5 ,7 ]
Saria, Suchi [1 ,6 ,8 ,9 ]
机构
[1] Johns Hopkins Univ, Dept Comp Sci, Baltimore, MD 21218 USA
[2] Northwestern Univ, Feinberg Sch Med, Dept Prevent Med, Chicago, IL USA
[3] Northwestern Univ, Ctr Behav Intervent Technol, Chicago, IL 60611 USA
[4] Howard Cty Gen Hosp, Columbia, MD USA
[5] Univ Wisconsin, Dept Ind Engn, Madison, WI 53706 USA
[6] Johns Hopkins Bloomberg Sch Publ Hlth, Dept Hlth Policy & Management, Baltimore, MD 21205 USA
[7] Univ Wisconsin, Dept Comp Sci, Madison, WI 53706 USA
[8] Johns Hopkins Univ, Dept Appl Math & Stat, Baltimore, MD 21218 USA
[9] Bayesian Hlth, New York, NY 10005 USA
基金
美国国家科学基金会;
关键词
DECISION-SUPPORT; BIG DATA; SEPSIS; MEDICINE; MODELS;
D O I
10.1038/s41746-022-00597-7
中图分类号
R19 [保健组织与事业(卫生事业管理)];
学科分类号
摘要
While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians' autonomy and support them across their entire workflow.
引用
收藏
页数:6
相关论文
共 44 条
[1]  
[Anonymous], 2018, ABS180200682 CORR
[2]  
[Anonymous], NAT MED, DOI [10.1038/s41591-022-01894-0(2022), DOI 10.1038/S41591-022-01894-0(2022)]
[3]   Big Data In Health Care: Using Analytics To Identify And Manage High-Risk And High-Cost Patients [J].
Bates, David W. ;
Saria, Suchi ;
Ohno-Machado, Lucila ;
Shah, Anand ;
Escobar, Gabriel .
HEALTH AFFAIRS, 2014, 33 (07) :1123-1131
[4]   Identifying Patients With Sepsis on the Hospital Wards [J].
Bhattacharjee, Poushali ;
Edelson, Dana P. ;
Churpek, Matthew M. .
CHEST, 2017, 151 (04) :898-907
[5]  
Centers for Medicare and Medicaid Services, 2016, CMS ANN UPD SEP 1 VA
[6]   Machine Learning and Prediction in Medicine - Beyond the Peak of Inflated Expectations [J].
Chen, Jonathan H. ;
Asch, Steven M. .
NEW ENGLAND JOURNAL OF MEDICINE, 2017, 376 (26) :2507-2509
[7]   Towards a Theory of Longitudinal Trust Calibration in Human-Robot Teams [J].
de Visser, Ewart J. ;
Peeters, Marieke M. M. ;
Jung, Malte F. ;
Kohn, Spencer ;
Shaw, Tyler H. ;
Pak, Richard ;
Neerincx, Mark A. .
INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS, 2020, 12 (02) :459-478
[8]   Understanding human-robot teams in light of all-human teams: Aspects of team interaction and shared cognition [J].
Demir, Mustafa ;
McNeese, Nathan J. ;
Cooke, Nancy J. .
INTERNATIONAL JOURNAL OF HUMAN-COMPUTER STUDIES, 2020, 140
[9]   Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err [J].
Dietvorst, Berkeley J. ;
Simmons, Joseph P. ;
Massey, Cade .
JOURNAL OF EXPERIMENTAL PSYCHOLOGY-GENERAL, 2015, 144 (01) :114-126
[10]  
Edmondson AC, 2007, ACAD MANAGE REV, V32, P1155, DOI 10.5465/AMR.2007.26586086