Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system

被引:107
作者
Henry, Katharine E. [1 ]
Kornfield, Rachel [2 ,3 ]
Sridharan, Anirudh [4 ]
Linton, Robert C. [4 ]
Groh, Catherine [5 ]
Wang, Tony [1 ]
Wu, Albert [6 ]
Mutlu, Bilge [5 ,7 ]
Saria, Suchi [1 ,6 ,8 ,9 ]
机构
[1] Johns Hopkins Univ, Dept Comp Sci, Baltimore, MD 21218 USA
[2] Northwestern Univ, Feinberg Sch Med, Dept Prevent Med, Chicago, IL USA
[3] Northwestern Univ, Ctr Behav Intervent Technol, Chicago, IL 60611 USA
[4] Howard Cty Gen Hosp, Columbia, MD USA
[5] Univ Wisconsin, Dept Ind Engn, Madison, WI 53706 USA
[6] Johns Hopkins Bloomberg Sch Publ Hlth, Dept Hlth Policy & Management, Baltimore, MD 21205 USA
[7] Univ Wisconsin, Dept Comp Sci, Madison, WI 53706 USA
[8] Johns Hopkins Univ, Dept Appl Math & Stat, Baltimore, MD 21218 USA
[9] Bayesian Hlth, New York, NY 10005 USA
基金
美国国家科学基金会;
关键词
DECISION-SUPPORT; BIG DATA; SEPSIS; MEDICINE; MODELS;
D O I
10.1038/s41746-022-00597-7
中图分类号
R19 [保健组织与事业(卫生事业管理)];
学科分类号
摘要
While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians' autonomy and support them across their entire workflow.
引用
收藏
页数:6
相关论文
共 44 条
[21]   Trust in automation: Designing for appropriate reliance [J].
Lee, JD ;
See, KA .
HUMAN FACTORS, 2004, 46 (01) :50-80
[22]   Hospital Deaths in Patients With Sepsis From 2 Independent Cohorts [J].
Liu, Vincent ;
Escobar, Gabriel J. ;
Greene, John D. ;
Soule, Jay ;
Whippy, Alan ;
Angus, Derek C. ;
Iwashyna, Theodore J. .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2014, 312 (01) :90-92
[23]   Adaptive design of a clinical decision support tool: What the impact on utilization rates means for future CDS research [J].
Mann, Devin ;
Hess, Rachel ;
McGinn, Thomas ;
Mishuris, Rebecca ;
Chokshi, Sara ;
McCullagh, Lauren ;
Smith, Paul D. ;
Palmisano, Joseph ;
Richardson, Safiya ;
Feldstein, David A. .
DIGITAL HEALTH, 2019, 5
[24]   Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice [J].
McDonald, Nora ;
Schoenebeck, Sarita ;
Forte, Andrea .
Proceedings of the ACM on Human-Computer Interaction, 2019, 3 (CSCW)
[25]   From Annoying to Appreciated Turning clinical decision support systems into a medical professional's best friend [J].
Mertz, Leslie .
IEEE PULSE, 2015, 6 (05) :4-9
[26]   Standards for Reporting Qualitative Research: A Synthesis of Recommendations [J].
O'Brien, Bridget C. ;
Harris, Ilene B. ;
Beckman, Thomas J. ;
Reed, Darcy A. ;
Cook, David A. .
ACADEMIC MEDICINE, 2014, 89 (09) :1245-1251
[27]   Predicting the Future - Big Data, Machine Learning, and Clinical Medicine [J].
Obermeyer, Ziad ;
Emanuel, Ezekiel J. .
NEW ENGLAND JOURNAL OF MEDICINE, 2016, 375 (13) :1216-1219
[28]   Epidemiology and Costs of Sepsis in the United StatesAn Analysis Based on Timing of Diagnosis and Severity Level* [J].
Paoli, Carly J. ;
Reynolds, Mark A. ;
Sinha, Meenal ;
Gitlin, Matthew ;
Crouser, Elliott .
CRITICAL CARE MEDICINE, 2018, 46 (12) :1889-1897
[29]   Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014 [J].
Rhee, Chanu ;
Dantes, Raymund ;
Epstein, Lauren ;
Murphy, David J. ;
Seymour, Christopher W. ;
Iwashyna, Theodore J. ;
Kadri, Sameer S. ;
Angus, Derek C. ;
Danner, Robert L. ;
Fiore, Anthony E. ;
Jernigan, John A. ;
Martin, Greg S. ;
Septimus, Edward ;
Warren, David K. ;
Karcz, Anita ;
Chan, Christina ;
Menchaca, John T. ;
Wang, Rui ;
Gruber, Susan ;
Klompas, Michael .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2017, 318 (13) :1241-1249
[30]   To catch a killer: electronic sepsis alert tools reaching a fever pitch? [J].
Ruppel, Halley ;
Liu, Vincent .
BMJ QUALITY & SAFETY, 2019, 28 (09) :693-696