Toward a responsible future: recommendations for AI-enabled clinical decision support

被引：20

作者：

Labkoff, Steven ^{[1
,2
]}

Oladimeji, Bilikis ^{[3
]}

Kannry, Joseph ^{[4
]}

Solomonides, Anthony ^{[5
]}

Leftwich, Russell ^{[6
]}

Koski, Eileen ^{[7
]}

Joseph, Amanda L. ^{[8
]}

Lopez-Gonzalez, Monica ^{[9
]}

Fleisher, Lee A. ^{[10
]}

Nolen, Kimberly ^{[11
]}

Dutta, Sayon ^{[12
,13
,14
]}

Levy, Deborah R. ^{[15
,16
]}

Price, Amy ^{[17
,18
]}

Barr, Paul J. ^{[17
]}

Hron, Jonathan D. ^{[19
,20
]}

Lin, Baihan ^{[21
,22
]}

Srivastava, Gyana ^{[2
,23
]}

Pastor, Nuria ^{[24
]}

Luque, Unai Sanchez ^{[24
]}

Bui, Tien Thi Thuy ^{[25
]}

Singh, Reva ^{[26
]}

Williams, Tayler ^{[26
]}

Weiner, Mark G. ^{[27
]}

Naumann, Tristan ^{[28
]}

Sittig, Dean F. ^{[29
]}

Jackson, Gretchen Purcell ^{[30
,31
]}

Quintana, Yuri ^{[2
,8
,12
,32
]}

机构：

[1] Quantori, Boston, MA 02142 USA

[2] Beth Israel Deaconess Med Ctr, Dept Med, Div Clin Informat, Boston, MA 02215 USA

[3] UnitedHlth Grp, Minnetonka, MN USA

[4] Icahn Sch Med Mt Sinai, Dept Med, Div Gen Internal Med, New York, NY 10029 USA

[5] Endeavor Hlth, Res Inst, Evanston, IL 60035 USA

[6] Vanderbilt Univ, Dept Biomed Informat, Nashville, TN USA

[7] IBM Res, Yorktown Hts, NY USA

[8] Univ Victoria, Sch Hlth Informat Sci, Victoria, BC, Canada

[9] Cognit Insights Artificial Intelligence, Baltimore, MD USA

[10] Univ Penn, Anesthesiol & Crit Care, Perelman Sch Med, Philadelphia, PA USA

[11] Pfizer Inc, New York, NY USA

[12] Massachusetts Gen Hosp, Dept Emergency Med, Boston, MA USA

[13] Mass Gen Brigham Digital, Clin Informat, Boston, MA USA

[14] Harvard Med Sch, Boston, MA USA

[15] VA Connecticut Healthcare Syst, Pain Res Informat Multimorbid & Epidemiol PRIME Ct, Dept Med, West Haven, CT USA

[16] Yale Sch Med, Dept Biomed Informat & Data Sci, New Haven, CT USA

[17] Geisel Sch Med Dartmouth, Dartmouth Inst Hlth Policy & Clin Practice, Hanover, NH USA

[18] BMJ, London, England

[19] Boston Childrens Hosp, Dept Pediat, Div Gen Pediat, Boston, MA 02115 USA

[20] Harvard Med Sch, Dept Pediat, Boston, MA 02115 USA

[21] Icahn Sch Med Mt Sinai, Dept Psychiat & Neurosci, New York, NY USA

[22] Harvard Law Sch, Berkman Klein Ctr Internet & Soc, Cambridge, MA USA

[23] Harvard Sch Publ Hlth, Boston, MA 02115 USA

[24] Vitalera, Barcelona 08028, Spain

[25] Massachusetts Coll Pharm & Hlth Sci, Boston, MA USA

[26] Amer Med Informat Assoc, Washington, DC USA

[27] Weill Cornell Med, Dept Populat Hlth Sci, New York, NY USA

[28] Microsoft Res, Redmond, WA USA

[29] Univ Texas Hlth Sci Ctr Houston, Dept Clin & Hlth Informat, Houston, TX USA

[30] Intuit Surg, Nashville, TN USA

[31] Vanderbilt Univ, Med Ctr, Dept Pediat & Biomed Informat, Nashville, TN USA

[32] Homewood Res Inst, Guelph, ON, Canada

来源：

JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION | 2024年 / 31卷 / 11期

关键词：

clinical decision support; artificial intelligence; clinician AI competencies; patient safety; algorithmic transparency; HEALTH; OUTCOMES; SAFETY; BASE; BIAS;

D O I：

10.1093/jamia/ocae209

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Background Integrating artificial intelligence (AI) in healthcare settings has the potential to benefit clinical decision-making. Addressing challenges such as ensuring trustworthiness, mitigating bias, and maintaining safety is paramount. The lack of established methodologies for pre- and post-deployment evaluation of AI tools regarding crucial attributes such as transparency, performance monitoring, and adverse event reporting makes this situation challenging.Objectives This paper aims to make practical suggestions for creating methods, rules, and guidelines to ensure that the development, testing, supervision, and use of AI in clinical decision support (CDS) systems are done well and safely for patients.Materials and Methods In May 2023, the Division of Clinical Informatics at Beth Israel Deaconess Medical Center and the American Medical Informatics Association co-sponsored a working group on AI in healthcare. In August 2023, there were 4 webinars on AI topics and a 2-day workshop in September 2023 for consensus-building. The event included over 200 industry stakeholders, including clinicians, software developers, academics, ethicists, attorneys, government policy experts, scientists, and patients. The goal was to identify challenges associated with the trusted use of AI-enabled CDS in medical practice. Key issues were identified, and solutions were proposed through qualitative analysis and a 4-month iterative consensus process.Results Our work culminated in several key recommendations: (1) building safe and trustworthy systems; (2) developing validation, verification, and certification processes for AI-CDS systems; (3) providing a means of safety monitoring and reporting at the national level; and (4) ensuring that appropriate documentation and end-user training are provided.Discussion AI-enabled Clinical Decision Support (AI-CDS) systems promise to revolutionize healthcare decision-making, necessitating a comprehensive framework for their development, implementation, and regulation that emphasizes trustworthiness, transparency, and safety. This framework encompasses various aspects including model training, explainability, validation, certification, monitoring, and continuous evaluation, while also addressing challenges such as data privacy, fairness, and the need for regulatory oversight to ensure responsible integration of AI into clinical workflow.Conclusions Achieving responsible AI-CDS systems requires a collective effort from many healthcare stakeholders. This involves implementing robust safety, monitoring, and transparency measures while fostering innovation. Future steps include testing and piloting proposed trust mechanisms, such as safety reporting protocols, and establishing best practice guidelines.

引用

页码：2730 / 2739

页数：10

共 65 条

[21]

dcinetwork, AI HEALTHCARE RISK M

[22]

digital-strategy.ec.europa, REGULATORY FRAMEWORK

[23]

Donald M., MPP MORAL DETERMINAN

[24] Algorithmovigilance-Advancing Methods to Analyze and Monitor Artificial Intelligence-Driven Health Care for Effectiveness and Equity [J].

Embi, Peter J. .

JAMA NETWORK OPEN, 2021, 4 (04)

[25]

European Union, EU ARTIFICIAL INTELL

[26]

FDA, Artificial intelligence and machine learning (ai/ml)-enabled medical devices

[27]

fda, Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SAMD)-Discussion Paper and Request for Feedback

[28] Considering Biased Data as Informative Artifacts in AI-Assisted Health Care [J].

Ferryman, Kadija ;

Mackintosh, Maxine ;

Ghassemi, Marzyeh .

NEW ENGLAND JOURNAL OF MEDICINE, 2023, 389 (09) :833-838

[29]

Finnell J T., 2022, Clinical Informatics Study Guide: Text and Review

[30]

Finnell JT., 2022, CLIN INFORM STUDY GU, DOI [10.1007/978-3-030-93765-2, DOI 10.1007/978-3-030-93765-2]

← 1 2 3 4 5 6 7 →