Can ChatGPT Accurately Answer a PICOT Question? Assessing AI Response to a Clinical Question

被引：34

作者：

Branum, Candise ^{[1
]}

Schiavenato, Martin ^{[2
]}

机构：

[1] Gonzaga Univ, Foley Ctr Lib, Foley Lib AD Box 95,502 E Boone Ave, Spokane, WA 99258 USA

[2] Gonzaga Univ, Sch Nursing & Human Physiol, Spokane, WA USA

来源：

NURSE EDUCATOR | 2023年 / 48卷 / 05期

关键词：

artificial intelligence; ChatGPT; information literacy; information storage and retrieval; machine learning; natural language processing; BLOOD-PRESSURE;

D O I：

10.1097/NNE.0000000000001436

中图分类号：

R47 [护理学];

学科分类号：

1011 ;

摘要：

Background:ChatGPT, an artificial intelligence (AI) text generator trained to predict correct words, can provide answers to questions but has shown mixed results in answering medical questions.Purpose:To assess the reliability and accuracy of ChatGPT in providing answers to a complex clinical question.Methods:A Population, Intervention, Comparison, Outcome, and Time (PICOT) formatted question was queried, along with a request for references. Full-text articles were reviewed to verify the accuracy of the evidence summary provided by the chatbot.Results:ChatGPT was unable to provide a certifiable response to a PICOT question. The references cited as evidence included incorrect journal information, and many study details summarized by ChatGPT proved to be patently false, including providing fabricated data.Conclusions:ChatGPT provides answers that appear legitimate but may be factually incorrect. The system is not transparent in how it gathers data to answer questions and sometimes fabricates information that looks plausible, making it an unreliable tool for clinical questions.

引用

页码：231 / 233

页数：3

共 11 条

[1] Effects of nurse-managed telemonitoring on blood pressure at 12-month follow-up among urban African Americans [J].

Artinian, Nancy T. ;

Flack, John M. ;

Nordstrom, Cheryl K. ;

Hockman, Eaine M. ;

Washington, Olivia G. M. ;

Jen, Kai-Lin Catherine ;

Fathy, Maryam .

NURSING RESEARCH, 2007, 56 (05) :312-322

[2] Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models [J].

Kung, Tiffany H. ;

Cheatham, Morgan ;

Medenilla, Arielle ;

Sillos, Czarina ;

De Leon, Lorie ;

Elepano, Camille ;

Madriaga, Maria ;

Aggabao, Rimel ;

Diaz-Candido, Giezel ;

Maningo, James ;

Tseng, Victor .

PLOS DIGITAL HEALTH, 2023, 2 (02)

[3] An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools [J].

Lauriola, Ivano ;

Lavelli, Alberto ;

Aiolli, Fabio .

NEUROCOMPUTING, 2022, 470 :443-456

[4]

Montti R., 2022, Search Engine J, P1

[5] A cluster-randomized trial of task shifting and blood pressure control in Ghana: study protocol [J].

Ogedegbe, Gbenga ;

Plange-Rhule, Jacob ;

Gyamfi, Joyce ;

Chaplin, William ;

Ntim, Michael ;

Apusiga, Kingsley ;

Khurshid, Kiran ;

Cooper, Richard .

IMPLEMENTATION SCIENCE, 2014, 9

[6] Evidence and Recommendations on the Use of Telemedicine for the Management of Arterial Hypertension An International Expert Position Paper [J].

Omboni, Stefano ;

McManus, Richard J. ;

Bosworth, Hayden B. ;

Chappell, Lucy C. ;

Green, Beverly B. ;

Kario, Kazuomi ;

Logan, Alexander G. ;

Magid, David J. ;

Mckinstry, Brian ;

Margolis, Karen L. ;

Parati, Gianfranco ;

Wakefield, Bonnie J. .

HYPERTENSION, 2020, 76 (05) :1368-1383

[7]

OpenAI, 2022, Introducing chatgpt

[8] Appropriateness of Cardiovascular Disease Prevention Recommendations Obtained From a Popular Online Chat-Based Artificial Intelligence Model [J].

Sarraju, Ashish ;

Bruemmer, Dennis ;

Van Iterson, Erik ;

Cho, Leslie ;

Rodriguez, Fatima ;

Laffin, Luke .

JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2023, 329 (10) :842-844

[9] PICO: What it is and what it is not [J].

Schiavenato, Martin ;

Chu, Frances .

NURSE EDUCATION IN PRACTICE, 2021, 56

[10]

Stokel-Walker Chris, 2022, Nature, DOI 10.1038/d41586-022-04397-7

← 1 2 →