ChatGPT provides acceptable responses to patient questions regarding common shoulder pathology

被引:2
作者
Ghilzai, Umar [1 ]
Fiedler, Benjamin [1 ]
Ghali, Abdullah [1 ]
Singh, Aaron [2 ]
Cass, Benjamin [3 ]
Young, Allan [3 ]
Ahmed, Adil Shahzad [1 ]
机构
[1] Baylor Coll Med, Dept Orthoped Surg, 7200 Cambridge St, Suite 10A, Houston, TX 77030 USA
[2] UT Hlth San Antonio, Dept Orthopaed, San Antonio, TX USA
[3] Sydney Shoulder Res Inst, Sydney Shoulder Specialists, Greenwich, NSW, Australia
关键词
Artificial intelligence; machine learning; ChatGPT; shoulder; large language model; INFORMATION; GOOGLE; ONLINE;
D O I
10.1177/17585732241283971
中图分类号
R826.8 [整形外科学]; R782.2 [口腔颌面部整形外科学]; R726.2 [小儿整形外科学]; R62 [整形外科学(修复外科学)];
学科分类号
摘要
Background ChatGPT is rapidly becoming a source of medical knowledge for patients. This study aims to assess the completeness and accuracy of ChatGPT's answers to the most frequently asked patients' questions about shoulder pathology.Methods ChatGPT (version 3.5) was queried to produce the five most common shoulder pathologies: biceps tendonitis, rotator cuff tears, shoulder arthritis, shoulder dislocation and adhesive capsulitis. Subsequently, it generated the five most common patient questions regarding these pathologies and was queried to respond. Responses were evaluated by three shoulder and elbow fellowship-trained orthopedic surgeons with a mean of 9 years of independent practice, on Likert scales for accuracy (1-6) and completeness (rated 1-3).Results For all questions, responses were deemed acceptable, rated at least "nearly all correct," indicated by a score of 5 or greater for accuracy, and "adequately complete," indicated by a minimum of 2 for completeness. The mean scores for accuracy and completeness, respectively, were 5.5 and 2.6 for rotator cuff tears, 5.8 and 2.7 for shoulder arthritis, 5.5 and 2.3 for shoulder dislocations, 5.1 and 2.4 for adhesive capsulitis, 5.8 and 2.9 for biceps tendonitis.Conclusion ChatGPT provides both accurate and complete responses to the most common patients' questions about shoulder pathology. These findings suggest that Large Language Models might play a role as a patient resource; however, patients should always verify online information with their physician.Level of Evidence Level V Expert Opinion.
引用
收藏
页数:6
相关论文
共 32 条
[1]   Artificial Hallucinations in ChatGPT: Implications in Scientific Writing [J].
Alkaissi, Hussam ;
McFarlane, Samy I. .
CUREUS JOURNAL OF MEDICAL SCIENCE, 2023, 15 (02)
[2]   Can Patients Trust Online Health Information? A Meta-narrative Systematic Review Addressing the Quality of Health Information on the Internet [J].
Daraz, Lubna ;
Morrow, Allison S. ;
Ponce, Oscar J. ;
Beuschel, Bradley ;
Farah, Magdoleen H. ;
Katabi, Abdulrahman ;
Alsawas, Mouaz ;
Majzoub, Abdul M. ;
Benkhadra, Raed ;
Seisa, Mohamed O. ;
Ding, Jingyi ;
Prokop, Larry ;
Murad, M. Hassan .
JOURNAL OF GENERAL INTERNAL MEDICINE, 2019, 34 (09) :1884-1891
[3]   ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health [J].
De Angelis, Luigi ;
Baglivo, Francesco ;
Arzilli, Guglielmo ;
Privitera, Gaetano Pierpaolo ;
Ferragina, Paolo ;
Tozzi, Alberto Eugenio ;
Rizzo, Caterina .
FRONTIERS IN PUBLIC HEALTH, 2023, 11
[4]   Using a Google Web Search Analysis to Assess the Utility of ChatGPT in Total Joint Arthroplasty [J].
Dubin, Jeremy A. ;
Bains, Sandeep S. ;
Chen, Zhongming ;
Hameed, Daniel ;
Nace, James ;
Mont, Michael A. ;
Delanois, Ronald E. .
JOURNAL OF ARTHROPLASTY, 2023, 38 (07) :1195-1202
[5]   ChatGPT: these are not hallucinations - they're fabrications and falsifications [J].
Emsley, Robin .
SCHIZOPHRENIA, 2023, 9 (01)
[6]   ChatGPT performance on the American Shoulder and Elbow Surgeons maintenance of certification exam [J].
Fiedler, Benjamin ;
Azua, Eric N. ;
Phillips, Todd ;
Ahmed, Adil Shahzad .
JOURNAL OF SHOULDER AND ELBOW SURGERY, 2024, 33 (09) :1888-1893
[7]   Comparison of Diagnostic and Triage Accuracy of Ada Health and WebMD Symptom Checkers, ChatGPT, and Physicians for Patients in an Emergency Department: Clinical Data Analysis Study [J].
Fraser, Hamish ;
Crossland, Daven ;
Bacher, Ian ;
Ranney, Megan ;
Madsen, Tracy ;
Hilliard, Ross .
JMIR MHEALTH AND UHEALTH, 2023, 11
[8]   Hallucinations in ChatGPT: A Cautionary Tale for Biomedical Researchers [J].
Goddard, Jerome .
AMERICAN JOURNAL OF MEDICINE, 2023, 136 (11) :1059-1060
[9]   Accuracy and Reliability of Chatbot Responses to Physician Questions [J].
Goodman, Rachel S. ;
Patrinely, J. Randall ;
Stone, Cosby A. ;
Zimmerman, Eli ;
Donald, Rebecca R. ;
Chang, Sam S. ;
Berkowitz, Sean T. ;
Finn, Avni P. ;
Jahangir, Eiman ;
Scoville, Elizabeth A. ;
Reese, Tyler S. ;
Friedman, Debra L. ;
Bastarache, Julie A. ;
van der Heijden, Yuri F. ;
Wright, Jordan J. ;
Ye, Fei ;
Carter, Nicholas ;
Alexander, Matthew R. ;
Choe, Jennifer H. ;
Chastain, Cody A. ;
Zic, John A. ;
Horst, Sara N. ;
Turker, Isik ;
Agarwal, Rajiv ;
Osmundson, Evan ;
Idrees, Kamran ;
Kiernan, Colleen M. ;
Padmanabhan, Chandrasekhar ;
Bailey, Christina E. ;
Schlegel, Cameron E. ;
Chambless, Lola B. ;
Gibson, Michael K. ;
Osterman, Travis J. ;
Wheless, Lee E. ;
Johnson, Douglas B. .
JAMA NETWORK OPEN, 2023, 6 (10)
[10]   A Call to Address AI "Hallucinations" and How Healthcare Professionals Can Mitigate Their Risks [J].
Hatem, Rami ;
Simmons, Brianna ;
Thornton, Joseph E. .
CUREUS JOURNAL OF MEDICAL SCIENCE, 2023, 15 (09)