Evaluating Large Language Learning Models' Accuracy and Reliability in Addressing Consumer Health Queries

被引:0
作者
Chung, Sunny [1 ]
Koos, Jessica [1 ]
机构
[1] SUNY Stony Brook, Stony Brook, NY USA
关键词
Consumer health; health sciences; large language models; medical librarianship;
D O I
10.1080/15398285.2024.2418777
中图分类号
R1 [预防医学、卫生学];
学科分类号
1004 ; 120402 ;
摘要
Individuals with no medical background or training may turn to a variety of resources when in need of health-related information. One new potential source of information is large language models (LLMs). This article describes a preliminary assessment of the accuracy and reliability of several of these models when used to obtain consumer health information. The references provided by the LLM's for each query are also examined for reliability based on the source of information.
引用
收藏
页码:395 / 402
页数:8
相关论文
共 11 条
[1]   Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References [J].
Athaluri, Sai Anirudh ;
Manthena, Sandeep Varma ;
Kesapragada, V. S. R. Krishna Manoj ;
Yarlagadda, Vineel ;
Dave, Tirth ;
Duddumpudi, Rama Tulasi Siri .
CUREUS JOURNAL OF MEDICAL SCIENCE, 2023, 15 (04)
[2]   Performance of ChatGPT-4 and Bard chatbots in responding to common patient questions on prostate cancer 177Lu-PSMA-617 therapy [J].
Bilgin, Gokce Belge ;
Bilgin, Cem ;
Childs, Daniel S. ;
Orme, Jacob J. ;
Burkett, Brian J. ;
Packard, Ann T. ;
Johnson, Derek R. ;
Thorpe, Matthew P. ;
Riaz, Irbaz Bin ;
Halfdanarson, Thorvardur R. ;
Johnson, Geoffrey B. ;
Sartor, Oliver ;
Kendi, Ayse Tuba .
FRONTIERS IN ONCOLOGY, 2024, 14
[3]   Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry Exam Questions [J].
Clark, Ted M. .
JOURNAL OF CHEMICAL EDUCATION, 2023, 100 (05) :1905-1916
[4]   Dr. Google vs. Dr. ChatGPT: Exploring the Use of Artificial Intelligence in Ophthalmology by Comparing the Accuracy, Safety, and Readability of Responses to Frequently Asked Patient Questions Regarding Cataracts and Cataract Surgery [J].
Cohen, Samuel A. ;
Brant, Arthur ;
Fisher, Ann Caroline ;
Pershing, Suzann ;
Do, Diana ;
Pan, Carolyn .
SEMINARS IN OPHTHALMOLOGY, 2024, 39 (06) :472-479
[5]   A Systematic Review of the Limitations and Associated Opportunities of ChatGPT [J].
Cong-Lem, Ngo ;
Soyoof, Ali ;
Tsering, Diki .
INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION, 2025, 41 (07) :3851-3866
[6]   Evaluating Academic Answers Generated Using ChatGPT [J].
Fergus, Suzanne ;
Botha, Michelle ;
Ostovar, Mehrnoosh .
JOURNAL OF CHEMICAL EDUCATION, 2023, 100 (04) :1672-1675
[7]  
Merriam Webster, 2024, Large language model Merriam-Webster.com
[8]   ChatGPT Output Regarding Compulsory Vaccination and COVID-19 Vaccine Conspiracy: A Descriptive Study at the Outset of a Paradigm Shift in Online Search for Information [J].
Sallam, Malik ;
Salim, Nesreen A. ;
Al-Tammemi, Ala'a B. ;
Barakat, Muna ;
Fayyad, Diaa ;
Hallit, Souheil ;
Harapan, Harapan ;
Hallit, Rabih ;
Mahafzah, Azmi .
CUREUS JOURNAL OF MEDICAL SCIENCE, 2023, 15 (02)
[9]   ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns [J].
Sallam, Malik .
HEALTHCARE, 2023, 11 (06)
[10]   A large language model artificial intelligence for patient queries in atopic dermatitis [J].
Sulejmani, Pranvera ;
Negris, Olivia ;
Aoki, Valeria ;
Chu, Chia-Yu ;
Eichenfield, Lawrence ;
Misery, Laurent ;
Mosca, Ana ;
Orfali, Raquel Leao ;
Aroman, Marketa Saint ;
Stalder, Jean-Francois ;
Trzeciak, Magdalena ;
Wollenberg, Andreas ;
Lio, Peter .
JOURNAL OF THE EUROPEAN ACADEMY OF DERMATOLOGY AND VENEREOLOGY, 2024, 38 (06) :e531-e535