Both Patients and Plastic Surgeons Prefer Artificial Intelligence-Generated Microsurgical Information

被引:7
作者
Berry, Charlotte E. [1 ]
Fazilat, Alexander Z. [1 ]
Lavin, Christopher
Lintel, Hendrik
Cole, Naomi [1 ]
Stingl, Cybil S.
Valencia, Caleb
Morgan, Annah G. [1 ]
Momeni, Arash [1 ,3 ]
Wan, Derrick C. [1 ,2 ]
机构
[1] Stanford Univ, Div Plast & Reconstruct Surg, Dept Surg, Hagey Lab Pediat Regenerat Med,Sch Med, Stanford, CA 94305 USA
[2] Stanford Univ, Lucile Packard Childrens Hosp, Hagey Family Fac Scholar Stem Cell & Regenerat Med, Div Plast & Reconstruct Surg,Sch Med, 257 Campus Dr West, Stanford, CA 94305 USA
[3] Stanford Univ, Sch Med, Div Plast & Reconstruct Surg, 700 Welch Rd, Suite 400, Stanford, CA 94305 USA
关键词
artificial intelligence; accuracy; comprehensiveness; clarity; readability; quality; online resources; American Society of Reproductive Medicine; HEALTH LITERACY; CHATGPT;
D O I
10.1055/a-2273-4163
中图分类号
R61 [外科手术学];
学科分类号
摘要
Background With the growing relevance of artificial intelligence (AI)-based patient-facing information, microsurgical-specific online information provided by professional organizations was compared with that of ChatGPT (Chat Generative Pre-Trained Transformer) and assessed for accuracy, comprehensiveness, clarity, and readability. Methods Six plastic and reconstructive surgeons blindly assessed responses to 10 microsurgery-related medical questions written either by the American Society of Reconstructive Microsurgery (ASRM) or ChatGPT based on accuracy, comprehensiveness, and clarity. Surgeons were asked to choose which source provided the overall highest-quality microsurgical patient-facing information. Additionally, 30 individuals with no medical background (ages: 18-81, mu = 49.8) were asked to determine a preference when blindly comparing materials. Readability scores were calculated, and all numerical scores were analyzed using the following six reliability formulas: Flesch-Kincaid Grade Level, Flesch-Kincaid Readability Ease, Gunning Fog Index, Simple Measure of Gobbledygook Index, Coleman-Liau Index, Linsear Write Formula, and Automated Readability Index. Statistical analysis of microsurgical-specific online sources was conducted utilizing paired t -tests. Results Statistically significant differences in comprehensiveness and clarity were seen in favor of ChatGPT. Surgeons, 70.7% of the time, blindly choose ChatGPT as the source that overall provided the highest-quality microsurgical patient-facing information. Nonmedical individuals 55.9% of the time selected AI-generated microsurgical materials as well. Neither ChatGPT nor ASRM-generated materials were found to contain inaccuracies. Readability scores for both ChatGPT and ASRM materials were found to exceed recommended levels for patient proficiency across six readability formulas, with AI-based material scored as more complex. Conclusion AI-generated patient-facing materials were preferred by surgeons in terms of comprehensiveness and clarity when blindly compared with online material provided by ASRM. Studied AI-generated material was not found to contain inaccuracies. Additionally, surgeons and nonmedical individuals consistently indicated an overall preference for AI-generated material. A readability analysis suggested that both materials sourced from ChatGPT and ASRM surpassed recommended reading levels across six readability scores.
引用
收藏
页码:657 / 664
页数:8
相关论文
共 19 条
[1]   Large Language Models and Artificial Intelligence: A Primer for Plastic Surgeons on the Demonstrated and Potential Applications, Promises, and Limitations of ChatGPT [J].
Abi-Rafeh, Jad ;
Xu, Hong Hao ;
Kazan, Roy ;
Tevlin, Ruth ;
Furnas, Heather .
AESTHETIC SURGERY JOURNAL, 2024, 44 (03) :329-343
[2]   Quality Assessment of Online Resources for Gender-affirming Surgery [J].
Berry, Charlotte E. ;
Fazilat, Alexander Z. ;
Churukian, Andrew A. ;
Abbas, Darren B. ;
Griffin, Michelle ;
Downer, Mauricio ;
Januszyk, Micheal ;
Momeni, Arash ;
Morrison, Shane D. ;
Wan, Derrick C. .
PLASTIC AND RECONSTRUCTIVE SURGERY-GLOBAL OPEN, 2023, 11 (10) :E5306
[3]   A Multimetric Health Literacy Analysis of Autologous Versus Implant-Based Breast Reconstruction [J].
Chen, Daniel H. ;
Johnson, Anna Rose ;
Ayyala, Haripriya ;
Lee, Edward S. ;
Lee, Bernard T. ;
Tran, Bao Ngoc N. .
ANNALS OF PLASTIC SURGERY, 2020, 85 :S102-S108
[4]   Can Patients Trust Online Health Information? A Meta-narrative Systematic Review Addressing the Quality of Health Information on the Internet [J].
Daraz, Lubna ;
Morrow, Allison S. ;
Ponce, Oscar J. ;
Beuschel, Bradley ;
Farah, Magdoleen H. ;
Katabi, Abdulrahman ;
Alsawas, Mouaz ;
Majzoub, Abdul M. ;
Benkhadra, Raed ;
Seisa, Mohamed O. ;
Ding, Jingyi ;
Prokop, Larry ;
Murad, M. Hassan .
JOURNAL OF GENERAL INTERNAL MEDICINE, 2019, 34 (09) :1884-1891
[5]   ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations [J].
Dave, Tirth ;
Athaluri, Sai Anirudh ;
Singh, Satyam .
FRONTIERS IN ARTIFICIAL INTELLIGENCE, 2023, 6
[6]   Opportunities and risks of ChatGPT in medicine, science, and academic publishing: a modern Promethean dilemma [J].
Homolak, Jan .
CROATIAN MEDICAL JOURNAL, 2023, 64 (01) :1-3
[7]   Toward a Conversational Agent to Support the Self-Management of Adults and Young Adults With Sickle Cell Disease: Usability and Usefulness Study [J].
Issom, David-Zacharie ;
Hardy-Dessources, Marie-Dominique ;
Romana, Marc ;
Hartvigsen, Gunnar ;
Lovis, Christian .
FRONTIERS IN DIGITAL HEALTH, 2021, 3
[8]  
Javaid M., 2023, BenchCouncil Trans. Benchmarks Stand. Eval, V3, P100105, DOI DOI 10.1016/J.TBENCH.2023.100105
[9]  
Johnson Douglas, 2023, Res Sq, DOI 10.21203/rs.3.rs-2566942/v1
[10]   Decoding radiology reports: Potential application of OpenAI ChatGPT to enhance patient understanding of diagnostic reports [J].
Li , Hanzhou ;
Moon, John T. ;
Iyer, Deepak ;
Balthazar, Patricia ;
Krupinski, Elizabeth A. ;
Bercu, Zachary L. ;
Newsome, Janice M. ;
Banerjee, Imon ;
Gichoya, Judy W. ;
Trivedi, Hari M. .
CLINICAL IMAGING, 2023, 101 :137-141