Evaluating ChatGPT on Orbital and Oculofacial Disorders: Accuracy and Readability Insights

被引：9

作者：

Balas, Michael ^{[1
]}

Janic, Ana ^{[1
]}

Daigle, Patrick ^{[2
]}

Nijhawan, Navdeep ^{[2
]}

Hussain, Ahsen ^{[3
]}

Gill, Harmeet ^{[2
]}

Lahaie, Gabriela L. ^{[4
]}

Belliveau, Michel J. ^{[5
,6
]}

Crawford, Sean A. ^{[1
,7
]}

Parnian, Arjmand ^{[8
]}

Ing, Edsel B. ^{[2
,9
,10
]}

机构：

[1] Univ Toronto, Temerty Fac Med, Toronto, ON, Canada

[2] Univ Toronto, Dept Ophthalmol & Vis Sci, Toronto, ON, Canada

[3] Dalhousie Univ, Dept Ophthalmol & Visual Sci, Halifax, NS, Canada

[4] Queens Univ, Dept Ophthalmol, Kingston, ON, Canada

[5] Univ Ottawa, Dept Ophthalmol, Ottawa, ON, Canada

[6] Ottawa Hosp, Res Inst, Ottawa, ON, Canada

[7] Univ Toronto, Dept Surg, Div Vasc Surg, Toronto, ON, Canada

[8] Mississauga Retina Inst, Toronto, ON, Canada

[9] Univ Alberta, Dept Ophthalmol & Vis Sci, Edmonton, AB, Canada

[10] Royal Alexandra Hosp, Rm 2320, 10240 Kingsway Ave NW, Edmonton, AB T5H 3V9, Canada

来源：

OPHTHALMIC PLASTIC AND RECONSTRUCTIVE SURGERY | 2024年 / 40卷 / 02期

关键词：

EXOPHTHALMOMETRY; NAUGLE;

D O I：

10.1097/IOP.0000000000002552

中图分类号：

R77 [眼科学];

学科分类号：

100212 ;

摘要：

Purpose:To assess the accuracy and readability of responses generated by the artificial intelligence model, ChatGPT (version 4.0), to questions related to 10 essential domains of orbital and oculofacial disease.Methods:A set of 100 questions related to the diagnosis, treatment, and interpretation of orbital and oculofacial diseases was posed to ChatGPT 4.0. Responses were evaluated by a panel of 7 experts based on appropriateness and accuracy, with performance scores measured on a 7-item Likert scale. Inter-rater reliability was determined via the intraclass correlation coefficient.Results:The artificial intelligence model demonstrated accurate and consistent performance across all 10 domains of orbital and oculofacial disease, with an average appropriateness score of 5.3/6.0 ("mostly appropriate" to "completely appropriate"). Domains of cavernous sinus fistula, retrobulbar hemorrhage, and blepharospasm had the highest domain scores (average scores of 5.5 to 5.6), while the proptosis domain had the lowest (average score of 5.0/6.0). The intraclass correlation coefficient was 0.64 (95% CI: 0.52 to 0.74), reflecting moderate inter-rater reliability. The responses exhibited a high reading-level complexity, representing the comprehension levels of a college or graduate education.Conclusions:This study demonstrates the potential of ChatGPT 4.0 to provide accurate information in the field of ophthalmology, specifically orbital and oculofacial disease. However, challenges remain in ensuring accurate and comprehensive responses across all disease domains. Future improvements should focus on refining the model's correctness and eventually expanding the scope to visual data interpretation. Our results highlight the vast potential for artificial intelligence in educational and clinical ophthalmology contexts. ChatGPT-4 demonstrated accurate and consistent performance in addressing 100 questions related to 10 domains of orbital and oculoplastic disease, highlighting its future potential for educational and clinical use in the field.

引用

页码：217 / 222

页数：6

共 39 条

[1]

Abraham C, 2011, WRITING HLTH COMMUNI

[2] ChatGPT and Lacrimal Drainage Disorders: Performance and Scope of Improvement [J].