An Evaluation of Current Trends in AI-Generated Text in Otolaryngology Publications

被引：0

作者：

Kutler, Rachel B. ^{[1
]}

Setzen, Sean A. ^{[1
]}

Tsai, Samantha ^{[1
]}

Rameau, Anais ^{[1
]}

机构：

[1] Weill Cornell Med Coll, Sean Parker Inst Voice, Dept Otolaryngol Head & Neck Surg, New York, NY 10021 USA

来源：

LARYNGOSCOPE | 2025年

关键词：

artificial intelligence; large language models; otolaryngology;

D O I：

10.1002/lary.32202

中图分类号：

R-3 [医学研究方法]; R3 [基础医学];

学科分类号：

1001 ;

摘要：

Objectives: Since the release of ChatGPT-4 in March 2023, large language models (LLMs) application in biomedical manuscript production has been widespread. GPT-modified text detectors, such as GPTzero, lack sensitivity and reliability and do not quantify the amount of AI-generated text. However, recent work has identified certain adjectives more frequently used by LLMs that can help identify and quantify LLM-modified text. The aim of this study is to utilize these adjectives to identify LLM-generated text in otolaryngology publications. Study design: Meta-research. Methods: Twenty-five otolaryngology journals were studied between November 2022 and July 2024, encompassing 8751 published works. Articles from countries where ChatGPT-4 is not available were removed, yielding 7702 articles for study inclusion. These publications were analyzed using a Python script to determine the frequency of the top 100 adjectives disproportionately generated by ChatGPT-4. Results: A significant increase in the frequency of adjectives associated with GPT use was observed from November 2023 to July 2024 across all journals (p < 0.001), with a significant difference before and after the release of ChatGPT in March 2023. Journals with higher impact factors had significantly lower usage of GPT-associated adjectives than those with lower impact factors (p < 0.001). There was no significant difference in GPT-associated adjective use by first authors with a doctoral degree versus those without. Publications by authors from English-speaking countries demonstrated a significantly more frequent use of LLM-associated adjectives (p < 0.001). Conclusions: This study suggests that ChatGPT use in otolaryngology manuscript production has significantly increased since the release of ChatGPT-4. Future research should be aimed at further characterizing the landscape of AI-generated text in otolaryngology and developing tools that encourage authors' transparency regarding the use of LLMs.

引用

页数：9