Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information

被引：181

作者：

Johnson, Skyler B. ^{[1
,2
,9
]}

King, Andy J. ^{[2
,3
]}

Warner, Echo L. ^{[2
,4
]}

Aneja, Sanjay ^{[5
,6
]}

Kann, Benjamin H. ^{[7
]}

Bylund, Carma L. ^{[8
]}

机构：

[1] Univ Utah, Huntsman Canc Inst, Dept Radiat Oncol, Sch Med, Salt Lake City, UT USA

[2] Huntsman Canc Inst, Canc Control & Populat Sci, Salt Lake City, UT USA

[3] Univ Utah, Dept Commun, Salt Lake City, UT USA

[4] Univ Utah, Coll Nursing, Salt Lake City, UT USA

[5] Yale Sch Med, Ctr Outcomes Res & Evaluat, New Haven, CT USA

[6] Yale Sch Med, Dept Therapeut Radiol, New Haven, CT USA

[7] Harvard Med Sch, Brigham & Womens Hosp, Dana Farber Canc Inst, Dept Radiat Oncol, Boston, MA USA

[8] Univ Florida, Dept Hlth Outcomes & Biomed Informat, Coll Med, Gainesville, FL USA

[9] Univ Utah, Dept Radiat Oncol, Huntsman Canc Inst, 1950 Circle Hope Dr, Salt Lake City, UT 84112 USA

来源：

JNCI CANCER SPECTRUM | 2023年 / 7卷 / 02期

关键词：

D O I：

10.1093/jncics/pkad015

中图分类号：

R73 [肿瘤学];

学科分类号：

100214 ;

摘要：

Data about the quality of cancer information that chatbots and other artificial intelligence systems provide are limited. Here, we evaluate the accuracy of cancer information on ChatGPT compared with the National Cancer Institute's (NCI's) answers by using the questions on the "Common Cancer Myths and Misconceptions" web page. The NCI's answers and ChatGPT answers to each question were blinded, and then evaluated for accuracy (accurate: yes vs no). Ratings were evaluated independently for each question, and then compared between the blinded NCI and ChatGPT answers. Additionally, word count and Flesch-Kincaid readability grade level for each individual response were evaluated. Following expert review, the percentage of overall agreement for accuracy was 100% for NCI answers and 96.9% for ChatGPT outputs for questions 1 through 13 (kappa = -0.03, standard error = 0.08). There were few noticeable differences in the number of words or the readability of the answers from NCI or ChatGPT. Overall, the results suggest that ChatGPT provides accurate information about common cancer myths and misconceptions.

引用

页数：9

共 10 条

[1]

[Anonymous], COMM CANC MYTHS MISC

[2] Social and Demographic Patterns of Health-Related Internet Use Among Adults in the United States: A Secondary Data Analysis of the Health Information National Trends Survey [J].

Calixte, Rose ;

Rivera, Argelis ;

Oridota, Olutobi ;

Beauchamp, William ;

Camacho-Rivera, Marlene .

INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH, 2020, 17 (18) :1-16

[3]

FLEISS JL, 1971, PSYCHOL BULL, V76, P378, DOI 10.1037/h0031619

[4]

Hundt Andrew, 2022, FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency, P743, DOI 10.1145/3531146.3533138

[5] Cancer Misinformation and Harmful Information on Facebook and Other Social Media: A Brief Report [J].

Johnson, Skyler B. ;

Parsons, Matthew ;

Dorff, Tanya ;

Moran, Meena S. ;

Ward, John H. ;

Cohen, Stacey A. ;

Akerley, Wallace ;

Bauman, Jessica ;

Hubbard, Joleen ;

Spratt, Daniel E. ;

Bylund, Carma L. ;

Swire-Thompson, Briony ;

Onega, Tracy ;

Scherer, Laura D. ;

Tward, Jonathan ;

Fagerlin, Angela .

JNCI-JOURNAL OF THE NATIONAL CANCER INSTITUTE, 2022, 114 (07) :1036-1039

[6] Chatbots in the fight against the COVID-19 pandemic [J].

Miner, Adam S. ;

Laranjo, Liliana ;

Kocaballi, A. Baki .

NPJ DIGITAL MEDICINE, 2020, 3 (01)

[7] An Initial Model of Trust in Chatbots for Customer Service-Findings from a Questionnaire Study [J].

Nordheim, Cecilie Bertinussen ;

Folstad, Asbjorn ;

Bjorkli, Cato Alexander .

INTERACTING WITH COMPUTERS, 2019, 31 (03) :317-335

[8] Dissecting racial bias in an algorithm used to manage the health of populations [J].

Obermeyer, Ziad ;

Powers, Brian ;

Vogeli, Christine ;

Mullainathan, Sendhil .

SCIENCE, 2019, 366 (6464) :447-+

[9]

Pearl Mike, 2022, MASHABLEDec. 03

[10]

Vincent J., 2016, VERGE

← 1 →