Evaluating Academic Answers Generated Using ChatGPT

被引:148
作者
Fergus, Suzanne [1 ]
Botha, Michelle [1 ]
Ostovar, Mehrnoosh [1 ]
机构
[1] Univ Hertfordshire, Sch Life & Med Sci, Hatfield AL10 9AB, England
关键词
integrity design; First-Year Undergraduate; General; Public Understanding; Outreach; Internet; Web-Based Learning; Applications of Chemistry;
D O I
10.1021/acs.jchemed.3c00087
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
The integration of technology in education has become ever more prioritized since the COVID-19 pandemic. Chat Generative Pre Trained Transformer (ChatGPT) is an artificial intelligence technology that generates conversational interactions to user prompts. The trained model can answer follow-up questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests. The functionality of ChatGPT in answering chemistry assessment questions requires investigation to ascertain its potential impact on learning and assessment. Two chemistry-focused modules in year 1 and year 2 of a pharmaceutical science program are used to study and evaluate ChatGPT-generated responses in relation to the end-of-year exam assessments. For questions that focused on knowledge and understanding with "describe" and "discuss" verbs, the ChatGPT generated responses. For questions that focused on application of knowledge and interpretation with nontext information, the ChatGPT technology reached a limitation. A further analysis of the quality of responses is reported in this study. ChatGPT is not considered a high-risk technology tool in relation to cheating. Similar to the COVID-19 disruption, ChatGPT is expected to provide a catalyst for educational discussions on academic integrity and assessment design.
引用
收藏
页码:1672 / 1675
页数:4
相关论文
共 16 条
[1]  
Anderson L., 2001, TAXONOMY LEARNING TE
[2]  
[Anonymous], About us
[3]  
[Anonymous], About Us
[4]   Contract cheating and assessment design: exploring the relationship [J].
Bretag, Tracey ;
Harper, Rowena ;
Burton, Michael ;
Ellis, Cath ;
Newton, Philip ;
van Haeringen, Karen ;
Saddiqui, Sonia ;
Rozenberg, Pearl .
ASSESSMENT & EVALUATION IN HIGHER EDUCATION, 2019, 44 (05) :676-691
[5]   Lessons Learned from the COVID-19 Crisis: Adjusting Assessment Approaches within Introductory Organic Courses [J].
Dicks, Andrew P. ;
Morra, Barbora ;
Quinlan, Kristine B. .
JOURNAL OF CHEMICAL EDUCATION, 2020, 97 (09) :3406-3412
[6]   Does authentic assessment assure academic integrity? Evidence from contract cheating data [J].
Ellis, Cath ;
van Haeringen, Karen ;
Harper, Rowena ;
Bretag, Tracey ;
Zucker, Ian ;
McBride, Scott ;
Rozenberg, Pearl ;
Newton, Phil ;
Saddiqui, Sonia .
HIGHER EDUCATION RESEARCH & DEVELOPMENT, 2020, 39 (03) :454-469
[7]   Insights Gained During COVID-19: Refocusing Laboratory Assessments Online [J].
Fergus, Suzanne ;
Botha, Michelle ;
Scott, Mark .
JOURNAL OF CHEMICAL EDUCATION, 2020, 97 (09) :3106-3109
[8]   Introduction to the Journal of Chemical Education Special Issue on Insights Gained While Teaching Chemistry in the Time of COVID-19 [J].
Holme, Thomas A. .
JOURNAL OF CHEMICAL EDUCATION, 2020, 97 (09) :2375-2377
[9]   How Can We Know What Language Models Know? [J].
Jiang, Zhengbao ;
Xu, Frank F. ;
Araki, Jun ;
Neubig, Graham .
TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, 2020, 8 :423-438
[10]   Shifting online during COVID-19: A systematic review of teaching and learning strategies and their outcomes [J].
Koh, Joyce Hwee Ling ;
Daniel, Ben Kei .
INTERNATIONAL JOURNAL OF EDUCATIONAL TECHNOLOGY IN HIGHER EDUCATION, 2022, 19 (01)