Revolutionising Essay Evaluation: A Cutting-Edge Rubric for AI-Assisted Writing

This study aims to develop a robust rubric for evaluating artificial intelligence (AI)-assisted essay writing in English as a Foreign Language (EFL) contexts. Employing a modified Delphi technique, we conducted a comprehensive literature review and administered Likert scale questionnaires. This process yielded nine key evaluation criteria, forming the initial rubric. The rubric was applied to evaluate 33 AI-assisted essays written by students as part of an intensive course assignment. Statistical analysis revealed significant inter-rater reliability and convergent validity coefficients, supporting the adoption and further development of such rubrics across higher education institutions. The developed rubric was subsequently used to evaluate these essays using two AI tools: ChatGPT and Claude. The results indicated that both AI tools evaluated the essays with similar scores, demonstrating consistency in their assessment capabilities.