Enhancing peer assessment with artificial intelligence

This paper surveys research and practice on enhancing peer assessment with artificial intelligence. Its objectives are to give the structure of the theoretical framework underpinning the study, synopsize a scoping review of the literature that illustrates this structure, and provide a case study which further illustrates this structure. The theoretical framework has six areas: (i) Assigning Peer Assessors, (ii) Enhancing Individual Reviews, (iii) Deriving Peer Grades/Feedback, (iv) Analyzing Student Feedback, (v) Facilitating Instructor Oversight and (vi) Peer Assessment Systems. The vast majority of the 79 papers in the review found that artificial intelligence improved peer assessment. However, the focus of many papers was on diversity in grades and feedback, fuzzy logic and the analysis of feedback with a view to equalizing its quality. Relatively few papers focused on automated assignment, automated assessment, calibration, teamwork effectiveness and automated feedback and these merit further research. This picture suggests AI is making inroads into peer assessment, but there is still a considerable way to go, particularly in the under-researched areas. The paper incorporates a case study of the RIPPLE peer-assessment tool, which harnesses student wisdom, insights from the learning sciences and AI to enable time-constrained educators to immerse their students in deep and personalized learning experiences that effectively prepare them to serve as assessors. Once trained, they use a comprehensive rubric to vet learning resources submitted by other students. They thereby create pools of high-quality learning resources which can be used to recommend personalized content to students. RIPPLE engages students in a trio of intertwined activities: creation, review and personalized practice, generating many resource types. AI-driven real-time feedback is given but students are counseled to assess whether it is accurate. Affordances and challenges for researchers and practitioners were identified.