Increasing Peer Review Quality in Online Learning Systems

Lecturers face an on-going struggle to keep up-to-date with their students' learning progress in large university courses. This hurts especially when it comes to identifying and supporting the diverse needs of each individual student. One way to approach this challenge is to introduce peer reviewing as a means to provide students with individual feedback throughout the semester. However, the quality of feedback written by peers can vary immensely and some students intentionally avoid putting work into writing reviews. We addressed these issues by calculating a Review Karma (RK), a value indicating how helpful students are in giving feedback to their colleagues and in helping them improve. While this approach shows much promise, especially in identifying different groups of students and enhancing their learning experience, we also identified trends that negatively impact the way students approach reviewing and provide their honest opinions of their colleagues work. The main contributions of this paper are the design of and lessons learned from the introduction of the RK and its initial evaluation via a survey.