Effectiveness of Real-time Feedback and Instructive Hints in Graduate CS Courses via Automated Grading System

In this paper, we present our findings on enhancing personalized learning with the help of our real-time automated grading system for a graduate-level CS course. In this course, students learned how to use widely-used open-source libraries, and worked on 15 programming assignments with the ultimate goal of building an end-to-end big data processing pipeline in the cloud. We implemented our automated system to evaluate the correctness and efficiency of students' Java projects and to dynamically provide appropriate hints. The instructor complemented this real-time feedback with detailed, personalized feedback through frequent code reviews. In addition, a live, anonymous scoreboard allowed students to access grading results as well as to see the progress of their peers. The scoreboard and instructive hint system motivated the students to make incremental changes and also to spend enough time on testing one's own code before submitting it. In a survey, 97% of the students agreed that our grading system and personalized feedback enhanced their learning experiences. Given the overwhelmingly positive feedback from students, we believe that our findings could be helpful for those who are using or consider adopting auto-grading systems with the ultimate goal of providing enhanced learning experiences.