Comparing Feedback from Large Language Models and Instructors: Teaching Computer Science at Scale

Large language models (LLMs) can provide formative feedback in programming to help students improve the code they have written. We investigate the use of LLMs (GPT-4) to provide formative code feedback in a sophomore-level computer science (CS) course on data structures and algorithms. In three quizzes on recursion, half of the students randomly received GPT-4's feedback, while the other half received feedback from the course instructor. Students resubmitted their code based on the provided feedback. We found that students in the LLM-feedback condition scored higher in resubmissions than those receiving feedback from the instructor. Students perceived the two types of feedback as equally supportive of guiding resubmissions. We discuss the implications of using LLMs to provide formative feedback at scale in CS instruction.