RETRACTED: TA Marking Parties: Worth the Price of Pizza? Evaluating TA Confidence and Efficacy in Group vs. Individual Marking Scenarios (Retracted Article)

Student teaching assistants marking examinations is a reality for many undergraduate computer science courses, and with the explosion in enrolments in CS programs, and the increase in class sizes, it is becoming ever more common. Many institutions employ a "marking party" model in which instructors and TAs gather together to mark exams collectively. This system is naturally more logistically difficult than simply dividing up the papers and allowing TAs to mark in their own time. However the reasoning is that the group nature of the marking party makes it easier for markers to ask for clarification and second opinions on tricky cases, and results in better, more consistent marking. In this work we evaluate the marking party model by performing an experiment in which TAs are randomly assigned to either a marking party or solo marking when grading the final exam of a CS1 course. However, in addition to the student papers, each TA receives "fake" papers, constructed to lead to marking errors if TAs are not attentive or do not carefully follow the assigned marking rubric. We also evaluate the time that each method of marking takes, as well as survey the TAs as to their personal opinions on the two marking methods. Our results show that the marking party not only allows TAs to mark faster, but produces more consistent marking, with fewer errors, and better intra and inter-marker reliability. There is clear evidence that organizing marking parties is likely worth the effort (and cost of providing lunch), as the benefits to students are significant, and the overhead of the logistics may be less than that of fixing marking errors and dealing with re-mark requests. And as an added bonus, the TAs seem to enjoy it.