Enhancing Tutoring Effectiveness Through Automated Feedback: Preliminary Findings from a Pilot Randomized Controlled Trial on SAT Tutoring

To address educational inequities, high-quality SAT tutoring is crucial for students who need support. Many tutors, however, are novices and require coaching to be effective. To examine this issue, we conducted a pilot randomized controlled trial (RCT) to evaluate the effectiveness of personalized, automated feedback for novice tutors. This feedback aimed to enhance their tutoring skills and, consequently, improve student outcomes. In our RCT, we not only assessed the effectiveness of the feedback provided to novice tutors but also examined the impact of extending this feedback to both tutors and their students, compared to just the tutors alone. Furthermore, we explored how the use of social versus personal goal-oriented language in the feedback influences educational outcomes. Our preliminary findings from this pilot indicate that providing feedback to both tutors and learners led to a statistically significant improvement in average SAT practice test scores. Additionally, this approach significantly increased the tutor talk time ratio, an outcome that was somewhat unexpected and requires further investigation. These initial results form the foundation for a more comprehensive RCT currently underway. This ongoing study aims to delve deeper into these initial findings and refine our understanding of the effectiveness of feedback in tutoring settings.