An Empirical Evaluation of Live Coding in CS1

Background and Context. Live coding is a teaching method in which an instructor dynamically writes code in front of students in an effort to impart skills such as incremental development and debugging. By contrast, traditional, static-code examples typically involve an instructor annotating or explaining components of pre-written code. Despite recommendations to use live coding and a wealth of qualitative analyses that identify perceived learning benefits of it, there are a lack of empirical evaluations to confirm those learning benefits, especially with respect to students' programming processes. Objectives. Our work aims to provide a holistic, empirical comparison of a live-coding pedagogy with a static-code one. We evaluated the impact of a live-coding pedagogy on three main areas: 1) students' adherence to effective programming processes, 2) their performance on exams and assignments, and 3) their lecture experiences, such as engagement during lecture and perceptions of code examples. Method. In our treatment-control quasi-experimental setup, one lecture group saw live-coding examples while the other saw only static-code ones. Both lecture groups were taught by the same instructor, were taught the exact same content, and completed the same assignments and exams. We collected compilation-level programming process data, student performance on exam and homework questions, and feedback via a survey and course evaluations. Findings. Our findings showed no statistically significant differences between the live-coding and static-code groups on programming process metrics related to incremental development, debugging, and productivity. Similarly, there was no difference between the groups on course performance on assignments and exams. Finally, student feedback suggests that more students in the live-coding group reported that lectures were too fast and failed to facilitate note-taking, potentially mitigating the perceived benefits of live coding. Implications. Live coding alone may not lead to many of the perceived and intended benefits that prior work identifies, but future work may investigate how to realize these benefits while minimizing the drawbacks we identified.