Semi-automated Analysis of Reflections as a Continuous Course Improvement Tool

This work-in-progress paper proposes a semi-automated method to analyze students' reflections. It is challenging to include reflection activities in computing classes because of the amount of time required from students to answer the reflection questions and the amount of effort required for instructors to review the students' responses. These challenges inspired us to adopt Digital Minute Paper (DMP) as a way to give students multiple, quick opportunities to stop and reflect on their experiences in class. In this way, students are given an opportunity to develop metacognitive skills and to potentially improve their performance in the class. In addition, we used these DMPs as formative feedback for the instructors to address students' problems in the class and to continuously improve the course design. Reading reflections is tedious, time-consuming, and does not scale to large classes. To extract insights from the DMPs, we created a semi-automated process for analyzing DMPs by applying natural language processing (NLP). Our process extracts unigrams and bigrams from the reflections and then visualizes related quotes from the reflections using a treemap visualization. We found that this semi-automatic analysis of the reflections is a good, low-effort way to capture student feedback in addition to helping students be more self-regulating learners.