Understanding Student Learning Trajectories Using Multimodal Learning Analytics within an Embodied-Interaction Learning Environment

The aim of this paper is to show how multimodal learning analytics (MMLA) can help understand how elementary students explore the concept of feedback loops while controlling an embodied simulation of a predator-prey ecosystem using hand movements as an interface with the computer simulation. We represent student motion patterns from fine-grained logs of hands and gaze data, and then map these observed motion patterns against levels of student performance to make inferences about how embodiment plays a role in the learning process. Results show five distinct motion sequences in students' embodied interactions, and these motion patterns are statistically associated with initial and post-tutorial levels of students' understanding of feedback loops. Analysis of student gaze also shows distinctive patterns as to how low- and high-performing students attended to information presented in the simulation. Using MMLA, we show how students' explanations of feedback loops look differently according to cluster membership, which provides evidence that embodiment interacts with conceptual understanding.