An evaluation of multimodal interactions with technology while learning science concepts

This paper explores the value of employing multiple modalities to facilitate science learning with technology. In particular, it is argued that when multiple modalities are employed, learners construct strong relations between physical movement and visual representations of motion. Body interactions with visual representations, enabled by interactive technologies, can encourage rhythmic cycles of engagement and reflection. A study was carried out to investigate how students interpret distance-time and velocity-time graphs created through hand movement. It explored the dynamic coupling between their body movements and graphs, and their subsequent interpretation and production of graphs on paper. The results show that physical manipulation of kinematics graphs has a significantly greater effect on students' ability to relate graphs to movement than observing the graphs being produced by someone else.