Facial expressions when learning with a Queer History App: Application of the Control Value Theory of Achievement Emotions

Learning analytics (LA) incorporates analyzing cognitive, social and emotional processes in learning scenarios to make informed decisions regarding instructional design and delivery. Research has highlighted important roles that emotions play in learning. We have extended this field of research by exploring the role of emotions in a relatively uncommon learning scenario: learning about queer history with a multimedia mobile app. Specifically, we used an automatic facial recognition software (FaceReader 7) to measure learners' discrete emotions and a counter-balanced multiple-choice quiz to assess learning. We also used an eye tracker (EyeLink 1000) to identify the emotions learners experienced while they read specific content, as opposed to the emotions they experienced over the course of the entire learning session. A total of 33 out of 57 of the learners' data were eligible to be analyzed. Results revealed that learners expressed more negative-activating emotions (ie, anger, anxiety) and negative-deactivating emotions (ie, sadness) than positive-activating emotions (ie, happiness). Learners with an angry emotion profile had the highest learning gains. The importance of examining typically undesirable emotions in learning, such as anger, is discussed using the control-value theory of achievement emotions. Further, this study describes a multimodal methodology to integrate behavioral trace data into learning analytics research. Practitioner Notes What is already known about this topic Multimodal analytics have increasingly gained traction, accompanied by more advanced methodologies and tools. Emotions play a critical role in learning, impacting learners' cognitive, motivational and regulatory processes. The Control Value Theory of Achievement Emotion predicts positive-activating emotions (eg, enjoyment) should lead to better performance. What this paper adds Application of Control Value Theory of Achievement Emotion to a less studied subject domain (ie, history) and issues (ie, LGBTQ rights). Preliminary evidence that negative-activating emotions (eg, anger) can facilitate learning in certain contexts, and that emotions may play different roles depending on the subject matter and domain. Insights into methods of aligning facial recognition data from facial expression analysis software with eye tracking data dealing with dynamic content. Insights into frequency and the types of emotions elicited in learning scenarios dealing with sensitive topics. Implications for practice and/or policy Educators should be aware of different types of emotions, and their roles in learning scenarios. Educators should critically evaluate whether emotions with positive valence always have a positive impact on learning, and vice versa. Learners may not always behaviorally express emotions through facial expressions, including when gazing at sections of learning material directly connected to assessment; therefore, it is helpful to supplement granular with larger timeframe analyses to examine emotional profiles.