3DLIM: Intelligent analysis of students' learning interest by using multimodal fusion technology

Learning interest affects the way of learning and its process, which is an important factor that affects the learning effect. At present, students' learning interest in a teaching environment is mainly based on a traditional questionnaire or case analysis, which is not conducive for teachers to promptly access students' interest in class to effectively improve the teaching behavior. To intelligently analyze students' learning interest, a Three-Dimensional Learning Interest Model (3DLIM) is proposed based on educational psychology angle, which includes cognitive attention (Attention), learning emotion (Emotion), and thinking activities (Thinking). The proposed approach consisted of multimodal information recognition and fusion on head pose as well as facial expression and class interaction analysis, so as to comprehensively analyze students' interest in a teaching environment. To verify the validity and feasibility of the model, full-time graduate students participated in the experiments in real classroom scenarios. The Super Star interactive platform was used to collect interactive information from students in a class. Surveillance video tracked the learning process of the whole professional theoretical course. Experimental results show that the proposed approach can reflect the students' learning interest in learning as well as the differences among individuals.