Towards real-time speech emotion recognition for affective e-learning

被引：40
作者：
Bahreini K. [1 ]
Nadolski R. [1 ]
Westera W. [1 ]
机构：
[1] Welten Institute, Research Centre for Learning, Teaching and Technology, Faculty of Psychology and Educational Sciences, Open University of the Netherlands, Valkenburgerweg 177, Heerlen
来源：
Education and Information Technologies | 2016年 / 21卷 / 5期
关键词：
Affective computing; E-learning; Empirical study of user behaviour; Evaluation methodology; Microphone; Real-time software development; Speech emotion recognition; Speech interaction;
D O I：
10.1007/s10639-015-9388-2
中图分类号：
学科分类号：
摘要：
This paper presents the voice emotion recognition part of the FILTWAM framework for real-time emotion recognition in affective e-learning settings. FILTWAM (Framework for Improving Learning Through Webcams And Microphones) intends to offer timely and appropriate online feedback based upon learner’s vocal intonations and facial expressions in order to foster their learning. Whereas the facial emotion recognition part has been successfully tested in a previous study, the here presented study describes the development and testing of FILTWAM’s vocal emotion recognition software artefact. The main goal of this study was to show the valid use of computer microphone data for real-time and adequate interpretation of vocal intonations into extracted emotional states. The software that was developed was tested in a study with 12 participants. All participants individually received the same computer-based tasks in which they were requested 80 times to mimic specific vocal expressions (960 occurrences in total). Each individual session was recorded on video. For the validation of the voice emotion recognition software artefact, two experts annotated and rated participants’ recorded behaviours. Expert findings were then compared with the software recognition results and showed an overall accuracy of Kappa of 0.743. The overall accuracy of the voice emotion recognition software artefact is 67 % based on the requested emotions and the recognized emotions. Our FILTWAM-software allows to continually and unobtrusively observing learners’ behaviours and transforms these behaviours into emotional states. This paves the way for unobtrusive and real-time capturing of learners’ emotional states for enhancing adaptive e-learning approaches. © 2015, The Author(s).
引用
收藏
页码：1367 / 1386
页数：19
相关论文
共 37 条
[1]  
Bachiller C., Hernandez C., Sastre J., Collaborative learning, research and science promotion in a multidisciplinary scenario: information and communications technology and music. Proceedings of the International Conference on Engineering Education (pp, 1–8), (2010)
[2]  
Bahreini K., Nadolski R., Qi W., Westera W., FILTWAM - A framework for online game-based communication skills training - Using webcams and microphones for enhancing learner support, The 6th European conference on games based learning (ECGBL), pp. 39-48, (2012)
[3]  
Bahreini K., Nadolski R., Westera W., FLITWAM and Voice Emotion Recognition. Games and Learning Alliance (GaLA) Conference, Paris, France, pp. 23-25, (2013)
[4]  
Bahreini K., Nadolski R., Westera W., Towards Multimodal Emotion Recognition in E-learning Environments, Interactive Learning Environments, pp. 1-16, (2014)
[5]  
Beale R., Creed C., Affective interaction: How emotional agents affect users, International Journal of Human-Computer Studies, 67, 9, pp. 755-776, (2009)
[6]  
Batliner A., Fischer K., Hubera R., Spilkera J., Noth E., How to find trouble in communication, Speech Communication, 40, pp. 117-143, (2003)
[7]  
Ben Ammar M., Neji M., Alimi A.M., Gouarderes G., The affective tutoring system, Expert Systems with Applications, 37, 4, pp. 3013-3023, (2010)
[8]  
Burkhardt F., Paeschke A., Rolfes M., Sendlmeier W., Weiss B., A database of German emotional speech. In proceedings of the Inter speech Lissabon, Portugal, pp. 1517-1520, (2005)
[9]  
Chen L., Mao X., Xue Y., Cheng L.L., Speech emotion recognition: features and classification models, Digital Signal Processing, 22, 6, pp. 1154-1160, (2012)
[10]  
Chibelushi C.C., Bourel F., Facial expression recognition: a brief tutorial overview, Available Online in Compendium of Computer Vision, (2003)
1234