Enhancement of Spoken Dialogue Systems by Means of User Emotion Recognition

In this paper we propose a new technique to enhance the performance of spoken dialogue systems by means of recognising users' emotional states. The technique employs two fusion modules that combine emotional predictions. The former employs a number of fusion methods to combine predictions made by classifiers that deal with different types of information regarding each sentence uttered by the user. These predictions are the input to the second fusion modules, which employs a fusion method to combine the predictions and obtain the most likely emotional category. This category represents the final decision of our technique regarding the emotional state of the user. We have carried out experiments considering two emotional categories ('Non-negative' and 'Negative') and classifiers to deal with information regarding prosody, acoustics, lexical items and dialogue acts. The results obtained employing an emotional corpus collected in our University show that the first fusion module clearly outperforms the classifiers, and so it does regarding a baseline system. The second fusion module, which represents the novelty of our study, enables enhancing the accuracy of the former fusion method by 2.25% absolutely.