Emotion Recognition System via Facial Expressions and Speech Using Machine Learning and Deep Learning Techniques

Patients in hospitals frequently exhibit psychological issues such as sadness, pessimism, eccentricity, and anxiety. However, hospitals normally lack tools and facilities to continuously monitor the psychological health of patients. It is desirable to identify depression in patients so that it can be managed by instantly providing better therapy. This can be possible by advances in machine learning for image processing with notable applications in the domain of emotion recognition using facial expressions. In this paper, we have proposed two different methods, i.e. facial expression detection and voice analysis, to predict emotions. For facial expression recognition, we have used two approaches, one is the use of Gabor filters for feature extraction with support vector machine for classification and another is using convolutional neural network (CNN). For voice analysis, we extracted mel-frequency cepstral coefficients from speech data and, based on those features, predicted the emotions of the speech using a CNN model. Experimental results show that our proposed emotion recognition methods obtained high accuracy and thus could be potentially deployed to real-world applications. © 2023, The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd.