Acoustic Characteristics of Emotional Speech Using Spectrogram Image Classification

One of the problems limiting the accuracy of speech emotion recognition (SER) is difficulty in the differentiation between acoustically-similar emotions. Since it is not clear how emotions differ in acoustic terms, it is difficult to design new, more efficient SER strategies. In this study, amplitude-frequency analysis of emotional speech was performed to determine relative differences between seven emotional categories of speech in the Berlin Emotional Speech (EMO-DB) database. The analysis transformed short 1-second blocks of speech into RGB images of spectrograms using four different frequency scales. The images were used to train a convolutional neural network (CNN) to recognize emotions. By training the network with different combinations of frequency scales and color components of the RGB images that emphasized different frequency and spectral amplitude values, links between different emotions and corresponding amplitude-frequency characteristics of speech were determined.