BLSTM and CNN Stacking Architecture for Speech Emotion Recognition

被引:0
作者
Dongdong Li
Linyu Sun
Xinlei Xu
Zhe Wang
Jing Zhang
Wenli Du
机构
[1] Ministry of Education,Key Laboratory of Advanced Control and Optimization for Chemical Processes
[2] East China University of Science and Technology,Department of Computer Science and Engineering
[3] East China University of Science and Technology,Provincial Key Laboratory for Computer Information Processing Technology
[4] Soochow University,undefined
来源
Neural Processing Letters | 2021年 / 53卷
关键词
Speech emotion recognition; Convolutional neural network; Bidirectional long short term memory; Stacking;
D O I
暂无
中图分类号
学科分类号
摘要
Speech Emotion Recognition (SER) is a huge challenge for distinguishing and interpreting the sentiments carried in speech. Fortunately, deep learning is proved to have great ability to deal with acoustic features. For instance, Bidirectional Long Short Term Memory (BLSTM) has an advantage of solving time series acoustic features and Convolutional Neural Network (CNN) can discover the local structure among different features. This paper proposed the BLSTM and CNN Stacking Architecture (BCSA) to enhance the ability to recognition emotions. In order to match the input formats of BLSTM and CNN, slicing feature matrices is necessary. For utilizing the different roles of the BLSTM and CNN, the Stacking is employed to integrate the BLSTM and CNN. In detail, taking into account overfitting problem, the estimates of probabilistic quantities from BLSTM and CNN are combined as new data using K-fold cross validation. Finally, based on the Stacking models, the logistic regression is used to recognize emotions effectively by fitting the new data. The experiment results demonstrate that the performance of proposed architecture is better than that of single model. Furthermore, compared with the state-of-the-art model on SER in our knowledge, the proposed method BCSA may be more suitable for SER by integrating time series acoustic features and the local structure among different features.
引用
收藏
页码:4097 / 4115
页数:18
相关论文
共 50 条
[21]   Boosting RUL Prediction Using a Hybrid Deep CNN-BLSTM Architecture [J].
Remadna, I ;
Terrissa, S. L. ;
Sayah, M. ;
Ayad, S. ;
Zerhouni, N. .
AUTOMATIC CONTROL AND COMPUTER SCIENCES, 2022, 56 (04) :300-310
[22]   SPEECH EMOTION RECOGNITION WITH DUAL-SEQUENCE LSTM ARCHITECTURE [J].
Wang, Jianyou ;
Xue, Michael ;
Culhane, Ryan ;
Diao, Enmao ;
Ding, Jie ;
Tarokh, Vahid .
2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2020, :6474-6478
[23]   Static, Dynamic and Acceleration Features for CNN-Based Speech Emotion Recognition [J].
Khalifa, Intissar ;
Ejbali, Ridha ;
Napoletano, Paolo ;
Schettini, Raimondo ;
Zaied, Mourad .
AIXIA 2021 - ADVANCES IN ARTIFICIAL INTELLIGENCE, 2022, 13196 :348-358
[24]   Speech emotion recognition and classification using hybrid deep CNN and BiLSTM model [J].
Mishra, Swami ;
Bhatnagar, Nehal ;
Prakasam, P. ;
Sureshkumar, T. R. .
MULTIMEDIA TOOLS AND APPLICATIONS, 2023, 83 (13) :37603-37620
[25]   Speech emotion recognition and classification using hybrid deep CNN and BiLSTM model [J].
Swami Mishra ;
Nehal Bhatnagar ;
Prakasam P ;
Sureshkumar T. R .
Multimedia Tools and Applications, 2024, 83 :37603-37620
[26]   EFFICIENT SPEECH EMOTION RECOGNITION USING MULTI-SCALE CNN AND ATTENTION [J].
Peng, Zixuan ;
Lu, Yu ;
Pan, Shengfeng ;
Liu, Yunfeng .
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021), 2021, :3020-3024
[27]   Hybrid LSTM-Attention and CNN Model for Enhanced Speech Emotion Recognition [J].
Makhmudov, Fazliddin ;
Kutlimuratov, Alpamis ;
Cho, Young-Im .
APPLIED SCIENCES-BASEL, 2024, 14 (23)
[28]   Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition [J].
Seo, Minji ;
Kim, Myungho .
SENSORS, 2020, 20 (19) :1-21
[29]   Real Time Emotion Recognition from Facial Expressions Using CNN Architecture [J].
Ozdemir, Mehmet Akif ;
Elagoz, Berkay ;
Alaybeyoglu, Aysegul ;
Sadighzadeh, Reza ;
Akan, Aydin .
2019 MEDICAL TECHNOLOGIES CONGRESS (TIPTEKNO), 2019, :417-420
[30]   Emotion Prompting for Speech Emotion Recognition [J].
Zhou, Xingfa ;
Li, Min ;
Yang, Lan ;
Sun, Rui ;
Wang, Xin ;
Zhan, Huayi .
INTERSPEECH 2023, 2023, :3108-3112