The Acoustically Emotion-Aware Conversational Agent With Speech Emotion Recognition and Empathetic Responses

被引：5
作者：
Hu, Jiaxiong [1 ]
Huang, Yun [2 ]
Hu, Xiaozhu [3 ]
Xu, Yingqing [1 ]
机构：
[1] Tsinghua Univ, Acad Arts & Design, Beijing 100084, Peoples R China
[2] Univ Illinois, Sch Informat Sci, Champaign, IL 61820 USA
[3] Tsinghua Univ, Dept Comp Sci & Technol, Beijing 100084, Peoples R China
来源：
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING | 2023年 / 14卷 / 01期
基金：
中国国家自然科学基金;
关键词：
Emotion recognition; Speech recognition; Databases; Sentiment analysis; Games; Electronic mail; Convolutional neural networks; Human-centered computing; emotion in human-computer interaction; influencing human emotional states; intelligent agents;
D O I：
10.1109/TAFFC.2022.3205919
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Emotion is important for the conversational user interface. In prior research, conversational agents (CAs) employ natural language process techniques to create affective interaction based on text. However, the use of acoustic features of speech for voice-based CAs is under exploration. This work presents an acoustically emotion-aware CA that enables speech emotion recognition and stylizes responses with empathetic feedback and interjections. We conducted an experiment with 75 participants to evaluate their perceived emotional intelligence (PEI) after interacting with the CA. Our results show that the acoustical emotion-awareness increased the participants' PEI of the CA, and the empathetic responses from the CA helped alleviate some participants' negative emotions. Our work provides implications for designing future CAs with better PEI.
引用
收藏
页码：17 / 30
页数：14
相关论文
共 50 条
[31]   Semi-Supervised Speech Emotion Recognition With Ladder Networks [J].
Parthasarathy, Srinivas ;
Busso, Carlos .
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2020, 28 :2697-2709
[32]   Speech emotion recognition using emotion perception spectral feature [J].
Jiang, Lin ;
Tan, Ping ;
Yang, Junfeng ;
Liu, Xingbao ;
Wang, Chao .
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2021, 33 (11)
[33]   Improving Empathetic Response Generation by Emotion Recognition and Information Filtration [J].
Shi, Guodong ;
Hou, Hongxu ;
Chen, Wei ;
Sun, Shuo ;
Zhao, Yuan ;
Ma, Jipeng .
ADVANCED INTELLIGENT COMPUTING TECHNOLOGY AND APPLICATIONS, PT III, ICIC 2024, 2024, 14877 :163-176
[34]   Efficient Feature-Aware Hybrid Model of Deep Learning Architectures for Speech Emotion Recognition [J].
Ezz-Eldin, Mai ;
Khalaf, Ashraf A. M. ;
Hamed, Hesham F. A. ;
Hussein, Aziza, I .
IEEE ACCESS, 2021, 9 :19999-20011
[35]   On the Correlation and Transferability of Features between Automatic Speech Recognition and Speech Emotion Recognition [J].
Fayek, Haytham M. ;
Lech, Margaret ;
Cavedon, Lawrence .
17TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2016), VOLS 1-5: UNDERSTANDING SPEECH PROCESSING IN HUMANS AND MACHINES, 2016, :3618-3622
[36]   Advanced differential evolution for gender-aware English speech emotion recognition [J].
Yue, Liya ;
Hu, Pei ;
Zhu, Jiulong .
SCIENTIFIC REPORTS, 2024, 14 (01)
[37]   Speech Emotion Recognition using DWT [J].
Lalitha, S. ;
Mudupu, Anoop ;
Nandyala, Bala Visali ;
Munagala, Renuka .
2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND COMPUTING RESEARCH (ICCIC), 2015, :20-23
[38]   Speech Emotion Recognition: A Comprehensive Survey [J].
Mohammed Jawad Al-Dujaili ;
Abbas Ebrahimi-Moghadam .
Wireless Personal Communications, 2023, 129 :2525-2561
[39]   Robust recognition of emotion from speech [J].
Hoque, Mohammed E. ;
Yeasin, Mohammed ;
Louwerse, Max M. .
INTELLIGENT VIRTUAL AGENTS, PROCEEDINGS, 2006, 4133 :42-53
[40]   Automatic emotion recognition by the speech signal [J].
Schuller, B ;
Lang, M ;
Rigoll, G .
6TH WORLD MULTICONFERENCE ON SYSTEMICS, CYBERNETICS AND INFORMATICS, VOL IX, PROCEEDINGS: IMAGE, ACOUSTIC, SPEECH AND SIGNAL PROCESSING II, 2002, :367-372
12345