Classifying human emotions in HRI: applying global optimization model to EEG brain signals

被引:4
作者
Staffa, Mariacarla [1 ]
D'Errico, Lorenzo [2 ]
Sansalone, Simone [3 ]
Alimardani, Maryam [4 ]
机构
[1] Univ Naples Parthenope, Dept Sci & Technol, Naples, Italy
[2] Univ Naples Federico II, Dept Elect Engn & Informat Technol, Naples, Italy
[3] Univ Naples Federico II, Dept Phys, Naples, Italy
[4] Tilburg Univ, Dept Cognit Sci & Artificial Intelligence, Tilburg, Netherlands
关键词
human-robot interaction (HRI); EEG signals; brain-computer interface (BCI); frontal brain asymmetry (FBA); Theory of Mind (ToM); Global Optimization Model (GOM); machine learning; deep learning; RECOGNITION; ASYMMETRY; STRESS;
D O I
10.3389/fnbot.2023.1191127
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Significant efforts have been made in the past decade to humanize both the form and function of social robots to increase their acceptance among humans. To this end, social robots have recently been combined with brain-computer interface (BCI) systems in an attempt to give them an understanding of human mental states, particularly emotions. However, emotion recognition using BCIs poses several challenges, such as subjectivity of emotions, contextual dependency, and a lack of reliable neuro-metrics for real-time processing of emotions. Furthermore, the use of BCI systems introduces its own set of limitations, such as the bias-variance trade-off, dimensionality, and noise in the input data space. In this study, we sought to address some of these challenges by detecting human emotional states from EEG brain activity during human-robot interaction (HRI). EEG signals were collected from 10 participants who interacted with a Pepper robot that demonstrated either a positive or negative personality. Using emotion valence and arousal measures derived from frontal brain asymmetry (FBA), several machine learning models were trained to classify human's mental states in response to the robot personality. To improve classification accuracy, all proposed classifiers were subjected to a Global Optimization Model (GOM) based on feature selection and hyperparameter optimization techniques. The results showed that it is possible to classify a user's emotional responses to the robot's behavior from the EEG signals with an accuracy of up to 92%. The outcome of the current study contributes to the first level of the Theory of Mind (ToM) in Human-Robot Interaction, enabling robots to comprehend users' emotional responses and attribute mental states to them. Our work advances the field of social and assistive robotics by paving the way for the development of more empathetic and responsive HRI in the future.
引用
收藏
页数:12
相关论文
共 57 条
[21]  
Iengo S., 2012, 2012 RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, P1135, DOI 10.1109/ROMAN.2012.6343901
[22]  
Khosrowabadi R, 2011, 2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), P757, DOI 10.1109/IJCNN.2011.6033297
[23]   Affective Body Expression Perception and Recognition: A Survey [J].
Kleinsmith, Andrea ;
Bianchi-Berthouze, Nadia .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2013, 4 (01) :15-33
[24]   DEAP: A Database for Emotion Analysis Using Physiological Signals [J].
Koelstra, Sander ;
Muhl, Christian ;
Soleymani, Mohammad ;
Lee, Jong-Seok ;
Yazdani, Ashkan ;
Ebrahimi, Touradj ;
Pun, Thierry ;
Nijholt, Anton ;
Patras, Ioannis .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2012, 3 (01) :18-31
[25]  
Kuijt A., 2020, 2020 IEEE International Conference on Human-Machine Systems (ICHMS), P1
[26]   The effect of a naturalistic stressor on frontal EEG asymmetry, stress, and health [J].
Lewis, Richard S. ;
Weekes, Nicole Y. ;
Wang, Tracy H. .
BIOLOGICAL PSYCHOLOGY, 2007, 75 (03) :239-247
[27]   Real-Time Movie-Induced Discrete Emotion Recognition from EEG Signals [J].
Liu, Yong-Jin ;
Yu, Minjing ;
Zhao, Guozhen ;
Song, Jinjing ;
Ge, Yan ;
Shi, Yuanchun .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2018, 9 (04) :550-562
[28]  
Menon Shraddha, 2018, Computational Signal Processing and Analysis. Select Proceedings of ICNETS2: LNEE 490, P211, DOI 10.1007/978-981-10-8354-9_19
[29]   A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges [J].
Muehl, Christian ;
Allison, Brendan ;
Nijholt, Anton ;
Chanel, Guillaume .
BRAIN-COMPUTER INTERFACES, 2014, 1 (02) :66-84
[30]   Real-Time Neuroimaging and Cognitive Monitoring Using Wearable Dry EEG [J].
Mullen, Tim R. ;
Kothe, Christian A. E. ;
Chi, Yu Mike ;
Ojeda, Alejandro ;
Kerth, Trevor ;
Makeig, Scott ;
Jung, Tzyy-Ping ;
Cauwenberghs, Gert .
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, 2015, 62 (11) :2553-2567