Spatio-Temporal Image-Based Encoded Atlases for EEG Emotion Recognition

被引:7
作者
Avola, Danilo [1 ]
Cinque, Luigi [1 ]
Mambro, Angelo Di [1 ]
Fagioli, Alessio [1 ]
Marini, Marco Raoul [1 ]
Pannone, Daniele [1 ]
Fanini, Bruno [2 ]
Foresti, Gian Luca [3 ]
机构
[1] Sapienza Univ Rome, Dept Comp Sci, Via Salaria 113, I-00198 Rome, Italy
[2] CNR, Inst Heritage Sci, Area Ric Roma 1,SP35d,9, I-00010 Montelibretti, Italy
[3] Univ Udine, Dept Comp Sci Math & Phys, Via Sci 206, I-33100 Udine, Italy
关键词
Emotion recognition; image encoding; spatio-temporal atlases; multi-branch architecture; EEG; PRISMIN framework; CNN; LSTM; GRU; ViT; AUTISM SPECTRUM DISORDER; CLASSIFICATION; NETWORK; ATTENTION; NET; OSCILLATIONS; PERFORMANCE; SIGNAL; ALPHA; LSTM;
D O I
10.1142/S0129065724500242
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Emotion recognition plays an essential role in human-human interaction since it is a key to understanding the emotional states and reactions of human beings when they are subject to events and engagements in everyday life. Moving towards human-computer interaction, the study of emotions becomes fundamental because it is at the basis of the design of advanced systems to support a broad spectrum of application areas, including forensic, rehabilitative, educational, and many others. An effective method for discriminating emotions is based on ElectroEncephaloGraphy (EEG) data analysis, which is used as input for classification systems. Collecting brain signals on several channels and for a wide range of emotions produces cumbersome datasets that are hard to manage, transmit, and use in varied applications. In this context, the paper introduces the Empatheia system, which explores a different EEG representation by encoding EEG signals into images prior to their classification. In particular, the proposed system extracts spatio-temporal image encodings, or atlases, from EEG data through the Processing and transfeR of Interaction States and Mappings through Image-based eNcoding (PRISMIN) framework, thus obtaining a compact representation of the input signals. The atlases are then classified through the Empatheia architecture, which comprises branches based on convolutional, recurrent, and transformer models designed and tuned to capture the spatial and temporal aspects of emotions. Extensive experiments were conducted on the Shanghai Jiao Tong University (SJTU) Emotion EEG Dataset (SEED) public dataset, where the proposed system significantly reduced its size while retaining high performance. The results obtained highlight the effectiveness of the proposed approach and suggest new avenues for data representation in emotion recognition from EEG signals.
引用
收藏
页数:18
相关论文
共 136 条
[1]  
Adeli H, 2010, AUTOMATED EEG-BASED DIAGNOSIS OF NEUROLOGICAL DISORDERS: INVENTING THE FUTURE OF NEUROLOGY, P1
[2]   Comparison of wavelet transform and FFT methods in the analysis of EEG signals [J].
Akin M. .
Journal of Medical Systems, 2002, 26 (3) :241-247
[3]   A dynamic ensemble learning algorithm for neural networks [J].
Alam, Kazi Md Rokibul ;
Siddique, Nazmul ;
Adeli, Hojjat .
NEURAL COMPUTING & APPLICATIONS, 2020, 32 (12) :8675-8690
[4]  
Alom M.Z., 2018, arXiv
[5]   Analyzing EEG Data with Machine and Deep Learning: A Benchmark [J].
Avola, Danilo ;
Cascio, Marco ;
Cinque, Luigi ;
Fagioli, Alessio ;
Foresti, Gian Luca ;
Marini, Marco Raoul ;
Pannone, Daniele .
IMAGE ANALYSIS AND PROCESSING, ICIAP 2022, PT I, 2022, 13231 :335-345
[6]   Affective Action and Interaction Recognition by Multi-View Representation Learning from Handcrafted Low-Level Skeleton Features [J].
Avola, Danilo ;
Cascio, Marco ;
Cinque, Luigi ;
Fagioli, Alessio ;
Foresti, Gian Luca .
INTERNATIONAL JOURNAL OF NEURAL SYSTEMS, 2022, 32 (10)
[7]   LieToMe: An Ensemble Approach for Deception Detection from Facial Cues [J].
Avola, Danilo ;
Cascio, Marco ;
Cinque, Luigi ;
Fagioli, Alessio ;
Foresti, Gian Luca .
INTERNATIONAL JOURNAL OF NEURAL SYSTEMS, 2021, 31 (02)
[8]   Deep Temporal Analysis for Non-Acted Body Affect Recognition [J].
Avola, Danilo ;
Cinque, Luigi ;
Fagioli, Alessio ;
Foresti, Gian Luca ;
Massaroni, Cristiano .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2022, 13 (03) :1366-1377
[9]   LieToMe: Preliminary study on hand gestures for deception detection via Fisher-LSTM [J].
Avola, Danilo ;
Cinque, Luigi ;
De Marsico, Maria ;
Fagioli, Alessio ;
Foresti, Gian Luca .
PATTERN RECOGNITION LETTERS, 2020, 138 :455-461
[10]   Glandular Morphometrics for Objective Grading of Colorectal Adenocarcinoma Histology Images [J].
Awan, Ruqayya ;
Sirinukunwattana, Korsuk ;
Epstein, David ;
Jefferyes, Samuel ;
Qidwai, Uvais ;
Aftab, Zia ;
Mujeeb, Imaad ;
Snead, David ;
Rajpoot, Nasir .
SCIENTIFIC REPORTS, 2017, 7 :2220-2243