DEAP: A Database for Emotion Analysis Using Physiological Signals

被引:3046
作者
Koelstra, Sander [1 ]
Muhl, Christian [2 ]
Soleymani, Mohammad [3 ]
Lee, Jong-Seok [4 ]
Yazdani, Ashkan [5 ]
Ebrahimi, Touradj [5 ]
Pun, Thierry [3 ]
Nijholt, Anton [2 ]
Patras, Ioannis [1 ]
机构
[1] Queen Mary Univ London, Sch Elect Engn & Comp Sci, London E1 4NS, England
[2] Univ Twente, Human Media Interact Grp, NL-7500 AE Enschede, Netherlands
[3] Univ Geneva, Dept Comp Sci, CH-1227 Geneva, Switzerland
[4] Yonsei Univ, Sch Integrated Technol, Seoul 120749, South Korea
[5] Ecole Polytech Fed Lausanne, Inst Elect Engn IEL, Multimedia Signal Proc Grp, CH-1015 Lausanne, Switzerland
关键词
Emotion classification; EEG; physiological signals; signal processing; pattern classification; affective computing; SELF-ASSESSMENT MANNEQUIN; EEG; AUDIO;
D O I
10.1109/T-AFFC.2011.15
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/ dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.
引用
收藏
页码:18 / 31
页数:14
相关论文
共 59 条
[11]   EEG CORRELATES OF EMOTIONAL TASKS RELATED TO ATTENTIONAL DEMANDS [J].
COLE, HW ;
RAY, WJ .
INTERNATIONAL JOURNAL OF PSYCHOPHYSIOLOGY, 1985, 3 (01) :33-41
[12]  
Cornelius R.R., 1996, SCI EMOTION RES TRAD
[13]   UNIVERSALS AND CULTURAL-DIFFERENCES IN THE JUDGMENTS OF FACIAL EXPRESSIONS OF EMOTION [J].
EKMAN, P ;
FRIESEN, WV ;
OSULLIVAN, M ;
CHAN, A ;
DIACOYANNITARLATZIS, I ;
HEIDER, K ;
KRAUSE, R ;
LECOMPTE, WA ;
PITCAIRN, T ;
RICCIBITTI, PE ;
SCHERER, K ;
TOMITA, M ;
TZAVARAS, A .
JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY, 1987, 53 (04) :712-717
[14]   A 3-D Audio-Visual Corpus of Affective Communication [J].
Fanelli, Gabriele ;
Gall, Juergen ;
Romsdorfer, Harald ;
Weise, Thibaut ;
Van Gool, Luc .
IEEE TRANSACTIONS ON MULTIMEDIA, 2010, 12 (06) :591-598
[15]   EMG contamination of EEG: spectral and topographical characteristics [J].
Goncharova, II ;
McFarland, DJ ;
Vaughan, TM ;
Wolpaw, JR .
CLINICAL NEUROPHYSIOLOGY, 2003, 114 (09) :1580-1593
[16]   THE VERA AM MITTAG GERMAN AUDIO-VISUAL EMOTIONAL SPEECH DATABASE [J].
Grimm, Michael ;
Kroschel, Kristian ;
Narayanan, Shrikanth .
2008 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOLS 1-4, 2008, :865-+
[17]  
Gunes H, 2006, INT C PATT RECOG, P1148
[18]   Affective video content representation and modeling [J].
Hanjalic, A ;
Xu, LQ .
IEEE TRANSACTIONS ON MULTIMEDIA, 2005, 7 (01) :143-154
[19]   Clarifying the emotive functions of asymmetrical frontal cortical activity [J].
Harmon-Jones, E .
PSYCHOPHYSIOLOGY, 2003, 40 (06) :838-848
[20]  
Healey J. A., 2000, Ph.D. thesis