Synchronized Video and EEG Based Childhood Epilepsy Seizure Detection

Childhood epilepsy seriously affects the nervous system development of children. Electroencephalogram (EEG) based epilepsy analysis is common in the past, but the inconvenient acquisition of EEG is the main challenge. In this paper, we firstly explored seizure detection performance of multi-modal synchronized video and EEG method. Further, we explore seizure detection only using video modal data. A novel childhood multi-modal epilepsy seizure detection algorithm using YOLOv3 for object detection, hybrid discriminate video and EEG feature representation is developed in the paper. After screening out interferences in video sequence by YOLOv3, the space-time interest points (STIPs) are extracted to characterize the body movement. The space-time interest points (STIPs) are extracted to characterize the body movement. Then, 4 popular features, Histogram of Oriented Gradient (HOG), Histograms of Oriented Optical Flow (HOF), Local Binary Pattern (LBP), and Motion Boundary Histogram (MBH) around STIPs are extracted. The Histograms of Word Frequency (HWF) features derived from a bag of words (BOW) model on HOG, HOF, LBP and MBH are developed for video representation. Meanwhile, the Mel-Frequency Cepstral Coefficients (MFCC) and Linear Predictive Cepstral Coefficients (LPCC) are extracted for EEG characterization. Multi-modal data of 13 childhood epilepsy patients from the Children's Hospital, Zhejiang University School of Medicine (CHZU) are studied. The fused EEG+Video feature based method could achieve an overall accuracy of 98.33%. Moreover, only using video feature, the method can achieve an overall accuracy of 93.30%.