Evaluating ANN Efficiency in Recognizing EEG and Eye-Tracking Evoked Potentials in Visual-Game-Events

EEG and Eye-tracking signals have customarily been analyzed and inspected visually in order to be correlated to the controlled stimuli. This process has proven to yield valid results as long as the stimuli of the experiment are under complete control (e.g.: the order of presentation). In this study, we have recorded the subject's electroencephalogram and eye-tracking data while they were exposed to a 2D platform game. In the game we had control over the design of each level by choosing the diversity of actions (i.e. events) afforded to the player. However we had no control over the order in which these actions were undertaken. The psychophysiological signals were synchronized to these game events and used to train and test an artificial neural network in order to evaluate how efficiently such a tool can help us in establishing the correlation, and therefore differentiating among the different categories of events. The highest average accuracies were between 60.25%-72.07%, hinting that it is feasible to recognize reactions to complex uncontrolled stimuli, like game events, using artificial neural networks.